- Email: [email protected]
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy for lesions detected at MRI screening and recommended for biopsy in a high-risk population?
Clinical Radiology (2009) 64, 1166e1174
ORIGINAL PAPER
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy for lesions detected at MRI screening and recommended for biopsy in a high-risk population?* T. Arazi-Kleinmana,c,*, P.A. Causera, R.A. Jonga, K. Hillb, E. Warnerb a
Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada, bDivision of Medical Oncology, Department of Medicine, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada, and cDepartment of Medical Imaging Tel Aviv Sourasky Medical Centre, Sackler School of Medicine Tel Aviv University, Tel Aviv, Israel Received 18 April 2009; received in revised form 2 August 2009; accepted 6 August 2009
AIM: To evaluate the sensitivity and specificity of magnetic resonance imaging (MRI) computer-aided detection (CAD) for breast MRI screen-detected lesions recommended for biopsy in a high-risk population. MATERIAL AND METHODS: Fifty-six consecutive Breast Imaging Reporting and Data System (BI-RADS) 3e5 lesions with histopathological correlation [nine invasive cancers, 13 ductal carcinoma in situ (DCIS) and 34 benign] were retrospectively evaluated using a breast MRI CAD prototype (CAD-Gaea). CAD evaluation was performed separately and in consensus by two radiologists specializing in breast imaging, blinded to the histopathology. Thresholds of 50, 80, and 100% and delayed enhancement were independently assessed with CAD. Lesions were rated as malignant or benign according to threshold and delayed enhancement only and in combination. Sensitivities, specificities, and negative predictive values (NPV) were determined for CAD assessments versus pathology. Initial MRI BI-RADS interpretation without CAD versus CAD assessments were compared using paired binary diagnostic tests. RESULTS: Threshold levels for lesion enhancement were: 50% to include all malignant (and all benign) lesions; and 100% for all invasive cancer and high-grade DCIS. Combined use of threshold and enhancement patterns for CAD assessment was best (73% sensitivity, 56% specificity and 76% NPV for all cancer). Sensitivities and NPV were better for invasive cancer (100%/100%) than for all malignancies (54%/76%). Radiologists’ MRI interpretation was more sensitive than CAD (p ¼ 0.05), but less specific (p ¼ 0.001) for cancer detection. CONCLUSION: The breast MRI CAD system used could not improve the radiologists’ accuracy for distinguishing all malignant from benign lesions, due to the poor sensitivity for DCIS detection. ª 2009 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Introduction The use of magnetic resonance imaging (MRI) in the detection and management of breast cancer is *
This research was presented at the RSNA 2007. * Guarantor and correspondent: T. Arazi-Kleinman, Department of Medical Imaging, Tel Aviv Sourasky Medical Center, 6 Weitzman St, Tel Aviv 64239, Israel. Tel.: þ972 3 697 3504; fax: þ972 3 697 3077. E-mail address: [email protected] (T. Arazi-Kleinman).
increasing. Breast MRI is highly sensitive for identifying breast malignancy that is occult on physical examination, mammography, and ultrasound.1e8 However, breast MRI has demonstrated variable specificity, resulting in increased false-positive examinations with their associated anxiety provoking call backs, diagnostic workups, and expensive, time-consuming breast MRI-guided biopsies.2,3 Computer-aided detection (CAD) programs have the potential to address the specificity limitation by automatically performing image processing
0009-9260/$ - see front matter ª 2009 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.crad.2009.08.003
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
and analysis, which may improve the diagnostic accuracy of breast MRI image interpretation.9e13 The aim of this study was to evaluate the sensitivity and specificity of a new MRI CAD software prototype (CAD-Gaea, version 0.9.497.814) for breast MRI screen-detected lesions recommended for biopsy in a high-risk population.
Material and methods The database from an ongoing MRI screening study of high-risk patients for breast cancer 14 was retrospectively searched from November 1997 to August 2006 for BI-RADS 3e5 lesions that were MRI detected and had histopathological correlation. Of a total of 516 women enrolled in the study over that time period there were 222 women with BRCA1 mutations, 192 with BRCA2 mutations, and 102 with a calculated lifetime risk of being a mutation carrier of 25% or higher based on having a first-degree relative with a mutation or a strong family history of earlyonset breast or ovarian cancer. From a total of 1548 breast MRI screening studies performed, there were 91 MRI-detected BI-RADS 3e5 lesions that underwent biopsy. BI-RADS 4 and 5 MRI lesions underwent biopsy based on MRI findings. BI-RADS 3 lesions underwent biopsy based on more concerning mammographic or ultrasound imaging findings of the correlate lesion. The initial bilateral screening MRI examinations were performed using institutional specific, research pulse sequences and software. From 1997 until 2001 these bilateral images were obtained without fat suppression. As the colourization algorithm of the CAD-Gaea software prototype was tailored heavily towards fatsuppressed data, and in order to standardize the MRI technique assessed in this study, only those BI-RADS 3e5 MRI-detected lesions subsequently evaluated with a unilateral diagnostic fat-suppressed (FS) MRI protocol were included. A total of 56 consecutive such lesions were included. The MRI screening study was approved by the institutional review board and informed written patient consent was obtained, to include retrospective review of imaging and medical records relevant to the study. These lesions were retrospectively evaluated with the CAD software. The prospective MRI reports were used for morphology assessment and BI-RADS category assessment.
Breast MRI technique MRI was performed using a 1.5T system (Signa; General Electric Medical Systems, Milwaukee, WI, USA) using a unilateral breast-dedicated MRI phased-array coil. The protocol includes:
1167
a localizing sequence; sagittal, FS, T2-weighted fast spin echo (FSE) [repetition time (TR) 4000 ms/ echo time (TE) 102 ms; matrix 128 256; field of view (FOV) 20 cm]; sagittal, T1-weighted, two-dimensional (2D), fast-spoiled gradient-recalled (FSPGR) FS (TR/TE 150 ms/4.2 ms; 50 flip angle; 256 128 matrix; 18e20 cm FOV; 3e4 mm section thickness) images, three series before and 11 after an intravenous injection of 0.1 mmol/kg gadodiamide (Omniscan; Amersham Health, Oakville, Ontario, Canada) at a rate of 2 ml/s; followed by a saline flush. Sagittal three-dimensional (3D) FSPGR FS (TR/TE 50 ms/4.2 ms; 50 flip angle; 256 512 matrix; 28e32 cm FOV; 1 mm section thickness) images, 6 min and 50 s and three dynamic runs of delayed sagittal T1-weighted 2D-FSPGR FS images as above were obtained of the breast.
MRI examination interpretation Over the eight and three quarter year period, all MRI examinations were prospectively interpreted by radiologists specializing in breast imaging. From 2001 to 2006, the MRI examinations were prospectively scored according to the preliminary and subsequently final BI-RADS classification for breast MRI.15e17 Prior to 2001, MRI examinations were scored using a combination of morphology and enhancement kinetics; the BIRADS Mammography lexicon was used as a template for lesion morphology.18,19 Details of lesion assessment have been previously described.14 In summary assessment was based primarily on morphology (mass versus non-mass enhancement, and symmetry of non-mass enhancement) using enhancement kinetics for indeterminate lesions.19 Three of four BI-RADS MRI category 3 lesions included were reclassified as BIRADS 4 at the 6-month follow-up MRI and, therefore, recommended for biopsy. One BI-RADS MRI category 3 lesion was biopsied based on its independent categorization as a BI-RADS 4 lesion based on screening ultrasound findings. For all BI-RADS category 4 (suspicious abnormality) and category 5 (highly suggestive of malignancy) lesions, correlation with mammography and breast ultrasound was performed in order to determine the easiest method for biopsy guidance. Lesions visible sonographically were biopsied using ultrasound guidance, mammographically with stereotactic mammographic guidance and those lesions that were only visible on MRI were biopsied using MRI guidance.
Automated software program All MRI examinations were retrospectively processed by the CAD-Gaea prototype version
1168
0.9.497.814 (Sentinelle Medical, Toronto, Canada; commercially released under the name Sentinelle Aegis software). The software provides tools for visualization and measurement of breast MR data in both 2D and 3D, as well as CAD analysis tools for visualizing contrast medium uptake dynamics. The CAD system produces colour overlay maps over lesions found to have enhancement meeting a user specified minimum threshold. In addition, it provides an automated interactive display of kinetic enhancement curves and details of all regions meeting the specified threshold on enhancement. For the purpose of this study to assess threshold and delayed enhancement independently, initial thresholds were displayed as one uniform colour. Once threshold assessments were performed, the delayed enhancement was displayed using three colours. The CAD-Gaea software provides all the analysis tools typically available in other breast CAD systems, such as visualizing and comparing image sets, viewing subtracted image data, contrast uptake colourization and curve analysis, region on interest (ROI) and measurement tools, and report generation tools. Other software features include true, 3D visualization and image processing in real-time and allowing instant, on-the-fly MPR (multiplanar reformatting) and real-time viewing of minimum intensity projections and surface volumes in any orientation. In addition, the software also includes a comprehensive set of interventional tools to aid in lesion targeting and biopsy needle guidance. For breast MRI examinations, the CAD system incorporates the unenhanced T1-weighted series and as many contrast-enhanced T1-weighted series as a specific examination includes. To determine whether areas on the MRI examination have ‘‘significant’’ enhancement, pixel intensity values on the unenhanced and subsequent contrastenhanced series are compared. If a pixel value increases by a specified minimum enhancement threshold, the enhancement is significant and the pixel is colour enhanced on the monitor. The minimum threshold selected can range from 30e200%. The optimum minimum enhancement threshold to use has not yet been determined. If a pixel value does not increase by the established threshold, no colour enhancement appears. For the delayed enhancement pattern the CAD program then assigns a specific colour to each pixel that meets threshold enhancement, based on the slope of a best least-squares fit of a line of the delayed enhancement intensity data. If the best-fit line decreases at a rate greater than 10%/240 s relative to the peak uptake value, the pixel is
T. Arazi-Kleinman et al.
colour-coded red, indicating a washout enhancement pattern. If the line increases at a rate greater than 10%/240 s relative to the peak uptake value, the pixel is colour-coded blue, indicating a persistent enhancement pattern. If the slope of the best-fit line is between these two values, the pixel is colour-coded green, indicating a plateau enhancement pattern. The end result is a colour overlay on each MRI image over regions of significant enhancement. The radiologist then selects a specific area of enhancement and the CAD program automatically generates a synopsis with details of the enhancement type and extent. The CAD prototype used for this study is based on the signal enhancement ratio method, which is also employed in other commercially available breast MRI CAD products. All MRI examinations were retrospectively processed by CAD-Gaea (Sentinelle Medical) and assessed by two breast radiologists, who had 5.5 (P.A.C.) and 5 years (R.A.J) of breast MRI experience, blinded to the lesion’s histopathology. Initial interpretations were performed independently and discrepant cases were resolved in consensus. Initially the lesion was located on the MR examination by the radiologists (Fig. 1a and b). Then the presence or absence of significant enhancement was recorded for each lesion at the 50, 80, and 100% enhancement threshold levels. Subsequently, delayed enhancement patterns both with colour mapping and curve analysis made by the radiologists were also recorded for each lesion. Each lesion was then scored as malignant or benign based on thresholds, delayed enhancement pattern and a combination of both, without the knowledge of histopathology. Lesion enhancement was compared to the background parenchymal enhancement to determine significance (Fig. 1cee). For each threshold, a lesion enhancement was required to be greater than the background to be considered significant. Similarly for delayed enhancement pattern, the pattern was required to be distinguishable in colour display from the background enhancement to be considered significant (Fig. 1feh). Plateau and washout delayed enhancement pattern were ranked malignant and continuous was ranked benign.19 Finally, the radiologists manually created an ROI curve for each lesion (Fig. 1i).
Data analysis Sensitivities, specificities, negative predictive values (NPV), and positive predictive values (PPV) were calculated for lesion malignancy assessment versus the presence of malignancy based on the
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1169
Figure 1 MRI images without and with CAD applied in a 33-year-old BRCA-1 genetic mutation carrier with a left breast MRI-detected invasive ductal carcinoma (IDC), diagnosed at ultrasound-guided core biopsy. (a) Dynamic, sagittal, T1weighted, FS 2D FSPGR image with contrast enhancement image shows an oval, 8 mm circumscribed mass (arrow) at the 12 o’clock position of the left breast, middle third. (b) High spatial resolution, sagittal, T1-weighted, 3D FSPGR FS image shows the same mass as a heterogeneous, rim enhancing mass (arrow). Single CAD colour mapping in purple demonstrates areas in the breast reaching threshold levels of (c) 50%, (d) 80%, and (e) 100%. CAD colour mapping of the delayed enhancement pattern at: (f) 50% threshold, (g) 80% threshold and (h) 100% threshold. Blue represents a persistent delayed enhancement pattern, green represents a plateau delayed enhancement pattern, and red represents a washout delayed enhancement pattern. (i) Radiologist-generated lesion kinetic curve, made by the manual placement of a region of interest on the red aspect of the lesion. A washout delayed enhancement pattern of this IDC was confirmed by curve analysis. The colour mapping of red within the mass in (feh) (arrow) also suggested a washout delayed enhancement pattern.
histopathology, for significant enhancement versus backgrounds at 50, 80,and 100% thresholds, delayed enhancement pattern (based on the CAD colour mapping and the manual ROI created by the
radiologists), as well as an integrated assessment considering the initial and delayed enhancement pattern of the lesion. Fisher’s exact test was used to calculate the association between the lesion
1170
histopathology and the MRI morphology category of mass versus non-mass. Chi-square tests were used to calculate the association between the delayed enhancement pattern and the histopathology of the MRI-detected lesions. The paired binary diagnostic tests for comparison of the sensitivity and specificity of two tests were used to compare the initial radiologist MRI interpretation as the prospective BI-RADS category assessment versus the CAD assessment with respect to the presence of malignancy.18 All statistical calculations were performed using SAS (Statistical Analysis System),20 Rosner’s book, Fundamentals of Biostatistics 21 and NCSS (Number Cruncher Statistical System).22
Results Patients and lesion characteristics Fifty-six consecutive lesions were found in 53 women (age range 26e68; mean 47 years). All women were at high risk for hereditary breast cancer. Thirty-two (57%) lesions were found in BRCA-1 mutation carriers, 17 (30%) in BRCA-2 mutation carriers and seven (13%) lesions were found in patients with a strong family history calculated to have at least a 25% risk of carrying a BRCA mutation as computed using the BRCA PRO software. Twenty-two of the lesions (39%) were malignant (13 DCIS, eight IDC, and one ILC) and 34 lesions were benign (Table 1). Most of the DCIS lesions had an intermediate nuclear grading (9/13, 69%), three had a high nuclear grade (3/13, 23%) and one had a low nuclear grade (1/13, 8%). Table 2 lists the correlation between the lesion pathology and the patient risk level. Details of MRI lesion characteristics, BI-RADS prospective categories, and method of biopsy guidance are listed in Table 3. Malignant lesions were larger than benign lesions (mean 17.7 mm versus 14.1 mm). Thirty-four (61%) were non-mass compared with 22 (39%) masses. A significant relationship did not exist between the MRI morphology categories of mass versus non-mass and benign versus malignant pathology (p ¼ 0.47). Of the malignancies, most masses (6/8, 75%) were invasive and most of the non-mass lesions (11/14, 79%) were DCIS.
Lesion enhancement and accuracy of CAD lesion’s assessment The invasive cancers had significant enhancement at all the threshold levels. However, the benign and DCIS lesions had variably significant enhancement
T. Arazi-Kleinman et al.
Table 1 Magnetic resonance imaging-detected lesions according to histopathological findings Histopathology
Total-n (%)
Benign Fibroadenoma Fibrocystic changes Other benign lesions Normal breast tissue/no pathological diagnosis Malignant Ductal carcinoma in situ Low grade Intermediate grade High grade Invasive ductal carcinoma Invasive lobular carcinoma
34 2 13 8 11
(6) (38) (24) (32)
22 13 1 9 3 8 1
(59) (8) (69) (23) (36) (5)
at the different threshold levels. All lesions (malignant and benign) enhanced at 50% threshold. At 80% threshold only six lesions did not enhance, of which five were benign and one was DCIS of intermediate nuclear grade. At 100% threshold all the invasive cancers enhanced, but only (7/13, 54%) of the DCIS and (19/34, 56%) of the benign lesions enhanced (Fig. 2). Of the six DCIS lesions that did not enhance at 100% threshold, one had a low nuclear grade and five were intermediate nuclear grade. The three DCIS lesions with a high nuclear grade enhanced at all the threshold levels. Most lesions demonstrated plateau delayed enhancement either by CAD colour mapping (38/56, 68%) or by a radiologist generated manual curve (35/56, 62.5%). Although most lesions with persistent enhancement were benign (7/8, 87%), one lesion (1/8, 13%) was malignant (DCIS with a high nuclear grade). CAD colour mapping of the delayed enhancement differed from the manual curve for three lesions including: two malignancies (one DCIS and one invasive cancer) for which a washout curve was obtained by the manual curve generated and the CAD colour mapped a plateau
Table 2 Lesion
Lesions according to high-risk patient status BRCA 1 mutation carrier
Ductal carcinoma in 6 (19%) situ (13/56, 23%) Invasive ductal 5 (16%) carcinoma (8/56, 14%) Invasive lobular 0 carcinoma (1/56, 2%) Benign (34/56, 61%) 21 (65%) Total ¼ 56
32 (57%)
BRCA 2 mutation carrier
Family history
6 (35%)
1 (14%)
3 (18%)
0
0
1 (14%)
8 (47%)
5 (72%)
17 (30%)
7 (13%)
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1171
Table 3 Magnetic resonance imaging (MRI) lesion characteristics, lesion prospective BI-RADS category assessment, lesion biopsy guidance method, and patient risk factor according to benign or malignant histopathology Characteristic
Benign (n ¼ 34)
Malignant (n ¼ 22)
Mean size (size range, mm) 14.1 (4e50) 17.7 (5e69) Morphology Mass 14 (42) 8 (36) Non-mass 20 (58) 14 (64) Radiologists’ prospective BI-RADS 3 2 (6) 2 (9) 4 31 (91) 18 (82) 5 1 (3) 2 (9) Biopsy method Ultrasound-guided percutaneous 9 (26) 7 (32) biopsy MRI-guided percutaneous biopsy 16 (48) 4 (18) MRI preoperative 8 (23) 8 (36) wire localization Stereotactic-guided 1 (3) 3 (14) percutaneous biopsy Risk factor BRCA-1 21 11 BRCA-2 8 9 Strong family history 5 2 Data are numbers of lesions. Numbers in parentheses are percentages. BI-RADS, Breast Imaging Reporting and Data System.
delayed enhancement; and one benign lesion for which a persistent delayed enhancement was obtained from the manual curve while CAD colour mapped a plateau delayed enhancement. Chisquare tests were not significant for an association between the delayed enhancement pattern and the histopathology of the MRI-detected lesions (Table 4). As no significant relationship between lesion MRI morphology of mass versus non-mass and histology was found, the accuracy of CAD based on MRI morphology was not performed. Finally, the lesions were categorized as benign or malignant based on the CAD interpretation including: at each threshold of 50, 80, and 100%; overall threshold; delayed enhancement pattern and the integrated evaluation of threshold and delayed enhancement information. Malignancy assessment based on the delayed enhancement pattern was more sensitive (73% versus 41%) and had a higher NPV (73% versus 65%) but was less specific (47% versus 57%) compared with overall threshold of enhancement. The final integrated evaluation of threshold and delayed enhancement pattern optimized sensitivity (73%), NPV (76%), and specificity (56%) for detecting malignant lesions. Sensitivity increased to 100% when only invasive cancers were included, but was lower for all DCIS detection (73%; Table 5).
Figure 2 Significant enhancement at different threshold levels according to lesion histopathology. All the invasive cancers enhanced at all threshold levels. There are decreasing numbers of benign and DCIS lesions enhancing at the 80 and 100% threshold levels, the greatest decrease occurring between the 80e100% threshold levels.
The paired binary diagnostic tests for comparison of sensitivity and specificity of two tests, showed that the prospective radiologist interpretation was significantly more sensitive than the CAD lesion interpretation (p ¼ 0.05); however, the overall CAD lesion analysis was more specific than radiologist MRI lesion assessment (p ¼ 0.001).
Discussion This study evaluated a new commercial CAD software prototype for breast MR examination in the high risk screening setting. The software was very sensitive and specific for invasive cancer; however, it was less sensitive and specific if DCIS was included, demonstrating 73% sensitivity, 56% specificity, and 76% NPV. Current breast MRI CAD using lesion enhancement information only cannot improve the accuracy of the radiologist for distinguishing all malignant from benign screening MRI detected lesions, due to the poor sensitivity for DCIS detection. At a peak enhancement threshold of 100% all the invasive cancers enhanced but only about half the DCIS and benign lesions reached the 100% enhancement threshold level. Therefore, the initial peak enhancement is predictive of invasive cancer, but not of all non-invasive cancers. If considering invasive cancer only, the initial portion of the timeesignal intensity curve provided by the CAD system is most predictive of malignancy similar to the previous report of Williams et al.12
1172
T. Arazi-Kleinman et al.
Table 4 Magnetic resonance imaging lesion delayed enhancement pattern according to computed-aided detection (CAD) colour mapping, region of interest (ROI) created curve and histopathological findings CAD colour mapping
Persistent e curve 1 Plateau e curve 2 Washout e curve 3
ROI radiologist-created curve
Malignant (n ¼ 22)
Benign (n ¼ 34)
Total (n ¼ 56)
Malignant (n ¼ 22)
Benign (n ¼ 34)
Total (n ¼ 56)
1 (5) 17 (77) 4 (18)
6 (17) 21 (62) 7 (21)
7 (13) 38 (68) 11 (19)
1 (5) 15 (68) 6 (27)
7 (21) 20 (58) 7 (21)
8 (14) 35 (63) 13 (23)
Data are numbers of lesions. Numbers in parentheses are percentages.
only including invasive cancers. Also similar to previous reports, no significant association was found between the delayed enhancement pattern of lesions and malignancy.9,12,23,24 However, malignancy assessment of the lesions with regard to the background enhancement had a better sensitivity and NPV based on the delayed enhancement pattern compared to malignancy assessment based on the initial peak enhancement, but was less specific. The colour mapping helps differentiate the lesion from the background in demonstrating the differences in the delayed enhancement pattern of the lesion compared to the background (Fig. 1). However, as the different delayed enhancement patterns by themselves are not associated with histopathology results, they are not specific for malignancy (Figs. 1 and 3). Lesion malignancy assessment based on the delayed enhancement pattern with respect to the background enhancement was more sensitive and had a higher NPV compared with the malignancy assessment based on the threshold enhancement (73 versus 41%, 73 versus 65%). However, the specificity of the delayed enhancement was lower (47 versus 71%). Other studies demonstrated an association between the delayed enhancement pattern and histology.19 Kuhl et al.20 evaluated the timeeintensity curve of 226 lesions seen on MRI. Their results showed that a pattern of delayed washout curve was more likely to be associated with malignant lesions and that a pattern of persistent enhancement was significantly more likely associated with benign lesions. However, there was overlap in the patterns of benign and malignant lesions. One of the explanations for this
difference could be related to the percentage of non-invasive cancers, which was 23% in the present study compared with 4.1% in the study by Kuhl et al.19 Hamilton and colleagues25 found that BRCA 2 mutation carriers are more likely to present with DCIS compared with BRCA 1 mutation carriers. In the present series, an equal number of cases of DCIS among women with BRCA1 and BRCA2 mutations was found; however, DCIS was proportionately more common among the BRCA2 mutation carriers who represented only one third of the patients in the present study. Other studies have also found kinetics to be less predictive of invasive disease. Schnall and his colleagues26 in their study of 995 lesions found that the margins of a mass (Az ¼ 0.76), distribution of non-mass enhancement (Az ¼ 0.78) and signal intensity (Az ¼ 0.70) were more predictive of malignant disease than the qualitative enhancement kinetics of the lesion (Az ¼ 0.66).26 Investigating the outcome of MRI-guided breast biopsies based on imaging features and indication for MRI, Han and colleagues27 found that of 150 biopsied lesions, the probability of malignancy was not statistically significant for the different kinetic features. The present breast MRI CAD prototype assessing enhancement information only is very sensitive for detecting invasive cancers with a high NPV for invasive cancers, with the highest values seen in the lesion colour assessment with respect to the background and in the final integrated analysis of all the CAD components. However, this is not true with respect to DCIS, hence the total assessment of the CAD for malignant disease is not very sensitive.
Table 5 Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of magnetic resonance imaging lesion CAD assessments
Threshold Sensitivity Specificity PPV NPV
50%
80%
100%
Final
0.14 0.97 0.75 0.64
0.23 0.82 0.46 0.62
0.27 0.71 0.38 0.60
0.41 0.71 0.47 0.65
Final colour
Final Integrated
0.73 0.47 0.47 0.73
0.73 0.56 0.52 0.76
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1173
Figure 3 MR images without and with CAD applied in a 57-year-old BRCA-2 genetic mutation carrier with a left breast MRI-detected DCIS, diagnosed at MRI-guided wire localization and surgical excisional biopsy. (a) Sagittal, T1-weighted, 3D, FSPGR FS image shows a spiculate 13 mm mass (arrow) in the lower inner quadrant left breast. (b) CAD assessment of the lesion demonstrated enhancement only at the 50% threshold level (arrow). CAD colour mapping of the delayed enhancement pattern shows green within the lesion, indicating a plateau delayed enhancement pattern that did not differ from the remaining background enhancement (arrowheads) and was, therefore, assessed as benign based on CAD.
There are several limitations to this study. Being a single-site study with one MRI technique, the results may not extrapolate to different sites using different MRI equipment, techniques, and interpretation by different radiologists. As with interpretation of the MRI examination, there may be interobserver interpretation differences of the CAD application that were not addressed in this or prior studies. This was a retrospective study of CAD assessment of lesions that were considered suspicious enough for biopsy and the results can not necessarily extrapolate to prospective applications of CAD for screening MRI. This commercially available CAD system for breast MRI examinations using lesion enhancement information only can be a helpful tool for radiologist interpretation, creating a colour mapping of whole breast enhancement, improving the specificity of the radiologist interpretation to detect invasive cancer. The 100% threshold optimizes the diagnostic accuracy for MRI screen-detected invasive cancers and high-grade DCIS. However, using enhancement information only is unreliable for detecting all non-invasive cancers, limiting the overall sensitivity of the CAD for all malignant lesions. New CAD systems for breast MRI incorporating lesion morphological characteristics are
already being tested, and may optimize the accuracy of breast MRI CAD systems for all cancers.
Acknowledgements The present study was supported by the following grants: Canadian Breast Cancer Research Initiative, Terry Fox Foundation of the National Cancer Institute of Canada, the Ontario Research and Development; and the Challenge Fund and Amersham Health
References 1. Morris EA. Breast cancer imaging with MRI. Radiol Clin North Am 2002;40:443e66. 2. Heywang SH, Wolf A, Pruss E, et al. MR Imaging of the breast with Gd-DTPA: use and limitations. Radiology 1989;171:95e103. 3. Kaiser WA, Zeitler E. MR Imaging of the breast: fast imaging sequences with and without Gd-DTPA. Preliminary observations. Radiology 1989;170:681e6. 4. Kriege M, Brekelmans C, Boetes C, et al. Efficacy of MRI and mammography for breast-cancer screening in women with a familial or genetic predisposition. N Engl J Med 2004;351:427e37. 5. Kuhl CK, Schmutzler RK, Leutner CC, et al. Breast MR imaging screening in 192 women proved or suspected to be carriers of a breast cancer susceptibility gene: preliminary results. Radiology 2000;215:267e79.
1174
6. Tilanus-Linthorst MM, Obdeijn IM, Bartels KC, et al. First experiences in screening women at high risk for breast cancer. Breast Cancer Res Treat 2000;63:53e60. 7. Warner E, Plewes DB, Shumak RS, et al. Comparison of breast magnetic resonance imaging, mammography and ultrasound for surveillance of women at high risk for hereditary breast cancer. J Clin Oncol 2001;19:3524e31. 8. Stoutjesdijk MJ, Boetes C, Jager GJ, et al. Magnetic resonance imaging and mammography in women with a hereditary risk of breast cancer. J Natl Cancer Inst 2001;93:1095e102. 9. Lehman CD, Peacock S, DeMartini WB, et al. A new automated software system to evaluate breast MR examinations: improved specificity without decreased sensitivity. AJR Am J Roentgenol 2006;187:51e6. 10. DeMartini WB, Lehman CD, Peacock S, et al. Computer-aided detection applied to breast MRI: assessment of CAD-generated enhancement and tumor sizes in breast cancers before and after neoadjuvant chemotherapy. Acad Radiol 2005;12:806e14. 11. Wood C. Computer aided detection (CAD) for breast MRI. Technol Cancer Res Treat 2005;4:49e53. 12. Williams TC, DeMartini WB, Partridge SC, et al. Breast MR imaging: computer-aided evaluation program for discriminating benign from malignant lesions. Radiology 2007;244:94e103. 13. Levman J, Leung T, Causer P, et al. Classification of dynamic contrast-enhanced magnetic resonance breast lesions by support vector machines. IEEE Trans Med Imaging 2008;27:688e96. 14. Warner E, Plewes DB, Hill KA, et al. Surveillance of BRCA1 and BRCA2 mutation carriers with magnetic resonance imaging, ultrasound, mammography, and clinical breast examination. J Am Med Assoc 2004;292:1317e25. 15. Ikeda DM, Hylton NM, Kinkel K, et al. Development, standardization and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies. J Magn Reson Imaging 2001;14:889e95.
T. Arazi-Kleinman et al.
16. Morris EA. Illustrated breast MRI lexicon. Semin Roentgenol 2001;36:238e49. 17. American College of Radiology. BI-RADS: MRI. In: Breast imaging reporting and data system: BI-RADS atlas. 4th ed. Reston, VA: American College of Radiology; 2003. 18. American College of Radiology. BI-RADS: mammography. In: Illustrated breast imaging reporting and data system: BIRADS. 3rd ed. Reston, VA: American College of Radiology; 1998. 19. Kuhl CK, Meilcareck P, Klaschik S, et al. Dynamic breast MR imaging: are signal intensity time course data useful for differential diagnosis of enhancing lesions? Radiology 1999; 211:101e10. 20. SAS, Statistical Analysis Stystem, Version 8.2, 2002; SAS Institute Inc, Cary, NC, USA. 21. Rosner B. Fundamentals of Biostatistics. Thomson Higher Education, Belmont CA, USA. 6th edition, 2006. 22. Hintze JL. Number Cruncher Statistical System (NCSS). User’s guide. V. Kaysville: Utah; 2003. pp. 5071e5078. 23. Orel SG, Shnall MD, LiVolsi VA, et al. Suspicious breast lesions: MR imaging with radiologicepathologic correlation. Radiology 1994;190:485e93. 24. Stomper PC, Herman S, Klippenstein DL, et al. Suspect breast lesions: finding at dynamic gadolinium-enhanced MR imaging correlated with mammography and pathologic features. Radiology 1995;197:387e95. 25. Hamilton LJ, Evans AJ, Wilson ARM, et al. Breast imaging finding in women with BRCA-1 and BRCA-2 associated breast carcinoma. Clin Radiol 2004;59:895e902. 26. Schnall MD, Blume J, Bluemke DA, et al. Diagnostic architectural and dynamic features at breast MR imaging: multicentric study. Radiology 2006;238:42e53. 27. Han BK, Schnall MD, Orel SG, et al. Outcome of MRI guided breast biopsy. AJR Am J Roentgenol 2008;191: 1798e804.
ORIGINAL PAPER
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy for lesions detected at MRI screening and recommended for biopsy in a high-risk population?* T. Arazi-Kleinmana,c,*, P.A. Causera, R.A. Jonga, K. Hillb, E. Warnerb a
Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada, bDivision of Medical Oncology, Department of Medicine, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada, and cDepartment of Medical Imaging Tel Aviv Sourasky Medical Centre, Sackler School of Medicine Tel Aviv University, Tel Aviv, Israel Received 18 April 2009; received in revised form 2 August 2009; accepted 6 August 2009
AIM: To evaluate the sensitivity and specificity of magnetic resonance imaging (MRI) computer-aided detection (CAD) for breast MRI screen-detected lesions recommended for biopsy in a high-risk population. MATERIAL AND METHODS: Fifty-six consecutive Breast Imaging Reporting and Data System (BI-RADS) 3e5 lesions with histopathological correlation [nine invasive cancers, 13 ductal carcinoma in situ (DCIS) and 34 benign] were retrospectively evaluated using a breast MRI CAD prototype (CAD-Gaea). CAD evaluation was performed separately and in consensus by two radiologists specializing in breast imaging, blinded to the histopathology. Thresholds of 50, 80, and 100% and delayed enhancement were independently assessed with CAD. Lesions were rated as malignant or benign according to threshold and delayed enhancement only and in combination. Sensitivities, specificities, and negative predictive values (NPV) were determined for CAD assessments versus pathology. Initial MRI BI-RADS interpretation without CAD versus CAD assessments were compared using paired binary diagnostic tests. RESULTS: Threshold levels for lesion enhancement were: 50% to include all malignant (and all benign) lesions; and 100% for all invasive cancer and high-grade DCIS. Combined use of threshold and enhancement patterns for CAD assessment was best (73% sensitivity, 56% specificity and 76% NPV for all cancer). Sensitivities and NPV were better for invasive cancer (100%/100%) than for all malignancies (54%/76%). Radiologists’ MRI interpretation was more sensitive than CAD (p ¼ 0.05), but less specific (p ¼ 0.001) for cancer detection. CONCLUSION: The breast MRI CAD system used could not improve the radiologists’ accuracy for distinguishing all malignant from benign lesions, due to the poor sensitivity for DCIS detection. ª 2009 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Introduction The use of magnetic resonance imaging (MRI) in the detection and management of breast cancer is *
This research was presented at the RSNA 2007. * Guarantor and correspondent: T. Arazi-Kleinman, Department of Medical Imaging, Tel Aviv Sourasky Medical Center, 6 Weitzman St, Tel Aviv 64239, Israel. Tel.: þ972 3 697 3504; fax: þ972 3 697 3077. E-mail address: [email protected] (T. Arazi-Kleinman).
increasing. Breast MRI is highly sensitive for identifying breast malignancy that is occult on physical examination, mammography, and ultrasound.1e8 However, breast MRI has demonstrated variable specificity, resulting in increased false-positive examinations with their associated anxiety provoking call backs, diagnostic workups, and expensive, time-consuming breast MRI-guided biopsies.2,3 Computer-aided detection (CAD) programs have the potential to address the specificity limitation by automatically performing image processing
0009-9260/$ - see front matter ª 2009 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.crad.2009.08.003
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
and analysis, which may improve the diagnostic accuracy of breast MRI image interpretation.9e13 The aim of this study was to evaluate the sensitivity and specificity of a new MRI CAD software prototype (CAD-Gaea, version 0.9.497.814) for breast MRI screen-detected lesions recommended for biopsy in a high-risk population.
Material and methods The database from an ongoing MRI screening study of high-risk patients for breast cancer 14 was retrospectively searched from November 1997 to August 2006 for BI-RADS 3e5 lesions that were MRI detected and had histopathological correlation. Of a total of 516 women enrolled in the study over that time period there were 222 women with BRCA1 mutations, 192 with BRCA2 mutations, and 102 with a calculated lifetime risk of being a mutation carrier of 25% or higher based on having a first-degree relative with a mutation or a strong family history of earlyonset breast or ovarian cancer. From a total of 1548 breast MRI screening studies performed, there were 91 MRI-detected BI-RADS 3e5 lesions that underwent biopsy. BI-RADS 4 and 5 MRI lesions underwent biopsy based on MRI findings. BI-RADS 3 lesions underwent biopsy based on more concerning mammographic or ultrasound imaging findings of the correlate lesion. The initial bilateral screening MRI examinations were performed using institutional specific, research pulse sequences and software. From 1997 until 2001 these bilateral images were obtained without fat suppression. As the colourization algorithm of the CAD-Gaea software prototype was tailored heavily towards fatsuppressed data, and in order to standardize the MRI technique assessed in this study, only those BI-RADS 3e5 MRI-detected lesions subsequently evaluated with a unilateral diagnostic fat-suppressed (FS) MRI protocol were included. A total of 56 consecutive such lesions were included. The MRI screening study was approved by the institutional review board and informed written patient consent was obtained, to include retrospective review of imaging and medical records relevant to the study. These lesions were retrospectively evaluated with the CAD software. The prospective MRI reports were used for morphology assessment and BI-RADS category assessment.
Breast MRI technique MRI was performed using a 1.5T system (Signa; General Electric Medical Systems, Milwaukee, WI, USA) using a unilateral breast-dedicated MRI phased-array coil. The protocol includes:
1167
a localizing sequence; sagittal, FS, T2-weighted fast spin echo (FSE) [repetition time (TR) 4000 ms/ echo time (TE) 102 ms; matrix 128 256; field of view (FOV) 20 cm]; sagittal, T1-weighted, two-dimensional (2D), fast-spoiled gradient-recalled (FSPGR) FS (TR/TE 150 ms/4.2 ms; 50 flip angle; 256 128 matrix; 18e20 cm FOV; 3e4 mm section thickness) images, three series before and 11 after an intravenous injection of 0.1 mmol/kg gadodiamide (Omniscan; Amersham Health, Oakville, Ontario, Canada) at a rate of 2 ml/s; followed by a saline flush. Sagittal three-dimensional (3D) FSPGR FS (TR/TE 50 ms/4.2 ms; 50 flip angle; 256 512 matrix; 28e32 cm FOV; 1 mm section thickness) images, 6 min and 50 s and three dynamic runs of delayed sagittal T1-weighted 2D-FSPGR FS images as above were obtained of the breast.
MRI examination interpretation Over the eight and three quarter year period, all MRI examinations were prospectively interpreted by radiologists specializing in breast imaging. From 2001 to 2006, the MRI examinations were prospectively scored according to the preliminary and subsequently final BI-RADS classification for breast MRI.15e17 Prior to 2001, MRI examinations were scored using a combination of morphology and enhancement kinetics; the BIRADS Mammography lexicon was used as a template for lesion morphology.18,19 Details of lesion assessment have been previously described.14 In summary assessment was based primarily on morphology (mass versus non-mass enhancement, and symmetry of non-mass enhancement) using enhancement kinetics for indeterminate lesions.19 Three of four BI-RADS MRI category 3 lesions included were reclassified as BIRADS 4 at the 6-month follow-up MRI and, therefore, recommended for biopsy. One BI-RADS MRI category 3 lesion was biopsied based on its independent categorization as a BI-RADS 4 lesion based on screening ultrasound findings. For all BI-RADS category 4 (suspicious abnormality) and category 5 (highly suggestive of malignancy) lesions, correlation with mammography and breast ultrasound was performed in order to determine the easiest method for biopsy guidance. Lesions visible sonographically were biopsied using ultrasound guidance, mammographically with stereotactic mammographic guidance and those lesions that were only visible on MRI were biopsied using MRI guidance.
Automated software program All MRI examinations were retrospectively processed by the CAD-Gaea prototype version
1168
0.9.497.814 (Sentinelle Medical, Toronto, Canada; commercially released under the name Sentinelle Aegis software). The software provides tools for visualization and measurement of breast MR data in both 2D and 3D, as well as CAD analysis tools for visualizing contrast medium uptake dynamics. The CAD system produces colour overlay maps over lesions found to have enhancement meeting a user specified minimum threshold. In addition, it provides an automated interactive display of kinetic enhancement curves and details of all regions meeting the specified threshold on enhancement. For the purpose of this study to assess threshold and delayed enhancement independently, initial thresholds were displayed as one uniform colour. Once threshold assessments were performed, the delayed enhancement was displayed using three colours. The CAD-Gaea software provides all the analysis tools typically available in other breast CAD systems, such as visualizing and comparing image sets, viewing subtracted image data, contrast uptake colourization and curve analysis, region on interest (ROI) and measurement tools, and report generation tools. Other software features include true, 3D visualization and image processing in real-time and allowing instant, on-the-fly MPR (multiplanar reformatting) and real-time viewing of minimum intensity projections and surface volumes in any orientation. In addition, the software also includes a comprehensive set of interventional tools to aid in lesion targeting and biopsy needle guidance. For breast MRI examinations, the CAD system incorporates the unenhanced T1-weighted series and as many contrast-enhanced T1-weighted series as a specific examination includes. To determine whether areas on the MRI examination have ‘‘significant’’ enhancement, pixel intensity values on the unenhanced and subsequent contrastenhanced series are compared. If a pixel value increases by a specified minimum enhancement threshold, the enhancement is significant and the pixel is colour enhanced on the monitor. The minimum threshold selected can range from 30e200%. The optimum minimum enhancement threshold to use has not yet been determined. If a pixel value does not increase by the established threshold, no colour enhancement appears. For the delayed enhancement pattern the CAD program then assigns a specific colour to each pixel that meets threshold enhancement, based on the slope of a best least-squares fit of a line of the delayed enhancement intensity data. If the best-fit line decreases at a rate greater than 10%/240 s relative to the peak uptake value, the pixel is
T. Arazi-Kleinman et al.
colour-coded red, indicating a washout enhancement pattern. If the line increases at a rate greater than 10%/240 s relative to the peak uptake value, the pixel is colour-coded blue, indicating a persistent enhancement pattern. If the slope of the best-fit line is between these two values, the pixel is colour-coded green, indicating a plateau enhancement pattern. The end result is a colour overlay on each MRI image over regions of significant enhancement. The radiologist then selects a specific area of enhancement and the CAD program automatically generates a synopsis with details of the enhancement type and extent. The CAD prototype used for this study is based on the signal enhancement ratio method, which is also employed in other commercially available breast MRI CAD products. All MRI examinations were retrospectively processed by CAD-Gaea (Sentinelle Medical) and assessed by two breast radiologists, who had 5.5 (P.A.C.) and 5 years (R.A.J) of breast MRI experience, blinded to the lesion’s histopathology. Initial interpretations were performed independently and discrepant cases were resolved in consensus. Initially the lesion was located on the MR examination by the radiologists (Fig. 1a and b). Then the presence or absence of significant enhancement was recorded for each lesion at the 50, 80, and 100% enhancement threshold levels. Subsequently, delayed enhancement patterns both with colour mapping and curve analysis made by the radiologists were also recorded for each lesion. Each lesion was then scored as malignant or benign based on thresholds, delayed enhancement pattern and a combination of both, without the knowledge of histopathology. Lesion enhancement was compared to the background parenchymal enhancement to determine significance (Fig. 1cee). For each threshold, a lesion enhancement was required to be greater than the background to be considered significant. Similarly for delayed enhancement pattern, the pattern was required to be distinguishable in colour display from the background enhancement to be considered significant (Fig. 1feh). Plateau and washout delayed enhancement pattern were ranked malignant and continuous was ranked benign.19 Finally, the radiologists manually created an ROI curve for each lesion (Fig. 1i).
Data analysis Sensitivities, specificities, negative predictive values (NPV), and positive predictive values (PPV) were calculated for lesion malignancy assessment versus the presence of malignancy based on the
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1169
Figure 1 MRI images without and with CAD applied in a 33-year-old BRCA-1 genetic mutation carrier with a left breast MRI-detected invasive ductal carcinoma (IDC), diagnosed at ultrasound-guided core biopsy. (a) Dynamic, sagittal, T1weighted, FS 2D FSPGR image with contrast enhancement image shows an oval, 8 mm circumscribed mass (arrow) at the 12 o’clock position of the left breast, middle third. (b) High spatial resolution, sagittal, T1-weighted, 3D FSPGR FS image shows the same mass as a heterogeneous, rim enhancing mass (arrow). Single CAD colour mapping in purple demonstrates areas in the breast reaching threshold levels of (c) 50%, (d) 80%, and (e) 100%. CAD colour mapping of the delayed enhancement pattern at: (f) 50% threshold, (g) 80% threshold and (h) 100% threshold. Blue represents a persistent delayed enhancement pattern, green represents a plateau delayed enhancement pattern, and red represents a washout delayed enhancement pattern. (i) Radiologist-generated lesion kinetic curve, made by the manual placement of a region of interest on the red aspect of the lesion. A washout delayed enhancement pattern of this IDC was confirmed by curve analysis. The colour mapping of red within the mass in (feh) (arrow) also suggested a washout delayed enhancement pattern.
histopathology, for significant enhancement versus backgrounds at 50, 80,and 100% thresholds, delayed enhancement pattern (based on the CAD colour mapping and the manual ROI created by the
radiologists), as well as an integrated assessment considering the initial and delayed enhancement pattern of the lesion. Fisher’s exact test was used to calculate the association between the lesion
1170
histopathology and the MRI morphology category of mass versus non-mass. Chi-square tests were used to calculate the association between the delayed enhancement pattern and the histopathology of the MRI-detected lesions. The paired binary diagnostic tests for comparison of the sensitivity and specificity of two tests were used to compare the initial radiologist MRI interpretation as the prospective BI-RADS category assessment versus the CAD assessment with respect to the presence of malignancy.18 All statistical calculations were performed using SAS (Statistical Analysis System),20 Rosner’s book, Fundamentals of Biostatistics 21 and NCSS (Number Cruncher Statistical System).22
Results Patients and lesion characteristics Fifty-six consecutive lesions were found in 53 women (age range 26e68; mean 47 years). All women were at high risk for hereditary breast cancer. Thirty-two (57%) lesions were found in BRCA-1 mutation carriers, 17 (30%) in BRCA-2 mutation carriers and seven (13%) lesions were found in patients with a strong family history calculated to have at least a 25% risk of carrying a BRCA mutation as computed using the BRCA PRO software. Twenty-two of the lesions (39%) were malignant (13 DCIS, eight IDC, and one ILC) and 34 lesions were benign (Table 1). Most of the DCIS lesions had an intermediate nuclear grading (9/13, 69%), three had a high nuclear grade (3/13, 23%) and one had a low nuclear grade (1/13, 8%). Table 2 lists the correlation between the lesion pathology and the patient risk level. Details of MRI lesion characteristics, BI-RADS prospective categories, and method of biopsy guidance are listed in Table 3. Malignant lesions were larger than benign lesions (mean 17.7 mm versus 14.1 mm). Thirty-four (61%) were non-mass compared with 22 (39%) masses. A significant relationship did not exist between the MRI morphology categories of mass versus non-mass and benign versus malignant pathology (p ¼ 0.47). Of the malignancies, most masses (6/8, 75%) were invasive and most of the non-mass lesions (11/14, 79%) were DCIS.
Lesion enhancement and accuracy of CAD lesion’s assessment The invasive cancers had significant enhancement at all the threshold levels. However, the benign and DCIS lesions had variably significant enhancement
T. Arazi-Kleinman et al.
Table 1 Magnetic resonance imaging-detected lesions according to histopathological findings Histopathology
Total-n (%)
Benign Fibroadenoma Fibrocystic changes Other benign lesions Normal breast tissue/no pathological diagnosis Malignant Ductal carcinoma in situ Low grade Intermediate grade High grade Invasive ductal carcinoma Invasive lobular carcinoma
34 2 13 8 11
(6) (38) (24) (32)
22 13 1 9 3 8 1
(59) (8) (69) (23) (36) (5)
at the different threshold levels. All lesions (malignant and benign) enhanced at 50% threshold. At 80% threshold only six lesions did not enhance, of which five were benign and one was DCIS of intermediate nuclear grade. At 100% threshold all the invasive cancers enhanced, but only (7/13, 54%) of the DCIS and (19/34, 56%) of the benign lesions enhanced (Fig. 2). Of the six DCIS lesions that did not enhance at 100% threshold, one had a low nuclear grade and five were intermediate nuclear grade. The three DCIS lesions with a high nuclear grade enhanced at all the threshold levels. Most lesions demonstrated plateau delayed enhancement either by CAD colour mapping (38/56, 68%) or by a radiologist generated manual curve (35/56, 62.5%). Although most lesions with persistent enhancement were benign (7/8, 87%), one lesion (1/8, 13%) was malignant (DCIS with a high nuclear grade). CAD colour mapping of the delayed enhancement differed from the manual curve for three lesions including: two malignancies (one DCIS and one invasive cancer) for which a washout curve was obtained by the manual curve generated and the CAD colour mapped a plateau
Table 2 Lesion
Lesions according to high-risk patient status BRCA 1 mutation carrier
Ductal carcinoma in 6 (19%) situ (13/56, 23%) Invasive ductal 5 (16%) carcinoma (8/56, 14%) Invasive lobular 0 carcinoma (1/56, 2%) Benign (34/56, 61%) 21 (65%) Total ¼ 56
32 (57%)
BRCA 2 mutation carrier
Family history
6 (35%)
1 (14%)
3 (18%)
0
0
1 (14%)
8 (47%)
5 (72%)
17 (30%)
7 (13%)
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1171
Table 3 Magnetic resonance imaging (MRI) lesion characteristics, lesion prospective BI-RADS category assessment, lesion biopsy guidance method, and patient risk factor according to benign or malignant histopathology Characteristic
Benign (n ¼ 34)
Malignant (n ¼ 22)
Mean size (size range, mm) 14.1 (4e50) 17.7 (5e69) Morphology Mass 14 (42) 8 (36) Non-mass 20 (58) 14 (64) Radiologists’ prospective BI-RADS 3 2 (6) 2 (9) 4 31 (91) 18 (82) 5 1 (3) 2 (9) Biopsy method Ultrasound-guided percutaneous 9 (26) 7 (32) biopsy MRI-guided percutaneous biopsy 16 (48) 4 (18) MRI preoperative 8 (23) 8 (36) wire localization Stereotactic-guided 1 (3) 3 (14) percutaneous biopsy Risk factor BRCA-1 21 11 BRCA-2 8 9 Strong family history 5 2 Data are numbers of lesions. Numbers in parentheses are percentages. BI-RADS, Breast Imaging Reporting and Data System.
delayed enhancement; and one benign lesion for which a persistent delayed enhancement was obtained from the manual curve while CAD colour mapped a plateau delayed enhancement. Chisquare tests were not significant for an association between the delayed enhancement pattern and the histopathology of the MRI-detected lesions (Table 4). As no significant relationship between lesion MRI morphology of mass versus non-mass and histology was found, the accuracy of CAD based on MRI morphology was not performed. Finally, the lesions were categorized as benign or malignant based on the CAD interpretation including: at each threshold of 50, 80, and 100%; overall threshold; delayed enhancement pattern and the integrated evaluation of threshold and delayed enhancement information. Malignancy assessment based on the delayed enhancement pattern was more sensitive (73% versus 41%) and had a higher NPV (73% versus 65%) but was less specific (47% versus 57%) compared with overall threshold of enhancement. The final integrated evaluation of threshold and delayed enhancement pattern optimized sensitivity (73%), NPV (76%), and specificity (56%) for detecting malignant lesions. Sensitivity increased to 100% when only invasive cancers were included, but was lower for all DCIS detection (73%; Table 5).
Figure 2 Significant enhancement at different threshold levels according to lesion histopathology. All the invasive cancers enhanced at all threshold levels. There are decreasing numbers of benign and DCIS lesions enhancing at the 80 and 100% threshold levels, the greatest decrease occurring between the 80e100% threshold levels.
The paired binary diagnostic tests for comparison of sensitivity and specificity of two tests, showed that the prospective radiologist interpretation was significantly more sensitive than the CAD lesion interpretation (p ¼ 0.05); however, the overall CAD lesion analysis was more specific than radiologist MRI lesion assessment (p ¼ 0.001).
Discussion This study evaluated a new commercial CAD software prototype for breast MR examination in the high risk screening setting. The software was very sensitive and specific for invasive cancer; however, it was less sensitive and specific if DCIS was included, demonstrating 73% sensitivity, 56% specificity, and 76% NPV. Current breast MRI CAD using lesion enhancement information only cannot improve the accuracy of the radiologist for distinguishing all malignant from benign screening MRI detected lesions, due to the poor sensitivity for DCIS detection. At a peak enhancement threshold of 100% all the invasive cancers enhanced but only about half the DCIS and benign lesions reached the 100% enhancement threshold level. Therefore, the initial peak enhancement is predictive of invasive cancer, but not of all non-invasive cancers. If considering invasive cancer only, the initial portion of the timeesignal intensity curve provided by the CAD system is most predictive of malignancy similar to the previous report of Williams et al.12
1172
T. Arazi-Kleinman et al.
Table 4 Magnetic resonance imaging lesion delayed enhancement pattern according to computed-aided detection (CAD) colour mapping, region of interest (ROI) created curve and histopathological findings CAD colour mapping
Persistent e curve 1 Plateau e curve 2 Washout e curve 3
ROI radiologist-created curve
Malignant (n ¼ 22)
Benign (n ¼ 34)
Total (n ¼ 56)
Malignant (n ¼ 22)
Benign (n ¼ 34)
Total (n ¼ 56)
1 (5) 17 (77) 4 (18)
6 (17) 21 (62) 7 (21)
7 (13) 38 (68) 11 (19)
1 (5) 15 (68) 6 (27)
7 (21) 20 (58) 7 (21)
8 (14) 35 (63) 13 (23)
Data are numbers of lesions. Numbers in parentheses are percentages.
only including invasive cancers. Also similar to previous reports, no significant association was found between the delayed enhancement pattern of lesions and malignancy.9,12,23,24 However, malignancy assessment of the lesions with regard to the background enhancement had a better sensitivity and NPV based on the delayed enhancement pattern compared to malignancy assessment based on the initial peak enhancement, but was less specific. The colour mapping helps differentiate the lesion from the background in demonstrating the differences in the delayed enhancement pattern of the lesion compared to the background (Fig. 1). However, as the different delayed enhancement patterns by themselves are not associated with histopathology results, they are not specific for malignancy (Figs. 1 and 3). Lesion malignancy assessment based on the delayed enhancement pattern with respect to the background enhancement was more sensitive and had a higher NPV compared with the malignancy assessment based on the threshold enhancement (73 versus 41%, 73 versus 65%). However, the specificity of the delayed enhancement was lower (47 versus 71%). Other studies demonstrated an association between the delayed enhancement pattern and histology.19 Kuhl et al.20 evaluated the timeeintensity curve of 226 lesions seen on MRI. Their results showed that a pattern of delayed washout curve was more likely to be associated with malignant lesions and that a pattern of persistent enhancement was significantly more likely associated with benign lesions. However, there was overlap in the patterns of benign and malignant lesions. One of the explanations for this
difference could be related to the percentage of non-invasive cancers, which was 23% in the present study compared with 4.1% in the study by Kuhl et al.19 Hamilton and colleagues25 found that BRCA 2 mutation carriers are more likely to present with DCIS compared with BRCA 1 mutation carriers. In the present series, an equal number of cases of DCIS among women with BRCA1 and BRCA2 mutations was found; however, DCIS was proportionately more common among the BRCA2 mutation carriers who represented only one third of the patients in the present study. Other studies have also found kinetics to be less predictive of invasive disease. Schnall and his colleagues26 in their study of 995 lesions found that the margins of a mass (Az ¼ 0.76), distribution of non-mass enhancement (Az ¼ 0.78) and signal intensity (Az ¼ 0.70) were more predictive of malignant disease than the qualitative enhancement kinetics of the lesion (Az ¼ 0.66).26 Investigating the outcome of MRI-guided breast biopsies based on imaging features and indication for MRI, Han and colleagues27 found that of 150 biopsied lesions, the probability of malignancy was not statistically significant for the different kinetic features. The present breast MRI CAD prototype assessing enhancement information only is very sensitive for detecting invasive cancers with a high NPV for invasive cancers, with the highest values seen in the lesion colour assessment with respect to the background and in the final integrated analysis of all the CAD components. However, this is not true with respect to DCIS, hence the total assessment of the CAD for malignant disease is not very sensitive.
Table 5 Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of magnetic resonance imaging lesion CAD assessments
Threshold Sensitivity Specificity PPV NPV
50%
80%
100%
Final
0.14 0.97 0.75 0.64
0.23 0.82 0.46 0.62
0.27 0.71 0.38 0.60
0.41 0.71 0.47 0.65
Final colour
Final Integrated
0.73 0.47 0.47 0.73
0.73 0.56 0.52 0.76
Can breast MRI computer-aided detection (CAD) improve radiologist accuracy
1173
Figure 3 MR images without and with CAD applied in a 57-year-old BRCA-2 genetic mutation carrier with a left breast MRI-detected DCIS, diagnosed at MRI-guided wire localization and surgical excisional biopsy. (a) Sagittal, T1-weighted, 3D, FSPGR FS image shows a spiculate 13 mm mass (arrow) in the lower inner quadrant left breast. (b) CAD assessment of the lesion demonstrated enhancement only at the 50% threshold level (arrow). CAD colour mapping of the delayed enhancement pattern shows green within the lesion, indicating a plateau delayed enhancement pattern that did not differ from the remaining background enhancement (arrowheads) and was, therefore, assessed as benign based on CAD.
There are several limitations to this study. Being a single-site study with one MRI technique, the results may not extrapolate to different sites using different MRI equipment, techniques, and interpretation by different radiologists. As with interpretation of the MRI examination, there may be interobserver interpretation differences of the CAD application that were not addressed in this or prior studies. This was a retrospective study of CAD assessment of lesions that were considered suspicious enough for biopsy and the results can not necessarily extrapolate to prospective applications of CAD for screening MRI. This commercially available CAD system for breast MRI examinations using lesion enhancement information only can be a helpful tool for radiologist interpretation, creating a colour mapping of whole breast enhancement, improving the specificity of the radiologist interpretation to detect invasive cancer. The 100% threshold optimizes the diagnostic accuracy for MRI screen-detected invasive cancers and high-grade DCIS. However, using enhancement information only is unreliable for detecting all non-invasive cancers, limiting the overall sensitivity of the CAD for all malignant lesions. New CAD systems for breast MRI incorporating lesion morphological characteristics are
already being tested, and may optimize the accuracy of breast MRI CAD systems for all cancers.
Acknowledgements The present study was supported by the following grants: Canadian Breast Cancer Research Initiative, Terry Fox Foundation of the National Cancer Institute of Canada, the Ontario Research and Development; and the Challenge Fund and Amersham Health
References 1. Morris EA. Breast cancer imaging with MRI. Radiol Clin North Am 2002;40:443e66. 2. Heywang SH, Wolf A, Pruss E, et al. MR Imaging of the breast with Gd-DTPA: use and limitations. Radiology 1989;171:95e103. 3. Kaiser WA, Zeitler E. MR Imaging of the breast: fast imaging sequences with and without Gd-DTPA. Preliminary observations. Radiology 1989;170:681e6. 4. Kriege M, Brekelmans C, Boetes C, et al. Efficacy of MRI and mammography for breast-cancer screening in women with a familial or genetic predisposition. N Engl J Med 2004;351:427e37. 5. Kuhl CK, Schmutzler RK, Leutner CC, et al. Breast MR imaging screening in 192 women proved or suspected to be carriers of a breast cancer susceptibility gene: preliminary results. Radiology 2000;215:267e79.
1174
6. Tilanus-Linthorst MM, Obdeijn IM, Bartels KC, et al. First experiences in screening women at high risk for breast cancer. Breast Cancer Res Treat 2000;63:53e60. 7. Warner E, Plewes DB, Shumak RS, et al. Comparison of breast magnetic resonance imaging, mammography and ultrasound for surveillance of women at high risk for hereditary breast cancer. J Clin Oncol 2001;19:3524e31. 8. Stoutjesdijk MJ, Boetes C, Jager GJ, et al. Magnetic resonance imaging and mammography in women with a hereditary risk of breast cancer. J Natl Cancer Inst 2001;93:1095e102. 9. Lehman CD, Peacock S, DeMartini WB, et al. A new automated software system to evaluate breast MR examinations: improved specificity without decreased sensitivity. AJR Am J Roentgenol 2006;187:51e6. 10. DeMartini WB, Lehman CD, Peacock S, et al. Computer-aided detection applied to breast MRI: assessment of CAD-generated enhancement and tumor sizes in breast cancers before and after neoadjuvant chemotherapy. Acad Radiol 2005;12:806e14. 11. Wood C. Computer aided detection (CAD) for breast MRI. Technol Cancer Res Treat 2005;4:49e53. 12. Williams TC, DeMartini WB, Partridge SC, et al. Breast MR imaging: computer-aided evaluation program for discriminating benign from malignant lesions. Radiology 2007;244:94e103. 13. Levman J, Leung T, Causer P, et al. Classification of dynamic contrast-enhanced magnetic resonance breast lesions by support vector machines. IEEE Trans Med Imaging 2008;27:688e96. 14. Warner E, Plewes DB, Hill KA, et al. Surveillance of BRCA1 and BRCA2 mutation carriers with magnetic resonance imaging, ultrasound, mammography, and clinical breast examination. J Am Med Assoc 2004;292:1317e25. 15. Ikeda DM, Hylton NM, Kinkel K, et al. Development, standardization and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies. J Magn Reson Imaging 2001;14:889e95.
T. Arazi-Kleinman et al.
16. Morris EA. Illustrated breast MRI lexicon. Semin Roentgenol 2001;36:238e49. 17. American College of Radiology. BI-RADS: MRI. In: Breast imaging reporting and data system: BI-RADS atlas. 4th ed. Reston, VA: American College of Radiology; 2003. 18. American College of Radiology. BI-RADS: mammography. In: Illustrated breast imaging reporting and data system: BIRADS. 3rd ed. Reston, VA: American College of Radiology; 1998. 19. Kuhl CK, Meilcareck P, Klaschik S, et al. Dynamic breast MR imaging: are signal intensity time course data useful for differential diagnosis of enhancing lesions? Radiology 1999; 211:101e10. 20. SAS, Statistical Analysis Stystem, Version 8.2, 2002; SAS Institute Inc, Cary, NC, USA. 21. Rosner B. Fundamentals of Biostatistics. Thomson Higher Education, Belmont CA, USA. 6th edition, 2006. 22. Hintze JL. Number Cruncher Statistical System (NCSS). User’s guide. V. Kaysville: Utah; 2003. pp. 5071e5078. 23. Orel SG, Shnall MD, LiVolsi VA, et al. Suspicious breast lesions: MR imaging with radiologicepathologic correlation. Radiology 1994;190:485e93. 24. Stomper PC, Herman S, Klippenstein DL, et al. Suspect breast lesions: finding at dynamic gadolinium-enhanced MR imaging correlated with mammography and pathologic features. Radiology 1995;197:387e95. 25. Hamilton LJ, Evans AJ, Wilson ARM, et al. Breast imaging finding in women with BRCA-1 and BRCA-2 associated breast carcinoma. Clin Radiol 2004;59:895e902. 26. Schnall MD, Blume J, Bluemke DA, et al. Diagnostic architectural and dynamic features at breast MR imaging: multicentric study. Radiology 2006;238:42e53. 27. Han BK, Schnall MD, Orel SG, et al. Outcome of MRI guided breast biopsy. AJR Am J Roentgenol 2008;191: 1798e804.