- Email: [email protected]
Simulation model of mammographic calcifications based on the american college of radiology breast imaging reporting and data system, or BIRADS
Simulation Model of Mammographic Calcifications Based on the American College of Radiology Breast Imaging Reporting and Data System, or BIRADS Maria Kallergi, PhD, Marios A. Gavrielides, MS, Li He, MS Claudia G. Berman, MD, Jihai J. Kim, MD, Robert A. Clark, MD
Acad Radiol 1998; 5:670-679 From the Department of Radiology, University of South Florida, 12901 Bruce B. Downs Blvd, Box 17, Tampa, FL 33612-4799. Received October 20, 1997; revision requested February 20, 1998; revision received May 4; a c c e p t e d May 9. Supported in part by the U,S. Public Health Service National Institutes of Health grant no, 1 R29 CA71479. Address reprint requests to M,K. ©AUR, 1998
670
Statistical analyses have recently shown that the incidence of breast cancer has increased, whereas the number of deaths due to breast cancer has remained the same (1). The early identification of breast cancers with mammography, which has become increasingly accessible, has substantially contributed to this outcome (2). Statistics also show that calcifications are found in almost 50% of women who undergo mammography (3). Most of the calcifications are associated with benign disease, but the ability to diagnose these lesions is poor; thus, a large number of biopsies are performed (3-5). Morpbologic and distributional characteristics of calcifications, such as shape, size, grouping, and location, are important elements in their diagnosis and are included in guidelines for calcification analysis (2,4). Computer-aided diagnosis (CAD) has been proposed as a way to improve detection and classification of mammographic abnormalities, and preliminary results have been promising (6-8). Most CAD methods are aimed at the detection and categorization of calcifications. Detection methods often include segmentation algorithms, which are used to extract the objects of interest from the image. These algorithms can provide information about the location, size, and shape of each object (9). Morphologic and distributional characteristics of calcifications are often used to classify them as benign or malignant (8). These characteristics are usually determined by human observers. It would be advantageous to use the output of the segmentation process for classification if the segmentation algorithm preserved the morphologic characteristics and distribution of the objects of interest; a fully automatic detection and classification scheme could
vol 5, No 10, October 1998
SIMULATION MODEL OF MAMMOGRAPHIC CALCIFICATIONS
then be formed. However, how can the accuracy of the segmentation and subsequent detection and classification techniques be evaluated and compared? How can one determine the effect of parameters such as image pixel size and depth on CAD performance? Generally, the proposed methods of comparison of CAD algorithms include the use of common databases, observer studies, or simulation studies with receiver operating characteristic analysis, free-response receiver operating characteristic analysis, or mathematic criteria (6,10-14). Simulation is the only objective and quantitative way of evaluating the absolute performance of a CAD method or of comparing two methods that have similar outputs. Several investigators have proposed the use of simulated microcalcifications for various functions: testing CAD detection algorithms (13-15), evaluating the relationship between parenchymal background type and failure to identify clusters (16), studying the dependence of CAD performance on the physical characteristics of microcalcifications (17), and studying observer performance (18). The methods that have been used for simulation include Monte Carlo techniques (18), graphical generation of irregular objects (16), generation of elliptical or spherical objects (15,17), and statistical analysis or feature extraction of real cases (13,14). Almost all simulation studies have used normal mammograms as the background on which to superimpose simulated objects so that parenchymal complexity would be preserved. Furthermore, almost all studies have generated one type of calcification morphology, namely, clusters of pleomorphic calcifications. Use of one type of calcification may be adequate for testing detection algorithms but is insufficient for testing segmentation- or morphology-based classification techniques. The goal of this study was to design a model for the simulation of single or clustered calcifications that would take morphologic characteristics into consideration. The model was based on the morphologic characteristics of calcifications described in the American College of Radiology Breast Imaging Reporting and Data System (BIRADS) (19). The incentive for this study was the need to objectively and quantitatively evaluate and compare segmentation- and morphology-based classification algorithms. To perform such an evaluation, the exact size, shape, number, and distribution of calcifications need to be known. The proposed simulation model was evaluated by three mammographers (C.G.B., JJ.K., R.A.C.) in a blinded study. Here, the model is evaluated and its implementation is discussed.
MATERIALS AND METHODS Selection of M a m m o g r a m s Thirty mammographic views were selected to provide a background for the simulation. The images were from 30 cases categorized as negative (BIRADS reporting code, N) with at least a 2-year clinical follow-up in which no cancer or other abnormality was detected. Parenchyreal densities on these mammograms were assessed by an expert mammographer and categorized according to the BIRADS. Five (17%) of the 30 mammograms depicted breast composed almost entirely of fat (category 1), six (20%) of 30 mammograms depicted scattered fibroglandular densities (category 2), five (17%) of 30 mammograms depicted breast that was heterogeneously dense (category 3), and 14 (47%) of 30 mammograms depicted breast that was extremely dense (category 4).
Digitization of M a m m o g r a m s Films were digitized at 30 gm and 16 bits per pixel with a commercially available scanner (ImageClear R3000; DBA, Melbourne, Fla). This resolution was selected because it matches the spatial resolution of mammographic screen-film systems and it allows the generation of simulated clusters that can serve multiple purposes in detection and classification studies (20,21). Model for Simulation of Calcifications Generation of calcification clusters.--At this stage, only calcifications arranged in clusters were simulated, but other distribution types (diffuse, regional, segmental, or linear) can be similarly generated. The criteria used for the generation of a cluster were based on published reports (2,4,13,19) and statistical findings derived from a set of I00 real clusters (6). The following criteria were used: (a) A cluster should have at least three objects (calcifications) within 1 cm 2, an area that corresponds to 333 × 333 pixel in the 30 gm/pixel images. (b) The number (N) of calcifications within a cluster should be determined by randomly selecting a number less than or equal to 15; the maximum number of calcifications was empirically selected to put emphasis on relatively small clusters. (c) The shapes of the calcifications within a cluster should represent types described in the BIRADS. Calcifications within benignlike clusters should have uniform shapes and sizes. Calcification size will depend on the type. Hence, calcifications were divided into two groups according to their size. Group A included benignlike calcifications (punctate, coarse, large rodlike, round, dystro-
671
Figure I. Binary imagesof simulated calcification clusters designed to represent benign conditions (group A) as described in the BIRADSand several published reports (4,21). The simulated morphologic characteristics are listed in Table 1,
672
(1)
(2)
Figure 2, Binary images of simulated calcification clusters designed to represent malignant or indeterminate conditions (group B) as described in the BIRADSand several published reports (4,21), The simulated morphologic characteristics are listed in Table 1.
(3)
L
•
lJ
t
(4)
(5)
(6)
(7)
(8)
(9)
"
,%
~
t
B
(lO)
(11)
(12) "v S
lit
(13)
(14)
.J
"
(15)
2
o
673
phic) with sizes of 100-1,000 gm. Group B included malignantlike calcifications (fine linear, fine linear branching, heterogeneous, amorphous) with sizes of 30500 gm. (d) The location and orientation of calcifications within a cluster, as well as the distances between calcifications, should be randomly determined. (e) The intensifies (gray values) of the pixels of each calcification should follow a Gaussian distribution. The peak value and standard deviation of the Gaussian curve should be different and randomly selected for each calcification within a cluster. The gray values of the calcification pixels should be determined from the values of the background-tissue pixels. Clusters were generated with Interactive Data Language (IDL) (Research Systems, Boulder, Colo) by first creating a 256 x 256-pixel image matrix. Then, contours of calcifications were manually drawn at a random location inside this matrix. The contours were based on outlines of real calcifications that matched the BIRADS morphologic description and published examples of various lesions (4,19). Different contours and, consequently, different shapes were generated to represent the various types of calcifications in groups A and B. A binary image was generated with N objects (3 < N < 15). Figures 1 and 2 show the morphologic characteristics and distribution of the clusters in groups A and B, respectively. The simulated morphologic characteristics of each lesion are listed in Table 1 for both groups. In Figure 3, a bar graph shows the distribution of cluster sizes (number of calcifications within the cluster). In Figure 4, a similar bar graph shows the distribution of the mean area of the calcifications in a cluster. Intensity and contrast of simulated calcifications.The intensity profile of each simulated calcification was determined by determining the intensity (gray values of pixels) of the area of the breast image on which the calcification was to be superimposed and a contrast value. For each calcification pixel, a different contrast value was randomly selected from a range of 0.04 to 0.20 (Fig 5). This range was determined from a statistical analysis of 100 real calcification clusters and data reported in the literature (6,13). The simulated clusters were then embedded in real mammograms. First, a location on the full digitized mammogram was selected for placement of the simulated cluster. A 256 x 256-pixel area centered at the selected location was then defined and considered as the background image. A 256 x 256-pixel matrix (P) that contains the final result of the simulation can be expressed as
674
Table I Characteristics of M a m m o g r a m s and Simulated Calcification Clusters Cluster No.
Simulated Morphology
Breast Composition*
Benign Calcifications (Group A) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Punctate Punctate Punctate Coarse Coarse Coarse Rodlike Rodlike Rodlike Round Round Round Dystrophic Dystrophic Dystrophic
4 4 2 2 2 2 3 4 3 1 4 2 4 1 3
Malignant Calcifications (Group B) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Amorphous Amorphous Amorphous Amorphous Amorphous Casting Casting
4 3 4 4 3 4 4 1 4 4 1 1 2 4 4
Note.--Clusters are shown graphically in Figures 1 and 2. *Numbers indicate the BIRADS categories.
the sum of the background image (Pb) and the clustercontaining image (Pc): P~ = Pb + Po " From this equation and the definition of the contrast value (C) as C = (Ps Pb)/(P~ + Pu), the nonzero elements of the intermediate matrix (Pc) were determined by Pc = P~ - Pb = Pb[2C/(1 -C)]. Note that image matrix Pc had nonzero elements only in the areas where the simulated calcification objects existed. The location of nonzero elements was determined by comparing matrix Pb to the binary image that contained the hand-drawn objects. Finally, the image Pc was smoothed (convolved) by a Ganssian template and added to the original mammogram at the selected location. The
14I
12 14 ¢n
12 U 0
0 I0
"8
e~
.0
E .-¢
Z
Z
0
~ 3-5
6-8
9=11
12-16
Number of Calcifications Per Cluster < 60
Figure 3. Bar g r a p h of t h e size ( n u m b e r of calcifications per cluster) of the 30 simulated clusters.
141
12
60 -- 100
101 -- 160
> 160
Mean Area (pixels) Figure 4. Bar g r a p h of t h e m e a n a r e a (in pixels) of t h e 30 simulated clusters. The m e a n a r e a is d e f i n e d as the sum of t h e areas of t h e individual calcifications in pixels d i v i d e d b y t h e n u m b e r of calcifications in t h e cluster. The minimum a n d m a x i m u m a r e a values w e r e 24 a n d 357 pixels, respectively, a t a resolution of 30 gm.
~1 ~
==
"6 .{3
E
Z
< 0.090
0.091 - 0.120
0.121 - 0,150
> 0.150
Mean Local Contrast
Figure 5. Bar g r a p h of t h e m e a n local contrast of t h e 30 simulated calcification clusters. The local contrast of a single c a l c i f i c a t i o n is d e f i n e d as t h e diff e r e n c e in g r a y values b e t w e e n a point in t h e calcification a n d a point in the a r e a surrounding t h e calcification div i d e d b y their sum, The five brightest calcifications in a cluster w e r e used to c a l c u l a t e t h e m e a n local contrast of t h e cluster. The minimum a n d m a x i m u m m e a n local contrast values w e r e 0.037 a n d O. 170, respectively,
standard deviation of the Gaussian template was randomly selected from the range of 1-2; this range was determined from statistical analysis of 100 real calcification cases (6). Positioning of the simulated cluster in the mammogram.--The location of the cluster was determined by the user, who, with the help of a computer mouse, selected a pixel on the digitized mammogram as the center of the window. The center of a specified cluster image was then automatically superimposed on the center of the window after the intensity of each object in the cluster was adjusted as described previously. Locations were selected based on reported statistical analysis regarding the relative incidence of breast carcinoma at various locations (22). It is reported that most breast cancers occur laterally and centrally; about 45% of the cancers occur on the upper outer quadrant, 25% are at the center of the breast, 15% are on the upper inner quadrant, 10% are in the lower outer quadrant, and only 5% are in the lower inner quadrant (22). We assumed that the same statistics would apply to calcifications (benign or malignant) and adapted these percentages to single-view mammograms; thus, the following distribution of clusters was maintained: 15 of 30 clusters (50%) were located in
675
KALLERGI ET AL
b
a
4._._•____
f-'--'-~
Vol 5, No 10, October 1998
!1
~'t J '
"---~._..
V k 11
iI
tI "~LI
/'/a,, I J,, ,"~ a. b. c. Figure 6. Generation of a cluster that simulates heterogeneous (pleomorphic) calcifications. (a) Binary image. (b) Gaussian filtered image (Pc). (c) Section of a digitized mammogram with the superimposed simulated cluster of calcifications. The arrows in~licate the location used for the intensity profiles plotted at the bottom of the images.
the outer half of the breast, nine of 30 clusters (30%) were at the center, and six of 30 clusters (20%) were at the inner half. A second criterion used in selecting a location for calcifications was that the background consist primarily of parenchymal tissue (bright areas in the radiograph) or have a gradient of intensities. Dark areas of fat and areas of uniform intensity were avoided. Finally, we ensured that positioning of the simulated objects was anatomically and physically correct; for example, linear branching calcifications were oriented radially from the nipple to the chest wall with the branching end toward the chest wall. Figure 6 shows the three steps followed in the generation of a cluster with pleomorphic calcifications. Intensity profiles are included to show the intensities of the calcifications before and after Gaussian filtering and after superimposing the cluster on the digitized mammogram. E v a l u a t i o n of S i m u l a t e d C a l c i f i c a t i o n s
Three mammographers evaluated the simulated calcifications in a blinded study in separate sessions. Sixty mammograms were used in the study: 30 with real calcification clusters (one per image) and 30 with simulated calcification clusters. The real cases matched the simulated ones in terms of morphologic characteristics and distribution of calcifications. Half of the real cases were benign and half were malignant. Diagnoses were proved by means of biopsy, and the composition of breast tissue was similar to that in the cases selected for the simulation. One high-resolution (2,000 x 2,000 pixel) monitor
676
QUESTIONS A. Is a calcification cluster present and where (yes/no)? If yes, give location. B. Classify the detected cluster according to BIRADS character type (give letter) C. Do you think the cluster is real (yes/no)? If no, answer question D D. Cluster is found simulated due to incorrect (select all that apply): size, brightness, shape, margins, orientation, position, distribution, no-reason, other (specify) Figure 7. Questionnaire completed by each of the three observers during the evaluation process.
with an interface developed in-house was used in the evaluation (23). Interactive gray-scale adjustments (window width, window level, gamma correction) and magnification tools (digital zoom, magnifying glass) were available (23). A table of the BIRADS lesion codes was provided on an adjacent monitor for easy reference in the assignment of calcification types (eg, round, punctate, amorphous). A training session for using the computerized viewing system preceded the evaluation. All readers had at least 1 year of experience using the BIRADS lexicon; no training was done in regard to its implementation. The real and simulated images were mixed randomly and shown to the observers, who completed the questionnaire shown in Figure 7 for each case. Two observers participated in the first stage of the evaluation. A simulation was considered successful if one or both of the radiologists classified it as real in the assessment. A simulation
Table 2 Results of Initial Assessment of Calcification Clusters by Observer I
Table 3 Results of Initial Assessment of Calcification Clusters by Observer 2
Actual Calcification Type Observer 1 Responses Real Simulated
Real
Simulated
16 14
12 18
Actual Calcification Type Observer 2 Responses Real Simulated
Real
Simulated
27 3
22 8
Note.--Z2= 1.071, P = .30i. Yates correction yielded X2= 0.603 a n d P--- .438.
Note.--Z2= 2.783 and P = .095. Yates correction yielded %2= 1.781 a n d P=.182,
Table 4 Results of Assessment of Modified Calcification Clusters by Observer 3
Table 5 C o m b i n e d Resultsof All Three Observers
Actual CalcificationType Actual Calcification Type - Observer 3 Responses Real Simulated
Real
Simulated
16 14
13 17
Note.--Z 2= 0.601 a n d P = .438. Yates correction yielded X2= 0.267 a n d P = .605.
was considered unsuccessful if both radiologists classified it as simulated. All unsuccessful simulations were modified according to the observers' recommendations, and the image set was reevaluated by the third radiologist in a similar manner. The first two observers were also asked to comment on (a) the benign or malignant appearance of the cluster, (b) the number of calcifications seen in the cluster, and (c) the subtlety of the cluster (low, moderate, high). The responses of the observers were analyzed for agreement with the kappa statistic and for independence with the X2 test statistic.
All observers correctly identified the location of all clusters. Observer 1 described only 12 of the 30 simulated cases as realistic, but this observer also described 14 of the 30 real cases as simulated (Table 2). Observer 2 interpreted all but eight of the 30 simulated cases as realistic and described three of the 30 real cases as simulated (Table 3). These two observers disagreed in 12 of the simulated cases; more specifically, one observer classified a case as real while the other observer said it was simulated. They agreed in seven of the simulated cases,
Combined Responses Real Simulated
Real 29 1
Simulated 27 3
Note.--For this result, a cluster was considered real ("Yes" c o m b i n e d response) if a t least one of the observers found it to b e real. A cluster was considered simulated ("No" c o m bined response) if all three observers found it to b e simulated. The %2 value for the c o m b i n e d responses was 1.071 a n d P = .301. Yates correction yielded Z2=0.268 a n d P = .605.
that is, they both thought that the clusters were simulated. The appearances of these seven simulations were changed according to comments by the observers, and the modified set of 60 images was evaluated by a third radiologist. The responses of observer 3 are listed in Table 4. Three of the seven modified cases were again identified as simulated (Table 5). The kappa coefficient, estimated for pairs of observers, was in the range of 0-0.2, which indicates that there was no evidence of agreement in any of the three pairs of readers (results were significant at the .05 level). The results from each observer and the combined responses of all observers were analyzed for independence with the Z2 test at the P = .05 significance level. The Z2 test statistic and the one-tailed probability of obtaining a value of ~2 or greater are listed in Tables 2-5. The Z2 and P results failed to show a difference in observer responses between the tested real and simulated calcification clusters. A much larger sample set should be tested, however, to reach definitive conclusions on differences between the real and simulated cases. With the present sample set, there is no reason to reject the independence hypothesis at the P = .05 significance level; any differences may be attributed to chance. The Yates correction for continuity
677
was also used due to the small sample size, but results did not differ from those of the other tests. The four most frequently given reasons for interpreting a cluster as "not real" were relative intensity, relative position, orientation, and size of the calcifications within the cluster. Substantial variability among observers was seen in assigning a type to the calcifications. This variability indicates primarily the difficulty in implementing the BIRADS lexicon. To test the hypothesis that the observers' type assignments were an accurate approximation of the expected cluster types, the observed frequencies per type for each reader were estimated. Then a )~2 goodness of fit test was performed to compare the observed and expected frequencies. The Z 2 test statistic and the onetailed probability of obtaining a value of )~2 or greater are listed in Table 6 for the three observers. In general, at the P = .05 significance level there is no reason to reject the hypothesis. The P value for observer 3 was only .065, which, although greater than .05, does not provide statistically significant evidence that the observed frequencies are a good approximation of the expected frequencies. As mentioned previously, two of the observers also classified the simulated clusters as benign or malignant, assessed the subtlety of the cases, and estimated the number of calcifications in the cluster. The observers' findings were in agreement in 90% of the decisions. On average, a third of the simulated clusters were classified as malignant and a third were classified as benign, a finding that is in agreement with the intent of the study design. The remaining third of the clusters were classified as indeterminate. From the indeterminate cases, half were designed to be benign and half were designed to be malignant. The majority of the indeterminate clusters were of the heterogeneous type. In terms of difficulty, eight (mean value) of the 30 simulated clusters were rated as difficult to detect. The remaining 22 clusters were considered easy to detect. No case was considered moderately difficult to detect, possibly because difficulty was determined by whether the cluster could be seen in a casual inspection of the reduced image (easy) or only after careful examination of all the sections at high resolution (difficult). As a result, a binary response was more likely. In the easy cases, the observers identified either all the calcifications in the cluster or more calcifications than were actually present, probably because of the scaling of the digital images and the graininess of the digital display. In the difficult cases, fewer calcifications were usually identified. Not all diffi-
678
Table 6 Results from the X2 Goodness of Fit between the Observed Frequencies of the Various Cluster Types and the Expected Frequencies from the Simulation Design Parameters X2 P*
Observer 1 5.074 .166
Observer 2
Observer 3
4.431 .219
7.225 .065
*One-tailed probability of obtaining a value of X2or greater.
cult cases were misclassified, a finding that suggests that detection difficulty is not equivalent to diagnostic difficulty.
We have developed a method for the simulation of calcification clusters based on the BIRADS guidelines for morphology and distribution of mammographic calcifications. Among the observers who evaluated the accuracy of reproduction of the various shapes and sizes of calcifications, substantial variability was observed, as shown by the kappa coefficients. A Z2 analysis of the individual and combined observer responses, however, did not indicate any statistically significant difference between real and simulated calcifications. This outcome supported the use of the proposed simulation model for the generation of test calcifications and the development of a larger test set as required for quantitative assessment of any differences between real and simulated calcifications. The generation of a large set of simulated calcification cases can be performed mathematically as long as there exists a representative basic set. Bootstrapping with "jittering" is a mathematic approach that has already been used in our laboratory for the generation of 1,000 pseudoclusters from a set of 50 real clusters (23). It can be used in a similar way to generate a large number of simulated clusters from the initial set described here. Additional morphologic characteristics and distributions of calcifications can be simulated by first creating a representative set with the model presented here and then applying jittering. In regard to the issue of representativeness, ideally, a representative set should consist of cases on which there is "ground truth" of the morphologic characteristics of the calcifications. Such ground truth can only come from an expert mammographer, but the large interobserver variability that may occur poses a problem. A possible solution may be the use of a panel of expert
Vol 5, No 10, October 1998
SIMULATION MODEL OF M A M M O G R A P H I C CALCIFICATIONS
mammographers who are in full agreement on the type of simulated calcifications that are part of the representative set. This set can then be expanded and used for clinical training or other applications as mentioned later. Further evaluation showed that the simulation model accurately reproduced the properties (intensity, size, shape, distribution) of punctate, round, rodlike, heterogeneous, and casting calcifications. The agreement between any two observers or between an observer and the simulation was the highest for these types. Very poor agreement was observed for coarse, dystrophic, and amorphous calcifications, a finding that suggests particular difficulty in the BIRADS interpretation of these types of clusters, as well as modeling weaknesses. Regardless of the reason, incorrect identification of the morphologic characteristics and distribution of the calcifications had a direct effect on the diagnosis. This finding indicates the importance of correctly identifying the type of the calcification. It also indicates a role for CAD, namely, in the assignment of BIRADS categories to detected clusters. The use of simulated cases similar to those presented here may aid in the development and optimization of such CAD methods. The responses of the observers on a cluster-by-cluster basis, as well as the problems encountered during modeling, led us to the conclusion that generally benignlike calcifications are more difficult to simulate than malignantlike ones because of their specific morphologic characteristics. The random properties of several of the malignantlike calcifications turned out to be an advantage in their simulation. Rodlike and casting calcifications posed special difficulties in reproduction owing to their particular orientation and location requirements. Potential applications of the simulated calcification clusters generated in this study may include (a) the evaluation of segmentation algorithms in terms of both detection performance and shape preservation; (b) the evaluation and ranking of morphologic features used in classification methods; (c) the evaluation of the effects of different image parameters (pixel size and depth) on the morphology and distribution of calcifications, as well as on the performance of detection and classification algorithms; and (d) the development of computer methods to assist in the implementation of BIRADS and in the consistent reporting of mammographic findings.
IEFERENCE,c
1, Parker SL, Tong T, Bolden S, Wingo PA. Cancer statistics, 1996. CA Cancer J Ciin 1996; 46:5-28, 2, Sickles EA. Breast calcifications: mammographic evaluation. Radiology 1986; 160:289-293. 3. Millis RR, Davis R, Stacey A J. The detection and significance of calcifications in the breast: a radiological and pathological sign. BrJ Radio11976; 49:12-26, 4. Lanyi M. Diagnosis and differential diagnosis of breast calcifications. New York, NY: Springer-Verlag, 1986. 5. Clark RA, Nemec L, Love N. Breast imaging: a practical look at its capabilities and its limitations. Postgrad Med 1992; 92:117-134, 6. Kallergi M, Clarke LP, Qian W, et al. Interpretation of calcifications in screen/film, digitized, and wavelet-enhanced, monitor displayed mammograms: an ROC study. Acad Radio11996; 3:285-293. 7. Doi K, Giger ML, Nishikawa RM, et al. Digital radiography: a useful clinical tool for computer-aided diagnosis by quantitative analysis of radiographic images. Acta Radiol 1993; 34:426-439. 8, Lo JY, Baker JA, Kornguth PJ, Floyd CE. Computer-aided diagnosis of breast cancer: artificial neural network app r o a c h for optimized merging of mammographic features. Acad Radio11995; 2:841-850. 9. Qian W, Kallergi M, Clarke LP, et al. Tree structured w a v e l e t transform segmentation of microcatcifications in digital mammography. Med Phys 1995; 22:1247-1254. 10. Li HD, Kallergi M, Clarke LP, Jain VK, Clark RA, Markov random field for tumor detection in digital mammography. IEEETrans Med Phys 1995; 14:565-576. 11. Chalana V, Kim Y. A methodology for evaluation of image segmentation algorithms on medical images. SPIE 1996; 2710:178189. 12. Eskicioglu AM. Application of multidimensional quality measures to reconstructed medical images. Opt Eng 1996; 35:778-785. 13. Lefebvre F, Benali H, Gilles R, Di Paola R. A simulation model of clustered breast microcalcifications, Med Phys 1994; 21:18651874. 14. Lado M J, Tahoces PG, Souto M, M6ndez A J, Vidal JJ. Real and simulated clustered microcalcifications in digital mammograms: ROC study of observer performance. Med Phys 1997; 24:13851394. 15. Brzakovic D, Brzakovic P, Neskovic M. An appr oa c h to automated screening of mammograms. SPIE 1993; 1905:690-701, 16. Astley S, Hutt I, Adamson S, et al. Automation in mammography: computer vision and human perception. SPIE 1993; 1905:716730. 17. Judy PF. Detection of clusters of simulated calcifications in lumpy noise backgrounds. SPIE 1996; 2716:39-46. 18. Chan HP, Doi K. Monte Carlo simulation in diagnostic radiology. In: Morin RL, ed. Monte Carlo simulation in the radiological sciences. Boca Raton, Fla: CRC, 1988; 103-192, 19. D'Orsi C J, Kopans DB. Mammographic feature analysis. Semin Roentgeno11993; 28:204-230. 20. Gavrielides MA, Kallergi M, Berman CG, Clarke LP. Automatic shape analysis and classification of mammographic calcifications. SPIE 1997; 3034:869-876. 21. Mammography: a user's guide--recommendations of the National Council on Radiation Protection and Measurements. NCRP report no. 85. Washington, DC: Government Printing Ofrice, 1986. 22. Gohel HJ, Kallergi M, Vossberg M, Gavrielides MA, Clarke LP. A workstation interface for ROC studies in digital mammography. SPIE 1997; 3031:440-447. 23. Ruth EJ. Estimating the minimal sample size of calcification clusters which approximates the Bayes risk of diagnosis. Thesis. University of South Florida, Tampa, Fla, 1997.
o
679
Acad Radiol 1998; 5:670-679 From the Department of Radiology, University of South Florida, 12901 Bruce B. Downs Blvd, Box 17, Tampa, FL 33612-4799. Received October 20, 1997; revision requested February 20, 1998; revision received May 4; a c c e p t e d May 9. Supported in part by the U,S. Public Health Service National Institutes of Health grant no, 1 R29 CA71479. Address reprint requests to M,K. ©AUR, 1998
670
Statistical analyses have recently shown that the incidence of breast cancer has increased, whereas the number of deaths due to breast cancer has remained the same (1). The early identification of breast cancers with mammography, which has become increasingly accessible, has substantially contributed to this outcome (2). Statistics also show that calcifications are found in almost 50% of women who undergo mammography (3). Most of the calcifications are associated with benign disease, but the ability to diagnose these lesions is poor; thus, a large number of biopsies are performed (3-5). Morpbologic and distributional characteristics of calcifications, such as shape, size, grouping, and location, are important elements in their diagnosis and are included in guidelines for calcification analysis (2,4). Computer-aided diagnosis (CAD) has been proposed as a way to improve detection and classification of mammographic abnormalities, and preliminary results have been promising (6-8). Most CAD methods are aimed at the detection and categorization of calcifications. Detection methods often include segmentation algorithms, which are used to extract the objects of interest from the image. These algorithms can provide information about the location, size, and shape of each object (9). Morphologic and distributional characteristics of calcifications are often used to classify them as benign or malignant (8). These characteristics are usually determined by human observers. It would be advantageous to use the output of the segmentation process for classification if the segmentation algorithm preserved the morphologic characteristics and distribution of the objects of interest; a fully automatic detection and classification scheme could
vol 5, No 10, October 1998
SIMULATION MODEL OF MAMMOGRAPHIC CALCIFICATIONS
then be formed. However, how can the accuracy of the segmentation and subsequent detection and classification techniques be evaluated and compared? How can one determine the effect of parameters such as image pixel size and depth on CAD performance? Generally, the proposed methods of comparison of CAD algorithms include the use of common databases, observer studies, or simulation studies with receiver operating characteristic analysis, free-response receiver operating characteristic analysis, or mathematic criteria (6,10-14). Simulation is the only objective and quantitative way of evaluating the absolute performance of a CAD method or of comparing two methods that have similar outputs. Several investigators have proposed the use of simulated microcalcifications for various functions: testing CAD detection algorithms (13-15), evaluating the relationship between parenchymal background type and failure to identify clusters (16), studying the dependence of CAD performance on the physical characteristics of microcalcifications (17), and studying observer performance (18). The methods that have been used for simulation include Monte Carlo techniques (18), graphical generation of irregular objects (16), generation of elliptical or spherical objects (15,17), and statistical analysis or feature extraction of real cases (13,14). Almost all simulation studies have used normal mammograms as the background on which to superimpose simulated objects so that parenchymal complexity would be preserved. Furthermore, almost all studies have generated one type of calcification morphology, namely, clusters of pleomorphic calcifications. Use of one type of calcification may be adequate for testing detection algorithms but is insufficient for testing segmentation- or morphology-based classification techniques. The goal of this study was to design a model for the simulation of single or clustered calcifications that would take morphologic characteristics into consideration. The model was based on the morphologic characteristics of calcifications described in the American College of Radiology Breast Imaging Reporting and Data System (BIRADS) (19). The incentive for this study was the need to objectively and quantitatively evaluate and compare segmentation- and morphology-based classification algorithms. To perform such an evaluation, the exact size, shape, number, and distribution of calcifications need to be known. The proposed simulation model was evaluated by three mammographers (C.G.B., JJ.K., R.A.C.) in a blinded study. Here, the model is evaluated and its implementation is discussed.
MATERIALS AND METHODS Selection of M a m m o g r a m s Thirty mammographic views were selected to provide a background for the simulation. The images were from 30 cases categorized as negative (BIRADS reporting code, N) with at least a 2-year clinical follow-up in which no cancer or other abnormality was detected. Parenchyreal densities on these mammograms were assessed by an expert mammographer and categorized according to the BIRADS. Five (17%) of the 30 mammograms depicted breast composed almost entirely of fat (category 1), six (20%) of 30 mammograms depicted scattered fibroglandular densities (category 2), five (17%) of 30 mammograms depicted breast that was heterogeneously dense (category 3), and 14 (47%) of 30 mammograms depicted breast that was extremely dense (category 4).
Digitization of M a m m o g r a m s Films were digitized at 30 gm and 16 bits per pixel with a commercially available scanner (ImageClear R3000; DBA, Melbourne, Fla). This resolution was selected because it matches the spatial resolution of mammographic screen-film systems and it allows the generation of simulated clusters that can serve multiple purposes in detection and classification studies (20,21). Model for Simulation of Calcifications Generation of calcification clusters.--At this stage, only calcifications arranged in clusters were simulated, but other distribution types (diffuse, regional, segmental, or linear) can be similarly generated. The criteria used for the generation of a cluster were based on published reports (2,4,13,19) and statistical findings derived from a set of I00 real clusters (6). The following criteria were used: (a) A cluster should have at least three objects (calcifications) within 1 cm 2, an area that corresponds to 333 × 333 pixel in the 30 gm/pixel images. (b) The number (N) of calcifications within a cluster should be determined by randomly selecting a number less than or equal to 15; the maximum number of calcifications was empirically selected to put emphasis on relatively small clusters. (c) The shapes of the calcifications within a cluster should represent types described in the BIRADS. Calcifications within benignlike clusters should have uniform shapes and sizes. Calcification size will depend on the type. Hence, calcifications were divided into two groups according to their size. Group A included benignlike calcifications (punctate, coarse, large rodlike, round, dystro-
671
Figure I. Binary imagesof simulated calcification clusters designed to represent benign conditions (group A) as described in the BIRADSand several published reports (4,21). The simulated morphologic characteristics are listed in Table 1,
672
(1)
(2)
Figure 2, Binary images of simulated calcification clusters designed to represent malignant or indeterminate conditions (group B) as described in the BIRADSand several published reports (4,21), The simulated morphologic characteristics are listed in Table 1.
(3)
L
•
lJ
t
(4)
(5)
(6)
(7)
(8)
(9)
"
,%
~
t
B
(lO)
(11)
(12) "v S
lit
(13)
(14)
.J
"
(15)
2
o
673
phic) with sizes of 100-1,000 gm. Group B included malignantlike calcifications (fine linear, fine linear branching, heterogeneous, amorphous) with sizes of 30500 gm. (d) The location and orientation of calcifications within a cluster, as well as the distances between calcifications, should be randomly determined. (e) The intensifies (gray values) of the pixels of each calcification should follow a Gaussian distribution. The peak value and standard deviation of the Gaussian curve should be different and randomly selected for each calcification within a cluster. The gray values of the calcification pixels should be determined from the values of the background-tissue pixels. Clusters were generated with Interactive Data Language (IDL) (Research Systems, Boulder, Colo) by first creating a 256 x 256-pixel image matrix. Then, contours of calcifications were manually drawn at a random location inside this matrix. The contours were based on outlines of real calcifications that matched the BIRADS morphologic description and published examples of various lesions (4,19). Different contours and, consequently, different shapes were generated to represent the various types of calcifications in groups A and B. A binary image was generated with N objects (3 < N < 15). Figures 1 and 2 show the morphologic characteristics and distribution of the clusters in groups A and B, respectively. The simulated morphologic characteristics of each lesion are listed in Table 1 for both groups. In Figure 3, a bar graph shows the distribution of cluster sizes (number of calcifications within the cluster). In Figure 4, a similar bar graph shows the distribution of the mean area of the calcifications in a cluster. Intensity and contrast of simulated calcifications.The intensity profile of each simulated calcification was determined by determining the intensity (gray values of pixels) of the area of the breast image on which the calcification was to be superimposed and a contrast value. For each calcification pixel, a different contrast value was randomly selected from a range of 0.04 to 0.20 (Fig 5). This range was determined from a statistical analysis of 100 real calcification clusters and data reported in the literature (6,13). The simulated clusters were then embedded in real mammograms. First, a location on the full digitized mammogram was selected for placement of the simulated cluster. A 256 x 256-pixel area centered at the selected location was then defined and considered as the background image. A 256 x 256-pixel matrix (P) that contains the final result of the simulation can be expressed as
674
Table I Characteristics of M a m m o g r a m s and Simulated Calcification Clusters Cluster No.
Simulated Morphology
Breast Composition*
Benign Calcifications (Group A) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Punctate Punctate Punctate Coarse Coarse Coarse Rodlike Rodlike Rodlike Round Round Round Dystrophic Dystrophic Dystrophic
4 4 2 2 2 2 3 4 3 1 4 2 4 1 3
Malignant Calcifications (Group B) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Heterogeneous Amorphous Amorphous Amorphous Amorphous Amorphous Casting Casting
4 3 4 4 3 4 4 1 4 4 1 1 2 4 4
Note.--Clusters are shown graphically in Figures 1 and 2. *Numbers indicate the BIRADS categories.
the sum of the background image (Pb) and the clustercontaining image (Pc): P~ = Pb + Po " From this equation and the definition of the contrast value (C) as C = (Ps Pb)/(P~ + Pu), the nonzero elements of the intermediate matrix (Pc) were determined by Pc = P~ - Pb = Pb[2C/(1 -C)]. Note that image matrix Pc had nonzero elements only in the areas where the simulated calcification objects existed. The location of nonzero elements was determined by comparing matrix Pb to the binary image that contained the hand-drawn objects. Finally, the image Pc was smoothed (convolved) by a Ganssian template and added to the original mammogram at the selected location. The
14I
12 14 ¢n
12 U 0
0 I0
"8
e~
.0
E .-¢
Z
Z
0
~ 3-5
6-8
9=11
12-16
Number of Calcifications Per Cluster < 60
Figure 3. Bar g r a p h of t h e size ( n u m b e r of calcifications per cluster) of the 30 simulated clusters.
141
12
60 -- 100
101 -- 160
> 160
Mean Area (pixels) Figure 4. Bar g r a p h of t h e m e a n a r e a (in pixels) of t h e 30 simulated clusters. The m e a n a r e a is d e f i n e d as the sum of t h e areas of t h e individual calcifications in pixels d i v i d e d b y t h e n u m b e r of calcifications in t h e cluster. The minimum a n d m a x i m u m a r e a values w e r e 24 a n d 357 pixels, respectively, a t a resolution of 30 gm.
~1 ~
==
"6 .{3
E
Z
< 0.090
0.091 - 0.120
0.121 - 0,150
> 0.150
Mean Local Contrast
Figure 5. Bar g r a p h of t h e m e a n local contrast of t h e 30 simulated calcification clusters. The local contrast of a single c a l c i f i c a t i o n is d e f i n e d as t h e diff e r e n c e in g r a y values b e t w e e n a point in t h e calcification a n d a point in the a r e a surrounding t h e calcification div i d e d b y their sum, The five brightest calcifications in a cluster w e r e used to c a l c u l a t e t h e m e a n local contrast of t h e cluster. The minimum a n d m a x i m u m m e a n local contrast values w e r e 0.037 a n d O. 170, respectively,
standard deviation of the Gaussian template was randomly selected from the range of 1-2; this range was determined from statistical analysis of 100 real calcification cases (6). Positioning of the simulated cluster in the mammogram.--The location of the cluster was determined by the user, who, with the help of a computer mouse, selected a pixel on the digitized mammogram as the center of the window. The center of a specified cluster image was then automatically superimposed on the center of the window after the intensity of each object in the cluster was adjusted as described previously. Locations were selected based on reported statistical analysis regarding the relative incidence of breast carcinoma at various locations (22). It is reported that most breast cancers occur laterally and centrally; about 45% of the cancers occur on the upper outer quadrant, 25% are at the center of the breast, 15% are on the upper inner quadrant, 10% are in the lower outer quadrant, and only 5% are in the lower inner quadrant (22). We assumed that the same statistics would apply to calcifications (benign or malignant) and adapted these percentages to single-view mammograms; thus, the following distribution of clusters was maintained: 15 of 30 clusters (50%) were located in
675
KALLERGI ET AL
b
a
4._._•____
f-'--'-~
Vol 5, No 10, October 1998
!1
~'t J '
"---~._..
V k 11
iI
tI "~LI
/'/a,, I J,, ,"~ a. b. c. Figure 6. Generation of a cluster that simulates heterogeneous (pleomorphic) calcifications. (a) Binary image. (b) Gaussian filtered image (Pc). (c) Section of a digitized mammogram with the superimposed simulated cluster of calcifications. The arrows in~licate the location used for the intensity profiles plotted at the bottom of the images.
the outer half of the breast, nine of 30 clusters (30%) were at the center, and six of 30 clusters (20%) were at the inner half. A second criterion used in selecting a location for calcifications was that the background consist primarily of parenchymal tissue (bright areas in the radiograph) or have a gradient of intensities. Dark areas of fat and areas of uniform intensity were avoided. Finally, we ensured that positioning of the simulated objects was anatomically and physically correct; for example, linear branching calcifications were oriented radially from the nipple to the chest wall with the branching end toward the chest wall. Figure 6 shows the three steps followed in the generation of a cluster with pleomorphic calcifications. Intensity profiles are included to show the intensities of the calcifications before and after Gaussian filtering and after superimposing the cluster on the digitized mammogram. E v a l u a t i o n of S i m u l a t e d C a l c i f i c a t i o n s
Three mammographers evaluated the simulated calcifications in a blinded study in separate sessions. Sixty mammograms were used in the study: 30 with real calcification clusters (one per image) and 30 with simulated calcification clusters. The real cases matched the simulated ones in terms of morphologic characteristics and distribution of calcifications. Half of the real cases were benign and half were malignant. Diagnoses were proved by means of biopsy, and the composition of breast tissue was similar to that in the cases selected for the simulation. One high-resolution (2,000 x 2,000 pixel) monitor
676
QUESTIONS A. Is a calcification cluster present and where (yes/no)? If yes, give location. B. Classify the detected cluster according to BIRADS character type (give letter) C. Do you think the cluster is real (yes/no)? If no, answer question D D. Cluster is found simulated due to incorrect (select all that apply): size, brightness, shape, margins, orientation, position, distribution, no-reason, other (specify) Figure 7. Questionnaire completed by each of the three observers during the evaluation process.
with an interface developed in-house was used in the evaluation (23). Interactive gray-scale adjustments (window width, window level, gamma correction) and magnification tools (digital zoom, magnifying glass) were available (23). A table of the BIRADS lesion codes was provided on an adjacent monitor for easy reference in the assignment of calcification types (eg, round, punctate, amorphous). A training session for using the computerized viewing system preceded the evaluation. All readers had at least 1 year of experience using the BIRADS lexicon; no training was done in regard to its implementation. The real and simulated images were mixed randomly and shown to the observers, who completed the questionnaire shown in Figure 7 for each case. Two observers participated in the first stage of the evaluation. A simulation was considered successful if one or both of the radiologists classified it as real in the assessment. A simulation
Table 2 Results of Initial Assessment of Calcification Clusters by Observer I
Table 3 Results of Initial Assessment of Calcification Clusters by Observer 2
Actual Calcification Type Observer 1 Responses Real Simulated
Real
Simulated
16 14
12 18
Actual Calcification Type Observer 2 Responses Real Simulated
Real
Simulated
27 3
22 8
Note.--Z2= 1.071, P = .30i. Yates correction yielded X2= 0.603 a n d P--- .438.
Note.--Z2= 2.783 and P = .095. Yates correction yielded %2= 1.781 a n d P=.182,
Table 4 Results of Assessment of Modified Calcification Clusters by Observer 3
Table 5 C o m b i n e d Resultsof All Three Observers
Actual CalcificationType Actual Calcification Type - Observer 3 Responses Real Simulated
Real
Simulated
16 14
13 17
Note.--Z 2= 0.601 a n d P = .438. Yates correction yielded X2= 0.267 a n d P = .605.
was considered unsuccessful if both radiologists classified it as simulated. All unsuccessful simulations were modified according to the observers' recommendations, and the image set was reevaluated by the third radiologist in a similar manner. The first two observers were also asked to comment on (a) the benign or malignant appearance of the cluster, (b) the number of calcifications seen in the cluster, and (c) the subtlety of the cluster (low, moderate, high). The responses of the observers were analyzed for agreement with the kappa statistic and for independence with the X2 test statistic.
All observers correctly identified the location of all clusters. Observer 1 described only 12 of the 30 simulated cases as realistic, but this observer also described 14 of the 30 real cases as simulated (Table 2). Observer 2 interpreted all but eight of the 30 simulated cases as realistic and described three of the 30 real cases as simulated (Table 3). These two observers disagreed in 12 of the simulated cases; more specifically, one observer classified a case as real while the other observer said it was simulated. They agreed in seven of the simulated cases,
Combined Responses Real Simulated
Real 29 1
Simulated 27 3
Note.--For this result, a cluster was considered real ("Yes" c o m b i n e d response) if a t least one of the observers found it to b e real. A cluster was considered simulated ("No" c o m bined response) if all three observers found it to b e simulated. The %2 value for the c o m b i n e d responses was 1.071 a n d P = .301. Yates correction yielded Z2=0.268 a n d P = .605.
that is, they both thought that the clusters were simulated. The appearances of these seven simulations were changed according to comments by the observers, and the modified set of 60 images was evaluated by a third radiologist. The responses of observer 3 are listed in Table 4. Three of the seven modified cases were again identified as simulated (Table 5). The kappa coefficient, estimated for pairs of observers, was in the range of 0-0.2, which indicates that there was no evidence of agreement in any of the three pairs of readers (results were significant at the .05 level). The results from each observer and the combined responses of all observers were analyzed for independence with the Z2 test at the P = .05 significance level. The Z2 test statistic and the one-tailed probability of obtaining a value of ~2 or greater are listed in Tables 2-5. The Z2 and P results failed to show a difference in observer responses between the tested real and simulated calcification clusters. A much larger sample set should be tested, however, to reach definitive conclusions on differences between the real and simulated cases. With the present sample set, there is no reason to reject the independence hypothesis at the P = .05 significance level; any differences may be attributed to chance. The Yates correction for continuity
677
was also used due to the small sample size, but results did not differ from those of the other tests. The four most frequently given reasons for interpreting a cluster as "not real" were relative intensity, relative position, orientation, and size of the calcifications within the cluster. Substantial variability among observers was seen in assigning a type to the calcifications. This variability indicates primarily the difficulty in implementing the BIRADS lexicon. To test the hypothesis that the observers' type assignments were an accurate approximation of the expected cluster types, the observed frequencies per type for each reader were estimated. Then a )~2 goodness of fit test was performed to compare the observed and expected frequencies. The Z 2 test statistic and the onetailed probability of obtaining a value of )~2 or greater are listed in Table 6 for the three observers. In general, at the P = .05 significance level there is no reason to reject the hypothesis. The P value for observer 3 was only .065, which, although greater than .05, does not provide statistically significant evidence that the observed frequencies are a good approximation of the expected frequencies. As mentioned previously, two of the observers also classified the simulated clusters as benign or malignant, assessed the subtlety of the cases, and estimated the number of calcifications in the cluster. The observers' findings were in agreement in 90% of the decisions. On average, a third of the simulated clusters were classified as malignant and a third were classified as benign, a finding that is in agreement with the intent of the study design. The remaining third of the clusters were classified as indeterminate. From the indeterminate cases, half were designed to be benign and half were designed to be malignant. The majority of the indeterminate clusters were of the heterogeneous type. In terms of difficulty, eight (mean value) of the 30 simulated clusters were rated as difficult to detect. The remaining 22 clusters were considered easy to detect. No case was considered moderately difficult to detect, possibly because difficulty was determined by whether the cluster could be seen in a casual inspection of the reduced image (easy) or only after careful examination of all the sections at high resolution (difficult). As a result, a binary response was more likely. In the easy cases, the observers identified either all the calcifications in the cluster or more calcifications than were actually present, probably because of the scaling of the digital images and the graininess of the digital display. In the difficult cases, fewer calcifications were usually identified. Not all diffi-
678
Table 6 Results from the X2 Goodness of Fit between the Observed Frequencies of the Various Cluster Types and the Expected Frequencies from the Simulation Design Parameters X2 P*
Observer 1 5.074 .166
Observer 2
Observer 3
4.431 .219
7.225 .065
*One-tailed probability of obtaining a value of X2or greater.
cult cases were misclassified, a finding that suggests that detection difficulty is not equivalent to diagnostic difficulty.
We have developed a method for the simulation of calcification clusters based on the BIRADS guidelines for morphology and distribution of mammographic calcifications. Among the observers who evaluated the accuracy of reproduction of the various shapes and sizes of calcifications, substantial variability was observed, as shown by the kappa coefficients. A Z2 analysis of the individual and combined observer responses, however, did not indicate any statistically significant difference between real and simulated calcifications. This outcome supported the use of the proposed simulation model for the generation of test calcifications and the development of a larger test set as required for quantitative assessment of any differences between real and simulated calcifications. The generation of a large set of simulated calcification cases can be performed mathematically as long as there exists a representative basic set. Bootstrapping with "jittering" is a mathematic approach that has already been used in our laboratory for the generation of 1,000 pseudoclusters from a set of 50 real clusters (23). It can be used in a similar way to generate a large number of simulated clusters from the initial set described here. Additional morphologic characteristics and distributions of calcifications can be simulated by first creating a representative set with the model presented here and then applying jittering. In regard to the issue of representativeness, ideally, a representative set should consist of cases on which there is "ground truth" of the morphologic characteristics of the calcifications. Such ground truth can only come from an expert mammographer, but the large interobserver variability that may occur poses a problem. A possible solution may be the use of a panel of expert
Vol 5, No 10, October 1998
SIMULATION MODEL OF M A M M O G R A P H I C CALCIFICATIONS
mammographers who are in full agreement on the type of simulated calcifications that are part of the representative set. This set can then be expanded and used for clinical training or other applications as mentioned later. Further evaluation showed that the simulation model accurately reproduced the properties (intensity, size, shape, distribution) of punctate, round, rodlike, heterogeneous, and casting calcifications. The agreement between any two observers or between an observer and the simulation was the highest for these types. Very poor agreement was observed for coarse, dystrophic, and amorphous calcifications, a finding that suggests particular difficulty in the BIRADS interpretation of these types of clusters, as well as modeling weaknesses. Regardless of the reason, incorrect identification of the morphologic characteristics and distribution of the calcifications had a direct effect on the diagnosis. This finding indicates the importance of correctly identifying the type of the calcification. It also indicates a role for CAD, namely, in the assignment of BIRADS categories to detected clusters. The use of simulated cases similar to those presented here may aid in the development and optimization of such CAD methods. The responses of the observers on a cluster-by-cluster basis, as well as the problems encountered during modeling, led us to the conclusion that generally benignlike calcifications are more difficult to simulate than malignantlike ones because of their specific morphologic characteristics. The random properties of several of the malignantlike calcifications turned out to be an advantage in their simulation. Rodlike and casting calcifications posed special difficulties in reproduction owing to their particular orientation and location requirements. Potential applications of the simulated calcification clusters generated in this study may include (a) the evaluation of segmentation algorithms in terms of both detection performance and shape preservation; (b) the evaluation and ranking of morphologic features used in classification methods; (c) the evaluation of the effects of different image parameters (pixel size and depth) on the morphology and distribution of calcifications, as well as on the performance of detection and classification algorithms; and (d) the development of computer methods to assist in the implementation of BIRADS and in the consistent reporting of mammographic findings.
IEFERENCE,c
1, Parker SL, Tong T, Bolden S, Wingo PA. Cancer statistics, 1996. CA Cancer J Ciin 1996; 46:5-28, 2, Sickles EA. Breast calcifications: mammographic evaluation. Radiology 1986; 160:289-293. 3. Millis RR, Davis R, Stacey A J. The detection and significance of calcifications in the breast: a radiological and pathological sign. BrJ Radio11976; 49:12-26, 4. Lanyi M. Diagnosis and differential diagnosis of breast calcifications. New York, NY: Springer-Verlag, 1986. 5. Clark RA, Nemec L, Love N. Breast imaging: a practical look at its capabilities and its limitations. Postgrad Med 1992; 92:117-134, 6. Kallergi M, Clarke LP, Qian W, et al. Interpretation of calcifications in screen/film, digitized, and wavelet-enhanced, monitor displayed mammograms: an ROC study. Acad Radio11996; 3:285-293. 7. Doi K, Giger ML, Nishikawa RM, et al. Digital radiography: a useful clinical tool for computer-aided diagnosis by quantitative analysis of radiographic images. Acta Radiol 1993; 34:426-439. 8, Lo JY, Baker JA, Kornguth PJ, Floyd CE. Computer-aided diagnosis of breast cancer: artificial neural network app r o a c h for optimized merging of mammographic features. Acad Radio11995; 2:841-850. 9. Qian W, Kallergi M, Clarke LP, et al. Tree structured w a v e l e t transform segmentation of microcatcifications in digital mammography. Med Phys 1995; 22:1247-1254. 10. Li HD, Kallergi M, Clarke LP, Jain VK, Clark RA, Markov random field for tumor detection in digital mammography. IEEETrans Med Phys 1995; 14:565-576. 11. Chalana V, Kim Y. A methodology for evaluation of image segmentation algorithms on medical images. SPIE 1996; 2710:178189. 12. Eskicioglu AM. Application of multidimensional quality measures to reconstructed medical images. Opt Eng 1996; 35:778-785. 13. Lefebvre F, Benali H, Gilles R, Di Paola R. A simulation model of clustered breast microcalcifications, Med Phys 1994; 21:18651874. 14. Lado M J, Tahoces PG, Souto M, M6ndez A J, Vidal JJ. Real and simulated clustered microcalcifications in digital mammograms: ROC study of observer performance. Med Phys 1997; 24:13851394. 15. Brzakovic D, Brzakovic P, Neskovic M. An appr oa c h to automated screening of mammograms. SPIE 1993; 1905:690-701, 16. Astley S, Hutt I, Adamson S, et al. Automation in mammography: computer vision and human perception. SPIE 1993; 1905:716730. 17. Judy PF. Detection of clusters of simulated calcifications in lumpy noise backgrounds. SPIE 1996; 2716:39-46. 18. Chan HP, Doi K. Monte Carlo simulation in diagnostic radiology. In: Morin RL, ed. Monte Carlo simulation in the radiological sciences. Boca Raton, Fla: CRC, 1988; 103-192, 19. D'Orsi C J, Kopans DB. Mammographic feature analysis. Semin Roentgeno11993; 28:204-230. 20. Gavrielides MA, Kallergi M, Berman CG, Clarke LP. Automatic shape analysis and classification of mammographic calcifications. SPIE 1997; 3034:869-876. 21. Mammography: a user's guide--recommendations of the National Council on Radiation Protection and Measurements. NCRP report no. 85. Washington, DC: Government Printing Ofrice, 1986. 22. Gohel HJ, Kallergi M, Vossberg M, Gavrielides MA, Clarke LP. A workstation interface for ROC studies in digital mammography. SPIE 1997; 3031:440-447. 23. Ruth EJ. Estimating the minimal sample size of calcification clusters which approximates the Bayes risk of diagnosis. Thesis. University of South Florida, Tampa, Fla, 1997.
o
679