- Email: [email protected]
PCNN-based level set method of automatic mammographic image segmentation
G Model
ARTICLE IN PRESS
IJLEO 56455 1–7
Optik xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
PCNN-based level set method of automatic mammographic image segmentation
1
2
3 4 5
Q1
Weiying Xie a , Yunsong Li a,∗ , Yide Ma b a b
State Key Laboratory of Integrated Service Network, Xidian University, Xi’an 710071, China School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China
6
7 19
a r t i c l e
i n f o
a b s t r a c t
8 9 10 11 12
Article history: Received 4 December 2014 Accepted 30 September 2015 Available online xxx
13
18
Keywords: Mammographic image Image segmentation Pulse coupled neural network Level set method
20
1. Introduction
14 15 16 17
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
A novel approach to mammographic image segmentation, termed as PCNN-based level set algorithm, is presented in this paper. As well known, it is difficult to robustly achieve mammogram image segmentation due to low contrast between normal and lesion tissues. Therefore, Pulse Coupled Neural Network (PCNN) algorithm is firstly employed to achieve mammary-specific and mass edge detection for subsequently extracting contour as the initial zero level set. The proposed scheme accurately obtains the initial contour for level set evolution, which does not suffer from the drawback that level set method is sensitive to the initial contour. Especially, an improved level set evolution is performed to segment the images and get the final results. A preliminary evaluation of the proposed method performs on a known public database, namely MIAS, which demonstrates that the proposed framework in this paper can potentially obtain better masses detection results than traditional CV and VFC model in terms of accuracy. © 2015 Elsevier GmbH. All rights reserved.
Recent statistics show that breast cancer is a serious disease with high incidence rate [1] and one of the leading causes of early mortality in women [2]. Breast cancer is the top cancer in women worldwide and is increasing particularly in developing countries where the majority of cases are diagnosed in late stages. Up to now, mammograms are effective screening technique for early detection of abnormalities like masses which are characterized by their shape, size and margin [3]. There is a clear evidence document which shows that early diagnosis and treatment of breast cancer can significantly increase the chance of survival for patients [4]. Therefore, it is suggested that women over the age of 40 get mammograms once a year. Mammography screening can depicts most of the significant changes of breast disease [5], which is a specific type of imaging that uses a low-dose X-ray system and high-contrast, high-resolution film for examination of the breasts. The main problem to analysis mammogram images is that low contrast between normal and lesions tissues and much noise lays in such images makes it very difficult to clearly segment these mammogram images [6]. In fact, image segmentation is one of the
∗ Corresponding author. Tel.: +86 15829262265; fax: +86 2988204271. E-mail address: [email protected] (Y. Li).
fundamental and significant tasks in image processing and computer vision. Osher and Sethian first introduced level set method in 1988 [7,8], which is widely used in image segmentation. There are large amounts of algorithms and techniques that have been developed to solve image segmentation problems. Paper [9] introduced shape modeling with front propagation based on level set method to implement image segmentation. Typically, an image segmentation study applying level set approach has been performed independently by Caselles et al. [10]. The fundamental principle of level set method is to represent a contour as the zero level set of a higher dimensional function, usually called a level set function (LSF), and to formulate the motion of the contour as the evolution of the level set function based on a partial differential equation (PDE). Li et al. [11,12] presented a new variational level set method to force the level set function to close to a signed distance function and eliminate the re-initialization procedure. Lately, the literature [13] introduced DRLSE model by incorporating a double-well potential function used in the geodesic active contour model. However, this method is a bit sensitive to the initial position of the evolving curve because it can just drive the curve evolving in the direction specified in advance. What is needed at the very beginning is a list of the coordinates of all required grid points together with their initial level set values. In addition, if the initial position is much close to the region of interest (ROI), the segmentation process will be automatic and the results will be better with a lower time-consumption.
http://dx.doi.org/10.1016/j.ijleo.2015.09.250 0030-4026/© 2015 Elsevier GmbH. All rights reserved.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
2
Fig. 1. Processing flow of breast mass segmentation algorithm.
91
Actually, only few works were developed in this research area. In order to overcome above drawbacks, we propose a new scheme. The paper presented a novel PCNN-based level set method to automatic detect and segment the breast masses with the intention to increase both sensitivity and specificity of the physicians interpretation of mammograms. Pulse Coupled Neural Network (PCNN) is an artificial neural network comes from the research of small mammals’ visual cortical properties [14]. This algorithm has proven to be highly effective when used in a diverse set of applications [15–20]. We use PCNN to segment the original mammographic image for the first time. This process aims to achieve Mammary-Specific scheme, so that we only make focus on whole breast region and ignore the background to detect the breast mass. After that, the PCNN algorithm is used again to detect breast mass only in the whole breast region. Then, an initial curve is evolved to approach the true boundary of the object by using the variational level set method. To date, there has been little published work on the combination of these two methods to automatically segment the region of interest (ROI). A general processing flow chart of the proposed method is outlined in Fig. 1. The following part of this paper is divided into five sections. Section 2 gives a brief description of the variational level set method. Section 3 mainly focuses on the determination of the initial location by PCNN model. Section 4 shows the experimental result by applying our proposed approach to segmenting mammographic images. In the last section, a conclusion is made.
92
2. Overview of level set model
93
2.1. Variational level set method
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
97
The energy functional is just the co-activation of the internal and external energies that make the zero level set curve C matching the boundaries well and reach a perfect effect of image segmentation, which can be defined as:
98
E = mEint + Eext = mP() + Lg () + ˛Ag ()
94 95 96
104
105
Lg () =
100 101 102 103
ı is the derivative of H , i.e. H = ı . The Gateaaux derivative of
107
the functional E can be written as:
108
∂ ∂E =− = m − div ∂t ∂
∇ |∇ |
∇
+ ı()div g
|∇ |
+ gı()
109
(4)
110
111
Here, is the Laplace operator. Therefore, the function minimizes this functional to satisfy the Euler–Lagrange equation.
2.2. Update the level set function The partial differential equation in the continuous domain defined in Eq. (4) can be solved by a finite difference method in numerical scheme. The numerical scheme of the gradient flow mentioned above using the forward difference can be simply written as follows: k+1 k i,j − i,j ∂ k = = L i,j ∂t
(5)
gı()|∇ |dx
(2)
gH(−)dx
(3)
˝
106
Ag () = ˝
112 113
114
(1)
where m is a parameter controlling the penalization effect of the internal energy. The energy functional Lg () computes the line integral of the function g along the zero level contour of . The energy functional Ag () computes a weighted area of the region inside the zero level set. , ˛ are the coefficient of the energy functional Lg () and Ag (), respectively. Lg () and Ag () are defined by
99
Fig. 2. Processing flow of coarse segmentation.
Fig. 3. Processing flow of refined segmentation.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
115 116 117 118 119
120
G Model
ARTICLE IN PRESS
IJLEO 56455 1–7
W. Xie et al. / Optik xxx (2015) xxx–xxx
3
Table 1 Detection results of the initial mask of suspicious regions per image. Class of lesions
Mammogram
Our method Range ˝0
Ground truth x
y
R
Range
CIRC CIRC SPIC SPIC MISC MISC ARCH ARCH ASYM ASYM
mdb028 mdb021 mdb184 mdb202 mbd134 mdb271 mdb155 mdb117 mdb081 mdb083
(310:380, 284:344) (480:525, 109:144) (300:390, 574:674) (535:580, 749:789) (450:490, 696:751) (750:820, 228:309) (434:462, 462:490) (470:530, 554:584) (420:560, 399:524) (520:580, 181:204)
338 493 352 557 469 784 448 480 492 544
314 125 624 772 728 270 480 576 473 194
56 49 114 37 49 68 95 84 131 38
(282:394, 258:370) (444:542, 76:174) (238:466, 510:738) (520:594, 735:809) (420:518, 679:777) (716:852, 202:338) (353:543, 385:575) (396:564, 492:660) (361:623, 342:604) (506:582, 156:232)
Fig. 4. Initial mask obtained by PCNN algorithm. Images in the column (a.1) and (a.2) are two output examples of CIRC lesions. Images in (b) column are the two examples of initial contour obtained in SPIC lesions. Images in (c), (d) and (e) column are the examples of MISC, ARCH and ASYM lesions, respectively.
121
where is the time-step, we choosea fixed step size in accordance
122
with each image, approximately. L i,j ) is the numerical approx-
123
imation. In L i,j ), the corresponding curvature is defined as:
For a sake of clarity, Eq. (4) can be implemented as follows:
k
k+1 k i,j − i,j n ∂ k n n n + i−1,j + i,j+1 + i,j−1 = L i,j = m i+1,j = ∂t
124
125
k
= div
∇ |∇ |
n
n
n −4i,j − + ı i,j g + vgı i,j
=
xx y2
− 2xy x y + yy x2 3/2 x2 + y2
126
(7)
127
(6) where is computed according to Eq. (6).
Fig. 5. Detection accuracy of our proposed method for initial contour. (a) Image-coordinates region of abnormality in x direction. (b) Image-coordinates region of abnormality in y direction.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
128
G Model IJLEO 56455 1–7 4
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
Fig. 6. Segmentation and extraction results of the PCNN-based level set method in narrowband implementation. Column A: the original image. Column B: the Mammaryspecific process using PCNN algorithm. Column C: the extraction of breast mass only in the whole breast region result using PCNN algorithm. Column D: the coordinate value of the whole breast and mass by Matlab command. Column E: the final segmentation result using level set model based on the obtained initial contour. Column F: the mass extraction result.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
5
Fig. 7. Segmentation results of three Mammographic images from the MIAS database. Each row illustrates the experiment of one image. Images in the first column are the original Mammographic images. The second column is the final segmentation results processed by the PCNN-based Level Set model. Images in the third column are the contrast segmentation results processed by the CV model. And images in the fourth column are the contrast segmentation results obtained by the classical VFC model. The last column shows ground truth.
129
3. Proposed mass segmentation method
130
3.1. Coarse segmentation to obtain initial mask
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149
As described above, the novel proposed model mainly contains coarse segmentation and refined segmentation. The main achievement is to obtain the initial mask for latterly level set method in the coarse segmentation process. PCNN model is significantly used in the coarse step to obtain the initial contour. The fundamental theory of segmentation by using PCNN is that each pixel is correctly assigned to the region it belongs to when all the pixels of an input image are fired by PCNN [21]. The processing flow of coarse segmentation is outlined as shown in Fig. 2. The first aim of coarse segmentation process is to achieve mammary-specific. It is obvious that the mammary-specific process considerably reduce the complexity of image segmentation. Subsequently, we make lesion detection in breast region, i.e. breast mass detection, by using PCNN algorithm again. The last step of coarse segmentation is sketching out the outline, in order to illustrate the coordinate values of the lesion in mammography. Meanwhile, the coordinate values of lesion can be obtained, which is the initial mask of level set algorithm for the refined segmentation of mammography images.
3.2. Refined segmentation with level set method
150
As previous mentioned, the initial mask for latterly level set segmentation has been obtained by PCNN algorithm. We propose to use a binary step function in (8) as the initial LSF, as it can be generated extremely efficiently. The binary initial level set function can be obtained precisely as an example shown in the processing flow. Practically, we can define the initial LSF in the rectangular region ˝0 = (450:490, 270:330) of the example in Fig. 2, such that ˝0 is close to the region to be segmented. The level set function evolves from a binary step function to an approximate signed distance function on a signed distance band (SDB). Because its values vary from –d to d across the band at the rate of | ∇ |=1, when the function becomes a signed distance function in the SDB. In this context, the initial value of d is usually set as 2:
0 (x, y) =
151 152 153 154 155 156 157 158 159 160 161 162 163
⎧ −2, (x, y) ∈ ˝0 − ∂˝0 ⎪ ⎨ ⎪ ⎩
0,
(x, y) ∈ ∂˝0
2,
(x, y) ∈ ˝ − ˝0
(8)
Here, we also illustrate refined segmentation by an example as follow figure. With the accurate region ˝0 , the segmentation step is automatic and efficient (Fig. 3). Q2
Fig. 8. Maximum distance detected results. From left to right: (x:265, y:472) is the detected results after PCNN-based level set model, (x:276, y:467) and (x:345, y:277) is the detected results after CV model, VFC model, respectively.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
164
165 166 167
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
6
Fig. 9. Applications of our method to two ultrasound images of gallstone and two iris location of iris images. Column 1: original image; Column 2: final zero level contours of level set function, i.e. the segmentation result; Column 3: extraction result.
168
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
4. Experimental results All the images used in this paper are belonging to a publicly available digital database of mammograms. The dataset used for evaluation of the proposed approaches for image segmentation are selected from the Mini Mammographic Image Analysis Society (MIAS) database in United Kingdom. The experiments are run on MATLAB 7.8.0 with Intel Core 2 Quad i5-2400 3.10 GHz PC. The coarse segmentation results by applying PCNN for detecting initial rectangular region ˝0 are illustrated in Table 1. The binary initial LSF can be defined in the rectangular region ˝0 , such that ˝0 is close to the region to be segmented. The annotations in MIAS are used as ground truth data. As shown in Table 1, the initial rectangular region ˝0 we obtained per image are all inside of the ground truth. Approximately, the breast mass can be extracted after the Mammary-Specific process by using PCNN algorithm. No matter for which suspicious types, the regions obtained in high accuracy are more reliable and reasonable as shown in Fig. 4. In terms of accuracy, the image-coordinates region of abnormality are shown in Fig. 5 in x and y direction, where the x-axes represent 10 different images. We can observe that the initial mask obtained per image is all inside of the ground truth, so that we can accurately determine the initial LSF in that region by PCNN. The next process is refined segmentation by using level set method, whose initial contour already obtained by PCNN model as previous mentioned. The ultimately extraction results are also shown in Fig. 6. In addition, a comparison of segmentation performance is conducted among three different methods to evaluate the validity of the proposed model. Two other common segmentation methods are employed. One is CV model proposed by Chan and Vese [22] and the other one is classical VFC model proposed by [23]. Moreover, the proposed PCNN-based level set model is automatic segmentation without re-initialization, but both CV model and VFC model are sensitive to initial location and noise. By using the PCNN algorithm, the initial contour for lately level set evolution is efficiently obtained, which also reduces the computation and improves accuracy. The smoothness of evolution surface has been maintained and the phenomenon of edge leakage has been avoided by the refined segmentation process using level set
method. Absolutely, the level set method based on PCNN algorithm can segment breast mass not only avoiding boundary leakage, but also without segmenting the backgrounds which is similar to the ROI. All these process are automatic without manual segmentation. It can draw a conclusion that the level set method based on PCNN algorithm has a clearer, more complete segmentation result and a higher accuracy (Figs. 7 and 8). The PCNN-based level set method has also been applied to real images. For examples, Fig. 9 shows the results of the PCNN-based level set method for other medical images: two ultrasound images of gallstone and two iris images. In this case, the choice of the number of regions is based on prior medical knowledge. The most important aim of this experiment is to demonstrate the ability of the proposed method to adapt automatically to this class of images acquired with special techniques. Because of the presence of speckle noise, low contrast and luminous in-homogeneity in ultrasound images, the available segmentation algorithms are general techniques and fail to detect gallstones in ultrasound images. As shown in first columns of Fig. 9, our proposed PCNN based level set model can accurately extract the gallstones in ultrasound images. Meanwhile, the proposed method by this paper also efficiently finds the location of the iris boundaries. 5. Conclusions A novel mammographic image segmentation method that combines the PCNN segmentation algorithm with the level set method is presented for the first time. Note that the most convincing evidence comes from the outstanding performance of mammographic image segmentation. The proposed scheme accurately obtains the initial contour for level set evolution, which does not suffer from the drawbacks that level set method is sensitive to the initial contour. Moreover, the numerous experiments which demonstrate that PCNN segmentation algorithm can effectively achieve the mammary-specific scheme and mass outline detection. This coarse segmentation process considerably reduces the complexity for mammographic image segmentation. The examples show that mass initial mask can be accurate detected by the proposed PCNN algorithm. Additionally, before dealing with the
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227
228
229 230 231 232 233 234 235 236 237 238 239 240 241 242
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
249
mammographic image, we obtain the negative image own to the mammographic image with predominantly dark region except the breast tumor. In terms of accuracy and robustness, experimental results have demonstrated superior performance of level set method based on the PCNN algorithm. Ultimately, this could lead to computer-assisted, quantitative assessment left ventricular function in clinical practice.
250
Acknowledgments
243 244 245 246 247 248
The authors would like to thank the reviewers and AE for their comments that have helped improve this paper. The authors 252 would like to thank the anonymous reviewers and AE for their 253 comments and constructive suggestions. This work was partially 254 Q3 supported by the National Nature Science Foundation of China (nos. 255 61222101, 61301287, 61301291 and 61350110239) and the 111 256 project (B08038). It was also partially supported by the Funda257 mental Research Funds for Central Universities (K5051201043 and 258 K5051301015). 259 251
260
261 262 263 264 265 266 267 268 269 270 271 272 273 274
References [1] R. Siegel, D. Naishadham, A. Jemal, CA: Cancer J. Clin. 63 (1) (2013) 11–30. [2] A. Jemal, et al., Global cancer statistics, CA: Cancer J. Clin. 61 (2) (2011) 69–90. [3] C.H. Lee, D.D. Dershaw, D. Kopans, P. Evans, B. Monsees, D. Monticciolo, R.J. Brenner, L. Bassett, W. Berg, S. Feig, Breast cancer screening with imaging: recommendations from the society of breast imaging and the ACR on the use of mammography, breast MRI, breast ultrasound, and other technologies for the detection of clinically occult breast cancer, J. Am. Coll. Radiol. 7 (1) (2010) 18–27. [4] J.S. Mandelblatt, K.A. Cronin, S. Bailey, D.A. Berry, H.J. de Koning, G. Draisma, H. Huang, S.J. Lee, M. Munsell, S.K. Plevritis, Effects of mammography screening under different screening schedules: model estimates of potential benefits and harms, Ann. Intern. Med. 151 (10) (2009) 738–747. [5] S. Timp, C. Varela, N. Karssemeijer, Temporal change analysis for characterization of mass lesions in mammography, IEEE Trans. Med. Imaging 26 (7) (2007) 945–953.
7
[6] Y. Wang, J. Li, X.B. Gao, Latent feature mining of spatial and marginal characteristics for mammographic mass classification, Neurocomputing 144 (2014) 107–118. [7] S. Osher, J. Sethian, Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi formulations, J. Comput. Phys. 79 (1) (1988) 12–49. [8] S. Osher, R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, SpringerVerlag, New York, 2002. [9] R. Malladi, J.A. Sethian, B.C. Vemuri, Shape modeling with front propagation: a level set approach, IEEE Trans. Pattern Anal. Mach. Intell. 17 (2) (1995) 158–175. [10] V. Caselles, R. Kimmel, G. Sapiro, Geodesic active contours, Int. J. Comput. Vis. 22 (1) (1997) 61–79. [11] C. Samson, L. Blanc-Feraud, G. Aubert, J. Zerubia, A variational model for image classification and restoration, IEEE Trans. Pattern Anal. Mach. Intell. 22 (5) (2000) 460–472. [12] C. Li, C. Kao, J.C. Gore, Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Trans. Image Process. 17 (10) (2008) 1940–1949. [13] C. Li, C. Xu, C. Gui, M.D. Fox, Distance regularized level set evolution and its application to image segmentation, IEEE Trans. Image Process. 19 (12) (2010) 3243–3254. [14] J.L. Johnson, M.L. Padgett, PCNN models and applications, IEEE Trans. Neural Netw. 10 (3) (1999) 480–498. [15] R. Eckhorn, H.J. Reitboeck, M. Arndt, P.W. Dicke, Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex, Neural Comput. 2 (3) (1990) 293–307. [16] R.C. Zhao, Y.D. Ma, A region segmentation method for region-oriented image compression, Neurocomputing 85 (2012) 45–52. [17] Y.D. Ma, C.L. Qi, Study of automated PCNN system based on genetic algorithm, J. Syst. Simul. 18 (3) (2006) 722–725. [18] Z. Wang, Y.D. Ma, F. Cheng, L. Yang, Review of pulse-coupled neural networks, Image Vis. Comput. 28 (1) (2010) 5–13. [19] Y.D. Ma, L. Liu, K. Zhan, Y.Q. Wu, Pulse-coupled neural networks and one-class support vector machines for geometry invariant texture retrieval, Image Vis. Comput. 28 (11) (2010) 1524–1529. [20] X.J. Li, Y.D. Ma, Z.B. Wang, W. Yu, Geometry-invariant texture retrieval using a dual-output pulse coupled neural network, Neural Comput. 24 (1) (2012) 194–216. [21] G. Kuntimad, H.S. Ranganath, Perfect image segmentation using pulse coupled neural networks, IEEE Trans. Neural Netw. 10 (3) (1999) 591–598. [22] T.F. Chan, L.A. Vese, Active contours without edges, IEEE Trans. Image Process. 10 (2) (2001) 266–277. [23] B. Li, S.T. Acton, Active contour external force using vector field convolution for image segmentation, IEEE Trans. Image Process. 16 (8) (2007) 2096–2106.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317
ARTICLE IN PRESS
IJLEO 56455 1–7
Optik xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
PCNN-based level set method of automatic mammographic image segmentation
1
2
3 4 5
Q1
Weiying Xie a , Yunsong Li a,∗ , Yide Ma b a b
State Key Laboratory of Integrated Service Network, Xidian University, Xi’an 710071, China School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China
6
7 19
a r t i c l e
i n f o
a b s t r a c t
8 9 10 11 12
Article history: Received 4 December 2014 Accepted 30 September 2015 Available online xxx
13
18
Keywords: Mammographic image Image segmentation Pulse coupled neural network Level set method
20
1. Introduction
14 15 16 17
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
A novel approach to mammographic image segmentation, termed as PCNN-based level set algorithm, is presented in this paper. As well known, it is difficult to robustly achieve mammogram image segmentation due to low contrast between normal and lesion tissues. Therefore, Pulse Coupled Neural Network (PCNN) algorithm is firstly employed to achieve mammary-specific and mass edge detection for subsequently extracting contour as the initial zero level set. The proposed scheme accurately obtains the initial contour for level set evolution, which does not suffer from the drawback that level set method is sensitive to the initial contour. Especially, an improved level set evolution is performed to segment the images and get the final results. A preliminary evaluation of the proposed method performs on a known public database, namely MIAS, which demonstrates that the proposed framework in this paper can potentially obtain better masses detection results than traditional CV and VFC model in terms of accuracy. © 2015 Elsevier GmbH. All rights reserved.
Recent statistics show that breast cancer is a serious disease with high incidence rate [1] and one of the leading causes of early mortality in women [2]. Breast cancer is the top cancer in women worldwide and is increasing particularly in developing countries where the majority of cases are diagnosed in late stages. Up to now, mammograms are effective screening technique for early detection of abnormalities like masses which are characterized by their shape, size and margin [3]. There is a clear evidence document which shows that early diagnosis and treatment of breast cancer can significantly increase the chance of survival for patients [4]. Therefore, it is suggested that women over the age of 40 get mammograms once a year. Mammography screening can depicts most of the significant changes of breast disease [5], which is a specific type of imaging that uses a low-dose X-ray system and high-contrast, high-resolution film for examination of the breasts. The main problem to analysis mammogram images is that low contrast between normal and lesions tissues and much noise lays in such images makes it very difficult to clearly segment these mammogram images [6]. In fact, image segmentation is one of the
∗ Corresponding author. Tel.: +86 15829262265; fax: +86 2988204271. E-mail address: [email protected] (Y. Li).
fundamental and significant tasks in image processing and computer vision. Osher and Sethian first introduced level set method in 1988 [7,8], which is widely used in image segmentation. There are large amounts of algorithms and techniques that have been developed to solve image segmentation problems. Paper [9] introduced shape modeling with front propagation based on level set method to implement image segmentation. Typically, an image segmentation study applying level set approach has been performed independently by Caselles et al. [10]. The fundamental principle of level set method is to represent a contour as the zero level set of a higher dimensional function, usually called a level set function (LSF), and to formulate the motion of the contour as the evolution of the level set function based on a partial differential equation (PDE). Li et al. [11,12] presented a new variational level set method to force the level set function to close to a signed distance function and eliminate the re-initialization procedure. Lately, the literature [13] introduced DRLSE model by incorporating a double-well potential function used in the geodesic active contour model. However, this method is a bit sensitive to the initial position of the evolving curve because it can just drive the curve evolving in the direction specified in advance. What is needed at the very beginning is a list of the coordinates of all required grid points together with their initial level set values. In addition, if the initial position is much close to the region of interest (ROI), the segmentation process will be automatic and the results will be better with a lower time-consumption.
http://dx.doi.org/10.1016/j.ijleo.2015.09.250 0030-4026/© 2015 Elsevier GmbH. All rights reserved.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
2
Fig. 1. Processing flow of breast mass segmentation algorithm.
91
Actually, only few works were developed in this research area. In order to overcome above drawbacks, we propose a new scheme. The paper presented a novel PCNN-based level set method to automatic detect and segment the breast masses with the intention to increase both sensitivity and specificity of the physicians interpretation of mammograms. Pulse Coupled Neural Network (PCNN) is an artificial neural network comes from the research of small mammals’ visual cortical properties [14]. This algorithm has proven to be highly effective when used in a diverse set of applications [15–20]. We use PCNN to segment the original mammographic image for the first time. This process aims to achieve Mammary-Specific scheme, so that we only make focus on whole breast region and ignore the background to detect the breast mass. After that, the PCNN algorithm is used again to detect breast mass only in the whole breast region. Then, an initial curve is evolved to approach the true boundary of the object by using the variational level set method. To date, there has been little published work on the combination of these two methods to automatically segment the region of interest (ROI). A general processing flow chart of the proposed method is outlined in Fig. 1. The following part of this paper is divided into five sections. Section 2 gives a brief description of the variational level set method. Section 3 mainly focuses on the determination of the initial location by PCNN model. Section 4 shows the experimental result by applying our proposed approach to segmenting mammographic images. In the last section, a conclusion is made.
92
2. Overview of level set model
93
2.1. Variational level set method
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
97
The energy functional is just the co-activation of the internal and external energies that make the zero level set curve C matching the boundaries well and reach a perfect effect of image segmentation, which can be defined as:
98
E = mEint + Eext = mP() + Lg () + ˛Ag ()
94 95 96
104
105
Lg () =
100 101 102 103
ı is the derivative of H , i.e. H = ı . The Gateaaux derivative of
107
the functional E can be written as:
108
∂ ∂E =− = m − div ∂t ∂
∇ |∇ |
∇
+ ı()div g
|∇ |
+ gı()
109
(4)
110
111
Here, is the Laplace operator. Therefore, the function minimizes this functional to satisfy the Euler–Lagrange equation.
2.2. Update the level set function The partial differential equation in the continuous domain defined in Eq. (4) can be solved by a finite difference method in numerical scheme. The numerical scheme of the gradient flow mentioned above using the forward difference can be simply written as follows: k+1 k i,j − i,j ∂ k = = L i,j ∂t
(5)
gı()|∇ |dx
(2)
gH(−)dx
(3)
˝
106
Ag () = ˝
112 113
114
(1)
where m is a parameter controlling the penalization effect of the internal energy. The energy functional Lg () computes the line integral of the function g along the zero level contour of . The energy functional Ag () computes a weighted area of the region inside the zero level set. , ˛ are the coefficient of the energy functional Lg () and Ag (), respectively. Lg () and Ag () are defined by
99
Fig. 2. Processing flow of coarse segmentation.
Fig. 3. Processing flow of refined segmentation.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
115 116 117 118 119
120
G Model
ARTICLE IN PRESS
IJLEO 56455 1–7
W. Xie et al. / Optik xxx (2015) xxx–xxx
3
Table 1 Detection results of the initial mask of suspicious regions per image. Class of lesions
Mammogram
Our method Range ˝0
Ground truth x
y
R
Range
CIRC CIRC SPIC SPIC MISC MISC ARCH ARCH ASYM ASYM
mdb028 mdb021 mdb184 mdb202 mbd134 mdb271 mdb155 mdb117 mdb081 mdb083
(310:380, 284:344) (480:525, 109:144) (300:390, 574:674) (535:580, 749:789) (450:490, 696:751) (750:820, 228:309) (434:462, 462:490) (470:530, 554:584) (420:560, 399:524) (520:580, 181:204)
338 493 352 557 469 784 448 480 492 544
314 125 624 772 728 270 480 576 473 194
56 49 114 37 49 68 95 84 131 38
(282:394, 258:370) (444:542, 76:174) (238:466, 510:738) (520:594, 735:809) (420:518, 679:777) (716:852, 202:338) (353:543, 385:575) (396:564, 492:660) (361:623, 342:604) (506:582, 156:232)
Fig. 4. Initial mask obtained by PCNN algorithm. Images in the column (a.1) and (a.2) are two output examples of CIRC lesions. Images in (b) column are the two examples of initial contour obtained in SPIC lesions. Images in (c), (d) and (e) column are the examples of MISC, ARCH and ASYM lesions, respectively.
121
where is the time-step, we choosea fixed step size in accordance
122
with each image, approximately. L i,j ) is the numerical approx-
123
imation. In L i,j ), the corresponding curvature is defined as:
For a sake of clarity, Eq. (4) can be implemented as follows:
k
k+1 k i,j − i,j n ∂ k n n n + i−1,j + i,j+1 + i,j−1 = L i,j = m i+1,j = ∂t
124
125
k
= div
∇ |∇ |
n
n
n −4i,j − + ı i,j g + vgı i,j
=
xx y2
− 2xy x y + yy x2 3/2 x2 + y2
126
(7)
127
(6) where is computed according to Eq. (6).
Fig. 5. Detection accuracy of our proposed method for initial contour. (a) Image-coordinates region of abnormality in x direction. (b) Image-coordinates region of abnormality in y direction.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
128
G Model IJLEO 56455 1–7 4
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
Fig. 6. Segmentation and extraction results of the PCNN-based level set method in narrowband implementation. Column A: the original image. Column B: the Mammaryspecific process using PCNN algorithm. Column C: the extraction of breast mass only in the whole breast region result using PCNN algorithm. Column D: the coordinate value of the whole breast and mass by Matlab command. Column E: the final segmentation result using level set model based on the obtained initial contour. Column F: the mass extraction result.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
5
Fig. 7. Segmentation results of three Mammographic images from the MIAS database. Each row illustrates the experiment of one image. Images in the first column are the original Mammographic images. The second column is the final segmentation results processed by the PCNN-based Level Set model. Images in the third column are the contrast segmentation results processed by the CV model. And images in the fourth column are the contrast segmentation results obtained by the classical VFC model. The last column shows ground truth.
129
3. Proposed mass segmentation method
130
3.1. Coarse segmentation to obtain initial mask
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149
As described above, the novel proposed model mainly contains coarse segmentation and refined segmentation. The main achievement is to obtain the initial mask for latterly level set method in the coarse segmentation process. PCNN model is significantly used in the coarse step to obtain the initial contour. The fundamental theory of segmentation by using PCNN is that each pixel is correctly assigned to the region it belongs to when all the pixels of an input image are fired by PCNN [21]. The processing flow of coarse segmentation is outlined as shown in Fig. 2. The first aim of coarse segmentation process is to achieve mammary-specific. It is obvious that the mammary-specific process considerably reduce the complexity of image segmentation. Subsequently, we make lesion detection in breast region, i.e. breast mass detection, by using PCNN algorithm again. The last step of coarse segmentation is sketching out the outline, in order to illustrate the coordinate values of the lesion in mammography. Meanwhile, the coordinate values of lesion can be obtained, which is the initial mask of level set algorithm for the refined segmentation of mammography images.
3.2. Refined segmentation with level set method
150
As previous mentioned, the initial mask for latterly level set segmentation has been obtained by PCNN algorithm. We propose to use a binary step function in (8) as the initial LSF, as it can be generated extremely efficiently. The binary initial level set function can be obtained precisely as an example shown in the processing flow. Practically, we can define the initial LSF in the rectangular region ˝0 = (450:490, 270:330) of the example in Fig. 2, such that ˝0 is close to the region to be segmented. The level set function evolves from a binary step function to an approximate signed distance function on a signed distance band (SDB). Because its values vary from –d to d across the band at the rate of | ∇ |=1, when the function becomes a signed distance function in the SDB. In this context, the initial value of d is usually set as 2:
0 (x, y) =
151 152 153 154 155 156 157 158 159 160 161 162 163
⎧ −2, (x, y) ∈ ˝0 − ∂˝0 ⎪ ⎨ ⎪ ⎩
0,
(x, y) ∈ ∂˝0
2,
(x, y) ∈ ˝ − ˝0
(8)
Here, we also illustrate refined segmentation by an example as follow figure. With the accurate region ˝0 , the segmentation step is automatic and efficient (Fig. 3). Q2
Fig. 8. Maximum distance detected results. From left to right: (x:265, y:472) is the detected results after PCNN-based level set model, (x:276, y:467) and (x:345, y:277) is the detected results after CV model, VFC model, respectively.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
164
165 166 167
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
6
Fig. 9. Applications of our method to two ultrasound images of gallstone and two iris location of iris images. Column 1: original image; Column 2: final zero level contours of level set function, i.e. the segmentation result; Column 3: extraction result.
168
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
4. Experimental results All the images used in this paper are belonging to a publicly available digital database of mammograms. The dataset used for evaluation of the proposed approaches for image segmentation are selected from the Mini Mammographic Image Analysis Society (MIAS) database in United Kingdom. The experiments are run on MATLAB 7.8.0 with Intel Core 2 Quad i5-2400 3.10 GHz PC. The coarse segmentation results by applying PCNN for detecting initial rectangular region ˝0 are illustrated in Table 1. The binary initial LSF can be defined in the rectangular region ˝0 , such that ˝0 is close to the region to be segmented. The annotations in MIAS are used as ground truth data. As shown in Table 1, the initial rectangular region ˝0 we obtained per image are all inside of the ground truth. Approximately, the breast mass can be extracted after the Mammary-Specific process by using PCNN algorithm. No matter for which suspicious types, the regions obtained in high accuracy are more reliable and reasonable as shown in Fig. 4. In terms of accuracy, the image-coordinates region of abnormality are shown in Fig. 5 in x and y direction, where the x-axes represent 10 different images. We can observe that the initial mask obtained per image is all inside of the ground truth, so that we can accurately determine the initial LSF in that region by PCNN. The next process is refined segmentation by using level set method, whose initial contour already obtained by PCNN model as previous mentioned. The ultimately extraction results are also shown in Fig. 6. In addition, a comparison of segmentation performance is conducted among three different methods to evaluate the validity of the proposed model. Two other common segmentation methods are employed. One is CV model proposed by Chan and Vese [22] and the other one is classical VFC model proposed by [23]. Moreover, the proposed PCNN-based level set model is automatic segmentation without re-initialization, but both CV model and VFC model are sensitive to initial location and noise. By using the PCNN algorithm, the initial contour for lately level set evolution is efficiently obtained, which also reduces the computation and improves accuracy. The smoothness of evolution surface has been maintained and the phenomenon of edge leakage has been avoided by the refined segmentation process using level set
method. Absolutely, the level set method based on PCNN algorithm can segment breast mass not only avoiding boundary leakage, but also without segmenting the backgrounds which is similar to the ROI. All these process are automatic without manual segmentation. It can draw a conclusion that the level set method based on PCNN algorithm has a clearer, more complete segmentation result and a higher accuracy (Figs. 7 and 8). The PCNN-based level set method has also been applied to real images. For examples, Fig. 9 shows the results of the PCNN-based level set method for other medical images: two ultrasound images of gallstone and two iris images. In this case, the choice of the number of regions is based on prior medical knowledge. The most important aim of this experiment is to demonstrate the ability of the proposed method to adapt automatically to this class of images acquired with special techniques. Because of the presence of speckle noise, low contrast and luminous in-homogeneity in ultrasound images, the available segmentation algorithms are general techniques and fail to detect gallstones in ultrasound images. As shown in first columns of Fig. 9, our proposed PCNN based level set model can accurately extract the gallstones in ultrasound images. Meanwhile, the proposed method by this paper also efficiently finds the location of the iris boundaries. 5. Conclusions A novel mammographic image segmentation method that combines the PCNN segmentation algorithm with the level set method is presented for the first time. Note that the most convincing evidence comes from the outstanding performance of mammographic image segmentation. The proposed scheme accurately obtains the initial contour for level set evolution, which does not suffer from the drawbacks that level set method is sensitive to the initial contour. Moreover, the numerous experiments which demonstrate that PCNN segmentation algorithm can effectively achieve the mammary-specific scheme and mass outline detection. This coarse segmentation process considerably reduces the complexity for mammographic image segmentation. The examples show that mass initial mask can be accurate detected by the proposed PCNN algorithm. Additionally, before dealing with the
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227
228
229 230 231 232 233 234 235 236 237 238 239 240 241 242
G Model IJLEO 56455 1–7
ARTICLE IN PRESS W. Xie et al. / Optik xxx (2015) xxx–xxx
249
mammographic image, we obtain the negative image own to the mammographic image with predominantly dark region except the breast tumor. In terms of accuracy and robustness, experimental results have demonstrated superior performance of level set method based on the PCNN algorithm. Ultimately, this could lead to computer-assisted, quantitative assessment left ventricular function in clinical practice.
250
Acknowledgments
243 244 245 246 247 248
The authors would like to thank the reviewers and AE for their comments that have helped improve this paper. The authors 252 would like to thank the anonymous reviewers and AE for their 253 comments and constructive suggestions. This work was partially 254 Q3 supported by the National Nature Science Foundation of China (nos. 255 61222101, 61301287, 61301291 and 61350110239) and the 111 256 project (B08038). It was also partially supported by the Funda257 mental Research Funds for Central Universities (K5051201043 and 258 K5051301015). 259 251
260
261 262 263 264 265 266 267 268 269 270 271 272 273 274
References [1] R. Siegel, D. Naishadham, A. Jemal, CA: Cancer J. Clin. 63 (1) (2013) 11–30. [2] A. Jemal, et al., Global cancer statistics, CA: Cancer J. Clin. 61 (2) (2011) 69–90. [3] C.H. Lee, D.D. Dershaw, D. Kopans, P. Evans, B. Monsees, D. Monticciolo, R.J. Brenner, L. Bassett, W. Berg, S. Feig, Breast cancer screening with imaging: recommendations from the society of breast imaging and the ACR on the use of mammography, breast MRI, breast ultrasound, and other technologies for the detection of clinically occult breast cancer, J. Am. Coll. Radiol. 7 (1) (2010) 18–27. [4] J.S. Mandelblatt, K.A. Cronin, S. Bailey, D.A. Berry, H.J. de Koning, G. Draisma, H. Huang, S.J. Lee, M. Munsell, S.K. Plevritis, Effects of mammography screening under different screening schedules: model estimates of potential benefits and harms, Ann. Intern. Med. 151 (10) (2009) 738–747. [5] S. Timp, C. Varela, N. Karssemeijer, Temporal change analysis for characterization of mass lesions in mammography, IEEE Trans. Med. Imaging 26 (7) (2007) 945–953.
7
[6] Y. Wang, J. Li, X.B. Gao, Latent feature mining of spatial and marginal characteristics for mammographic mass classification, Neurocomputing 144 (2014) 107–118. [7] S. Osher, J. Sethian, Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi formulations, J. Comput. Phys. 79 (1) (1988) 12–49. [8] S. Osher, R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, SpringerVerlag, New York, 2002. [9] R. Malladi, J.A. Sethian, B.C. Vemuri, Shape modeling with front propagation: a level set approach, IEEE Trans. Pattern Anal. Mach. Intell. 17 (2) (1995) 158–175. [10] V. Caselles, R. Kimmel, G. Sapiro, Geodesic active contours, Int. J. Comput. Vis. 22 (1) (1997) 61–79. [11] C. Samson, L. Blanc-Feraud, G. Aubert, J. Zerubia, A variational model for image classification and restoration, IEEE Trans. Pattern Anal. Mach. Intell. 22 (5) (2000) 460–472. [12] C. Li, C. Kao, J.C. Gore, Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Trans. Image Process. 17 (10) (2008) 1940–1949. [13] C. Li, C. Xu, C. Gui, M.D. Fox, Distance regularized level set evolution and its application to image segmentation, IEEE Trans. Image Process. 19 (12) (2010) 3243–3254. [14] J.L. Johnson, M.L. Padgett, PCNN models and applications, IEEE Trans. Neural Netw. 10 (3) (1999) 480–498. [15] R. Eckhorn, H.J. Reitboeck, M. Arndt, P.W. Dicke, Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex, Neural Comput. 2 (3) (1990) 293–307. [16] R.C. Zhao, Y.D. Ma, A region segmentation method for region-oriented image compression, Neurocomputing 85 (2012) 45–52. [17] Y.D. Ma, C.L. Qi, Study of automated PCNN system based on genetic algorithm, J. Syst. Simul. 18 (3) (2006) 722–725. [18] Z. Wang, Y.D. Ma, F. Cheng, L. Yang, Review of pulse-coupled neural networks, Image Vis. Comput. 28 (1) (2010) 5–13. [19] Y.D. Ma, L. Liu, K. Zhan, Y.Q. Wu, Pulse-coupled neural networks and one-class support vector machines for geometry invariant texture retrieval, Image Vis. Comput. 28 (11) (2010) 1524–1529. [20] X.J. Li, Y.D. Ma, Z.B. Wang, W. Yu, Geometry-invariant texture retrieval using a dual-output pulse coupled neural network, Neural Comput. 24 (1) (2012) 194–216. [21] G. Kuntimad, H.S. Ranganath, Perfect image segmentation using pulse coupled neural networks, IEEE Trans. Neural Netw. 10 (3) (1999) 591–598. [22] T.F. Chan, L.A. Vese, Active contours without edges, IEEE Trans. Image Process. 10 (2) (2001) 266–277. [23] B. Li, S.T. Acton, Active contour external force using vector field convolution for image segmentation, IEEE Trans. Image Process. 16 (8) (2007) 2096–2106.
Please cite this article in press as: W. Xie, et al., PCNN-based level set method of automatic mammographic image segmentation, Optik - Int. J. Light Electron Opt. (2015), http://dx.doi.org/10.1016/j.ijleo.2015.09.250
275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317