- Email: [email protected]
Watershed based intelligent scissors
Computerized Medical Imaging and Graphics 43 (2015) 122–129
Contents lists available at ScienceDirect
Computerized Medical Imaging and Graphics journal homepage: www.elsevier.com/locate/compmedimag
Watershed based intelligent scissors W. Wieclawek ∗ , E. Pietka Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland
a r t i c l e
i n f o
Article history: Received 31 July 2014 Received in revised form 26 November 2014 Accepted 9 January 2015 Keywords: Image segmentation Intelligent scissors Live-Wire Watershed Edge Boundary Corridor
a b s t r a c t Watershed based modification of intelligent scissors has been developed. This approach requires a preprocessing phase with anisotropic diffusion to reduce subtle edges. Then, the watershed transform enhances the corridors. Finally, a roaming procedure, developed in this study, delineates the edge selected by a user. Due to a very restrictive set of pixels, subjected to the analysis, this approach significantly reduces the computational complexity. Moreover, the accuracy of the algorithm performance makes often one click point to be sufficient for one edge delineation. The method has been evaluated on structures as different in shape and appearance as the retina layers in OCT exams, chest and abdomen in CT and knee in MR studies. The accuracy is comparable with the traditional Life-Wire approach, whereas the analysis time decreases due to the reduction of the user interaction and number of pixels processed by the method. © 2015 Elsevier Ltd. All rights reserved.
1. Introduction Segmentation of various structures is very important in medical image processing. There are many ways to detect edges in the image. Some solutions are global and rely on detection of the largest possible number of edges. As examples, spatial edge filters (i.e. Roberts, Prewitt, Sobel), detectors (i.e. Canny, Susan) and transforms (i.e. Fourier, wavelet, watershed) can be given. Some of them are highly sensitive and detect many – also very thin – edges. Therefore, they are low resistant to local noise. Others are more noise resistant but often lose edge continuity and thus do not provide correct edge detection of a desired structure. Some of the methods for edge detection employ the graph theory. Graph-based segmentation methods are a relatively large group of commonly known approaches [2,21,3,5,1,6,8,25–27]. The LW algorithm is the prototype of the title approach that belongs to the interactive methods [2,21,3,5,1,6,8]. The detection of the edge is based on the graph search procedure. Other solutions [25] use graphs to compute the fuzzy connectedness. Huang et al. [26,27] operate in graph domain to correctly segment breast tumours in ultrasound images. In our work semi-manual tool with user interaction are employed. The Live-Wire (LW) approach [1–3], called also intelligent scissors belongs to this class of methods. The original version
∗ Corresponding author. Tel.: +48 322371128. E-mail addresses: [email protected] (W. Wieclawek), [email protected] (E. Pietka). http://dx.doi.org/10.1016/j.compmedimag.2015.01.003 0895-6111/© 2015 Elsevier Ltd. All rights reserved.
of the algorithm features certain drawbacks including numerical complexity and multiple user interaction. Attempt to reduce these drawbacks results in many variations of this method. A typical version has got a bi-partial character. The first step defines the image cost map, which reflects the edge properties of the image pixel and influences the precision. This cost map is used to describe each vertex of the graph, created during the second stage of the algorithm. The graph size strongly depends on the image size as each image pixel is represented by the graph vertex. Subsequently, the graph searching stage, based on the Dijkstra algorithm [4], is performed. It requires a user interaction based on an edge pixel indication. Because different image objects have irregular edges, the number of marked points is often quite large. Additionally, in medical images of 512 × 512, this graph searching part is very time consuming. The analysis includes the entire image. In order to reduce the time, homogeneous regions could be excluded from the graph searching procedure. Therefore, many modifications that reduce the number of graph vertices is presented in the literature [5–8]. Farin et al. [5] introduce a new semi-automatic segmentation tool which uses a modified concept of user interaction. The segmentation tool is based on the shortest circular path searched within a corridor that is manually drawn by the user along the object boundary. This solution radically increases the user interaction and additionally limits the segmentation process to the manually selected structure. Semi-automatic method of the corridor generation is presented in [8]. In this method fuzzy C-means (FCM) algorithm is applied. After clustering the image into classes, only vertices corresponding
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
123
Fig. 1. Workflow.
to pixels whose neighbourhoods are assigned to various classes are subjected to the graph analysis. This action significantly reduces the number of graph vertices (even to several percent of the global number of image pixels), and speeds up graph searching procedure. The class numbers required by the FCM procedure is a drawback of this approach. The last approach presented in [6,7] reduces the number of pixels by their earlier grouping. Mortensen and Barrett compute the optimal path by over-segmenting the image using tobogganing and then impose a weighted planar graph on top of the resulting region boundaries. The region-based graph is significantly smaller than the pixel-based graph used previously, thus provide a faster graph search and immediate user interaction. Additionally, this approach decreases the number of points marked by the user, particularly for synthetic images (sharp structures with homogeneous background). In this paper a new approach – Live-Wire with watershed transform (LW-WT) – is discussed. It is a modification of a standard Live-Wire approach [2] where cost map definition has been changed. Moreover, knowing the benefits of corridor solutions [5–7,21,8] these approaches have been developed. Since corridors are obtained either manual [5,21] or by control parameters [8], Watershed transform has been proposed. This transformation draws watersheds lines automatically and is independent on any parameters. The use of corridors reduces the graph size and numerical complexity as well as accelerates the segmentation process. With such defined corridors the number of Seed-Points has been limited. As a corridor based method it has abandoned a cost map analysis usually required in regular intelligent scissors method. This paper is organized as follows. Section 2 introduces the method discussing all processing phases that include the image preprocessing, followed by the corridors generation and analysis yielding delineation of edges. The results and evaluation of the method are presented in Section 3, followed by a conclusion in Section 4.
2. Method The proposed application has a bipartite structure (Fig. 1). The first stage – after Image Reading and expected preprocessing – is followed by a watershed based corridor grid extraction (Watershed Algorithm) that locates all edges and image gradient analysis that yields the gradient vector flow. Then, a Roaming Procedure is performed. It expects a user to indicate the desirable
Fig. 2. Preprocessing procedure.
edges. Further processing is fully automatic. On the user request or when no edges are detected, a correction function may be launched. 2.1. Preprocessing The watershed algorithm to be implemented in the following phase requires the local noise suppression. If not reduced it causes over-segmentation or detects edges within homogeneous areas. In order to preserve the edges an adaptive filtering procedure has been employed (Fig. 2). In our approach anisotropic diffusion is applied [9,10]. It suppresses the noise (impulse and Gaussian) preserving the edges. The filtering kernel parameters are calculated for each image pixel separately. Then, an edge enhancement procedure (Unsharp Masking) is implemented as an arithmetic sum of gradient magnitude and an image subjected previously to an adaptive filtering procedure. The last stage (Filtering) employs a spatial average filtering with Gaussian kernel interspersed, followed by median filtering (with relatively small 7 × 7 neighbourhood mask) and a mathematical morphology filtering. The latest implements the sequential opening and closing with the smallest, symmetric structural element (defining 8-neighbourhood). The preprocessed image is subjected to more advanced processing that employs the watershed approach and the gradient vector flow. 2.2. Watershed transform In order to outline the corridors a watershed transform [11] is implemented. This is a region-based segmentation technique, whose intuitive idea comes from the geography. Watershed algorithm considers a gray level image as a topographic relief, which is flooded by water. One of approaches relies on immersion of the landscape starting from each of a local minimum, called the catchment basin [12]. Then, the water will fill up the local minima. At points where the water coming from different basins would meet, dams will be built. When the water level has reached the highest peak in the landscape, the process stops. As a result, the landscape is partitioned into regions or basins separated by dams, called watershed lines or simply watersheds [13]. Watershed algorithm provides a complete division of the image.
124
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
Fig. 3. Watershed lines (white corridors) and Cross-Points (circles).
Fig. 4. The field of image gradient (→) with watershed lines (white pixels).
Based on the image gradient for each pixel the angle defined as: When watershed lines cover the image edges in practice a watershed transform is applied to the image gradient magnitude. Unfortunately, image gradient calculation implicate local noise. In this consideration myriad of small regions are generated, which makes this result useless. To remove the over-segmentation problem, various techniques including preprocessing [14], marker controlled watershed segmentation approach [15,16], region merging [15,17], etc. are applied. Since a prominent reduction of watershed regions is undesirable in our approach, a simple limitation method has been employed. One of the plain approach is the preprocessing stage (discussed in Section 2.1), which precedes the watershed algorithm. It preserves the essential edges and simultaneously reduces the number of local minima or watershed regions. The watershed algorithm yields a binary image of watershed lines, that separate neighbouring regions (basins). These lines are located on the image edges. The watershed lines look like an irregular mesh (Fig. 3) with many Cross-Points (pixels marked by circles in Fig. 3). The Cross-Points are image pixels with more than two white neighbours (in 4-neighbourhood). Many various CrossPoint configurations are possible. In extreme cases, the number of neighbour Cross-Points may equal 4. Since watershed lines overlie image structure edges, in further analysis only white pixels are considered. This process involves passing the corridors from a marked by the user Seed-Point in the direction indicated by the image gradient or additionally other edge features.
= atan2
∂ ∂ I, I ∂y ∂x
(2)
is computed. Further processing is performed on pixels, which correspond to watershed corridors only. 2.3.1. Gradient vector flow The gradient vector has a large magnitude only in the immediate vicinity of the boundary. Furthermore, in homogeneous regions where the intensity is nearly constant, the magnitudes of gradient vectors are close to zero. Several researchers point out this limitations and propose alternative solutions to increase the capture range of the external force [18]. The most popular solution is the gradient vector flow [19]. The GVF approach extends the gradient map farther away from the edges into homogeneous regions using a gradient directional that preserve the diffusion process [20]. Using the calculus of variations, it can be shown that the GVF can be found by solving the following Euler equations: · ∇ 2 u − (u − Ix ) · (Ix2 + Iy2 ) = 0,
(3)
· ∇ 2 v − (v − Iy ) · (Ix2 + Iy2 ) = 0,
where ∇ 2 is the Laplacian operator and is a positive constant parameter governing the amount of the smoothness term. Eqs. (3) can be solved by treating u and v as functions of time: 2
2
ut (x, y, t) = · ∇ 2 u(x, y, t) − · · · − (u(x, y, t) − Ix (x, y)) · (Ix (x, y) + Iy (x, y) ), 2
2
vt (x, y, t) = · ∇ 2 v(x, y, t) − · · · − (v(x, y, t) − Iy (x, y)) · (Ix (x, y) + Iy (x, y) ),
(4)
2.4. Roaming procedure 2.3. Image gradient To indicate the direction of the edges corresponding to the watershed lines the field of image gradient is adopted. An image gradient illustrates a directional change of the intensity in either the x- or the y-direction:
∇I =
∂ ∂ I, I ∂x ∂y
(1)
The field of the image gradient can be presented as a set of arrows, which represent the direction of a local image gradient in each image pixel (Fig. 4). The figure shows small, strongly zoomed image segment (with resolution 9 × 11 pixels) with one edge. Therefore, there is one dominant gradient direction.
The watershed lines properly located and the field of gradient vector with angle are subjected to further processing based on the Roaming procedure presented in Fig. 5. First a user specifies the edge of the segmented structure by marking a single point located on the edge (Insert Seed-Point). Due to an imprecise Seed-Point location, three cases are possible (Fig. 6). First, indicated by the user point may hit the corridor making additional operation unnecessary (Fig. 6(a)). Second, a Seed-Point located along a corridor has to be shifted (along the corridor) in the local gradient direction towards the nearest CrossPoint (Fig. 6(b)). In the third case, the Seed-Point may be marked on the background. This requires two actions. The first one shifts the Seed-Point to the nearest (in terms of Euclidean distance) watershed line (corridor) and the second moves it along the corridor towards the nearest Cross-Point in the local gradient direction
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
125
Table 1 Comparison of features analysed in Cross-Points. Features
Fig. 5. Workflow of the Roaming procedure.
(Fig. 6(c)). These functions are referred to Attract to Nearest Corridor and Go to Cross-Points, respectively (Fig. 5). Continuation along a corridor is the most crucial and complex function (Corridor Continuation in Fig. 5) in this approach. A correct choice of the watershed line influences the quality and accuracy of the overall segmentation. The principle assumption requires that a corridor be chosen only at a Cross-Point. Between them the motion follows the watershed line direction with no possibility to change it. In terms of computational complexity this is a very advantageous and distinguishing feature of the proposed algorithm with respect to the standard of intelligent scissors. The analysed image pixels are limited not only to corridors pixels but exclusively to the Cross-Points. Therefore, only ends of corridors are subjected to a detailed analysis.
Geometrical
Gray scale
Corridor orientation Corridor length
Gradient direction: • Average • Standard deviation
The decision of further walking paths is based on a comparison of geometrical corridor features with image information features. The first group quantifies the geometric location of Cross-Points, whereas the second one relies on gray scale values of neighbour pixels surrounding the currently analysed Cross-Point. Both types of features are listed in Table 1. The algorithm is performed in 5 steps (Algorithm 2.1). In the first step the detection of all corridor segments which start with the current Cross-Point is performed (white corridors in Fig. 7 attached to Cross-Points A and B). Then (Step 2), geometrical features of selected segments are detected (the length and direction marked by gray dotted arrows in Fig. 7). When the geometrical feature indicates that two Cross-Points are located next to each other (point B in Fig. 7) the neighbour Cross-Point is skipped and the search is continued. Next, local gradients of pixels along corridor segments (short, thin, black arrows in Fig. 7) are selected from the global gradient field shown in Fig. 4. These gradient vector directions are averaged and compared with all geometrical vector directions that starts at each Cross-Point. The closest match indicates the corridor segment aligned to the delineated structure. Algorithm 2.1.
Corridor continuation
Input: Watershed lines, Image gradient Output: Best-fit corridor Step 1: Detection of all watershed lines (corridor segments) starting at the Cross-Point. Step 2: Geometrical features calculation for all corridors detected in Step 1. Step 3: Local features specification from gray scale image. Step 4: Comparison of features obtained in Step 2 and Step 3. Step 5: Choice of the best-fit corridor for continuation.
The analysis stops when: (1) image border is achieved, (2) the indicated for analysis Cross-Point has already been considered, (3) in any Cross-Point discrepancy between geometrical versus gradient vector will be too significant.
Fig. 6. Processing of Seed-Point located at various positions: (a) Seed-Point located at the Cross-Point – no action, (b) Seed-Point within a corridor – attraction to a Cross-Point, (c) Seed-Point within the background – attraction to a corridor and Cross-Point.
Fig. 7. The corridor selection.
126
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
Table 2 Algorithms comparison: Live-Wire, Live-Wire with FCM, corridor scissors, watershed based intelligent scissors. Feature
LW
LW-FCM
CS
LW-WT
Number of points Additional manual user operations Additional control parameter Number of significant pixels
Large N/A N/A All in the image
Small N/A Class number Small
Medium Corridor selection N/A Small
Small N/A N/A Small
Watershed approach usually considers an image as a relief. The analysis is searching paths of the largest slopes. These cases are known in the literature as tobbogan [11,6,21] or hill climbing [22]. In this study the corridors are referred as a ridges and the analysis is performed along them without climbing up and down the hills (solid lines in Fig. 8). The Cross-Points correspond to the points where two or more edges come together. Methods that search ridges [23] yield all edges without following a particular border. Our approach selects a single ridge marked by the user. 3. Experiments and results Proposed methodology has been evaluated based on different types of images. Additionally, obtained results have been compared with previous versions of the algorithms, known in the literature [2,8,5]. A description of relevant features has been given in Table 2. Traditional Live-Wire algorithm [2] takes into consideration all image pixels, thus it is much more time consuming than other methods. Additionally, since following all edges (even very subtle once), the main direction may be lost. This requires many more characteristic points marked by the user. Fuzzy clustering in intelligent scissors LW-FCM [8] significantly reduces the user interaction and considers only those image pixels which are situated along edges. To identify important pixels a fuzzy C-means algorithm is used. Therefore, specification of the class number is indispensable. For all tests in the current work a number of classes is related to the currently segmented structure. The lowest level of automation features a corridor scissors implementation [5]. This method provides additional manual tool for rough selection of the desirable edge. It returns coarse corridor which traces the segmented object. Then, automatic procedure finds the correct edge localization inside the previously selected corridor. Due to manual corridor selection this method has not been evaluated later in this study. Presented method relies on corridors, furthermore they are extracted automatically based on the watershed transform. This gains two advantages over the corridors scissors implementation. First, the user interaction is limited. On the other hand watershed lines cover the entire image. The evaluation of algorithms has been carried out based on medical images of various types. We started with optical coherence tomography (OCT) images from eyeball examination (Figs. 9 and 10). The segmentation procedure has been performed on the retina layers, in particular inner limiting membrane (ILM), outer
Fig. 8. Relief.
plexiform layer (OPL) and retinal pigment epithelium (RPE). Anatomical (Fig. 9) and pathological (Fig. 10) cases have been subjected to the analysis. Sometimes OCT images have low quality and include shadows of the blood vessels. The poor image quality does not effect the algorithm performance. The algorithm works also correctly when applied to pathological cases (Fig. 10) as vitreomacular traction, drusen, macular holes (cysts), etc. High efficiency of the solution at a relatively high level of automation has led us to verify the algorithm using other images. Among them are magnetic resonance (MR) knee joint examination (Fig. 11), computed tomography (CT) chest studies (Fig. 12) and abdomen (Fig. 16). The accuracy of segmentation and number of user points proof the robustness of this method. In the evaluation process the number of user interactions, number of pixels analysed and accuracy of the delineation have been tested. 3.1. Automation level The main goal of the first test is the automation level verification. It is expressed as a number of points marked by the user in order to obtain a correct delineation of a structure.
Fig. 9. Segmentation of retina layers: (a) ILM, (b) OPL, and (c)–(d) RPE.
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
127
Table 3 Number of characteristic points. Image
Structure
Retina OCT Retina OCT Retina OCT Knee joint MR Knee joint MR Chest CT
ILM OPL RPE Thigh bone Tibia Single lung
Number of points LW
LW-FCM
LW-WT
5–11 8–17 4–13 7–16 5–8 6–8
2–3 2–4 2–4 3–5 2–5 2–3
1 1–2 1 1–2 1–2 1
Fig. 10. Segmentation of retina layers in different images. Fig. 13. Analysed pixels in: (a) LW-FCM and (b) LW-WT algorithms.
Fig. 11. Segmentation of knee structures.
Fig. 12. Segmentation of lungs.
The significant decrease in number of user points can be seen when LW-WT is applied (Table 3). A correct segmentation has been obtained for both straight and curved edges (also for circular edges, see lungs, thigh bone, tibia). In many cases it is sufficient to indicate a single point per one edge.
pixels to be processed (n). Traditional LW approach takes into consideration all image pixels (100%) therefore the on-the-fly mechanism is frequently used. The edge search stops when the end point of the edge is reached. The obtained reduction level may be difficult to estimate. It has been shown [8] that in the worst case it does not exceed of 32% image pixels. Moreover, LW-FCM algorithm is as accurate as the traditional LW approach with and without applying the on-the-fly mechanism. It has also been proven that LW-FCM takes into consideration only pixels located along the corridor (Fig. 13(a)). Therefore, further reduction of analysed points is obtained and for an optimal number of classes in FCM it equals at most 5% of the global image pixel number [8]. Since in the LW-WT approach only some corridor pixels are processed, further reduction is obtained (Fig. 13(b)). In the case shown in Fig. 13 the numbers of pixels processed in LW-FCM and LW-WT equal 0.3403% and 0.0618%, respectively. The analysis performed on the set of clinical images is shown in Table 4. The number of pixels to be proceed by the LW-WT approach is usually about 10 times smaller than by implementing the LWFCM algorithm and many less when the LW method (even using on-the-fly mechanism) is applied. 3.3. Segmentation accuracy evaluation The third test compares the segmentation accuracy of LWFCM and LW-WT versus LW approach. The LW approach has been Table 4 Number of pixels subjected to the analysis. Image
Structure
3.2. Number of processed pixels evaluation Numerical complexity of the proposed methodology is the same as in the Dijkstra algorithm, i.e. O(n2 ), where n is number of graph vertices (in image processing domain number of analysed image pixels). Therefore, beside reducing the number of Seed-Points (see Section 3.1) the proposed methodology decreases the number of
Retina OCT Retina OCT Retina OCT Knee joint MR Knee joint MR Chest CT
ILM OPL RPE Thigh bone Tibia Single lung
Number of pixels (%) LW
LW-FCM
LW-WT
38.8306 49.1565 43.8737 11.21 31.31 20.11
0.1838 0.1511 0.1825 0.4913 0.3311 0.4274
0.0169 0.0129 0.0151 0.0748 0.0656 0.0796
128
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129 Table 5 Sensitivity (S) and specificity (P) of the abdominal structures segmentation.
Fig. 14. Accuracy of OCT segmentation.
Structure
No. of cases
S (%)
P (%)
Liver Left kidney Right kidney
96 198 171
94.73 96.89 95.90
99.95 99.96 99.96
Due to the nature of corridor solutions, the LW-FCM and LWWT methods have yield similar distance values comparing with LW approach. On the other hand the corridor approaches return smoother edges. Noisy edges that result from applying the LW approach are caused by the dense manual selection of user points. Due to low contrast of the OPL layer the FCM procedure has failed. Thus, the LW-FCM returns a non solid corridor. Therefore, in Fig. 14 the OPL segmentation result is missing. Previously presented tests show user-oriented results. The evaluation of the accuracy is based on the analysis of 200 clinical images. The liver, left and right kidneys have been delineated manually on 96, 198, 171 images, respectively. In order to compare the performance of the LW-WT with the manual expert delineation, the sensitivity (S) and specificity (P) have been found. Numerical results are presented in Table 5 and examples of the LW-WT performance are shown in Fig. 16. The results show very high sensitivity and specificity independently on the selected anatomical structure. Note, that all the delineations are obtained by just a single Seed-Point per structure marked by the user. 4. Conclusions
Fig. 15. Accuracy of MR and CT segmentation.
considered as a reference method because a user can correct the contour as many times, as required to obtain successful delineation. In order to compare all algorithms the Hausdorff distance [24] between contour points has been determined. Means and standard deviations for over a dozen images of each type have been shown in diagrams. Fig. 14 presents the delineation of three different layers in OCT images (ILM, OPL and RPE). Fig. 15 illustrates the knee structure segmentation results in MR studies (thigh bone and tibia) and CT lungs exams. Generally, segmentation results from corridor modifications approaches (LW-FCM and LW-WT) are similar to each other. In LW-FCM usually the mean distance value is larger and the standard deviation is lower, yet the differences are not significant. Additionally, it can be seen that the OCT segmentation returns higher values of distances than MR/CT studies. It is due to the higher resolution of OCT images with respect to MR/CT.
The method proposed in this paper demonstrates that intelligent scissors approach based on watershed transform can yield robust and accurate segmentation of various anatomical structures acquired with different modalities. The method can be successfully employed for semi-automatic delineation. It can handle structures as different in shape and appearance as the retina layers in OCT exams, chest, liver and kidney in CT and knee in MR studies. The overall segmentation accuracy with respect to the traditional Life-Wire approach shows comparable results in the range of single pixels. What is more important, the number of pixels to be processed decreases about 3000 times for OCT and about 300 times for CT and MR. This results in a significant reduction of the numerical complexity. Another important drawback of all semi-automatic method, namely the user interaction, has also been minimized and reduced to 1 or 2 click points. A thorough analysis led use to the conclusion that the watershed based intelligent scissors is a promising interactive tool to semi-automatic edge delineation. The method is related neither to a modality nor anatomical/pathological structures. It can be applied as a segmentation procedure in image manipulation systems that do not offer an automated segmentation options or when the automated analysis has failed. References
Fig. 16. Segmentation of (c) liver, (b) kidneys and (c) liver and both kidneys.
[1] Mortensen EN, Morse BS, Barrett WA, Udupa JK. Adaptive boundary detection using ‘Live-Wire’ two-dimensional dynamic programming. In: IEEE computers in cardiology. 1992. p. 635–8. [2] Mortensen EN, Barrett WA. Interactive segmentation with intelligent scissors. Graph Models Image Process 1998;60(5):349–84. [3] Barrett WA, Mortensen EN. Interactive Live-Wire boundary extraction. Med Image Anal 1997;1(4):331–41. [4] Dijkstra EW. A note on two problems in connexion with graphs. Numer Math 1959;1:269–70. [5] Farin D, Pfeffer M, de With PHN, Effelsberg W. Corridor scissors: a semiautomatic segmentation tool employing minimum-cost circular paths. In: International conference on image processing (ICIP), vol. 2. 2004. p. 1177–80.
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129 [6] Mortensen EN, Barrett WA. Toboggan-based intelligent scissors with a four parameter edge model. In: IEEE computer vision and pattern recognition, vol. 2. 1999. p. 452–8. [7] Barrett WA, Reese LJ, Mortensen EN. Intelligent segmentation tools. In: IEEE international symposium on biomedical imaging. 2002. p. 217–20. [8] Wieclawek W, Pietka E. Fuzzy clustering in intelligent scissors. Comput Med Imaging Graph 2012;36(5):396–409. [9] Seramani Z, Jiayin S, Luk CK, Malmurugan N, Nagappan A. Denoising of MR images using non linear anisotropic diffusion filtering as a preprocessing step. Int J Biosci Technol 2008;1:17–21. [10] Black MJ, Sapiro G, Marimont DH, Heeger D. Robust anisotropic diffusion. IEEE Trans Image Process 1998;7(3):421–32. [11] Vincent L, Soille P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 1991;13(6):583–98. [12] Roerdink JBTM, Meijster A. The watershed transform: definitions, algorithms and parallelization strategies. Fundam Inf 2001;41:187–228. [13] Acharjya PP, Ghoshal D. An approach to reduce oversegmentation in watershed ridge line observation. Int J Adv Res Technol 2013;2(6):290–4. [14] Karantzalos K, Argialas D. Improving edge detection and watershed segmentation with anisotropic diffusion and morphological levellings. Int J Remote Sens 2006;27(24):5427–34. [15] Mittal U, Anand S. Modified watershed segmentation with denoising of medical images. Int J Innov Res Sci Eng Technol 2013;2(4):982–7. [16] Parvati K, Prakasa Rao BS, Das MM. Image segmentation using gray-scale morphology and marker-controlled watershed transformation. Discrete Dyn Nat Soc 2008 [Article ID 384346].
129
[17] Haris K, Efstratiadis SN, Maglaveras N, Katsaggelos AK. Hybrid image segmentation using watersheds and fast region merging. IEEE Trans Image Process 1998;7(12):1684–99. [18] Boukerroui D. Efficient numerical schemes for gradient vector flow. Pattern Recognit 2012;45(1):626–36. [19] Xu C, Prince JL. Gradient vector flow: a new external force for snakes. In: CVPR IEEE computer society. 1997. p. 66–71. [20] Bauer G, Homm F, Walchshäusl L, Burschka D. Multi spectral pedestrian detection and localization. In: Advanced microsystems for automotive applications. Springer-Verlag; 2008. [21] Fairfield J. Toboggan contrast enhancement for contrast segmentation. In: Proceedings of 10th international conference on pattern recognition. 1990. p. 712–6. [22] Meyer F. Topographic distance and watershed lines. Signal Process 1994;38(1):113–25. [23] Lindeberg T. Edge detection and ridge detection with automatic scale selection. Int J Comput Vis 1996;30:465–70. [24] Dubuisson MP, Jain AK. A modified Hausdorff distance for object matching. In: Conference A: computer vision & image processing. Proceedings of the 12th IAPR international conference on pattern recognition. 1994. p. 566–8. [25] Badura P, Pietka E. Soft computing approach to 3D lung nodule segmentation in CT. Comput Biol Med 2014;53:230–43. [26] Qinghua H, Xiao B, Yingguang L, Lianwen J, Xuelong L. Optimized graph-based segmentation for ultrasound images. Neurocomputing 2014:216–24. [27] Qinghua H, Suying L, Longzhong L, Minhua L, Lianwen J, Anhua L. A robust graph-based segmentation method for breast tumors in ultrasound images. Ultrasonics 2012;52(2):266–75.
Contents lists available at ScienceDirect
Computerized Medical Imaging and Graphics journal homepage: www.elsevier.com/locate/compmedimag
Watershed based intelligent scissors W. Wieclawek ∗ , E. Pietka Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland
a r t i c l e
i n f o
Article history: Received 31 July 2014 Received in revised form 26 November 2014 Accepted 9 January 2015 Keywords: Image segmentation Intelligent scissors Live-Wire Watershed Edge Boundary Corridor
a b s t r a c t Watershed based modification of intelligent scissors has been developed. This approach requires a preprocessing phase with anisotropic diffusion to reduce subtle edges. Then, the watershed transform enhances the corridors. Finally, a roaming procedure, developed in this study, delineates the edge selected by a user. Due to a very restrictive set of pixels, subjected to the analysis, this approach significantly reduces the computational complexity. Moreover, the accuracy of the algorithm performance makes often one click point to be sufficient for one edge delineation. The method has been evaluated on structures as different in shape and appearance as the retina layers in OCT exams, chest and abdomen in CT and knee in MR studies. The accuracy is comparable with the traditional Life-Wire approach, whereas the analysis time decreases due to the reduction of the user interaction and number of pixels processed by the method. © 2015 Elsevier Ltd. All rights reserved.
1. Introduction Segmentation of various structures is very important in medical image processing. There are many ways to detect edges in the image. Some solutions are global and rely on detection of the largest possible number of edges. As examples, spatial edge filters (i.e. Roberts, Prewitt, Sobel), detectors (i.e. Canny, Susan) and transforms (i.e. Fourier, wavelet, watershed) can be given. Some of them are highly sensitive and detect many – also very thin – edges. Therefore, they are low resistant to local noise. Others are more noise resistant but often lose edge continuity and thus do not provide correct edge detection of a desired structure. Some of the methods for edge detection employ the graph theory. Graph-based segmentation methods are a relatively large group of commonly known approaches [2,21,3,5,1,6,8,25–27]. The LW algorithm is the prototype of the title approach that belongs to the interactive methods [2,21,3,5,1,6,8]. The detection of the edge is based on the graph search procedure. Other solutions [25] use graphs to compute the fuzzy connectedness. Huang et al. [26,27] operate in graph domain to correctly segment breast tumours in ultrasound images. In our work semi-manual tool with user interaction are employed. The Live-Wire (LW) approach [1–3], called also intelligent scissors belongs to this class of methods. The original version
∗ Corresponding author. Tel.: +48 322371128. E-mail addresses: [email protected] (W. Wieclawek), [email protected] (E. Pietka). http://dx.doi.org/10.1016/j.compmedimag.2015.01.003 0895-6111/© 2015 Elsevier Ltd. All rights reserved.
of the algorithm features certain drawbacks including numerical complexity and multiple user interaction. Attempt to reduce these drawbacks results in many variations of this method. A typical version has got a bi-partial character. The first step defines the image cost map, which reflects the edge properties of the image pixel and influences the precision. This cost map is used to describe each vertex of the graph, created during the second stage of the algorithm. The graph size strongly depends on the image size as each image pixel is represented by the graph vertex. Subsequently, the graph searching stage, based on the Dijkstra algorithm [4], is performed. It requires a user interaction based on an edge pixel indication. Because different image objects have irregular edges, the number of marked points is often quite large. Additionally, in medical images of 512 × 512, this graph searching part is very time consuming. The analysis includes the entire image. In order to reduce the time, homogeneous regions could be excluded from the graph searching procedure. Therefore, many modifications that reduce the number of graph vertices is presented in the literature [5–8]. Farin et al. [5] introduce a new semi-automatic segmentation tool which uses a modified concept of user interaction. The segmentation tool is based on the shortest circular path searched within a corridor that is manually drawn by the user along the object boundary. This solution radically increases the user interaction and additionally limits the segmentation process to the manually selected structure. Semi-automatic method of the corridor generation is presented in [8]. In this method fuzzy C-means (FCM) algorithm is applied. After clustering the image into classes, only vertices corresponding
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
123
Fig. 1. Workflow.
to pixels whose neighbourhoods are assigned to various classes are subjected to the graph analysis. This action significantly reduces the number of graph vertices (even to several percent of the global number of image pixels), and speeds up graph searching procedure. The class numbers required by the FCM procedure is a drawback of this approach. The last approach presented in [6,7] reduces the number of pixels by their earlier grouping. Mortensen and Barrett compute the optimal path by over-segmenting the image using tobogganing and then impose a weighted planar graph on top of the resulting region boundaries. The region-based graph is significantly smaller than the pixel-based graph used previously, thus provide a faster graph search and immediate user interaction. Additionally, this approach decreases the number of points marked by the user, particularly for synthetic images (sharp structures with homogeneous background). In this paper a new approach – Live-Wire with watershed transform (LW-WT) – is discussed. It is a modification of a standard Live-Wire approach [2] where cost map definition has been changed. Moreover, knowing the benefits of corridor solutions [5–7,21,8] these approaches have been developed. Since corridors are obtained either manual [5,21] or by control parameters [8], Watershed transform has been proposed. This transformation draws watersheds lines automatically and is independent on any parameters. The use of corridors reduces the graph size and numerical complexity as well as accelerates the segmentation process. With such defined corridors the number of Seed-Points has been limited. As a corridor based method it has abandoned a cost map analysis usually required in regular intelligent scissors method. This paper is organized as follows. Section 2 introduces the method discussing all processing phases that include the image preprocessing, followed by the corridors generation and analysis yielding delineation of edges. The results and evaluation of the method are presented in Section 3, followed by a conclusion in Section 4.
2. Method The proposed application has a bipartite structure (Fig. 1). The first stage – after Image Reading and expected preprocessing – is followed by a watershed based corridor grid extraction (Watershed Algorithm) that locates all edges and image gradient analysis that yields the gradient vector flow. Then, a Roaming Procedure is performed. It expects a user to indicate the desirable
Fig. 2. Preprocessing procedure.
edges. Further processing is fully automatic. On the user request or when no edges are detected, a correction function may be launched. 2.1. Preprocessing The watershed algorithm to be implemented in the following phase requires the local noise suppression. If not reduced it causes over-segmentation or detects edges within homogeneous areas. In order to preserve the edges an adaptive filtering procedure has been employed (Fig. 2). In our approach anisotropic diffusion is applied [9,10]. It suppresses the noise (impulse and Gaussian) preserving the edges. The filtering kernel parameters are calculated for each image pixel separately. Then, an edge enhancement procedure (Unsharp Masking) is implemented as an arithmetic sum of gradient magnitude and an image subjected previously to an adaptive filtering procedure. The last stage (Filtering) employs a spatial average filtering with Gaussian kernel interspersed, followed by median filtering (with relatively small 7 × 7 neighbourhood mask) and a mathematical morphology filtering. The latest implements the sequential opening and closing with the smallest, symmetric structural element (defining 8-neighbourhood). The preprocessed image is subjected to more advanced processing that employs the watershed approach and the gradient vector flow. 2.2. Watershed transform In order to outline the corridors a watershed transform [11] is implemented. This is a region-based segmentation technique, whose intuitive idea comes from the geography. Watershed algorithm considers a gray level image as a topographic relief, which is flooded by water. One of approaches relies on immersion of the landscape starting from each of a local minimum, called the catchment basin [12]. Then, the water will fill up the local minima. At points where the water coming from different basins would meet, dams will be built. When the water level has reached the highest peak in the landscape, the process stops. As a result, the landscape is partitioned into regions or basins separated by dams, called watershed lines or simply watersheds [13]. Watershed algorithm provides a complete division of the image.
124
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
Fig. 3. Watershed lines (white corridors) and Cross-Points (circles).
Fig. 4. The field of image gradient (→) with watershed lines (white pixels).
Based on the image gradient for each pixel the angle defined as: When watershed lines cover the image edges in practice a watershed transform is applied to the image gradient magnitude. Unfortunately, image gradient calculation implicate local noise. In this consideration myriad of small regions are generated, which makes this result useless. To remove the over-segmentation problem, various techniques including preprocessing [14], marker controlled watershed segmentation approach [15,16], region merging [15,17], etc. are applied. Since a prominent reduction of watershed regions is undesirable in our approach, a simple limitation method has been employed. One of the plain approach is the preprocessing stage (discussed in Section 2.1), which precedes the watershed algorithm. It preserves the essential edges and simultaneously reduces the number of local minima or watershed regions. The watershed algorithm yields a binary image of watershed lines, that separate neighbouring regions (basins). These lines are located on the image edges. The watershed lines look like an irregular mesh (Fig. 3) with many Cross-Points (pixels marked by circles in Fig. 3). The Cross-Points are image pixels with more than two white neighbours (in 4-neighbourhood). Many various CrossPoint configurations are possible. In extreme cases, the number of neighbour Cross-Points may equal 4. Since watershed lines overlie image structure edges, in further analysis only white pixels are considered. This process involves passing the corridors from a marked by the user Seed-Point in the direction indicated by the image gradient or additionally other edge features.
= atan2
∂ ∂ I, I ∂y ∂x
(2)
is computed. Further processing is performed on pixels, which correspond to watershed corridors only. 2.3.1. Gradient vector flow The gradient vector has a large magnitude only in the immediate vicinity of the boundary. Furthermore, in homogeneous regions where the intensity is nearly constant, the magnitudes of gradient vectors are close to zero. Several researchers point out this limitations and propose alternative solutions to increase the capture range of the external force [18]. The most popular solution is the gradient vector flow [19]. The GVF approach extends the gradient map farther away from the edges into homogeneous regions using a gradient directional that preserve the diffusion process [20]. Using the calculus of variations, it can be shown that the GVF can be found by solving the following Euler equations: · ∇ 2 u − (u − Ix ) · (Ix2 + Iy2 ) = 0,
(3)
· ∇ 2 v − (v − Iy ) · (Ix2 + Iy2 ) = 0,
where ∇ 2 is the Laplacian operator and is a positive constant parameter governing the amount of the smoothness term. Eqs. (3) can be solved by treating u and v as functions of time: 2
2
ut (x, y, t) = · ∇ 2 u(x, y, t) − · · · − (u(x, y, t) − Ix (x, y)) · (Ix (x, y) + Iy (x, y) ), 2
2
vt (x, y, t) = · ∇ 2 v(x, y, t) − · · · − (v(x, y, t) − Iy (x, y)) · (Ix (x, y) + Iy (x, y) ),
(4)
2.4. Roaming procedure 2.3. Image gradient To indicate the direction of the edges corresponding to the watershed lines the field of image gradient is adopted. An image gradient illustrates a directional change of the intensity in either the x- or the y-direction:
∇I =
∂ ∂ I, I ∂x ∂y
(1)
The field of the image gradient can be presented as a set of arrows, which represent the direction of a local image gradient in each image pixel (Fig. 4). The figure shows small, strongly zoomed image segment (with resolution 9 × 11 pixels) with one edge. Therefore, there is one dominant gradient direction.
The watershed lines properly located and the field of gradient vector with angle are subjected to further processing based on the Roaming procedure presented in Fig. 5. First a user specifies the edge of the segmented structure by marking a single point located on the edge (Insert Seed-Point). Due to an imprecise Seed-Point location, three cases are possible (Fig. 6). First, indicated by the user point may hit the corridor making additional operation unnecessary (Fig. 6(a)). Second, a Seed-Point located along a corridor has to be shifted (along the corridor) in the local gradient direction towards the nearest CrossPoint (Fig. 6(b)). In the third case, the Seed-Point may be marked on the background. This requires two actions. The first one shifts the Seed-Point to the nearest (in terms of Euclidean distance) watershed line (corridor) and the second moves it along the corridor towards the nearest Cross-Point in the local gradient direction
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
125
Table 1 Comparison of features analysed in Cross-Points. Features
Fig. 5. Workflow of the Roaming procedure.
(Fig. 6(c)). These functions are referred to Attract to Nearest Corridor and Go to Cross-Points, respectively (Fig. 5). Continuation along a corridor is the most crucial and complex function (Corridor Continuation in Fig. 5) in this approach. A correct choice of the watershed line influences the quality and accuracy of the overall segmentation. The principle assumption requires that a corridor be chosen only at a Cross-Point. Between them the motion follows the watershed line direction with no possibility to change it. In terms of computational complexity this is a very advantageous and distinguishing feature of the proposed algorithm with respect to the standard of intelligent scissors. The analysed image pixels are limited not only to corridors pixels but exclusively to the Cross-Points. Therefore, only ends of corridors are subjected to a detailed analysis.
Geometrical
Gray scale
Corridor orientation Corridor length
Gradient direction: • Average • Standard deviation
The decision of further walking paths is based on a comparison of geometrical corridor features with image information features. The first group quantifies the geometric location of Cross-Points, whereas the second one relies on gray scale values of neighbour pixels surrounding the currently analysed Cross-Point. Both types of features are listed in Table 1. The algorithm is performed in 5 steps (Algorithm 2.1). In the first step the detection of all corridor segments which start with the current Cross-Point is performed (white corridors in Fig. 7 attached to Cross-Points A and B). Then (Step 2), geometrical features of selected segments are detected (the length and direction marked by gray dotted arrows in Fig. 7). When the geometrical feature indicates that two Cross-Points are located next to each other (point B in Fig. 7) the neighbour Cross-Point is skipped and the search is continued. Next, local gradients of pixels along corridor segments (short, thin, black arrows in Fig. 7) are selected from the global gradient field shown in Fig. 4. These gradient vector directions are averaged and compared with all geometrical vector directions that starts at each Cross-Point. The closest match indicates the corridor segment aligned to the delineated structure. Algorithm 2.1.
Corridor continuation
Input: Watershed lines, Image gradient Output: Best-fit corridor Step 1: Detection of all watershed lines (corridor segments) starting at the Cross-Point. Step 2: Geometrical features calculation for all corridors detected in Step 1. Step 3: Local features specification from gray scale image. Step 4: Comparison of features obtained in Step 2 and Step 3. Step 5: Choice of the best-fit corridor for continuation.
The analysis stops when: (1) image border is achieved, (2) the indicated for analysis Cross-Point has already been considered, (3) in any Cross-Point discrepancy between geometrical versus gradient vector will be too significant.
Fig. 6. Processing of Seed-Point located at various positions: (a) Seed-Point located at the Cross-Point – no action, (b) Seed-Point within a corridor – attraction to a Cross-Point, (c) Seed-Point within the background – attraction to a corridor and Cross-Point.
Fig. 7. The corridor selection.
126
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
Table 2 Algorithms comparison: Live-Wire, Live-Wire with FCM, corridor scissors, watershed based intelligent scissors. Feature
LW
LW-FCM
CS
LW-WT
Number of points Additional manual user operations Additional control parameter Number of significant pixels
Large N/A N/A All in the image
Small N/A Class number Small
Medium Corridor selection N/A Small
Small N/A N/A Small
Watershed approach usually considers an image as a relief. The analysis is searching paths of the largest slopes. These cases are known in the literature as tobbogan [11,6,21] or hill climbing [22]. In this study the corridors are referred as a ridges and the analysis is performed along them without climbing up and down the hills (solid lines in Fig. 8). The Cross-Points correspond to the points where two or more edges come together. Methods that search ridges [23] yield all edges without following a particular border. Our approach selects a single ridge marked by the user. 3. Experiments and results Proposed methodology has been evaluated based on different types of images. Additionally, obtained results have been compared with previous versions of the algorithms, known in the literature [2,8,5]. A description of relevant features has been given in Table 2. Traditional Live-Wire algorithm [2] takes into consideration all image pixels, thus it is much more time consuming than other methods. Additionally, since following all edges (even very subtle once), the main direction may be lost. This requires many more characteristic points marked by the user. Fuzzy clustering in intelligent scissors LW-FCM [8] significantly reduces the user interaction and considers only those image pixels which are situated along edges. To identify important pixels a fuzzy C-means algorithm is used. Therefore, specification of the class number is indispensable. For all tests in the current work a number of classes is related to the currently segmented structure. The lowest level of automation features a corridor scissors implementation [5]. This method provides additional manual tool for rough selection of the desirable edge. It returns coarse corridor which traces the segmented object. Then, automatic procedure finds the correct edge localization inside the previously selected corridor. Due to manual corridor selection this method has not been evaluated later in this study. Presented method relies on corridors, furthermore they are extracted automatically based on the watershed transform. This gains two advantages over the corridors scissors implementation. First, the user interaction is limited. On the other hand watershed lines cover the entire image. The evaluation of algorithms has been carried out based on medical images of various types. We started with optical coherence tomography (OCT) images from eyeball examination (Figs. 9 and 10). The segmentation procedure has been performed on the retina layers, in particular inner limiting membrane (ILM), outer
Fig. 8. Relief.
plexiform layer (OPL) and retinal pigment epithelium (RPE). Anatomical (Fig. 9) and pathological (Fig. 10) cases have been subjected to the analysis. Sometimes OCT images have low quality and include shadows of the blood vessels. The poor image quality does not effect the algorithm performance. The algorithm works also correctly when applied to pathological cases (Fig. 10) as vitreomacular traction, drusen, macular holes (cysts), etc. High efficiency of the solution at a relatively high level of automation has led us to verify the algorithm using other images. Among them are magnetic resonance (MR) knee joint examination (Fig. 11), computed tomography (CT) chest studies (Fig. 12) and abdomen (Fig. 16). The accuracy of segmentation and number of user points proof the robustness of this method. In the evaluation process the number of user interactions, number of pixels analysed and accuracy of the delineation have been tested. 3.1. Automation level The main goal of the first test is the automation level verification. It is expressed as a number of points marked by the user in order to obtain a correct delineation of a structure.
Fig. 9. Segmentation of retina layers: (a) ILM, (b) OPL, and (c)–(d) RPE.
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129
127
Table 3 Number of characteristic points. Image
Structure
Retina OCT Retina OCT Retina OCT Knee joint MR Knee joint MR Chest CT
ILM OPL RPE Thigh bone Tibia Single lung
Number of points LW
LW-FCM
LW-WT
5–11 8–17 4–13 7–16 5–8 6–8
2–3 2–4 2–4 3–5 2–5 2–3
1 1–2 1 1–2 1–2 1
Fig. 10. Segmentation of retina layers in different images. Fig. 13. Analysed pixels in: (a) LW-FCM and (b) LW-WT algorithms.
Fig. 11. Segmentation of knee structures.
Fig. 12. Segmentation of lungs.
The significant decrease in number of user points can be seen when LW-WT is applied (Table 3). A correct segmentation has been obtained for both straight and curved edges (also for circular edges, see lungs, thigh bone, tibia). In many cases it is sufficient to indicate a single point per one edge.
pixels to be processed (n). Traditional LW approach takes into consideration all image pixels (100%) therefore the on-the-fly mechanism is frequently used. The edge search stops when the end point of the edge is reached. The obtained reduction level may be difficult to estimate. It has been shown [8] that in the worst case it does not exceed of 32% image pixels. Moreover, LW-FCM algorithm is as accurate as the traditional LW approach with and without applying the on-the-fly mechanism. It has also been proven that LW-FCM takes into consideration only pixels located along the corridor (Fig. 13(a)). Therefore, further reduction of analysed points is obtained and for an optimal number of classes in FCM it equals at most 5% of the global image pixel number [8]. Since in the LW-WT approach only some corridor pixels are processed, further reduction is obtained (Fig. 13(b)). In the case shown in Fig. 13 the numbers of pixels processed in LW-FCM and LW-WT equal 0.3403% and 0.0618%, respectively. The analysis performed on the set of clinical images is shown in Table 4. The number of pixels to be proceed by the LW-WT approach is usually about 10 times smaller than by implementing the LWFCM algorithm and many less when the LW method (even using on-the-fly mechanism) is applied. 3.3. Segmentation accuracy evaluation The third test compares the segmentation accuracy of LWFCM and LW-WT versus LW approach. The LW approach has been Table 4 Number of pixels subjected to the analysis. Image
Structure
3.2. Number of processed pixels evaluation Numerical complexity of the proposed methodology is the same as in the Dijkstra algorithm, i.e. O(n2 ), where n is number of graph vertices (in image processing domain number of analysed image pixels). Therefore, beside reducing the number of Seed-Points (see Section 3.1) the proposed methodology decreases the number of
Retina OCT Retina OCT Retina OCT Knee joint MR Knee joint MR Chest CT
ILM OPL RPE Thigh bone Tibia Single lung
Number of pixels (%) LW
LW-FCM
LW-WT
38.8306 49.1565 43.8737 11.21 31.31 20.11
0.1838 0.1511 0.1825 0.4913 0.3311 0.4274
0.0169 0.0129 0.0151 0.0748 0.0656 0.0796
128
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129 Table 5 Sensitivity (S) and specificity (P) of the abdominal structures segmentation.
Fig. 14. Accuracy of OCT segmentation.
Structure
No. of cases
S (%)
P (%)
Liver Left kidney Right kidney
96 198 171
94.73 96.89 95.90
99.95 99.96 99.96
Due to the nature of corridor solutions, the LW-FCM and LWWT methods have yield similar distance values comparing with LW approach. On the other hand the corridor approaches return smoother edges. Noisy edges that result from applying the LW approach are caused by the dense manual selection of user points. Due to low contrast of the OPL layer the FCM procedure has failed. Thus, the LW-FCM returns a non solid corridor. Therefore, in Fig. 14 the OPL segmentation result is missing. Previously presented tests show user-oriented results. The evaluation of the accuracy is based on the analysis of 200 clinical images. The liver, left and right kidneys have been delineated manually on 96, 198, 171 images, respectively. In order to compare the performance of the LW-WT with the manual expert delineation, the sensitivity (S) and specificity (P) have been found. Numerical results are presented in Table 5 and examples of the LW-WT performance are shown in Fig. 16. The results show very high sensitivity and specificity independently on the selected anatomical structure. Note, that all the delineations are obtained by just a single Seed-Point per structure marked by the user. 4. Conclusions
Fig. 15. Accuracy of MR and CT segmentation.
considered as a reference method because a user can correct the contour as many times, as required to obtain successful delineation. In order to compare all algorithms the Hausdorff distance [24] between contour points has been determined. Means and standard deviations for over a dozen images of each type have been shown in diagrams. Fig. 14 presents the delineation of three different layers in OCT images (ILM, OPL and RPE). Fig. 15 illustrates the knee structure segmentation results in MR studies (thigh bone and tibia) and CT lungs exams. Generally, segmentation results from corridor modifications approaches (LW-FCM and LW-WT) are similar to each other. In LW-FCM usually the mean distance value is larger and the standard deviation is lower, yet the differences are not significant. Additionally, it can be seen that the OCT segmentation returns higher values of distances than MR/CT studies. It is due to the higher resolution of OCT images with respect to MR/CT.
The method proposed in this paper demonstrates that intelligent scissors approach based on watershed transform can yield robust and accurate segmentation of various anatomical structures acquired with different modalities. The method can be successfully employed for semi-automatic delineation. It can handle structures as different in shape and appearance as the retina layers in OCT exams, chest, liver and kidney in CT and knee in MR studies. The overall segmentation accuracy with respect to the traditional Life-Wire approach shows comparable results in the range of single pixels. What is more important, the number of pixels to be processed decreases about 3000 times for OCT and about 300 times for CT and MR. This results in a significant reduction of the numerical complexity. Another important drawback of all semi-automatic method, namely the user interaction, has also been minimized and reduced to 1 or 2 click points. A thorough analysis led use to the conclusion that the watershed based intelligent scissors is a promising interactive tool to semi-automatic edge delineation. The method is related neither to a modality nor anatomical/pathological structures. It can be applied as a segmentation procedure in image manipulation systems that do not offer an automated segmentation options or when the automated analysis has failed. References
Fig. 16. Segmentation of (c) liver, (b) kidneys and (c) liver and both kidneys.
[1] Mortensen EN, Morse BS, Barrett WA, Udupa JK. Adaptive boundary detection using ‘Live-Wire’ two-dimensional dynamic programming. In: IEEE computers in cardiology. 1992. p. 635–8. [2] Mortensen EN, Barrett WA. Interactive segmentation with intelligent scissors. Graph Models Image Process 1998;60(5):349–84. [3] Barrett WA, Mortensen EN. Interactive Live-Wire boundary extraction. Med Image Anal 1997;1(4):331–41. [4] Dijkstra EW. A note on two problems in connexion with graphs. Numer Math 1959;1:269–70. [5] Farin D, Pfeffer M, de With PHN, Effelsberg W. Corridor scissors: a semiautomatic segmentation tool employing minimum-cost circular paths. In: International conference on image processing (ICIP), vol. 2. 2004. p. 1177–80.
W. Wieclawek, E. Pietka / Computerized Medical Imaging and Graphics 43 (2015) 122–129 [6] Mortensen EN, Barrett WA. Toboggan-based intelligent scissors with a four parameter edge model. In: IEEE computer vision and pattern recognition, vol. 2. 1999. p. 452–8. [7] Barrett WA, Reese LJ, Mortensen EN. Intelligent segmentation tools. In: IEEE international symposium on biomedical imaging. 2002. p. 217–20. [8] Wieclawek W, Pietka E. Fuzzy clustering in intelligent scissors. Comput Med Imaging Graph 2012;36(5):396–409. [9] Seramani Z, Jiayin S, Luk CK, Malmurugan N, Nagappan A. Denoising of MR images using non linear anisotropic diffusion filtering as a preprocessing step. Int J Biosci Technol 2008;1:17–21. [10] Black MJ, Sapiro G, Marimont DH, Heeger D. Robust anisotropic diffusion. IEEE Trans Image Process 1998;7(3):421–32. [11] Vincent L, Soille P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 1991;13(6):583–98. [12] Roerdink JBTM, Meijster A. The watershed transform: definitions, algorithms and parallelization strategies. Fundam Inf 2001;41:187–228. [13] Acharjya PP, Ghoshal D. An approach to reduce oversegmentation in watershed ridge line observation. Int J Adv Res Technol 2013;2(6):290–4. [14] Karantzalos K, Argialas D. Improving edge detection and watershed segmentation with anisotropic diffusion and morphological levellings. Int J Remote Sens 2006;27(24):5427–34. [15] Mittal U, Anand S. Modified watershed segmentation with denoising of medical images. Int J Innov Res Sci Eng Technol 2013;2(4):982–7. [16] Parvati K, Prakasa Rao BS, Das MM. Image segmentation using gray-scale morphology and marker-controlled watershed transformation. Discrete Dyn Nat Soc 2008 [Article ID 384346].
129
[17] Haris K, Efstratiadis SN, Maglaveras N, Katsaggelos AK. Hybrid image segmentation using watersheds and fast region merging. IEEE Trans Image Process 1998;7(12):1684–99. [18] Boukerroui D. Efficient numerical schemes for gradient vector flow. Pattern Recognit 2012;45(1):626–36. [19] Xu C, Prince JL. Gradient vector flow: a new external force for snakes. In: CVPR IEEE computer society. 1997. p. 66–71. [20] Bauer G, Homm F, Walchshäusl L, Burschka D. Multi spectral pedestrian detection and localization. In: Advanced microsystems for automotive applications. Springer-Verlag; 2008. [21] Fairfield J. Toboggan contrast enhancement for contrast segmentation. In: Proceedings of 10th international conference on pattern recognition. 1990. p. 712–6. [22] Meyer F. Topographic distance and watershed lines. Signal Process 1994;38(1):113–25. [23] Lindeberg T. Edge detection and ridge detection with automatic scale selection. Int J Comput Vis 1996;30:465–70. [24] Dubuisson MP, Jain AK. A modified Hausdorff distance for object matching. In: Conference A: computer vision & image processing. Proceedings of the 12th IAPR international conference on pattern recognition. 1994. p. 566–8. [25] Badura P, Pietka E. Soft computing approach to 3D lung nodule segmentation in CT. Comput Biol Med 2014;53:230–43. [26] Qinghua H, Xiao B, Yingguang L, Lianwen J, Xuelong L. Optimized graph-based segmentation for ultrasound images. Neurocomputing 2014:216–24. [27] Qinghua H, Suying L, Longzhong L, Minhua L, Lianwen J, Anhua L. A robust graph-based segmentation method for breast tumors in ultrasound images. Ultrasonics 2012;52(2):266–75.