NGVF: An improved external force field for active contour model

Pattern Recognition Letters 28 (2007) 58–63 www.elsevier.com/locate/patrec

NGVF: An improved external force field for active contour model Ning Jifeng a

a,b,* ,

Wu Chengke a, Liu Shigang a, Yang Shuqin

b

Xidian University, National key laboratory of ISN, No. 2 South TaiBai Road, Xi’an, Shaanxi 710071, China b Northwest A&F University, 3 Taicheng Road, Yangling, Shaanxi 712100, China Received 8 July 2005; received in revised form 26 April 2006 Available online 23 August 2006 Communicated by H.H.S. Ip

Abstract An improved external force field for active contour model, called as NGVF, is presented in this paper. Based on analyzing the diffusion process of the GVF and three interpolation functions, it is found that the generation of GVF contains diffusions in two orthogonal directions along the edge of image, one is the tangent direction and the other is the normal direction. Moreover, the diffusion in the normal direction plays the key role on the diffusion of GVF, while the diffusion in the tangent direction has little effect. So the GVF in the normal direction (NGVF) is taken as a new force field to study. Experiment results with several test images reveal that, compared with GVF, NGVF can enter into long, thin indention and has faster convergence speed towards the concavity and bigger time step.  2006 Elsevier B.V. All rights reserved. Keywords: Active contour model; Gradient vector flow; GVF in the normal direction; Image interpolation

1. Introduction The variational method has been a research focus of image processing in recent years (Kass et al., 1988; Osher and Sethian, 1988; Yuen et al., 1998; Jain et al., 1998; Caselles et al., 1998; Zhong et al., 2000; Chan and Vese, 2001; Wang and Chou, 2004). Notably, active contours, known as ‘‘snakes’’, have been widely studied and applied. And their applications mainly include edge detection, segmentation of objects, shape modeling and motion tracking. Active contours were first introduced in 1988 by Kass et al. (1988). They are closed curves or surfaces represented by parametric equation. An energy function is associated with these curves, which convert the problem of finding objects into the process of energy minimizing. Affected by both internal force and external force, the parametric curves move to the direction of minimum energy. The internal force is decided by the curves themselves, and the external force is decided by the image, so the external *

Corresponding author. Fax: +86 29 8823 2281. E-mail address: [email protected] (N. Jifeng).

0167-8655/$ - see front matter  2006 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2006.06.014

force is also called ‘‘image force’’. The traditional force field has small capture range, and is sensitive to the initial snake curve. In order to enable the curves to converge the edge of objects rapidly, many improved models of image force field were put forwarded. Cohen (1991) presented the balloon model, which enlarge the capture range of snakes, but could not enter into the concavities of the objects’ edge. Additionally, external forces defined as the negative gradient of a Euclidean distance map were widely used (Cohen and Cohen, 1993). Xu and Prince (1997, 1998) proposed a new external force model, known as GVF, which uses a spatial diffusion of the gradient of an edge map of the image to create a dynamic force field. It solves the problem of small capture range of traditional snakes’ model, and can go into the concavities of the objects’ edge in principle. There are two different implement methods of active contour model, namely parametric active contours and geometric active contours (or geodesic active contours) (Malladi et al., 1995; Caselles et al., 1997). Parametric active contours use a parametric representation of the curves, and geometric active contours utilize level set

N. Jifeng et al. / Pattern Recognition Letters 28 (2007) 58–63

methods for its implementation (Osher and Sethian, 1988). In general, parametric snakes have the advantage over geometric snakes in convergence speed. The virtue of geometric active contours is that, during evolution process, the application of level set methods makes the splitting or merging of the curves treated automatically, whereas parametric methods have difficulty in handling these situations (Sumengen, 2004). In recent years, some relations between these two models have been researched (Xu et al., 2000; Paragios et al., 2004). In (Paragios et al., 2004), GVF was integrated into the geodesic active contour, which improved the segmentation effects of geometric snakes. Therefore, constructing a good external force field would be of benefit to the segmentation performance of parametric snakes and geometric snakes. In this paper, we discuss at length the diffusion process of GVF force field and propose an improved external force field called as NGVF. There has close relationship between NGVF and GVF. However, compared with GVF, NGVF has some better performances than GVF in many cases. The rest of paper are organized as follows: In Section 2, we introduce briefly active contour and GVF model, in Section 3, further discussion on GVF and proposed new model is explained in detail. While, some experiment results appear in Section 4. Conclusions of this paper are presented in Section 5 and the acknowledgement.

59

GVF is the vector field V(x, y) = [u(x, y), v(x, y)], which minimizes the energy functional: ZZ 2 2 Q¼ lðu2x þ u2y þ v2x þ v2y Þ þ jrf j jV  rf j dx dy ð4Þ where f is edge map of original image and l is weighting parameters. Using the calculus of variations, it can be shown that the GVF field can be found by solving the following Euler equations: lr2 u  ðu  fx Þðfx2 þ fy2 Þ ¼ 0; lr2 v  ðv  fy Þðfx2 þ fy2 Þ ¼ 0

ð5Þ

2

where $ is the Laplacian operator. GVF is a dynamic force field, which diffuses along the directions of x and y of image gradient simultaneously, and could preserve the image’s edge information well after numerous iterations. GVF has favorable convergence and could enter into the concavities of the objects’ edge in principle, so it has been one of the models which are used most wildly. However, the diffusion speed of the edge information of GVF is low and so only through more iteration numbers can the force field go into the concavities of the objects’ edge. 3. Discussion of problem and proposed model 3.1. Discussion of gradient vector flow

2. Active contour model 2.1. Snakes In 2D, snakes are a curve c(s) = (x(s), y(s), s 2 [0,1]) that moves through the spatial domain of an image to minimize the energy functional: Z 1 1 2 2 Esnakes ¼ ½ajc0 ðsÞj þ bjc00 ðsÞj  þ Eext ðcðsÞÞ ds ð1Þ 2 0 where a and b are weighting parameters that control the snake’s tension and rigidity, respectively, and the two parameters of c 0 (s) and c00 (s) denote the first and second derivatives of c(s) with respect to s. The external energy function Eext is derived from the image, and it takes on its smaller values at the features of interest, such as boundaries. In (1), a snake that minimizes E must satisfy the Euler equation ac00 ðsÞ  bc0000 ðsÞ  rEext ¼ 0

ð2Þ

This can be viewed as a force balance equation F int þ F ext ¼ 0

ð3Þ 00

0000

where Fint = ac (s)  bc (s) and Fext = $Eext. 2.2. GVF (Gradient Vector Flow) Xu and Prince (1997, 1998) proposed an external force field, known as GVF, which has large capture range.

In Eq. (5), $u, $v are diffusion terms, and ðu  fx Þ ðfx2 þ fy2 Þ, ðv  fy Þðfx2 þ fy2 Þ are data attraction terms. It is well known that Laplacian operator has very strong isotropic smoothing properties and does not preserve edges, whereas ðu  fx Þðfx2 þ fy2 Þ and ðv  fy Þðfx2 þ fy2 Þ could preserve the edge map, l is a weight or regularization parameter adjusting between the first terms and the second terms. So the outcomes of Eq. (5) are equivalent to a progressive construction of the GVF starting from the object boundaries and moving towards the flat background. The above analyses show that the diffusion of GVF force field depends virtually on Laplacian terms, while data attraction terms only preserve the edge map, which is to say that the properties of GVF have close relationship with Laplacian terms. Accordingly, if Laplacian terms in Eq. (5) could be replaced with other better diffusion operator, the property of GVF force field would be improved. Therefore, we emphasize on analyzing the Laplacian terms next. In (Aubert and Pierre, 2002), Laplacian terms are decomposed using the local image structures, that is, the tangent and normal directions to the isophote lines (Fig. 1). Further, we rewrite Laplacian terms as r2 f ¼ fTT þ fNN 1 fTT ¼ ðfx2 fyy þ fy2 fxx  2fx fy fxy Þ jrf j2 1 ðfx2 fxx þ fy2 fyy þ 2fx fy fxy Þ fNN ¼ 2 jrf j

ð6Þ ð7Þ ð8Þ

60

N. Jifeng et al. / Pattern Recognition Letters 28 (2007) 58–63

process that edge maps spreads out. So they are similar to a great extent. Since Eq. (8) is better interpolation operator than Laplacian operator, it should be a better diffusion operator than Laplican operator. Then we replace Laplacian diffusion terms of Eq. (5) with Eq. (8) and get a new force field for active contour as follows. luNN  ðu  fx Þðfx2 þ fy2 Þ ¼ 0; lvNN  ðv  fy Þðfx2 þ fy2 Þ ¼ 0 Fig. 1. Tangent and normal directions of edge map of image.

where fTT and fNN denote the second derivatives of f in the tangent direction and normal direction, respectively. In fact, decomposing the Laplacian term as weighted sum of the two directional derivatives along T and N can be done for most classical diffusion operators. This allows us to see clearly the action of the Laplacian operator in the direction T and N. Accordingly, we rewrite Eq. (5) as lðauTT þ buNN Þ  ðu  fx Þðfx2 þ fy2 Þ ¼ 0; lðavTT þ bvNN Þ  ðv  fy Þðfx2 þ fy2 Þ ¼ 0

ð9Þ

where a and b denote weight of diffusion in the tangent direction and normal direction, respectively. 3.2. Discussion of three interpolation functions Caselles puts forward an axiomatic approach to image interpolation in (Caselles et al., 1998) where Eqs. (6)–(8) are considered as three image interpolation functions. As the interpolation functions, Eq. (6) is excellent and standard, Eq. (8) is somewhat new, whereas Eq. (7) has no solution. Generally speaking, fNN has the best interpolating effects, $2f the second, and fTT the third. 3.3. Proposed model Above discussions show that Laplacian operator is both a diffusion operators and an interpolation operator. Image interpolation and diffusion of force field is the diffusion

ð10Þ

From another point of view, Eq. (10) is different from Eq. (9) only in taking out of the diffusion terms in tangent direction. In other words, Eq. (10) just contains the diffusion terms in normal direction. Thus, we call this new force field as NGVF (GVF in the normal direction). Additionally, when a = 0 and b = 1 in (9), GVF becomes NGVF. So NGVF is special case of GVF. Moreover, unlike isotropic properties of Laplacian operator, diffusion term of NGVF is anisotropic. The partial differential Eq. (10) can be solved using explicit finite difference scheme. NGVF only with one direction diffusion is stable when the time step l < 12, while the time step of GVF is less than 14 (Xu and Prince, 1998). Bigger time step is the first advantage of NGVF. 4. Experimental results We compare the force filed properties and the segmentation effects of GVF with those of NGVF using Matlab6.5. 4.1. The comparison of performance between GVF and NGVF We test Fig. 2(a) with 64 · 64 pixels. Fig. 2(b)–(d) are the force fields of GVF and NGVF after 20 iterations, respectively. We can see that GVF and NGVF have similar properties. Both of them are global force fields and could enter into the concavity and maintain the image’s edge map. However, we know that an important performance of external force field for active contour mainly depends on the ability to enter concavity of edge. As for the concavities of each force fields, Fig. 2(b) and (c) infer that, in the same

Fig. 2. Force field vector (a) u1, (b) GVF force field (l = 0.2, force field iterations = 20), (c) NGVF force field (l = 0.2, force field iterations = 20), (d) NGVF force field (l = 0.45, force field iterations = 20).

N. Jifeng et al. / Pattern Recognition Letters 28 (2007) 58–63

time step, NGVF has slight advantage over GVF. While, Fig. 2(b) and (d) imply that NGVF is obviously better than GVF when it has greater time step. In other words, NGVF which only contain the diffusion in the normal direction have better property of force field. As mentioned previously, the diffusion term of GVF is Laplacian operator, which can be decomposed as the diffusion in the normal direction as well as in the tangent direction. Fig. 2 suggests that the former term weighs heavily in

61

the diffusion process of Laplacian operator, but the latter seems to weigh little or play ‘bad’ roles on the generation of GVF force field. Therefore, we can conclude that the diffusion of Laplacian operator in the tangent direction or along the edge restrains vector field from entering into the concavity. 4.2. Image segmentation Six typical images are selected to contrast the segmentation accuracy between GVF and NGVF.

Fig. 3. (a) u2, (b) GVF force field (l = 0.2, force field iterations = 100), (c) NGVF force field (l = 0.45, force field iterations = 30), (d) initial curve, (e) segmenting by GVF, (f) segmenting by NGVF.

Fig. 4. (a) Room image and initial curve (l = 0.2, force field iterations = 30), (b) segmenting by GVF, (c) segmenting by NGVF.

4.2.1. Image without noise Fig. 3(a) with long and thin concavity is used to test whether external force field has the ability to enter the stronger concavity. Seen from force field property of the concavity, Fig. 3(b) and (c) shows clearly the difference between GVF and NGVF. NGVF force field on the concavity points vertically to the bottom of the concavity, but GVF force field in the rectangle is opposite, which implies that GVF could not enter into the concavity. Fig. 3(e) and (f) are illustrated with experiment results clearly. Fig. 4(a) has a small gap. At the gap, there is the diffusion in the tangent direction for GVF but no such diffusion for NGVF. That is the main difference between GVF and NGVF. This diffusion promotes the closing of the gap and causes the different evolvement paths of NGVF and GVF at the gap. However, NGVF can still converge towards the gap without diffusion in the tangent direction. This indicates that the diffusion in the normal direction is the main diffusion factor for the generation of external force field. 4.2.2. Image with noise Fig. 5(a) and Fig. 6(a) are two images with different noises. Corresponding experiment results show that NGVF is slight better than GVF in segmentation results. But as for their evolvements towards the concavity, NGVF is obvious faster than GVF, which further approves our assumption that the diffusion terms of NGVF are better than Laplacian diffusion terms of GVF.

Fig. 5. (a) u1 with salt noise and initial curve, (b) blurred image f = Gr(x, y) * I(x, y), where r = 0.8, (c) segmenting by GVF (l = 0.23, force field iterations = 70), (d) segmenting by NGVF (l = 0.45, force field iterations = 50).

62

N. Jifeng et al. / Pattern Recognition Letters 28 (2007) 58–63

Fig. 6. (a) u1 with speckle noise and initial curve, (b) blurred image, f = Gr(x, y) * I(x, y), r = 1, (c) segmenting by GVF (l = 0.24, force field iterations = 80), (d) segmenting by NGVF (l = 0.45, force field iterations = 80).

Fig. 7. (a) Heart image and initial curve, (b) edge map, f = j$ (Gr(x, y) * I(x, y))j, r = 2.5, (c) segmenting by GVF (l = 0.24, force field iterations = 30), (d) segmenting by NGVF (l = 0.45, force field iterations = 20).

Fig. 8. (a) Dog image and initial curve, (b) edge map, f = j$(Gr(x,y)*I(x,y))j, r = 2.5, (c) segmenting by GVF (l = 0.24, force field iterations = 35), (d) segmenting by NGVF (l = 0.45, force field iterations = 15).

4.2.3. Gray image Figs. 7 and 8 show that NGVF can also converge well with fewer force field iterations and maintain the edge map of the image.

field, which might provide some insights to construct better force fields.

5. Conclusions

This paper uses MATLAB snake demo toolbox developed by Xu C. and Prince J., which helps author to learn active contour model. The valuable suggestions from reviewers are important for authors to improve the paper. This work is supported by NSFC (No. 60473119).

Based on analyzing the diffusion process of GVF, we present an improved external force field called as NGVF. NGVF is differentiated from GVF by diffusion term and can be also considered as a special case of GVF force field. In general, NGVF and GVF have similar property in many cases. NGVF is still insensitive to noises even without diffusion in the tangent direction. Moreover, compared with GVF, NGVF can enter into long and thin concavity. It is important that bigger time step makes NGVF more effective than GVF in some cases. In addition, we associate the interpolation function with diffusion process of force

Acknowledgements

References Aubert, G., Pierre, K., 2002. Mathematical Problems in Image Processing (partial differential equations and the calculus of variations. Springer. Caselles, V., Kimmel, R., Sapiro, G., 1997. Geodesic active contours. Internat. J. Comput. Vision 10 (1), 61–79.

N. Jifeng et al. / Pattern Recognition Letters 28 (2007) 58–63 Caselles, Vicent, Morel, Jean-Michel, Sbert, Catalina, 1998. An axiomatic approach to image interpolation. IEEE Trans. Image Process. 7 (3), 376–386. Chan, T.F., Vese, L.A., 2001. Active contours without edges. IEEE Trans. Image Process. 10 (2), 266–277. Cohen, T.F., 1991. On active contour models and balloons. Comput. Vision Graphics Image Process. (Image Understanding) 53 (2), 211– 218. Cohen, L.D., Cohen, I., 1993. Finite-element methods for active contour models and balloons for 2-D and 3-D images. IEEE Trans. Pattern Anal. Machine Intell. 15, 1131–1147. Jain, A., Zhong, Y., Dubuisson-Jolly, M., 1998. Deformable template models: A review. Signal Process. 71, 109–129. Kass, M., Withkin, A., Terzopoulos, D., 1988. Snakes: Active contour models[J]. Internat. J. Comput. Vision 1 (4), 321–331. Malladi, R., Sethian, J.A., Vermuri, B.C., 1995. Shape modeling with front propagation: A level set approach. IEEE Trans. Pattern Anal. Machine Intell. 17 (2), 158–175. Osher, S., Sethian, J.A., 1988. Fronts propagating with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. (79), 12–49.

63

Paragios, N., Mellina-Gottardo, O., Ramesh, V., 2004. Gradient vector flow fast geodesic active contours. IEEE Trans. Pattern Anal. Machine Intell. (26), 403–407. Sumengen, Baris, 2004. Variational image segmentation and curve evolution on natural images. Ph.D. thesis, University of California. Wang, Y.-C., Chou, J.-J., 2004. Automatic segmentation of touching rice kernels with an active contour model. Trans. ASAE 47 (5), 1803–1811. Xu, C., Prince, J.L., 1997. Gradient vector flow: A new external force model for snakes. In: IEEE Proc. Conf. on Computer Vision and Pattern Recognition, pp. 66–71. Xu, C., Prince, J.L., 1998. Snakes, shapes, and gradient vector flow. IEEE Trans. Image Process. 7, 359–369. Xu, C., Yezzi Jr., A., Prince, J.L., 2000. On the relationship between parametric and geometric active contours[A]. In: Asilomar Conference on Signals, Systems and Computers pp. 483–489. Yuen, P.C., Feng, G.C., Zhou, J.P., 1998. Contour detection method: Initialization and contour model. Pattern Recognition Lett. 20, 141– 148. Zhong, Y., Jain, A.K., Dubuisson-Jolly, M., 2000. Object tracking using deformable templates. IEEE Trans. Pattern Anal. Machine Intell. (22), 544–549.