- Email: [email protected]
An affine shape constraint for geometric active contours
CHAPTER
An affine shape constraint for geometric active contours
33
Mohamed Amine Mezghich, Maweheb Saidani, Slim M’Hiri, Faouzi Ghorbel GRIFT Research Group, CRISTAL Laboratory, Ecole Nationale des Sciences de l’Informatique (ENSI), Campus Universitaire de la Manouba, Manouba, Tunisia
1 INTRODUCTION Precise object of interest detection is of high need in many applications of image processing such as medical imaging, tracking of moving object, and 3D object reconstruction. There have been several works in this field of image segmentation and object detection. Among them we can distinguish the classical active contours which are intensity-based models [1–5] and the constrained models by a prior knowledge that are driven both by global geometrical [6–9] or statistical [10–14] shape information and a local image information like gradient or curvature. There have been promising results given that the classical snake models can detect object with smooth contours whereas the level set-based models have the additional ability to detect simultaneously many objects in the image. However, several work of the prior knowledge-driven models reached good segmentation results by detecting partially occluded shapes presenting missing part in low contrast or very noisy image. To define prior knowledge, the authors use either shape alignment [7] or registration [9] between the contour of the target shape and the shape of reference or a distance between invariant shape descriptors like [8]. All these work manage the case of Euclidean transformations (translation, rotation, and scale factor). But in real situations, the object of interest can submit large transformations (affine or projective cases) and distortions. Hence the class of rigid transformations is not appropriate to model this type of movement between the shape of reference and the target one. To our knowledge, there has been only the work of Foulonneau et al. [15] that treats the case of affine transformations. Although this approach, which is based on the distance between an invariant set of Legendre descriptors of the target and reference shapes, presented good results, this method suffers from the instability of the Legendre moments and the order that has to be fixed empirically. Besides, the method requires a heavy execution time to achieve satisfactory results. In order to extend the work of shape priors presented in Refs. [16,17], we propose in this research a geometric approach to manage the class of affine Emerging Trends in Image Processing, Computer Vision and Pattern Recognition http://dx.doi.org/10.1016/B978-0-12-802045-6.00033-8 © 2015 Elsevier Inc. All rights reserved.
499
500
CHAPTER 33 An affine shape constraint for geometric active
transformations [26] that can occur between the shape of interest and the reference one. Prior knowledge is obtained by aligning the evolving contour and the template using an alignment method which is based on Fourier descriptors. For this purpose, we will start by introducing our approach in the case of rigid transformations. Then we generalize the idea by treating the case of affine ones that often occur in real situations. We will also propose a geometric method based on a set of complete and stable set of invariant shape descriptors to manage the case of mutireferences. Combining the proposed shape prior information with the level set-based model, the improved model can retain all the advantages of the active contour model and have the additional ability of being able to represent the global shape of an object. The remainder of this paper is organized as follows: In Section 2, a description of the outlines of the used shape alignment method in both cases (Euclidean and affine transformations) will be presented. In Section 3, we will briefly recall the principle of the used edge-based active contours, then we will move to present the proposed shape prior. Experimental results are presented and commented in Section 4. Finally, we conclude the work and highlight some possible perspectives in Section 5.
2 SHAPE ALIGNMENT USING FOURIER DESCRIPTORS In this section, we recall the used alignment methods to define the proposed energy in both cases rigid and affine transformations.
2.1 EUCLIDEAN SHAPES ALIGNMENT It is well known that the closed contour of a planar object can be given by its parametric representation g : ½0,2p ! C t ! xðtÞ + iyðtÞ
(1)
Let g1and g2 be centered (according to the center of mass) and normalized arc length parameterizations of two closed planar curves having shapes F1and F2. Suppose that g1 and g2 have a similar shape under rigid transformation. In shape space this is equivalent to have: g1 ðtÞ ¼ aeiy g2 ðt + t0 Þ
(2)
where a is the scaling factor, y the rotation angle, t0 is the difference between the two starting points of g1 and g2. Rigid motion estimation is obtained by minimizing the following quantity introduced by [18]: Err ðg1 , g2 Þ ¼ min ða, y, t0 Þ g1 ðtÞ aeiy g2 ðt + t0 ÞL2
(3)
2 Shape alignment using fourier descriptors
If ais equals to 1 [19], showed that Equation (3) is a metric in shape space and corresponds to the Hausdorff distance between the two shapes. In practice, this result implies the uniqueness of the motion’s parameters so obtained. Using Fourier descriptors Ck(g1) and Ck(g2), minimizing Err becomes equivalent to minimize f(y, t0) in Fourier domain f ðy, t0 Þ ¼
X Ck ðg1 Þ aeiðkt0 + yÞ Ck ðg2 Þ2
(4)
k2Z
[18] proposed an analytical solution to compute t0 and y. t0 is one of the zeros of the following function: gðtÞ ¼
X
X rk sin ðck + ktÞ krk cos ðck + ktÞ kX k X krk sin ðck + ktÞ rk cos ðck + ktÞ k
ick
(5)
k
¼ Ck ðg1 ÞCk ðg2 Þ, y is chosen to satisfy Equation (6) and minimize f(y, t0)
where rk e where t0 is one of the roots of Equation (5).
P r sin ðck + kt0 Þ P tany ¼ k k r k k cos ðck + kt0 Þ
(6)
Having the value of y and t0 that minimize f(y, t0), the scaling factor is given by P
a¼
rk cos ðck + kt0 + yÞ kP k Ck ðg2 ÞCk ðg2 Þ
(7)
Based on Nyquist-Shannon theorem and B-spline approximation of the g(t)’s curve, Persoon and Fu [19] proposed the first implementation of this method of shape alignment and its application in rigid motion estimation for video compression. In Figure 1, we give an example of two aligned curves with the used method. The estimated motion’s parameters are presented in Table 1. The associated g(t)’s curve is presented in Figure 2. The zeros of g(t) are drawn with circles.
FIGURE 1 Example of two curves before (a) and after (b) shape alignment.
501
502
CHAPTER 33 An affine shape constraint for geometric active
Table 1 The Estimated Parameters of Synthetic Rigid Motion Rotation ( )
Scaling
Difference Between Starting Point
45
1
0
FIGURE 2 Example g(l)’s curve.
2.2 AFFINE SHAPE ALIGNMENT In order to define affine invariant shape prior from a reference shape, we have to perform shape matching between the target and the template shapes F1 and F2. To correctly estimate the affine motion parameters, we have to deal with the same number of points which is not relevant if we consider two different points of view. For this purpose, in Ref. [20], the author proposes to use the affine reparametrization of curve which is invariant under affine transformation. We will start by presenting the affine reparametrization procedure of a given curve. Then we will describe our method of affine motion’s parameters estimation.
2.2.1 Reparametrization of closed curve If we take the same object in two different camera sides, we found a different number of contour points in each image. Consequently, we proceed by normalization of curves under affine transformation. Given the shape front, its edge pixels are
2 Shape alignment using fourier descriptors
extracted and traversed to yield a discrete closed curve which is a parametric equation g(t) ¼ (u(t), v(t)), where t 2 {0,. . .,N 1} and g(N) ¼ g(0). We use the affine arc length reparametrization to normalize closed curves under affine transformations that can be expressed by: sa ð t Þ ¼
ð 2 1 0 ^ 00 3 g ðtÞ g ðtÞ dt, t 2 ½a, b La
La ¼
(8)
ð b rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ^ 3 g0 ðtÞ g00 ðtÞdt
(9)
a
where La is the total affine arc length of the considered curve, ^ represents the cross product between two vectors and, jjjj denotes the Euclidean norm.
2.2.2 Contours alignment using geometrical affine parameters estimation We consider two closed curves O1and O2which define the same shape. O1and O2 are said to be related by an affine transformation if and only if: hðlÞ ¼ aAf ðl + l0 Þ + B
(10)
where Bis a translation vector, A is a linear transformation, l0 is the shift value, f and h are the affine reparametrization of two contours having the same affine shape. In the Fourier space we get: Uk ðhÞ ¼ ae2ipkl0 AUk ð f Þ + bdk
(11)
where Uk(h) and Uk(f) are, respectively, the Fourier coefficients of f and h. So estimation of affine motion can be resumed to the estimation of its three parameters: the affine matrix A, the shift value l0, and finally the scale factor a if we consider a normalization under translation.
Estimation of the scale factor a The scale factor can be estimated using the following formula: a2 ¼
det Uk ðhÞ, Uk ðhÞ det Uk ð f Þ, Uk ð f Þ
(12)
where ðUk ðhÞ, Uk∗ ðhÞÞ and ðUk ðf Þ, Uk∗ ðf ÞÞ are 2 2 matrix formed by Fourier coefficients on some fixed index k and U* is the U complex conjugate.
Computation of the shift value l0
Let’s consider M1 ¼ ðUk1 ðhÞ, Uk2 ðhÞÞ and M2 ¼ ðUk1 ð f Þ, Uk2 ð f ÞÞ l0 ¼
arg ð det ðM2 ÞÞ arg ð det ðM1 ÞÞ ðk1 + k2 Þ
(13)
where k1 and k2 are two fixed index and arg(U) is the complex argument of U.
503
504
CHAPTER 33 An affine shape constraint for geometric active
Computation of the matrix A’s parameters Since scale factor a and shift value l0have been previously estimated (Equations 12 and 13), we are going to estimate affine matrix A parameters. Lets O1, O2 be two curves and Uf, Uh, respectively, their Fourier coefficients. In Fourier space we have (Equation 14) Uk ðhÞ ¼ ae2ipkl0 AUk ð f Þ
Using matrix, we have the following formula:
u2 ðkÞ v 2 ðk Þ
¼ ae2ipkl0
a1 a2 a3 a4
(14)
u1 ðkÞ v 1 ðk Þ
(15)
So we have to estimate a1, a2, a3, and a4. This system can be represented as following: 8 u2 ðk1 Þ ¼ ae2ipk1 l0 a1 u1 ðk1 Þ + e2ipk1 l0 a2 v1 ðk1 Þ > > > 2ipk1 l0 > a3 u1 ðk1 Þ + e2ipk1 l0 a4 v1 ðk1 Þ > < v2 ðk1 Þ ¼ ae .. . > > > > u2 ðkn Þ ¼ ae2ipkn l0 a1 u1 ðkn Þ + e2ipkn l0 a2 v1 ðkn Þ > : v2 ðkn Þ ¼ ae2ipkn l0 a3 u1 ðkn Þ + e2ipkn l0 a4 v1 ðkn Þ
(16)
Then to estimate matrix A’s parameters (Equation 14), we have to resolve a system with 2N equations and four unknown parameters that can be written as K2N4 A4 ¼ U2N
(17)
The solution of the system (Equation 16) can be obtained by KA U ¼ e
(18)
So we have to minimize the quadratic error e by using the pseudoinverse of K. We show by the following figure an example of affine motion estimation between two synthetic curves after contours resampling (Figure 3). The final result of curves alignment after affine motion estimation is presented by Figure 4. The estimated motion parameters are presented in Table 2.
2.3 DISCUSSION Having the parameters of the affine or Euclidean transformations between the two contours, we perform the alignment of the two curves to determine the regions of variability between shapes by computing the product function of the signed distance
FIGURE 3 Two different shapes after affine deformations ((a) and (c)) and curve resampling ((b) and (d)).
2 Shape alignment using fourier descriptors
FIGURE 4 Results of affine shape alignment.
Table 2 The Estimated Affine Motion Parameters l0
a
Shapes of (b)
0.002
1.003
Shapes of (d)
0
2.004
A
0:860 0:47 0:52 0:865 0:962 0:002 0:1037 0:966
functions associated to level set functions of, respectively, the target and the reference object after alignment given by fprod ðx, yÞ ¼ f∅ref ðx, yÞ f∅ ðx, yÞ
(19)
where f∅ref and f∅ are the two binary images associated, respectively, to ∅ ref and ∅. See Ref. [16] for more details. By construction, the product function fprod is negative in the areas of variability between the two binary images f∅ref and f∅ due to occlusion, clutter, or missing parts, whereas in positive regions, the objects are similar. Thus, in what follows, we propose to update the level set function ∅, only in regions of variability between shapes to make the evolving contours overpass the spurious edges and recover the desired shapes of objects. This property recalls the narrow band technique used to accelerate the evolution of the level set functions [4].
2.4 GLOBAL MATCHING USING AFFINE INVARIANTS DESCRIPTORS In presence of many templates, we have to choose the most suitable one according to the evolving curve. Let aand b be positive real numbers, and k0, k1, k2, and k3 four positive integers. Let Cxn and Cyn be the complex Fourier coefficients of the coordinates (u, v). △ denotes the determinant. x x C C ny m △m ¼ n C Cy n m
(20)
505
506
CHAPTER 33 An affine shape constraint for geometric active
In Ref. [20], the author introduced two sets of invariant descriptors I and J which are, respectively, given by (Equations 21 and 22). Dk1 k2 Dk2 k Dkk1 8k f0, k0 , k1 , k2 g, Ik1 ¼ jDk1 , k0 j, Ik2 ¼ jDk2 , k0 j, Ik ¼ k, k0 k1 , k0 k2 , k0 kk1 b Dkk21 ka , k0 Dk2 , k0
(21)
Dk1 k2 Dk2 k Dkk1 8k f0, k0 , k1 , k2 g, Jk1 ¼ jDk1 , k3 j, Jk2 ¼ jDk2 , k3 j, Jk ¼ k, k3 k1 , k3 k2 , k3 kk1 b Dkk21 ka , k3 Dk2 , k3
(22)
In Refs. [21,22], the authors have shown experimentally that such descriptors are complete and stable. The completeness guarantees the uniqueness of matching. The stability gives robustness under nonlinear shape distortions and numerical errors. In Ref. [20], the author demonstrates that the shape space S can be considered as a metric space with a set of metrics. Hence, the Euclidean distance (Equation 23) between the set of the presented invariants can be used to compare the evolving curve and the available templates. dp ðF, HÞ ¼ kfIn ð f Þg fIn ðhÞgklp ¼
X
1=p jIn ð f Þ In ðhÞjp
(23)
For any real number p such that p > 1. Where f and hare two normalized affine arc length reparametrization of two objects having, respectively, the shapes F and H. The shape having the minimum distance according to the evolving active contour is used as template.
3 SHAPE PRIOR FOR GEOMETRIC ACTIVE CONTOURS Geometric active contours are iterative segmentation methods which use the level set approach [2] to determine the evolving front at each iteration. Several models have been proposed in literature that we can classify into edge-based or region-based active contours. In Ref. [4], the level set approach is used to model the shape of objects using an evolving front. The evolution’s equation of the level set function ∅ is ∅t + Fjr∅ j ¼ 0;
(24)
F is a speed function of the form F ¼ F0 + F1(K) where F0 is a constant advection term equals to (1) depends of the object inside or outside the initial contour. The second term is of the form eK where K is the curvature at any point and e > 0. To detect the objects in the image, the authors proposed to use the following function which stops the level set function’s evolution at the object boundaries. gðx, yÞ ¼
1 1 + jrGs f ðx, yÞj
(25)
where f is the image and Gsis a Gaussian filter with a deviation equals to s. This stopping function has values that are closer to zero in regions of high image gradient
4 Experimental results
and values that are closer to unity in regions with relatively constant intensity. Hence, the discrete evolution equation is: ∅n + 1 ði, jÞ ∅n ði, jÞ ¼ gði, jÞFði, jÞjr∅n ði, jÞj △t
(26)
It’s obvious that the evolution is based on the stopping function g which depends on the image gradient. That’s why this model leads to unsatisfactory results in presence of occlusions, low contrast, and even noise. To make the level set function evolve in the regions of variability between the shape of reference and the target shape, we propose the new stopping function as follows:
gshape ðx, yÞ ¼
0, if ∅prod ðx, yÞ 0, sign ð∅ref ðx, yÞÞ, else
(27)
where ∅ prod(x, y) ¼ ∅ (x, y) c˙ ∅ ref(x, y), ∅ is the level set function associated to the evolving contour, while ∅ ref is the level set function associated to the shape of reference after alignment. As it can be seen, the new proposed stopping function only allows for updating the level set function in the regions of variability between shapes. In these regions, gshapeis either 1 or 1 because in the case of partial occlusions, the function is equal to 1 in order to push the edge inward (deflate) and in case of missing parts, this function is equal to 1 to push the contour toward the outside (inflate). This property recalls the Balloon snake’s model proposed by Cohen [23] in which the direction of evolution (inflate or deflate) should be precise from the beginning. In our work, the direction of evolution is handled automatically based on the sign of ∅ ref. The total discrete evolution’s equation that we propose is as follows: ∅n + 1 ði, jÞ ∅n ði, jÞ ¼ wgði, jÞ + ð1 wÞgshape ði, jÞ Fði, jÞjrfn ði, jÞj △t
(28)
w is a weighting factor between the image-based force and knowledge-driven force. See Ref. [16] for our proposed shape prior for a region-based active contours.
4 EXPERIMENTAL RESULTS 4.1 ROBUSTNESS OF THE PROPOSED SHAPE PRIORS We present in Figure 5 an example of successive evolutions between several shapes of different topologies under the proposed shape priors only (w ¼ 0). By the first row, the initial shape is a simple square while the target is a tree leaf. For the second row, the final shape obtained after several iterations in the first row is used as initial contour. The template in this case consists of two contours. This experiment proves that the proposed shape constraint can act on the topology of the original curve. Finally, by the third row, we show the ability of our shape prior to detect objects with holes. This simulation shows that the proposed shape priors can well constrain an active contour to take a given shape (known as reference) and handling nontrivial geometric shapes with holes and complex topologies.
507
508
CHAPTER 33 An affine shape constraint for geometric active
FIGURE 5 Curve evolution under the proposed shape prior only.
4.2 APPLICATION TO OBJECT DETECTION We will devote this section to present some satisfactory results obtained by the proposed model in the case of Euclidean transformations, then the general case of the affine ones will be treated. The algorithm is as follows: We first evolve the active contour without shape prior until convergence (i.e., w ¼ 1) to reduce the computational complexity and to have a good estimation of the parameters of the geometric transformation as in Refs. [8,11]. This first result provides an initialization for the model with prior knowledge. Then the model will evolve under both forces (data and prior forces) with more weight assigned to prior knowledge (generally w 0.5) to promote convergence toward the target shape.
4.2.1 Case of Euclidean transformation To assess the performance of the method, we have experimented the proposed method in medical imaging. The template is given by image 1 of Figure 6 which corresponds to the left ventricle of the heart. The aim is to detect the true contours of the left ventricle of the heart which are partially occluded by the valve. The second row of Figure 6 shows several iterations of curve evolution until convergence without prior knowledge. Starting with final this result, and based on the reference shape after rigid motion estimation, the evolving front continues its evolution until the curve reaches the desired contours (third row). The aligned curves which are used to compute the prior energy are presented in Figure 7. We present by Table 3, the estimated rigid motion parameters.
4 Experimental results
FIGURE 6 First row: The template, test image, initial curve. Second row: The active contours without shape prior. Third row: The active contours with shape prior.
FIGURE 7 First image: Curves before alignment, Second image: Curves after alignment.
509
510
CHAPTER 33 An affine shape constraint for geometric active
Table 3 The Estimated Motion’s Parameters Rotation ( )
Scaling
Difference Between Starting Points
0.802
1.571
1.312
FIGURE 8 Tracking of the mouse in video sequence. First row: The active contours model without shape prior. Second row: The active contours model with shape prior.
We end this part of experiments by applying the active contour with shape prior to object tracking. In Figure 8, the goal is to track the mouse in a video sequence with cluttered background. We suppose that the movement of the mouse through frames is rigid thus the presented method of shape alignment is used. The convergent contour in frame i is used as a template in frame i + 1. Results seem to be satisfactory. Although, the presented rigid motion estimation method solves some situations and leads to interesting detection results. Other types of transformations such as stretching can only be solved by the class of relevant transformations. Thus the goal of the next section is to present some obtained results within this framework.
4.2.2 Case of affine transformation Consider for this first experiment, the spider object, image (a) of Figure 9, obtained from MCD database (url: http://vision.ece.ucsb.edu/zuliani/Research/MCD/MCD. shtml). The (b) image is the object of interest which is obtained from (a) after affine transformation and partial occlusion. As a first step, we perform object (b) segmentation without prior knowledge. The obtained contour is then aligned with the contour of the (a) object to determine the regions of occlusions. We present in (c) the result of shape alignment. The estimated values are a ¼ 2, l0 ¼ 0, A ¼ [0.5, 0.2, 0, 2]. By Figure 10, we present the obtained results without and with shape prior. We
4 Experimental results
FIGURE 9 Spider’s shapes alignment.
FIGURE 10 Detection without, image (a), and with, image (b), affine prior knowledge.
FIGURE 11 Image (a): The available templates. Image (b): The transformed object.
considered in this experiment four templates (a spider, a chopper, a device, and a bird), see Figure 11. We were based on the set of the presented invariant shape descriptors to choose the suitable reference according to the occluded spider (image (b) of Figure 11).
511
512
CHAPTER 33 An affine shape constraint for geometric active
Table 4 Distances Between the Occluded Spider and the Used Templates The occluded spider
Spider
Chopper
Bird
Device
0.037
0.46
0.44
0.53
Table 4 presents the obtained Euclidean distance between the target shape and the available templates. We notice that the minimum distance can be easily identified because the suitable form in this experiment is distinguishable. In the next experiment, a mosaic application is considered. The mosaic images that are considered are taken from the Bardo Museum of Tunisia which contains the biggest collection of mosaic images in the world. In mosaic images, objects are composed of tessellas and are often partially occluded as they are shown by images (c) and (d) (Figure 12). So, given that in mosaic images the objects are often repeated and in order to study the robustness of our method, we try to find to true contour of an occluded object based on another one having the same shape. We have approximated the perspective projection to an affine transformation which is often used in the literature according to the acquisition conditions. Let’s consider the (c) image. As it’s shown, this image contains many forms. Among all forms available in these two images, (c) and (d), taken from two sides, we use the Euclidean distance between the affine invariant Fourier descriptors to localize two lions. The left one is partially occluded. We will use the right lion of the (d) image as template in order to have a better segmentation. In the last figure, images (a) and (b), we present the used curves to perform shape alignment and shape prior computation. The estimated values are a ¼ 1, l0 ¼ 0.52, A ¼ [0.002, 0.003, 2.003, 0.35]. We present by the (c) image the obtained curves alignment result and by the (d) image the segmentation result based on the proposed shape prior. Such results are particularly interesting since it can be used for mosaic images restoration under partial occlusions and missing parts. The underlying idea is to extract similar forms using minimal distance between descriptors in order to define prior knowledge for such occluded or cluttered objects based on the presented affine shape alignment method for the purpose of a good segmentation result (Figure 13).
FIGURE 12 Some mosaic images from the Bardo Museum of Tunisia.
5 Conclusions
FIGURE 13 Robust object detection in mosaic image.
5 CONCLUSIONS We presented in this paper an alternative approach to incorporate prior knowledge into a level set-based active contours in order to have robust object detection in case of large shape distortion that can be analyzed by the class of affine transformations. We presented also a geometric solution to choose the reference shape in case of many available templates, given that the statistical approach needs a training set and PCA. Then the application of a given classifier like Bayesian classifier to determine the appropriate reference shape like the work of Fang and Chan [24]. Given that the proposed approach invokes only pixels of the regions of variability between the shape of reference and the object of interest in the process of curve evolution and based on the fast Fourier transform for affine motion estimation and invariants computation, the method is faster compared to Refs. [15,25] where at each iteration shape descriptors are calculated for a given order that has to be set empirically. The obtained results are promising in the case of real and simulated data and the method can be used for the restoration of mosaic images in the archeological field. As future perspectives, we are working on integrating our model in the context of 3D object reconstruction from silhouettes sequence in order to refine the obtained 3D model.
513
514
CHAPTER 33 An affine shape constraint for geometric active
REFERENCES [1] Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. Int J Comput Vis 1988;1:321–31. [2] Osher S, Sethian JA. Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulation. J Comput Phys 1988;79:12–49. [3] Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis 1997;22:61–79. [4] Malladi R, Sethian J, Vemuri B. Shape modeling with front propagation: a level set approach. PAMI 1995;17:158–75. [5] Chan T, Vese L. Active contours without edges. IEEE Trans Imag Process 2001;10:266–77. [6] Charmi MA, Mezghich MA, M’Hiri S, Derrode S, Ghorbel F. Geometric shape prior to region-based active contours using Fourier-based shape alignment. In: IEEE international conference on imaging systems and techniques; 2010. p. 478–81. [7] Charmi MA, Derrode S, Ghorbel F. Using Fourier-based shape alignment to add geometric prior to snakes. In: International conference on acoustics, speech and signal processing; 2009. p. 1209–12. [8] Foulonneau A, Charbonnier P, Heitz F. Contraintes ge´ome´triques de formes pour les contours actifs oriente´s re´gion: une approche base´e sur les moments de Legendre. Traitement du signal 2004;21:109–27. [9] Mezghich MA, Sellami M, M’Hiri S, Ghorbel F. Shape prior for an edge-based active contours using phase correlation, In: European signal processing conference; 2013. p. 1–5. [10] Leventon M, Grimson E, Faugeras O. Statistical shape influence in geodesic active contours. In: Proc. IEEE conference on computer vision and pattern recognition; 2000. p. 316–23. [11] Fang W, Chan KL. Incorporating shape prior into geodesic active contours for detecting partially occluded object. Pattern Recogn 2007;40:2163–72. [12] Chen Y, Thiruvenkadam S, Tagare HD, Huang F, Wilson D, Geiser EA. On the incorporation of shape priors into geometric active contours. In: Proc. IEEE workshop on variational and level set methods in computer vision; 2001. p. 145–52. [13] Vandergheynst P, Bresson X, Thiran JP. A priori information in image segmentation: energy functional based on shape statistical model and image information, In: International conference on image processing; 2003. p. 425–8. [14] Duay V, Allal A, Houhou N, Lemkaddem A, Thiran JP. Shape prior based on statistical map for active contour segmentation. In: International conference on image processing; 2008. p. 2284–7. [15] Foulonneau A, Charbonnier P, Heitz F. Affine invariant geometric shape priors for region-based active contours. PAMI 2008;8:1352–7. [16] Mezghich MA, M’Hiri S, Ghorbel F. Fourier based multi-references shape prior for regionbased active contours. In: Mediterranean electrotechnical conference; 2012. p. 661–4. [17] Mezghich MA, M’Hiri S, Ghorbel F. Invariant shape prior knowledge for an edge-based active contours. In: International conference on computer vision theory and applications; 2014. p. 454–61. [18] Mokadem A, Avaro O, Ghorbel F, Daoudi M, Sanson H. Global planar rigid motion estimation, applied to object-oriented coding. In: International conference on pattern recognition; 1996. p. 641–5.
References
[19] Persoon E, Fu KS. Shape discrimination using Fourier descriptors. IEEE Trans Syst Man Cybern 1977;7:170–9. [20] Ghorbel F. Towards a unitary formulation for invariant image description: application to image coding. Ann Telecommun 1998;153:145–55. [21] Chaker F, Ghorbel F. Apparent motion estimation using planar contours and Fourier descriptors. In: International conference on computer vision theory and applications; 2010. [22] Ghorbel F, Chaker F, Bannour MT. Contour retrieval and matching by affine invariant Fourier descriptors, In: International association for pattern recognition; 2007. [23] Cohen L. On active contour models and balloons. Graph Model Im Proc 1991;53:211–8. [24] Fang W, Chan KL. Using statistical shape priors in geodesic active contours for robust object detection. In: International conference on pattern recognition; 2006. p. 304–7. [25] Derrode S, Charmi MA, Ghorbel F. Fourier-based shape prior for snakes. Pattern Recogn Lett 2008;29:897–904. [26] Mezghich MA, Saidani M, M’Hiri S, Ghorbel F. An affine shape constraint for geometric active contours. In: International conference on image processing, computer vision, and pattern recognition; 2014.
515
An affine shape constraint for geometric active contours
33
Mohamed Amine Mezghich, Maweheb Saidani, Slim M’Hiri, Faouzi Ghorbel GRIFT Research Group, CRISTAL Laboratory, Ecole Nationale des Sciences de l’Informatique (ENSI), Campus Universitaire de la Manouba, Manouba, Tunisia
1 INTRODUCTION Precise object of interest detection is of high need in many applications of image processing such as medical imaging, tracking of moving object, and 3D object reconstruction. There have been several works in this field of image segmentation and object detection. Among them we can distinguish the classical active contours which are intensity-based models [1–5] and the constrained models by a prior knowledge that are driven both by global geometrical [6–9] or statistical [10–14] shape information and a local image information like gradient or curvature. There have been promising results given that the classical snake models can detect object with smooth contours whereas the level set-based models have the additional ability to detect simultaneously many objects in the image. However, several work of the prior knowledge-driven models reached good segmentation results by detecting partially occluded shapes presenting missing part in low contrast or very noisy image. To define prior knowledge, the authors use either shape alignment [7] or registration [9] between the contour of the target shape and the shape of reference or a distance between invariant shape descriptors like [8]. All these work manage the case of Euclidean transformations (translation, rotation, and scale factor). But in real situations, the object of interest can submit large transformations (affine or projective cases) and distortions. Hence the class of rigid transformations is not appropriate to model this type of movement between the shape of reference and the target one. To our knowledge, there has been only the work of Foulonneau et al. [15] that treats the case of affine transformations. Although this approach, which is based on the distance between an invariant set of Legendre descriptors of the target and reference shapes, presented good results, this method suffers from the instability of the Legendre moments and the order that has to be fixed empirically. Besides, the method requires a heavy execution time to achieve satisfactory results. In order to extend the work of shape priors presented in Refs. [16,17], we propose in this research a geometric approach to manage the class of affine Emerging Trends in Image Processing, Computer Vision and Pattern Recognition http://dx.doi.org/10.1016/B978-0-12-802045-6.00033-8 © 2015 Elsevier Inc. All rights reserved.
499
500
CHAPTER 33 An affine shape constraint for geometric active
transformations [26] that can occur between the shape of interest and the reference one. Prior knowledge is obtained by aligning the evolving contour and the template using an alignment method which is based on Fourier descriptors. For this purpose, we will start by introducing our approach in the case of rigid transformations. Then we generalize the idea by treating the case of affine ones that often occur in real situations. We will also propose a geometric method based on a set of complete and stable set of invariant shape descriptors to manage the case of mutireferences. Combining the proposed shape prior information with the level set-based model, the improved model can retain all the advantages of the active contour model and have the additional ability of being able to represent the global shape of an object. The remainder of this paper is organized as follows: In Section 2, a description of the outlines of the used shape alignment method in both cases (Euclidean and affine transformations) will be presented. In Section 3, we will briefly recall the principle of the used edge-based active contours, then we will move to present the proposed shape prior. Experimental results are presented and commented in Section 4. Finally, we conclude the work and highlight some possible perspectives in Section 5.
2 SHAPE ALIGNMENT USING FOURIER DESCRIPTORS In this section, we recall the used alignment methods to define the proposed energy in both cases rigid and affine transformations.
2.1 EUCLIDEAN SHAPES ALIGNMENT It is well known that the closed contour of a planar object can be given by its parametric representation g : ½0,2p ! C t ! xðtÞ + iyðtÞ
(1)
Let g1and g2 be centered (according to the center of mass) and normalized arc length parameterizations of two closed planar curves having shapes F1and F2. Suppose that g1 and g2 have a similar shape under rigid transformation. In shape space this is equivalent to have: g1 ðtÞ ¼ aeiy g2 ðt + t0 Þ
(2)
where a is the scaling factor, y the rotation angle, t0 is the difference between the two starting points of g1 and g2. Rigid motion estimation is obtained by minimizing the following quantity introduced by [18]: Err ðg1 , g2 Þ ¼ min ða, y, t0 Þ g1 ðtÞ aeiy g2 ðt + t0 ÞL2
(3)
2 Shape alignment using fourier descriptors
If ais equals to 1 [19], showed that Equation (3) is a metric in shape space and corresponds to the Hausdorff distance between the two shapes. In practice, this result implies the uniqueness of the motion’s parameters so obtained. Using Fourier descriptors Ck(g1) and Ck(g2), minimizing Err becomes equivalent to minimize f(y, t0) in Fourier domain f ðy, t0 Þ ¼
X Ck ðg1 Þ aeiðkt0 + yÞ Ck ðg2 Þ2
(4)
k2Z
[18] proposed an analytical solution to compute t0 and y. t0 is one of the zeros of the following function: gðtÞ ¼
X
X rk sin ðck + ktÞ krk cos ðck + ktÞ kX k X krk sin ðck + ktÞ rk cos ðck + ktÞ k
ick
(5)
k
¼ Ck ðg1 ÞCk ðg2 Þ, y is chosen to satisfy Equation (6) and minimize f(y, t0)
where rk e where t0 is one of the roots of Equation (5).
P r sin ðck + kt0 Þ P tany ¼ k k r k k cos ðck + kt0 Þ
(6)
Having the value of y and t0 that minimize f(y, t0), the scaling factor is given by P
a¼
rk cos ðck + kt0 + yÞ kP k Ck ðg2 ÞCk ðg2 Þ
(7)
Based on Nyquist-Shannon theorem and B-spline approximation of the g(t)’s curve, Persoon and Fu [19] proposed the first implementation of this method of shape alignment and its application in rigid motion estimation for video compression. In Figure 1, we give an example of two aligned curves with the used method. The estimated motion’s parameters are presented in Table 1. The associated g(t)’s curve is presented in Figure 2. The zeros of g(t) are drawn with circles.
FIGURE 1 Example of two curves before (a) and after (b) shape alignment.
501
502
CHAPTER 33 An affine shape constraint for geometric active
Table 1 The Estimated Parameters of Synthetic Rigid Motion Rotation ( )
Scaling
Difference Between Starting Point
45
1
0
FIGURE 2 Example g(l)’s curve.
2.2 AFFINE SHAPE ALIGNMENT In order to define affine invariant shape prior from a reference shape, we have to perform shape matching between the target and the template shapes F1 and F2. To correctly estimate the affine motion parameters, we have to deal with the same number of points which is not relevant if we consider two different points of view. For this purpose, in Ref. [20], the author proposes to use the affine reparametrization of curve which is invariant under affine transformation. We will start by presenting the affine reparametrization procedure of a given curve. Then we will describe our method of affine motion’s parameters estimation.
2.2.1 Reparametrization of closed curve If we take the same object in two different camera sides, we found a different number of contour points in each image. Consequently, we proceed by normalization of curves under affine transformation. Given the shape front, its edge pixels are
2 Shape alignment using fourier descriptors
extracted and traversed to yield a discrete closed curve which is a parametric equation g(t) ¼ (u(t), v(t)), where t 2 {0,. . .,N 1} and g(N) ¼ g(0). We use the affine arc length reparametrization to normalize closed curves under affine transformations that can be expressed by: sa ð t Þ ¼
ð 2 1 0 ^ 00 3 g ðtÞ g ðtÞ dt, t 2 ½a, b La
La ¼
(8)
ð b rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ^ 3 g0 ðtÞ g00 ðtÞdt
(9)
a
where La is the total affine arc length of the considered curve, ^ represents the cross product between two vectors and, jjjj denotes the Euclidean norm.
2.2.2 Contours alignment using geometrical affine parameters estimation We consider two closed curves O1and O2which define the same shape. O1and O2 are said to be related by an affine transformation if and only if: hðlÞ ¼ aAf ðl + l0 Þ + B
(10)
where Bis a translation vector, A is a linear transformation, l0 is the shift value, f and h are the affine reparametrization of two contours having the same affine shape. In the Fourier space we get: Uk ðhÞ ¼ ae2ipkl0 AUk ð f Þ + bdk
(11)
where Uk(h) and Uk(f) are, respectively, the Fourier coefficients of f and h. So estimation of affine motion can be resumed to the estimation of its three parameters: the affine matrix A, the shift value l0, and finally the scale factor a if we consider a normalization under translation.
Estimation of the scale factor a The scale factor can be estimated using the following formula: a2 ¼
det Uk ðhÞ, Uk ðhÞ det Uk ð f Þ, Uk ð f Þ
(12)
where ðUk ðhÞ, Uk∗ ðhÞÞ and ðUk ðf Þ, Uk∗ ðf ÞÞ are 2 2 matrix formed by Fourier coefficients on some fixed index k and U* is the U complex conjugate.
Computation of the shift value l0
Let’s consider M1 ¼ ðUk1 ðhÞ, Uk2 ðhÞÞ and M2 ¼ ðUk1 ð f Þ, Uk2 ð f ÞÞ l0 ¼
arg ð det ðM2 ÞÞ arg ð det ðM1 ÞÞ ðk1 + k2 Þ
(13)
where k1 and k2 are two fixed index and arg(U) is the complex argument of U.
503
504
CHAPTER 33 An affine shape constraint for geometric active
Computation of the matrix A’s parameters Since scale factor a and shift value l0have been previously estimated (Equations 12 and 13), we are going to estimate affine matrix A parameters. Lets O1, O2 be two curves and Uf, Uh, respectively, their Fourier coefficients. In Fourier space we have (Equation 14) Uk ðhÞ ¼ ae2ipkl0 AUk ð f Þ
Using matrix, we have the following formula:
u2 ðkÞ v 2 ðk Þ
¼ ae2ipkl0
a1 a2 a3 a4
(14)
u1 ðkÞ v 1 ðk Þ
(15)
So we have to estimate a1, a2, a3, and a4. This system can be represented as following: 8 u2 ðk1 Þ ¼ ae2ipk1 l0 a1 u1 ðk1 Þ + e2ipk1 l0 a2 v1 ðk1 Þ > > > 2ipk1 l0 > a3 u1 ðk1 Þ + e2ipk1 l0 a4 v1 ðk1 Þ > < v2 ðk1 Þ ¼ ae .. . > > > > u2 ðkn Þ ¼ ae2ipkn l0 a1 u1 ðkn Þ + e2ipkn l0 a2 v1 ðkn Þ > : v2 ðkn Þ ¼ ae2ipkn l0 a3 u1 ðkn Þ + e2ipkn l0 a4 v1 ðkn Þ
(16)
Then to estimate matrix A’s parameters (Equation 14), we have to resolve a system with 2N equations and four unknown parameters that can be written as K2N4 A4 ¼ U2N
(17)
The solution of the system (Equation 16) can be obtained by KA U ¼ e
(18)
So we have to minimize the quadratic error e by using the pseudoinverse of K. We show by the following figure an example of affine motion estimation between two synthetic curves after contours resampling (Figure 3). The final result of curves alignment after affine motion estimation is presented by Figure 4. The estimated motion parameters are presented in Table 2.
2.3 DISCUSSION Having the parameters of the affine or Euclidean transformations between the two contours, we perform the alignment of the two curves to determine the regions of variability between shapes by computing the product function of the signed distance
FIGURE 3 Two different shapes after affine deformations ((a) and (c)) and curve resampling ((b) and (d)).
2 Shape alignment using fourier descriptors
FIGURE 4 Results of affine shape alignment.
Table 2 The Estimated Affine Motion Parameters l0
a
Shapes of (b)
0.002
1.003
Shapes of (d)
0
2.004
A
0:860 0:47 0:52 0:865 0:962 0:002 0:1037 0:966
functions associated to level set functions of, respectively, the target and the reference object after alignment given by fprod ðx, yÞ ¼ f∅ref ðx, yÞ f∅ ðx, yÞ
(19)
where f∅ref and f∅ are the two binary images associated, respectively, to ∅ ref and ∅. See Ref. [16] for more details. By construction, the product function fprod is negative in the areas of variability between the two binary images f∅ref and f∅ due to occlusion, clutter, or missing parts, whereas in positive regions, the objects are similar. Thus, in what follows, we propose to update the level set function ∅, only in regions of variability between shapes to make the evolving contours overpass the spurious edges and recover the desired shapes of objects. This property recalls the narrow band technique used to accelerate the evolution of the level set functions [4].
2.4 GLOBAL MATCHING USING AFFINE INVARIANTS DESCRIPTORS In presence of many templates, we have to choose the most suitable one according to the evolving curve. Let aand b be positive real numbers, and k0, k1, k2, and k3 four positive integers. Let Cxn and Cyn be the complex Fourier coefficients of the coordinates (u, v). △ denotes the determinant. x x C C ny m △m ¼ n C Cy n m
(20)
505
506
CHAPTER 33 An affine shape constraint for geometric active
In Ref. [20], the author introduced two sets of invariant descriptors I and J which are, respectively, given by (Equations 21 and 22). Dk1 k2 Dk2 k Dkk1 8k f0, k0 , k1 , k2 g, Ik1 ¼ jDk1 , k0 j, Ik2 ¼ jDk2 , k0 j, Ik ¼ k, k0 k1 , k0 k2 , k0 kk1 b Dkk21 ka , k0 Dk2 , k0
(21)
Dk1 k2 Dk2 k Dkk1 8k f0, k0 , k1 , k2 g, Jk1 ¼ jDk1 , k3 j, Jk2 ¼ jDk2 , k3 j, Jk ¼ k, k3 k1 , k3 k2 , k3 kk1 b Dkk21 ka , k3 Dk2 , k3
(22)
In Refs. [21,22], the authors have shown experimentally that such descriptors are complete and stable. The completeness guarantees the uniqueness of matching. The stability gives robustness under nonlinear shape distortions and numerical errors. In Ref. [20], the author demonstrates that the shape space S can be considered as a metric space with a set of metrics. Hence, the Euclidean distance (Equation 23) between the set of the presented invariants can be used to compare the evolving curve and the available templates. dp ðF, HÞ ¼ kfIn ð f Þg fIn ðhÞgklp ¼
X
1=p jIn ð f Þ In ðhÞjp
(23)
For any real number p such that p > 1. Where f and hare two normalized affine arc length reparametrization of two objects having, respectively, the shapes F and H. The shape having the minimum distance according to the evolving active contour is used as template.
3 SHAPE PRIOR FOR GEOMETRIC ACTIVE CONTOURS Geometric active contours are iterative segmentation methods which use the level set approach [2] to determine the evolving front at each iteration. Several models have been proposed in literature that we can classify into edge-based or region-based active contours. In Ref. [4], the level set approach is used to model the shape of objects using an evolving front. The evolution’s equation of the level set function ∅ is ∅t + Fjr∅ j ¼ 0;
(24)
F is a speed function of the form F ¼ F0 + F1(K) where F0 is a constant advection term equals to (1) depends of the object inside or outside the initial contour. The second term is of the form eK where K is the curvature at any point and e > 0. To detect the objects in the image, the authors proposed to use the following function which stops the level set function’s evolution at the object boundaries. gðx, yÞ ¼
1 1 + jrGs f ðx, yÞj
(25)
where f is the image and Gsis a Gaussian filter with a deviation equals to s. This stopping function has values that are closer to zero in regions of high image gradient
4 Experimental results
and values that are closer to unity in regions with relatively constant intensity. Hence, the discrete evolution equation is: ∅n + 1 ði, jÞ ∅n ði, jÞ ¼ gði, jÞFði, jÞjr∅n ði, jÞj △t
(26)
It’s obvious that the evolution is based on the stopping function g which depends on the image gradient. That’s why this model leads to unsatisfactory results in presence of occlusions, low contrast, and even noise. To make the level set function evolve in the regions of variability between the shape of reference and the target shape, we propose the new stopping function as follows:
gshape ðx, yÞ ¼
0, if ∅prod ðx, yÞ 0, sign ð∅ref ðx, yÞÞ, else
(27)
where ∅ prod(x, y) ¼ ∅ (x, y) c˙ ∅ ref(x, y), ∅ is the level set function associated to the evolving contour, while ∅ ref is the level set function associated to the shape of reference after alignment. As it can be seen, the new proposed stopping function only allows for updating the level set function in the regions of variability between shapes. In these regions, gshapeis either 1 or 1 because in the case of partial occlusions, the function is equal to 1 in order to push the edge inward (deflate) and in case of missing parts, this function is equal to 1 to push the contour toward the outside (inflate). This property recalls the Balloon snake’s model proposed by Cohen [23] in which the direction of evolution (inflate or deflate) should be precise from the beginning. In our work, the direction of evolution is handled automatically based on the sign of ∅ ref. The total discrete evolution’s equation that we propose is as follows: ∅n + 1 ði, jÞ ∅n ði, jÞ ¼ wgði, jÞ + ð1 wÞgshape ði, jÞ Fði, jÞjrfn ði, jÞj △t
(28)
w is a weighting factor between the image-based force and knowledge-driven force. See Ref. [16] for our proposed shape prior for a region-based active contours.
4 EXPERIMENTAL RESULTS 4.1 ROBUSTNESS OF THE PROPOSED SHAPE PRIORS We present in Figure 5 an example of successive evolutions between several shapes of different topologies under the proposed shape priors only (w ¼ 0). By the first row, the initial shape is a simple square while the target is a tree leaf. For the second row, the final shape obtained after several iterations in the first row is used as initial contour. The template in this case consists of two contours. This experiment proves that the proposed shape constraint can act on the topology of the original curve. Finally, by the third row, we show the ability of our shape prior to detect objects with holes. This simulation shows that the proposed shape priors can well constrain an active contour to take a given shape (known as reference) and handling nontrivial geometric shapes with holes and complex topologies.
507
508
CHAPTER 33 An affine shape constraint for geometric active
FIGURE 5 Curve evolution under the proposed shape prior only.
4.2 APPLICATION TO OBJECT DETECTION We will devote this section to present some satisfactory results obtained by the proposed model in the case of Euclidean transformations, then the general case of the affine ones will be treated. The algorithm is as follows: We first evolve the active contour without shape prior until convergence (i.e., w ¼ 1) to reduce the computational complexity and to have a good estimation of the parameters of the geometric transformation as in Refs. [8,11]. This first result provides an initialization for the model with prior knowledge. Then the model will evolve under both forces (data and prior forces) with more weight assigned to prior knowledge (generally w 0.5) to promote convergence toward the target shape.
4.2.1 Case of Euclidean transformation To assess the performance of the method, we have experimented the proposed method in medical imaging. The template is given by image 1 of Figure 6 which corresponds to the left ventricle of the heart. The aim is to detect the true contours of the left ventricle of the heart which are partially occluded by the valve. The second row of Figure 6 shows several iterations of curve evolution until convergence without prior knowledge. Starting with final this result, and based on the reference shape after rigid motion estimation, the evolving front continues its evolution until the curve reaches the desired contours (third row). The aligned curves which are used to compute the prior energy are presented in Figure 7. We present by Table 3, the estimated rigid motion parameters.
4 Experimental results
FIGURE 6 First row: The template, test image, initial curve. Second row: The active contours without shape prior. Third row: The active contours with shape prior.
FIGURE 7 First image: Curves before alignment, Second image: Curves after alignment.
509
510
CHAPTER 33 An affine shape constraint for geometric active
Table 3 The Estimated Motion’s Parameters Rotation ( )
Scaling
Difference Between Starting Points
0.802
1.571
1.312
FIGURE 8 Tracking of the mouse in video sequence. First row: The active contours model without shape prior. Second row: The active contours model with shape prior.
We end this part of experiments by applying the active contour with shape prior to object tracking. In Figure 8, the goal is to track the mouse in a video sequence with cluttered background. We suppose that the movement of the mouse through frames is rigid thus the presented method of shape alignment is used. The convergent contour in frame i is used as a template in frame i + 1. Results seem to be satisfactory. Although, the presented rigid motion estimation method solves some situations and leads to interesting detection results. Other types of transformations such as stretching can only be solved by the class of relevant transformations. Thus the goal of the next section is to present some obtained results within this framework.
4.2.2 Case of affine transformation Consider for this first experiment, the spider object, image (a) of Figure 9, obtained from MCD database (url: http://vision.ece.ucsb.edu/zuliani/Research/MCD/MCD. shtml). The (b) image is the object of interest which is obtained from (a) after affine transformation and partial occlusion. As a first step, we perform object (b) segmentation without prior knowledge. The obtained contour is then aligned with the contour of the (a) object to determine the regions of occlusions. We present in (c) the result of shape alignment. The estimated values are a ¼ 2, l0 ¼ 0, A ¼ [0.5, 0.2, 0, 2]. By Figure 10, we present the obtained results without and with shape prior. We
4 Experimental results
FIGURE 9 Spider’s shapes alignment.
FIGURE 10 Detection without, image (a), and with, image (b), affine prior knowledge.
FIGURE 11 Image (a): The available templates. Image (b): The transformed object.
considered in this experiment four templates (a spider, a chopper, a device, and a bird), see Figure 11. We were based on the set of the presented invariant shape descriptors to choose the suitable reference according to the occluded spider (image (b) of Figure 11).
511
512
CHAPTER 33 An affine shape constraint for geometric active
Table 4 Distances Between the Occluded Spider and the Used Templates The occluded spider
Spider
Chopper
Bird
Device
0.037
0.46
0.44
0.53
Table 4 presents the obtained Euclidean distance between the target shape and the available templates. We notice that the minimum distance can be easily identified because the suitable form in this experiment is distinguishable. In the next experiment, a mosaic application is considered. The mosaic images that are considered are taken from the Bardo Museum of Tunisia which contains the biggest collection of mosaic images in the world. In mosaic images, objects are composed of tessellas and are often partially occluded as they are shown by images (c) and (d) (Figure 12). So, given that in mosaic images the objects are often repeated and in order to study the robustness of our method, we try to find to true contour of an occluded object based on another one having the same shape. We have approximated the perspective projection to an affine transformation which is often used in the literature according to the acquisition conditions. Let’s consider the (c) image. As it’s shown, this image contains many forms. Among all forms available in these two images, (c) and (d), taken from two sides, we use the Euclidean distance between the affine invariant Fourier descriptors to localize two lions. The left one is partially occluded. We will use the right lion of the (d) image as template in order to have a better segmentation. In the last figure, images (a) and (b), we present the used curves to perform shape alignment and shape prior computation. The estimated values are a ¼ 1, l0 ¼ 0.52, A ¼ [0.002, 0.003, 2.003, 0.35]. We present by the (c) image the obtained curves alignment result and by the (d) image the segmentation result based on the proposed shape prior. Such results are particularly interesting since it can be used for mosaic images restoration under partial occlusions and missing parts. The underlying idea is to extract similar forms using minimal distance between descriptors in order to define prior knowledge for such occluded or cluttered objects based on the presented affine shape alignment method for the purpose of a good segmentation result (Figure 13).
FIGURE 12 Some mosaic images from the Bardo Museum of Tunisia.
5 Conclusions
FIGURE 13 Robust object detection in mosaic image.
5 CONCLUSIONS We presented in this paper an alternative approach to incorporate prior knowledge into a level set-based active contours in order to have robust object detection in case of large shape distortion that can be analyzed by the class of affine transformations. We presented also a geometric solution to choose the reference shape in case of many available templates, given that the statistical approach needs a training set and PCA. Then the application of a given classifier like Bayesian classifier to determine the appropriate reference shape like the work of Fang and Chan [24]. Given that the proposed approach invokes only pixels of the regions of variability between the shape of reference and the object of interest in the process of curve evolution and based on the fast Fourier transform for affine motion estimation and invariants computation, the method is faster compared to Refs. [15,25] where at each iteration shape descriptors are calculated for a given order that has to be set empirically. The obtained results are promising in the case of real and simulated data and the method can be used for the restoration of mosaic images in the archeological field. As future perspectives, we are working on integrating our model in the context of 3D object reconstruction from silhouettes sequence in order to refine the obtained 3D model.
513
514
CHAPTER 33 An affine shape constraint for geometric active
REFERENCES [1] Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. Int J Comput Vis 1988;1:321–31. [2] Osher S, Sethian JA. Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulation. J Comput Phys 1988;79:12–49. [3] Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis 1997;22:61–79. [4] Malladi R, Sethian J, Vemuri B. Shape modeling with front propagation: a level set approach. PAMI 1995;17:158–75. [5] Chan T, Vese L. Active contours without edges. IEEE Trans Imag Process 2001;10:266–77. [6] Charmi MA, Mezghich MA, M’Hiri S, Derrode S, Ghorbel F. Geometric shape prior to region-based active contours using Fourier-based shape alignment. In: IEEE international conference on imaging systems and techniques; 2010. p. 478–81. [7] Charmi MA, Derrode S, Ghorbel F. Using Fourier-based shape alignment to add geometric prior to snakes. In: International conference on acoustics, speech and signal processing; 2009. p. 1209–12. [8] Foulonneau A, Charbonnier P, Heitz F. Contraintes ge´ome´triques de formes pour les contours actifs oriente´s re´gion: une approche base´e sur les moments de Legendre. Traitement du signal 2004;21:109–27. [9] Mezghich MA, Sellami M, M’Hiri S, Ghorbel F. Shape prior for an edge-based active contours using phase correlation, In: European signal processing conference; 2013. p. 1–5. [10] Leventon M, Grimson E, Faugeras O. Statistical shape influence in geodesic active contours. In: Proc. IEEE conference on computer vision and pattern recognition; 2000. p. 316–23. [11] Fang W, Chan KL. Incorporating shape prior into geodesic active contours for detecting partially occluded object. Pattern Recogn 2007;40:2163–72. [12] Chen Y, Thiruvenkadam S, Tagare HD, Huang F, Wilson D, Geiser EA. On the incorporation of shape priors into geometric active contours. In: Proc. IEEE workshop on variational and level set methods in computer vision; 2001. p. 145–52. [13] Vandergheynst P, Bresson X, Thiran JP. A priori information in image segmentation: energy functional based on shape statistical model and image information, In: International conference on image processing; 2003. p. 425–8. [14] Duay V, Allal A, Houhou N, Lemkaddem A, Thiran JP. Shape prior based on statistical map for active contour segmentation. In: International conference on image processing; 2008. p. 2284–7. [15] Foulonneau A, Charbonnier P, Heitz F. Affine invariant geometric shape priors for region-based active contours. PAMI 2008;8:1352–7. [16] Mezghich MA, M’Hiri S, Ghorbel F. Fourier based multi-references shape prior for regionbased active contours. In: Mediterranean electrotechnical conference; 2012. p. 661–4. [17] Mezghich MA, M’Hiri S, Ghorbel F. Invariant shape prior knowledge for an edge-based active contours. In: International conference on computer vision theory and applications; 2014. p. 454–61. [18] Mokadem A, Avaro O, Ghorbel F, Daoudi M, Sanson H. Global planar rigid motion estimation, applied to object-oriented coding. In: International conference on pattern recognition; 1996. p. 641–5.
References
[19] Persoon E, Fu KS. Shape discrimination using Fourier descriptors. IEEE Trans Syst Man Cybern 1977;7:170–9. [20] Ghorbel F. Towards a unitary formulation for invariant image description: application to image coding. Ann Telecommun 1998;153:145–55. [21] Chaker F, Ghorbel F. Apparent motion estimation using planar contours and Fourier descriptors. In: International conference on computer vision theory and applications; 2010. [22] Ghorbel F, Chaker F, Bannour MT. Contour retrieval and matching by affine invariant Fourier descriptors, In: International association for pattern recognition; 2007. [23] Cohen L. On active contour models and balloons. Graph Model Im Proc 1991;53:211–8. [24] Fang W, Chan KL. Using statistical shape priors in geodesic active contours for robust object detection. In: International conference on pattern recognition; 2006. p. 304–7. [25] Derrode S, Charmi MA, Ghorbel F. Fourier-based shape prior for snakes. Pattern Recogn Lett 2008;29:897–904. [26] Mezghich MA, Saidani M, M’Hiri S, Ghorbel F. An affine shape constraint for geometric active contours. In: International conference on image processing, computer vision, and pattern recognition; 2014.
515