- Email: [email protected]
Multiprimitive segmentation based on meaningful breakpoints for fitting digital planar curves with line segments and conic arcs
Image and Vision Computing 23 (2005) 783–789 www.elsevier.com/locate/imavis
Multiprimitive segmentation based on meaningful breakpoints for fitting digital planar curves with line segments and conic arcs Wu-Chih Hu* Department of Computer Science and Information Engineering, National Penghu Institute of Technology, #300, Liu-Ho Road, Makung City, Penghu, Taiwan, ROC Received 30 January 2004; received in revised form 26 April 2005; accepted 5 May 2005
Abstract This paper presents a multiprimitive segmentation method with line segments and conic arcs based on the types of breakpoints. In this method, a joint tuning procedure is proposed to merge consecutive segments and adjust their locations to achieve more accurate and stable conic arcs. No threshold is required in the multiprimitive segmentation by using the proposed scheme. And, the types of breakpoints among line segments and conic arcs are defined and they are useful and meaningful for pattern recognition and shape analysis. Besides, the computational complexity of the proposed method is O(n log n) which is lower than most other conic fitting methods. Further, the concept of types of breakpoints can be easily extended to other primitives. q 2005 Elsevier B.V. All rights reserved. Keywords: Multiprimitive; Segmentation; Conic arcs; Breakpoint
1. Introduction Segmentation of planar curves is one of the most important jobs in image processing since a segmented contour can be used to describe an object in a meaningful and compact form for higher level vision processing, such as pattern recognition and shape analysis. The objective is to use a minimum number of primitive pieces to approximate curves with minimum distortion. Many techniques have been proposed for this purpose in the past two decades. Polygonal approximation is the simplest approach. The line segments are almost extracted from curves based on the corner detection [1–6] or dominant point detection [7–10]. But polygonal approximation is rarely used for further shape analysis. To go from segmentation to shape analysis, one could include higher order primitives such as circular arcs, elliptic arcs, splines etc. in the segmentation. Normally, longer computation time will be involved in segmentation with such primitives since more parameters need to be solved. Hence, curve segmentation using line segments and * Tel./fax: C886 6 9277361. E-mail address: [email protected].
0262-8856/$ - see front matter q 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2005.05.004
circular arcs is selected, and it is a better representation than polygonal approximation. Thus, many methods were proposed to obtain line segments and circular arcs [11– 14]. The main reason is that a circular arc is easily obtained based on three parameters (center (x0,y0) and radius r). High order analytic curves such as conic arcs (circular, elliptic, parabolic, and hyperbolic arcs) can produce more accurate representations than only using circular arcs. Thus, the conic arcs are considered in this research because of computation time and flexibility of description. Needless to say, conic arcs in our daily life and in industry are important. Some techniques for conic fitting were proposed [15–18]. The least-squares method is often applied to fitting of conic arcs. Least-square fitting minimizes the square sum of errorof-fit in predefined measures, and two main categories (algebraic and geometric fitting) are differentiated by the respective definition of the error distance. In algebraic fitting, a conic arc is described by implicit equation, and the error distance is defined with deviation of the implicit equation from the expected value at each given point. And, in geometric fitting, the error distance is defined with the orthogonal distance from the given point to the fitted conic arc. These approaches are almost focused on accurate and efficient the result of fitting conic arcs. However, these results of conic fitting are not enough to employ on pattern
784
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
recognition and shape analysis. That is, the fitting result is only flexible for curve description, but the shape analysis is not further obtained by these conic pieces (such as conic piece indexing). Hence, the multiprimitive segmentation based on conic piece indexing is very necessary to analyze the object shape. We proposed a method for segmenting curves into line segments and circular arcs by using types of breakpoints [13]. The advantages of this technique are that it is: † Threshold-free–No threshold within the algorithm, † Stable–Invariant to transformations of the data (rotation, translation, and scale), and † Extendible—Types of breakpoints can be extended to other primitives. Hence, using the types of breakpoints for fitting curves with line segments and conic arcs is proposed in this research. The proposed scheme is a modified/extended method based on the breakpoint classification and tuning approach [13]. In the proposed scheme, the breakpoints are first categorized as five types: c-ll, c-la, c-aa, s-la, and s-aa by using AKC function and PHF [13], where c indicates a corner and s is a smooth joint. Subsequently, the procedure of line/conic segmentation and merging is employed based on the above types of breakpoints. The breakpoints are further categorized as 29 types: c-ll, c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, c-hh, s-lc, s-le, s-lp, s-lh, s-cc, s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. And, these breakpoints are defined as meaningful breakpoints in this research. These meaningful breakpoints are very useful for describing planar curves, pattern recognition, and shape analysis. Hence, the main contribution of this paper is to present the concept of meaningful breakpoints to give conic pieces indexing for shape analysis. This concept proposed in this paper is not referred in the existent methods for conic fitting. In the remainder of this paper, the breakpoint classification is presented in Section 2 while multiprimitive segmentation based on lines and conic arcs is described in Section 3. In Section 4, the merging of conic arcs is proposed. Section 5 presents experimental examples and evaluations of the results. Finally, conclusion is made in Section 6.
2. Breakpoint classification In this section, the associated breakpoints are first detected by using the methods of dominant point or corner detection [1–10], and then the breakpoints are categorized as five types: corner-ll, corner-la, corner-aa, smooth joint-la or smooth joint-aa by using AKC (adaptive k-curvature) function and PHF (projective height function) [13].
The type ll means that the segments on both sides of the breakpoint are line segments; la stands for a joint of a line segment and an arc; aa represents a joint of two arcs; and corner is a discontinuous tangent and smooth joint is associated with continuous tangent, but discontinuous curvature [19]. The AKC function is briefly described as follows. Consider three consecutive breakpoints PiK1, Pi and PiC1, and two consecutive segments Si and SiC1 starting at PiK1, joining at Pi and ending at PiC1. Let lengths of Si and SiC1 be li and liC1, respectively, and kZmin(li,liC1). And, let the ~ Pi C k, ~ where kZ ~ k=2 region-of-support for Pi be ½Pi K k; ~ ~ and define the k-vector at a point Pj 2½Pi K k; Pi C k as aÐjk Z ðPjC ðxÞ K Pj ðxÞ;
PjC ðyÞ K Pj ðyÞÞ
(1a)
bÐjk Z ðPjK ðxÞ K Pj ðxÞ;
PjK ðyÞ K Pj ðyÞÞ
(1b)
~ and then the k-cosine where PjC Z Pj C k~ and PjK Z Pj K k, Ð Ð between a jk and b jk is cjk Z
aÐjk ,bÐjk kÐ a jk kkbÐjk k
(2)
The AKC function of Pi is defined as the values cjk ~ Pi C k. ~ cPj 2½Pi K k; And, the PHF is described as follows. Let the intervals for evaluating the projective heights (PHs) of the segments ~ Pi K k=2 ~ on both sides of the breakpoint Pi be ½Pi K 3k=2; ~ Pi C 3k=2 ~ for the segment for the segment Si, and ½Pi C k=2; SiC1, which insure that the PH evaluation does not exceed either PiK1 and Pi or Pi and PiC1. ~ Pi K k=2, ~ Consider a point Pj 2½Pi K 3k=2; and the ~ and P^ jC Z Pj C k=2. ~ associated endpoints P^ jK Z Pj K k=2 Then, the parametric form for the line segment between the endpoints is ( x Z P^ jK ðxÞ C lx t ; t 2½0; 1 (3) y Z P^ jK ðyÞ C ly t where lx Z P^ jC ðxÞK P^ jK ðxÞ and ly Z P^ jC ðyÞK P^ jK ðyÞ, and the projective height of Pj is sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ½lx ðPj ðyÞ K P^ jK ðyÞÞ K ly ðPj ðxÞ K P^ jK ðxÞÞ2 hðPj Þ Z (4) l2x C l2y The PHF is defined as the projective heights h(Pj) over ~ Pi K k=2 ~ or ½Pi C k=2; ~ Pi C 3k=2. ~ the interval of ½Pi K 3k=2; Define a line accumulator and an arc accumulator on the region of support of PHF and use 0.5 to discriminate a line from an arc, since it is the maximum possible deviation of the PH of a digitized line segment can make. If h(Pj)!0.5, then the line accumulator is incremented by one; otherwise add one to the arc accumulator. The type of segment is determined by comparing the values in the two accumulators. If the value in the line accumulator is greater, then the segment is identified as a line; otherwise it is an arc.
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
785
Fig. 2. Recovery of type s-la joints in the {c-ll, s-aa} case.
Fig. 1. Types of corner and smooth joint.
The procedure of breakpoint classification is described as follows. The breakpoints can be categorized as corners and smooth joints by testing a global maximum in the AKC function. Clearly, if there is a global maximum at the breakpoint in the AKC function, then it is a corner; otherwise it is a smooth joint. And, using the PHF scheme, the types of segments on both sides of a breakpoint can be efficiently detected and a breakpoint can be further divided into types ll, la (or al), and aa. Combining the results of AKC and PHF schemes, the breakpoints are categorized as five types: corner-ll (c-ll), corner-la (c-la), corner-aa (c-aa), smooth joint-la (s-la) and smooth joint-aa (s-aa). For understanding the geometry of corner and smooth joint, the breakpoint types can be illustrated clearly from Fig. 1. Fig. 1(a) is c-ll, Fig. 1(b) and Fig. 1(c) are c-la, Fig. 1(d)–(f) are c-aa, Fig. 1(g) is s-la, and Fig. 1(h) and 1(i) are s-aa breakpoints. The computational complexity of breakpoint classification is easily seen to be in O(n), where n is the number of points in curves.
3. Multiprimitive segmentation Segmentation of a planar curve into line segments and conic arcs is easily achieved if the types of breakpoints are identified. In this section, segmentation of lines and conic arcs is described in Sections 3.1 and 3.2, respectively.
etc. Practically, however, due to the errors in breakpointdetection, two situations may be brought about. The first case is that there is a type c-ll joint followed by a type c-aa (or s-aa) joint immediately, and there must be a type s-la joint between these two consecutive joints. To determine the location of such a type s-la joint, form a straight line by connecting the type c-ll and c-aa (or s-aa) joints. The point in the original segment that is farthest from this connected line segment is then considered as such a type s-la joint, following the idea by Duda and Hart [20]. The result of the first case is shown in Fig. 2. The second case is that there is a type c-la joint surrounded by two types c-aa (or s-aa) joint immediately, then there must be a type s-la joint preceding or following the type c-la joint. The location of such a type s-la joint can be obtained by using the PHF of the type c-la joint. That is, if there is a line segment on the forward side of c-la joint, then the new s-la joint should be on the forward segment, and vice versa. The exact location of the new s-la joint is also determined at the point where the distance from the line connecting the c-la and c-aa (or s-aa) joints is the maximum. The result of the second case is shown in Fig. 3. Using the above schemes, line segments in planar curves are extracted successfully and only conic arcs are not treated. 3.2. Segmentation of conic arcs After the breakpoint classification and line extraction, the preserved parts are only arc segments. Hence, the multiprimitive segmentation is proposed to extract conic arcs (circular, elliptic, parabolic, and hyperbolic arcs) from these arc segments. Segmentation of conic arcs can be easily obtained by using the approximation of parametric cubic curve [21]. Consider three consecutive control points Q1, Q2
3.1. Line extraction The procedure of line extraction is very easy and described as follows. Ideally, line segments exist between two consecutive type c-ll joints, a type c-ll and c-la (or s-la) joints or two consecutive type c-la (or s-la) joints, and where type c-ll (or c-la) joints must be followed by only type c-la (or s-la) joints, but not type c-aa (or s-aa) joints,
Fig. 3. Recovery of type s-la joints in the {s-aa, c-la, s-aa} case.
786
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
and Q3, and a scaling parameter r, the parametric cubic curve is defined by Eq. (5) and the model is described from Fig. 4. PðuÞ Z ð2u3 K 3u2 C 1ÞQ1 C ðK2u3 C 3u2 ÞQ3 C ðu3 K 2u2 C uÞQu1 C ðu3 K u2 ÞQu3
(5)
where u2[0,1], Qu1 Z 4rðQ2 K Q1 Þ, Qu3 Z 4rðQ3 K Q2 Þ, and rZ Qd Qm =Q2 Qm . If 0.5!r%1, then P(u) is a hyperbolic arc; if rZ0.5, then P(u) is a parabolic arc; if 0%r!0.5, then P(u) is an elliptic arc, where if it also satisfies Q1 Q2 Z Q2 Q3 and r=ð1K rÞZ Q1 Q3 =ð2Q1 Q2 Þ, then P(u) is a circular arc [22]. For preserving the smoothness of curves and reducing the error of digital quantization on slopes of the tangent lines at control points Q1 and Q3 in Fig. 4, a Fourier curve is used to approximate the arc segment. An arc segment characterized by Fourier descriptors is defined by Eq. (6). " # " # " #" # N X cosð2kpt=TÞ xðtÞ A0 Ak Bk FðtÞ : Z C yðtÞ C0 Dk sinð2kpt=TÞ kZ1 Ck
After substituting Eqs. (8a) and (8d) into Eq. (7) and manipulating, we obtain Eq. (9). PN 0 0 kZ1 Ck cosð2kpt=TÞ C Dk sinð2kpt=TÞ (9) mðtÞ Z PN 0 0 kZ1 Ak cosð2kpt=TÞ C Bk sinð2kpt=TÞ where Ak0 Z Ak ½cosð2kpDt=TÞ K 1 C Bk sinð2kpDt=TÞ
(10a)
Bk0 Z Bk ½cosð2kpDt=TÞ K 1 K Ak sinð2kpDt=TÞ
(10b)
Ck0 Z Ck ½cosð2kpDt=TÞ K 1 C Dk sinð2kpDt=TÞ
(10c)
Dk0 Z Dk ½cosð2kpDt=TÞ K 1 K Ck sinð2kpDt=TÞ
(10d)
where parameter t is such that t2[0,T/2]; T is the period of a Fourier function that describes the curve; and A0, C0, Ak, Bk, Ck and Dk are Fourier descriptors. In the procedure of obtaining the parametric cubic curve, it is necessary to calculate the slope of the tangent line at each point on the Fourier curve. By definition, the slope of F(t) is obtained by Eq. (7). PN yðt C DtÞ K yðtÞ ðC~ k C D~ k Þ mðtÞ Z Z PkZ1 (7) N ~ ~ xðt C DtÞ K xðtÞ kZ1 ðAk C Bk Þ
Segmentation of conic arcs based on a Fourier curve is done as described below. The first point of segmentation is the first data point on a Fourier curve (tZ0). At the beginning, the next point of segmentation will be the sixth data point (tZ5) because of at least six pixels for ellipse fitting [23] or hyperbola fitting. Let these two points be the candidate control points Q1 and Q3 (parametric cubic curve model in Fig. 4); then, the control point Q2 is the intersection of the tangent lines at Q1 and Q3. Calculate the maximum deviation between the parametric cubic curve P(u) and segmented curve, and then compute the measurement m by using Eq. (11), where the maximum deviation is obtained from orthogonal distance. Using the measurement m, the lower the measurement is, the more significant the conic arc is. That is, the longer the conic arc is, the greater the deviation that will be tolerated. Thus, the same shape of curve at different scales will have the same significance.
where
mZ
(6)
A~ k Z Ak ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8a)
B~ k Z Bk ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8b)
C~ k Z Ck ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8c)
D~ k Z Dk ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8d)
Fig. 4. Model of the parametric cubic curve.
maximum deviation segment length
(11)
Subsequently, the region of support is extended to make a longer segment as follows. Let moldZm, then the region of support is extended. The next data point along the Fourier curve (i.e. tZtC1) is chosen to replace Q3, and calculate m (let mnewZm). Compare mold and mnew, if mold!mnew and then an appropriate conic arc is obtained; otherwise let moldZmnew and extend the region of support. The above process is iterated until mold!mnew. The same procedure is repeated with Q3 in the previous segment assigned as Q1 for a new segment. The above process is continued until the endpoint of the Fourier curve (tZT/2) is reached. Using the above procedure, conic arcs are successfully segmented as circular, elliptic, parabolic, and hyperbolic arcs based on scaling parameter r. Using the types of segments on both sides of the breakpoint, these breakpoints obtained from Section 2 can be further classified into more meaningful signs. That is, c-la and c-aa can be categorized further as c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, and c-hh. The breakpoints s-la and s-aa can be classified further into s-lc, s-le, s-lp, s-lh, s-cc,
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789 Input Digital Curve
Corner or Dominant Point Detection
Corner and Smooth Joints Classification
Line segments and Arcs Detection
The First Stage for 5 Types of Breakpoint Classification Line Segment Extraction
Segmentation of Conic Arcs
The Second Stage for 29 Types of Breakpoint Classification Merging of Conic Arcs
Line and Conic Arcs Fitting
Fig. 5. The system block diagram.
s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. The parameters of conic arcs can be obtained based on Eqs (5.2)–(5.9) in the literature [17]. The computational complexity of multiprimitive segmentation is clearly of the order of O(n log n), where n is the number of points in arc segments.
4. Merging of conic arcs After extraction of lines and conic arcs, multiprimitive segmentation based on line segments and conic arcs is successfully obtained, but the limitation of parametric cubic curve model (Fig. 4) causes the conic arcs to become small pieces. That is, the region of support of conic segment is limited under the triangle of control points Q1, Q2 and Q3, it makes a lot of small pieces. For example, an extracted elliptical arc is always smaller than a half of an ellipse. Hence, the merging of conic arcs is necessary to obtain larger conic arcs. Two conditions must be obeyed in the merging of conic arcs for two consecutive conic arcs: (i) If
787
the breakpoint is a corner, then these arcs can’t be merged. (ii) The types of conic arcs must be same to merge, that is, circular arc and circular arc may possibly be merged to a new circular arc. The elliptical arc and elliptical (or circular) arc may possibly be merged to a new elliptical arc. The parabolic arc and parabolic arc may possibly be merged to a new parabolic arc. The hyperbolic arc and hyperbolic arc may possibly be merged to a new hyperbolic arc. If consecutive conic arcs satisfy the above two conditions, then the merging procedure is performed as follows. Calculate the measurement m of new conic arc for the two consecutive arcs, and then compare the original measurements of merged conic arcs. If the former is smaller than latter, then the consecutive arcs are successfully merged to a new conic arc. The next consecutive arc is added in merging procedure, and then the above process is iterated until the merging procedure is failed. The whole merging procedure is performed until no more merging of conic arcs occurring. Using the above procedure, conic arcs with same type are merged to avoid redundancy and the most significant representation is chosen. The computational complexity of merging of conic arcs is easily seen to be in O(n log n), where n is the number of points in the merged conic arcs. The overall system block diagram of the proposed method is illustrated as Fig. 5. There are three phase schemes in the proposed method, and the computational complexity of the proposed method can be derived as the following. In the first-phase scheme, the breakpoint classification takes O(n) complexity, where n is the number of points in curves. And, in the second-phase scheme, multiprimitive segmentation has O(n log n) complexity, where n is the number of points in arc segments. In the thirdphase scheme, merging of conic arcs is O(n log n) complexity, where n is the number of points in the merged conic arcs. The overall computational complexity of the proposed method can thus be concluded as O(n log n).
5. Experimental results Some experiments have been done to demonstrate the performance of the proposed multiprimitive segmentation. A model curve composed of line segments and arcs [24] is
Fig. 6. Multiprimitive segmentation of the model curve [24]. (a) The original curve; (b) The result using the segmentation method [13]; (c) The result using the proposed method.
788
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
Fig. 7. Multiprimitive segmentation of the benchmark. (a) The benchmark; (b) The result using the segmentation method [13]; (c) The result using the proposed method.
investigated. The original data is shown in Fig. 6(a). The results using the segmentation method [13] and the proposed method are indicated in Fig. 6(b) and Fig. 6(c), respectively. In Fig. 6(b), there are 11 conic pieces and they are merged as 3 conic pieces in Fig. 6(c). There are six circular arcs and elliptic arcs being merged as an elliptic arc, three elliptic arcs being merged as an elliptic arc, and two circular arcs being merged as a circular arc. An artificially created cock-head-like benchmark [13] is shown in Fig. 7(a). In the benchmark, all possible combination of joint-types including c-llCc-la, c-llCs-la, c-laCc-la, c-laCc-aa, c-laCs-la, c-laCs-aa, c-aaCc-aa, c-aaCs-la, c-aaCs-aa, s-laCs-la, s-laCs-aa and s-aaCsaa are involved. The locations and the joint-types are given in Fig. 7(b) based on using the segmentation method [13]. The result using the proposed method is indicated in Fig. 7(c). In the result using the proposed method, 19 segments (line segments and arcs) in Fig. 7(b) are merged and tuned as 15 segments (line segments and conic arcs) in Fig. 7(c). The image of a cutter [13] is shown in Fig. 8(a), and the contours are given in Fig. 8(b). The results using the segmentation with line segments and circular arcs [13] and
the proposed method are shown in Fig. 8(c) and Fig. 8(d), respectively. In Fig. 8(c), 7 segments (5 line segments and two circular arcs) and two circles are detected, and there are 6 segments (4 line segments, one elliptic arc, and one hyperbolic arc) and two ellipses detected as shown in Fig. 8(d). The result of Fig. 8(d) is more realistic than one of Fig. 8(c), especially on two holes in the cutter. The performance of curve fitting by using the proposed method is evaluated based on the following examples. The image of changes is shown in Fig. 9(a), and the one of a lamp is given in Fig. 10(a). The contours are given in Fig. 9(b) and Fig. 10(b), respectively. The results of curve fitting using the proposed method are shown in Fig. 9(c) and Fig. 10(c), respectively. In Fig. 9(c), two ellipses and two elliptic arcs are detected, and there are two elliptic arcs and four line segments detected as shown in Fig. 10(c). From the experimental results of Figs. 6(c), 7(c), and 8(d), using the proposed method can effectively avoid redundancy. And, using the proposed method also can obtain good results of curve fitting from Figs. 9(c) and 10(c).
Fig. 9. Curve fitting of changes. (a) Original image; (b) The contours; (c) The result using the proposed method.
Fig. 8. Multiprimitive segmentation of a cutter. (a) Original image; (b) The contours; (c) The result using the segmentation method [13]; (d) The result using the proposed method.
Fig. 10. Curve fitting of a lamp. (a) Original image; (b) The contours; (c) The result using the proposed method.
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
6. Conclusion The multiprimitive segmentation based on types of breakpoints is proposed successfully. The multiprimitive segmentation by using line segments and conic arcs can describe planar curves meaningfully. In the proposed method, the breakpoints are first categorized as five types: c-ll, c-la, c-aa, s-la, and s-aa by using AKC function and PHF, where c indicates corner and s is smooth joint. Subsequently, line segments are extracted by s-la breakpoints tuning. And, the extraction and merging of conic arcs are employed on the preserved parts without line segments. Using the above procedure, the breakpoints are further categorized as 29 types, such as c-ll, c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, c-hh, s-lc, s-le, s-lp, s-lh, s-cc, s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. The computational complexity of the proposed method is O(n log n) which is lower than most other conic fitting methods. And, the proposed method can avoid redundancy based on using the model of the parametric cubic curve or the guiding triangle of a conic section [17], and obtain more significant representation. The main contribution of this paper is to present the concept of types of breakpoints to give conic pieces indexing. The sequence of breakpoint-types is very useful for shape analysis and pattern recognition. And, this concept proposed in this paper is not referred in the existent conic fitting methods. In the proposed multiprimitive segmentation, since no threshold is required, the result is not influenced by the selection of thresholds. That is, the performance of the proposed scheme is threshold-free. Further, using the significance measurement (the ratio of the maximum deviation divided by the segment length) can obtain the performance that the same shape of curve at different scales will have the same significance.
Acknowledgements This paper has been supported partially by the National Science Council of the Republic of China under grant, NSC 89-2213-E-346-001. The author also wishes to express their appreciation for the anonymous reviewers for their helpful suggestions.
References [1] J. Sporring, O.F. Olsen, M. Nielsen, J. Weickert, Smoothing images creates corners, Image and Vision Computing 18 (2000) 261–266.
789
[2] F. Chabat, G.Z. Yang, D.M. Hansell, A corner orientation detector, Image and Vision Computing 17 (1999) 761–769. [3] L. Li, W. Chen, Corner detection and interpretation on planar curves using fuzzy reasoning, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 1204–1210. [4] H.T. Sheu, W.C. Hu, A rotationally invariant two-phase scheme for corner detection, Pattern Recognition 29 (1996) 819–828. [5] D.M. Tsai, Boundary-based corner detection using neural networks, Pattern Recognition 29 (1996) 85–97. [6] A. Rattarangsi, R.T. Chin, Scale-based detection of corner of planar curves, IEEE Transactions on Pattern Analysis and Machine Intelligence 14 (1992) 430–449. [7] W.Y. Wu, Dominant point detection using adaptive bending value, Image and Vision Computing 21 (2003) 517–525. [8] J.H. Horng, Improving fitting quality of polygonal approximation by using the dynamic programming technique, Pattern Recognition Letters 23 (2002) 1657–1673. [9] B. Sarkar, S. Roy, D. Sarkar, Hierarchical representation of digitized curves through dominant point detection, Pattern Recognition Letters 24 (2003) 2869–2882. [10] C.H. Teh, R.T. Chin, On the detection of dominant points on digital curves, IEEE Transactions on Pattern Analysis and Machine Intelligence 11 (1989) 859–872. [11] J.H. Horng, An adaptive smoothing approach for fitting digital planar curves with line segments and circular arcs, Pattern Recognition Letters 24 (2003) 565–577. [12] B. Sarkar, L.K. Singh, D. Sarkar, Approximation of digital curves with line segments and circular arcs using genetic algorithms, Pattern Recognition Letters 24 (2003) 2585–2595. [13] H.T. Sheu, W.C. Hu, Multiprimitive segmentation of planar curves— A two-level breakpoint classification and tuning approach, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 791–797. [14] P.L. Rosin, G.A.W. West, Segmentation of edges into lines and arcs, Image and Vision Computing 7 (1989) 109–114. [15] G.Y. Wang, Z. Houkes, B. Zheng, X. Li, A note on conic fitting by the gradient weighted least-squares estimation: refined eigenvector solution, Pattern Recognition Letters 23 (2002) 1695–1703. [16] S.J. Ahn, W. Rauh, H.J. Warnecke, Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola, Pattern Recognition 34 (2001) 2283–2303. [17] Q. Zhu, L. Peng, A new approach to conic section approximation of object boundaries, Image and Vision Computing 17 (1999) 645– 658. [18] Z. Zhang, Parameter estimation techniques: a tutorial with application to conic fitting, Image and Vision Computing 15 (1997) 59–76. [19] H. Asada, M. Brady, The curvature primal sketch, IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (1986) 2–14. [20] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, New York, 1973. [21] M.E. Mortenson, Computer Graphics Handbook: Geometry and Mathematics, Industrial Press, Inc., New York, 1990. [22] I. Zeid, CAD/CAM Theory and Practice, McGraw-Hill, Inc., Singapore, 1991. [23] P.L. Rosin, G.A.W. West, Segmenting curves into elliptic arcs and straight lines, Third Int’l Conference Computer Vision, 75–78, Osaka, Japan 1990. [24] P.L. Rosin, G.A.W. West, Nonparametric segmentation of curves into various representations, IEEE Transactions on Pattern Analysis and Machine Intelligence 17 (1995) 1140–1153.
Multiprimitive segmentation based on meaningful breakpoints for fitting digital planar curves with line segments and conic arcs Wu-Chih Hu* Department of Computer Science and Information Engineering, National Penghu Institute of Technology, #300, Liu-Ho Road, Makung City, Penghu, Taiwan, ROC Received 30 January 2004; received in revised form 26 April 2005; accepted 5 May 2005
Abstract This paper presents a multiprimitive segmentation method with line segments and conic arcs based on the types of breakpoints. In this method, a joint tuning procedure is proposed to merge consecutive segments and adjust their locations to achieve more accurate and stable conic arcs. No threshold is required in the multiprimitive segmentation by using the proposed scheme. And, the types of breakpoints among line segments and conic arcs are defined and they are useful and meaningful for pattern recognition and shape analysis. Besides, the computational complexity of the proposed method is O(n log n) which is lower than most other conic fitting methods. Further, the concept of types of breakpoints can be easily extended to other primitives. q 2005 Elsevier B.V. All rights reserved. Keywords: Multiprimitive; Segmentation; Conic arcs; Breakpoint
1. Introduction Segmentation of planar curves is one of the most important jobs in image processing since a segmented contour can be used to describe an object in a meaningful and compact form for higher level vision processing, such as pattern recognition and shape analysis. The objective is to use a minimum number of primitive pieces to approximate curves with minimum distortion. Many techniques have been proposed for this purpose in the past two decades. Polygonal approximation is the simplest approach. The line segments are almost extracted from curves based on the corner detection [1–6] or dominant point detection [7–10]. But polygonal approximation is rarely used for further shape analysis. To go from segmentation to shape analysis, one could include higher order primitives such as circular arcs, elliptic arcs, splines etc. in the segmentation. Normally, longer computation time will be involved in segmentation with such primitives since more parameters need to be solved. Hence, curve segmentation using line segments and * Tel./fax: C886 6 9277361. E-mail address: [email protected].
0262-8856/$ - see front matter q 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2005.05.004
circular arcs is selected, and it is a better representation than polygonal approximation. Thus, many methods were proposed to obtain line segments and circular arcs [11– 14]. The main reason is that a circular arc is easily obtained based on three parameters (center (x0,y0) and radius r). High order analytic curves such as conic arcs (circular, elliptic, parabolic, and hyperbolic arcs) can produce more accurate representations than only using circular arcs. Thus, the conic arcs are considered in this research because of computation time and flexibility of description. Needless to say, conic arcs in our daily life and in industry are important. Some techniques for conic fitting were proposed [15–18]. The least-squares method is often applied to fitting of conic arcs. Least-square fitting minimizes the square sum of errorof-fit in predefined measures, and two main categories (algebraic and geometric fitting) are differentiated by the respective definition of the error distance. In algebraic fitting, a conic arc is described by implicit equation, and the error distance is defined with deviation of the implicit equation from the expected value at each given point. And, in geometric fitting, the error distance is defined with the orthogonal distance from the given point to the fitted conic arc. These approaches are almost focused on accurate and efficient the result of fitting conic arcs. However, these results of conic fitting are not enough to employ on pattern
784
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
recognition and shape analysis. That is, the fitting result is only flexible for curve description, but the shape analysis is not further obtained by these conic pieces (such as conic piece indexing). Hence, the multiprimitive segmentation based on conic piece indexing is very necessary to analyze the object shape. We proposed a method for segmenting curves into line segments and circular arcs by using types of breakpoints [13]. The advantages of this technique are that it is: † Threshold-free–No threshold within the algorithm, † Stable–Invariant to transformations of the data (rotation, translation, and scale), and † Extendible—Types of breakpoints can be extended to other primitives. Hence, using the types of breakpoints for fitting curves with line segments and conic arcs is proposed in this research. The proposed scheme is a modified/extended method based on the breakpoint classification and tuning approach [13]. In the proposed scheme, the breakpoints are first categorized as five types: c-ll, c-la, c-aa, s-la, and s-aa by using AKC function and PHF [13], where c indicates a corner and s is a smooth joint. Subsequently, the procedure of line/conic segmentation and merging is employed based on the above types of breakpoints. The breakpoints are further categorized as 29 types: c-ll, c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, c-hh, s-lc, s-le, s-lp, s-lh, s-cc, s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. And, these breakpoints are defined as meaningful breakpoints in this research. These meaningful breakpoints are very useful for describing planar curves, pattern recognition, and shape analysis. Hence, the main contribution of this paper is to present the concept of meaningful breakpoints to give conic pieces indexing for shape analysis. This concept proposed in this paper is not referred in the existent methods for conic fitting. In the remainder of this paper, the breakpoint classification is presented in Section 2 while multiprimitive segmentation based on lines and conic arcs is described in Section 3. In Section 4, the merging of conic arcs is proposed. Section 5 presents experimental examples and evaluations of the results. Finally, conclusion is made in Section 6.
2. Breakpoint classification In this section, the associated breakpoints are first detected by using the methods of dominant point or corner detection [1–10], and then the breakpoints are categorized as five types: corner-ll, corner-la, corner-aa, smooth joint-la or smooth joint-aa by using AKC (adaptive k-curvature) function and PHF (projective height function) [13].
The type ll means that the segments on both sides of the breakpoint are line segments; la stands for a joint of a line segment and an arc; aa represents a joint of two arcs; and corner is a discontinuous tangent and smooth joint is associated with continuous tangent, but discontinuous curvature [19]. The AKC function is briefly described as follows. Consider three consecutive breakpoints PiK1, Pi and PiC1, and two consecutive segments Si and SiC1 starting at PiK1, joining at Pi and ending at PiC1. Let lengths of Si and SiC1 be li and liC1, respectively, and kZmin(li,liC1). And, let the ~ Pi C k, ~ where kZ ~ k=2 region-of-support for Pi be ½Pi K k; ~ ~ and define the k-vector at a point Pj 2½Pi K k; Pi C k as aÐjk Z ðPjC ðxÞ K Pj ðxÞ;
PjC ðyÞ K Pj ðyÞÞ
(1a)
bÐjk Z ðPjK ðxÞ K Pj ðxÞ;
PjK ðyÞ K Pj ðyÞÞ
(1b)
~ and then the k-cosine where PjC Z Pj C k~ and PjK Z Pj K k, Ð Ð between a jk and b jk is cjk Z
aÐjk ,bÐjk kÐ a jk kkbÐjk k
(2)
The AKC function of Pi is defined as the values cjk ~ Pi C k. ~ cPj 2½Pi K k; And, the PHF is described as follows. Let the intervals for evaluating the projective heights (PHs) of the segments ~ Pi K k=2 ~ on both sides of the breakpoint Pi be ½Pi K 3k=2; ~ Pi C 3k=2 ~ for the segment for the segment Si, and ½Pi C k=2; SiC1, which insure that the PH evaluation does not exceed either PiK1 and Pi or Pi and PiC1. ~ Pi K k=2, ~ Consider a point Pj 2½Pi K 3k=2; and the ~ and P^ jC Z Pj C k=2. ~ associated endpoints P^ jK Z Pj K k=2 Then, the parametric form for the line segment between the endpoints is ( x Z P^ jK ðxÞ C lx t ; t 2½0; 1 (3) y Z P^ jK ðyÞ C ly t where lx Z P^ jC ðxÞK P^ jK ðxÞ and ly Z P^ jC ðyÞK P^ jK ðyÞ, and the projective height of Pj is sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ½lx ðPj ðyÞ K P^ jK ðyÞÞ K ly ðPj ðxÞ K P^ jK ðxÞÞ2 hðPj Þ Z (4) l2x C l2y The PHF is defined as the projective heights h(Pj) over ~ Pi K k=2 ~ or ½Pi C k=2; ~ Pi C 3k=2. ~ the interval of ½Pi K 3k=2; Define a line accumulator and an arc accumulator on the region of support of PHF and use 0.5 to discriminate a line from an arc, since it is the maximum possible deviation of the PH of a digitized line segment can make. If h(Pj)!0.5, then the line accumulator is incremented by one; otherwise add one to the arc accumulator. The type of segment is determined by comparing the values in the two accumulators. If the value in the line accumulator is greater, then the segment is identified as a line; otherwise it is an arc.
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
785
Fig. 2. Recovery of type s-la joints in the {c-ll, s-aa} case.
Fig. 1. Types of corner and smooth joint.
The procedure of breakpoint classification is described as follows. The breakpoints can be categorized as corners and smooth joints by testing a global maximum in the AKC function. Clearly, if there is a global maximum at the breakpoint in the AKC function, then it is a corner; otherwise it is a smooth joint. And, using the PHF scheme, the types of segments on both sides of a breakpoint can be efficiently detected and a breakpoint can be further divided into types ll, la (or al), and aa. Combining the results of AKC and PHF schemes, the breakpoints are categorized as five types: corner-ll (c-ll), corner-la (c-la), corner-aa (c-aa), smooth joint-la (s-la) and smooth joint-aa (s-aa). For understanding the geometry of corner and smooth joint, the breakpoint types can be illustrated clearly from Fig. 1. Fig. 1(a) is c-ll, Fig. 1(b) and Fig. 1(c) are c-la, Fig. 1(d)–(f) are c-aa, Fig. 1(g) is s-la, and Fig. 1(h) and 1(i) are s-aa breakpoints. The computational complexity of breakpoint classification is easily seen to be in O(n), where n is the number of points in curves.
3. Multiprimitive segmentation Segmentation of a planar curve into line segments and conic arcs is easily achieved if the types of breakpoints are identified. In this section, segmentation of lines and conic arcs is described in Sections 3.1 and 3.2, respectively.
etc. Practically, however, due to the errors in breakpointdetection, two situations may be brought about. The first case is that there is a type c-ll joint followed by a type c-aa (or s-aa) joint immediately, and there must be a type s-la joint between these two consecutive joints. To determine the location of such a type s-la joint, form a straight line by connecting the type c-ll and c-aa (or s-aa) joints. The point in the original segment that is farthest from this connected line segment is then considered as such a type s-la joint, following the idea by Duda and Hart [20]. The result of the first case is shown in Fig. 2. The second case is that there is a type c-la joint surrounded by two types c-aa (or s-aa) joint immediately, then there must be a type s-la joint preceding or following the type c-la joint. The location of such a type s-la joint can be obtained by using the PHF of the type c-la joint. That is, if there is a line segment on the forward side of c-la joint, then the new s-la joint should be on the forward segment, and vice versa. The exact location of the new s-la joint is also determined at the point where the distance from the line connecting the c-la and c-aa (or s-aa) joints is the maximum. The result of the second case is shown in Fig. 3. Using the above schemes, line segments in planar curves are extracted successfully and only conic arcs are not treated. 3.2. Segmentation of conic arcs After the breakpoint classification and line extraction, the preserved parts are only arc segments. Hence, the multiprimitive segmentation is proposed to extract conic arcs (circular, elliptic, parabolic, and hyperbolic arcs) from these arc segments. Segmentation of conic arcs can be easily obtained by using the approximation of parametric cubic curve [21]. Consider three consecutive control points Q1, Q2
3.1. Line extraction The procedure of line extraction is very easy and described as follows. Ideally, line segments exist between two consecutive type c-ll joints, a type c-ll and c-la (or s-la) joints or two consecutive type c-la (or s-la) joints, and where type c-ll (or c-la) joints must be followed by only type c-la (or s-la) joints, but not type c-aa (or s-aa) joints,
Fig. 3. Recovery of type s-la joints in the {s-aa, c-la, s-aa} case.
786
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
and Q3, and a scaling parameter r, the parametric cubic curve is defined by Eq. (5) and the model is described from Fig. 4. PðuÞ Z ð2u3 K 3u2 C 1ÞQ1 C ðK2u3 C 3u2 ÞQ3 C ðu3 K 2u2 C uÞQu1 C ðu3 K u2 ÞQu3
(5)
where u2[0,1], Qu1 Z 4rðQ2 K Q1 Þ, Qu3 Z 4rðQ3 K Q2 Þ, and rZ Qd Qm =Q2 Qm . If 0.5!r%1, then P(u) is a hyperbolic arc; if rZ0.5, then P(u) is a parabolic arc; if 0%r!0.5, then P(u) is an elliptic arc, where if it also satisfies Q1 Q2 Z Q2 Q3 and r=ð1K rÞZ Q1 Q3 =ð2Q1 Q2 Þ, then P(u) is a circular arc [22]. For preserving the smoothness of curves and reducing the error of digital quantization on slopes of the tangent lines at control points Q1 and Q3 in Fig. 4, a Fourier curve is used to approximate the arc segment. An arc segment characterized by Fourier descriptors is defined by Eq. (6). " # " # " #" # N X cosð2kpt=TÞ xðtÞ A0 Ak Bk FðtÞ : Z C yðtÞ C0 Dk sinð2kpt=TÞ kZ1 Ck
After substituting Eqs. (8a) and (8d) into Eq. (7) and manipulating, we obtain Eq. (9). PN 0 0 kZ1 Ck cosð2kpt=TÞ C Dk sinð2kpt=TÞ (9) mðtÞ Z PN 0 0 kZ1 Ak cosð2kpt=TÞ C Bk sinð2kpt=TÞ where Ak0 Z Ak ½cosð2kpDt=TÞ K 1 C Bk sinð2kpDt=TÞ
(10a)
Bk0 Z Bk ½cosð2kpDt=TÞ K 1 K Ak sinð2kpDt=TÞ
(10b)
Ck0 Z Ck ½cosð2kpDt=TÞ K 1 C Dk sinð2kpDt=TÞ
(10c)
Dk0 Z Dk ½cosð2kpDt=TÞ K 1 K Ck sinð2kpDt=TÞ
(10d)
where parameter t is such that t2[0,T/2]; T is the period of a Fourier function that describes the curve; and A0, C0, Ak, Bk, Ck and Dk are Fourier descriptors. In the procedure of obtaining the parametric cubic curve, it is necessary to calculate the slope of the tangent line at each point on the Fourier curve. By definition, the slope of F(t) is obtained by Eq. (7). PN yðt C DtÞ K yðtÞ ðC~ k C D~ k Þ mðtÞ Z Z PkZ1 (7) N ~ ~ xðt C DtÞ K xðtÞ kZ1 ðAk C Bk Þ
Segmentation of conic arcs based on a Fourier curve is done as described below. The first point of segmentation is the first data point on a Fourier curve (tZ0). At the beginning, the next point of segmentation will be the sixth data point (tZ5) because of at least six pixels for ellipse fitting [23] or hyperbola fitting. Let these two points be the candidate control points Q1 and Q3 (parametric cubic curve model in Fig. 4); then, the control point Q2 is the intersection of the tangent lines at Q1 and Q3. Calculate the maximum deviation between the parametric cubic curve P(u) and segmented curve, and then compute the measurement m by using Eq. (11), where the maximum deviation is obtained from orthogonal distance. Using the measurement m, the lower the measurement is, the more significant the conic arc is. That is, the longer the conic arc is, the greater the deviation that will be tolerated. Thus, the same shape of curve at different scales will have the same significance.
where
mZ
(6)
A~ k Z Ak ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8a)
B~ k Z Bk ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8b)
C~ k Z Ck ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8c)
D~ k Z Dk ½cosð2kpðt C DtÞ=TÞ K cosð2kpt=TÞ
(8d)
Fig. 4. Model of the parametric cubic curve.
maximum deviation segment length
(11)
Subsequently, the region of support is extended to make a longer segment as follows. Let moldZm, then the region of support is extended. The next data point along the Fourier curve (i.e. tZtC1) is chosen to replace Q3, and calculate m (let mnewZm). Compare mold and mnew, if mold!mnew and then an appropriate conic arc is obtained; otherwise let moldZmnew and extend the region of support. The above process is iterated until mold!mnew. The same procedure is repeated with Q3 in the previous segment assigned as Q1 for a new segment. The above process is continued until the endpoint of the Fourier curve (tZT/2) is reached. Using the above procedure, conic arcs are successfully segmented as circular, elliptic, parabolic, and hyperbolic arcs based on scaling parameter r. Using the types of segments on both sides of the breakpoint, these breakpoints obtained from Section 2 can be further classified into more meaningful signs. That is, c-la and c-aa can be categorized further as c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, and c-hh. The breakpoints s-la and s-aa can be classified further into s-lc, s-le, s-lp, s-lh, s-cc,
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789 Input Digital Curve
Corner or Dominant Point Detection
Corner and Smooth Joints Classification
Line segments and Arcs Detection
The First Stage for 5 Types of Breakpoint Classification Line Segment Extraction
Segmentation of Conic Arcs
The Second Stage for 29 Types of Breakpoint Classification Merging of Conic Arcs
Line and Conic Arcs Fitting
Fig. 5. The system block diagram.
s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. The parameters of conic arcs can be obtained based on Eqs (5.2)–(5.9) in the literature [17]. The computational complexity of multiprimitive segmentation is clearly of the order of O(n log n), where n is the number of points in arc segments.
4. Merging of conic arcs After extraction of lines and conic arcs, multiprimitive segmentation based on line segments and conic arcs is successfully obtained, but the limitation of parametric cubic curve model (Fig. 4) causes the conic arcs to become small pieces. That is, the region of support of conic segment is limited under the triangle of control points Q1, Q2 and Q3, it makes a lot of small pieces. For example, an extracted elliptical arc is always smaller than a half of an ellipse. Hence, the merging of conic arcs is necessary to obtain larger conic arcs. Two conditions must be obeyed in the merging of conic arcs for two consecutive conic arcs: (i) If
787
the breakpoint is a corner, then these arcs can’t be merged. (ii) The types of conic arcs must be same to merge, that is, circular arc and circular arc may possibly be merged to a new circular arc. The elliptical arc and elliptical (or circular) arc may possibly be merged to a new elliptical arc. The parabolic arc and parabolic arc may possibly be merged to a new parabolic arc. The hyperbolic arc and hyperbolic arc may possibly be merged to a new hyperbolic arc. If consecutive conic arcs satisfy the above two conditions, then the merging procedure is performed as follows. Calculate the measurement m of new conic arc for the two consecutive arcs, and then compare the original measurements of merged conic arcs. If the former is smaller than latter, then the consecutive arcs are successfully merged to a new conic arc. The next consecutive arc is added in merging procedure, and then the above process is iterated until the merging procedure is failed. The whole merging procedure is performed until no more merging of conic arcs occurring. Using the above procedure, conic arcs with same type are merged to avoid redundancy and the most significant representation is chosen. The computational complexity of merging of conic arcs is easily seen to be in O(n log n), where n is the number of points in the merged conic arcs. The overall system block diagram of the proposed method is illustrated as Fig. 5. There are three phase schemes in the proposed method, and the computational complexity of the proposed method can be derived as the following. In the first-phase scheme, the breakpoint classification takes O(n) complexity, where n is the number of points in curves. And, in the second-phase scheme, multiprimitive segmentation has O(n log n) complexity, where n is the number of points in arc segments. In the thirdphase scheme, merging of conic arcs is O(n log n) complexity, where n is the number of points in the merged conic arcs. The overall computational complexity of the proposed method can thus be concluded as O(n log n).
5. Experimental results Some experiments have been done to demonstrate the performance of the proposed multiprimitive segmentation. A model curve composed of line segments and arcs [24] is
Fig. 6. Multiprimitive segmentation of the model curve [24]. (a) The original curve; (b) The result using the segmentation method [13]; (c) The result using the proposed method.
788
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
Fig. 7. Multiprimitive segmentation of the benchmark. (a) The benchmark; (b) The result using the segmentation method [13]; (c) The result using the proposed method.
investigated. The original data is shown in Fig. 6(a). The results using the segmentation method [13] and the proposed method are indicated in Fig. 6(b) and Fig. 6(c), respectively. In Fig. 6(b), there are 11 conic pieces and they are merged as 3 conic pieces in Fig. 6(c). There are six circular arcs and elliptic arcs being merged as an elliptic arc, three elliptic arcs being merged as an elliptic arc, and two circular arcs being merged as a circular arc. An artificially created cock-head-like benchmark [13] is shown in Fig. 7(a). In the benchmark, all possible combination of joint-types including c-llCc-la, c-llCs-la, c-laCc-la, c-laCc-aa, c-laCs-la, c-laCs-aa, c-aaCc-aa, c-aaCs-la, c-aaCs-aa, s-laCs-la, s-laCs-aa and s-aaCsaa are involved. The locations and the joint-types are given in Fig. 7(b) based on using the segmentation method [13]. The result using the proposed method is indicated in Fig. 7(c). In the result using the proposed method, 19 segments (line segments and arcs) in Fig. 7(b) are merged and tuned as 15 segments (line segments and conic arcs) in Fig. 7(c). The image of a cutter [13] is shown in Fig. 8(a), and the contours are given in Fig. 8(b). The results using the segmentation with line segments and circular arcs [13] and
the proposed method are shown in Fig. 8(c) and Fig. 8(d), respectively. In Fig. 8(c), 7 segments (5 line segments and two circular arcs) and two circles are detected, and there are 6 segments (4 line segments, one elliptic arc, and one hyperbolic arc) and two ellipses detected as shown in Fig. 8(d). The result of Fig. 8(d) is more realistic than one of Fig. 8(c), especially on two holes in the cutter. The performance of curve fitting by using the proposed method is evaluated based on the following examples. The image of changes is shown in Fig. 9(a), and the one of a lamp is given in Fig. 10(a). The contours are given in Fig. 9(b) and Fig. 10(b), respectively. The results of curve fitting using the proposed method are shown in Fig. 9(c) and Fig. 10(c), respectively. In Fig. 9(c), two ellipses and two elliptic arcs are detected, and there are two elliptic arcs and four line segments detected as shown in Fig. 10(c). From the experimental results of Figs. 6(c), 7(c), and 8(d), using the proposed method can effectively avoid redundancy. And, using the proposed method also can obtain good results of curve fitting from Figs. 9(c) and 10(c).
Fig. 9. Curve fitting of changes. (a) Original image; (b) The contours; (c) The result using the proposed method.
Fig. 8. Multiprimitive segmentation of a cutter. (a) Original image; (b) The contours; (c) The result using the segmentation method [13]; (d) The result using the proposed method.
Fig. 10. Curve fitting of a lamp. (a) Original image; (b) The contours; (c) The result using the proposed method.
W.-C. Hu / Image and Vision Computing 23 (2005) 783–789
6. Conclusion The multiprimitive segmentation based on types of breakpoints is proposed successfully. The multiprimitive segmentation by using line segments and conic arcs can describe planar curves meaningfully. In the proposed method, the breakpoints are first categorized as five types: c-ll, c-la, c-aa, s-la, and s-aa by using AKC function and PHF, where c indicates corner and s is smooth joint. Subsequently, line segments are extracted by s-la breakpoints tuning. And, the extraction and merging of conic arcs are employed on the preserved parts without line segments. Using the above procedure, the breakpoints are further categorized as 29 types, such as c-ll, c-lc, c-le, c-lp, c-lh, c-cc, c-ce, c-cp, c-ch, c-ee, c-ep, c-eh, c-pp, c-ph, c-hh, s-lc, s-le, s-lp, s-lh, s-cc, s-ce, s-cp, s-ch, s-ee, s-ep, s-eh, s-pp, s-ph, and s-hh, where c, e, p, and h indicate circular, elliptic, parabolic, and hyperbolic arcs, respectively. The computational complexity of the proposed method is O(n log n) which is lower than most other conic fitting methods. And, the proposed method can avoid redundancy based on using the model of the parametric cubic curve or the guiding triangle of a conic section [17], and obtain more significant representation. The main contribution of this paper is to present the concept of types of breakpoints to give conic pieces indexing. The sequence of breakpoint-types is very useful for shape analysis and pattern recognition. And, this concept proposed in this paper is not referred in the existent conic fitting methods. In the proposed multiprimitive segmentation, since no threshold is required, the result is not influenced by the selection of thresholds. That is, the performance of the proposed scheme is threshold-free. Further, using the significance measurement (the ratio of the maximum deviation divided by the segment length) can obtain the performance that the same shape of curve at different scales will have the same significance.
Acknowledgements This paper has been supported partially by the National Science Council of the Republic of China under grant, NSC 89-2213-E-346-001. The author also wishes to express their appreciation for the anonymous reviewers for their helpful suggestions.
References [1] J. Sporring, O.F. Olsen, M. Nielsen, J. Weickert, Smoothing images creates corners, Image and Vision Computing 18 (2000) 261–266.
789
[2] F. Chabat, G.Z. Yang, D.M. Hansell, A corner orientation detector, Image and Vision Computing 17 (1999) 761–769. [3] L. Li, W. Chen, Corner detection and interpretation on planar curves using fuzzy reasoning, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 1204–1210. [4] H.T. Sheu, W.C. Hu, A rotationally invariant two-phase scheme for corner detection, Pattern Recognition 29 (1996) 819–828. [5] D.M. Tsai, Boundary-based corner detection using neural networks, Pattern Recognition 29 (1996) 85–97. [6] A. Rattarangsi, R.T. Chin, Scale-based detection of corner of planar curves, IEEE Transactions on Pattern Analysis and Machine Intelligence 14 (1992) 430–449. [7] W.Y. Wu, Dominant point detection using adaptive bending value, Image and Vision Computing 21 (2003) 517–525. [8] J.H. Horng, Improving fitting quality of polygonal approximation by using the dynamic programming technique, Pattern Recognition Letters 23 (2002) 1657–1673. [9] B. Sarkar, S. Roy, D. Sarkar, Hierarchical representation of digitized curves through dominant point detection, Pattern Recognition Letters 24 (2003) 2869–2882. [10] C.H. Teh, R.T. Chin, On the detection of dominant points on digital curves, IEEE Transactions on Pattern Analysis and Machine Intelligence 11 (1989) 859–872. [11] J.H. Horng, An adaptive smoothing approach for fitting digital planar curves with line segments and circular arcs, Pattern Recognition Letters 24 (2003) 565–577. [12] B. Sarkar, L.K. Singh, D. Sarkar, Approximation of digital curves with line segments and circular arcs using genetic algorithms, Pattern Recognition Letters 24 (2003) 2585–2595. [13] H.T. Sheu, W.C. Hu, Multiprimitive segmentation of planar curves— A two-level breakpoint classification and tuning approach, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 791–797. [14] P.L. Rosin, G.A.W. West, Segmentation of edges into lines and arcs, Image and Vision Computing 7 (1989) 109–114. [15] G.Y. Wang, Z. Houkes, B. Zheng, X. Li, A note on conic fitting by the gradient weighted least-squares estimation: refined eigenvector solution, Pattern Recognition Letters 23 (2002) 1695–1703. [16] S.J. Ahn, W. Rauh, H.J. Warnecke, Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola, Pattern Recognition 34 (2001) 2283–2303. [17] Q. Zhu, L. Peng, A new approach to conic section approximation of object boundaries, Image and Vision Computing 17 (1999) 645– 658. [18] Z. Zhang, Parameter estimation techniques: a tutorial with application to conic fitting, Image and Vision Computing 15 (1997) 59–76. [19] H. Asada, M. Brady, The curvature primal sketch, IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (1986) 2–14. [20] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, New York, 1973. [21] M.E. Mortenson, Computer Graphics Handbook: Geometry and Mathematics, Industrial Press, Inc., New York, 1990. [22] I. Zeid, CAD/CAM Theory and Practice, McGraw-Hill, Inc., Singapore, 1991. [23] P.L. Rosin, G.A.W. West, Segmenting curves into elliptic arcs and straight lines, Third Int’l Conference Computer Vision, 75–78, Osaka, Japan 1990. [24] P.L. Rosin, G.A.W. West, Nonparametric segmentation of curves into various representations, IEEE Transactions on Pattern Analysis and Machine Intelligence 17 (1995) 1140–1153.