- Email: [email protected]
VICUR: A human-vision-based algorithm for curve reconstruction
ARTICLE IN PRESS Robotics and Computer-Integrated Manufacturing 24 (2008) 824– 834
Contents lists available at ScienceDirect
Robotics and Computer-Integrated Manufacturing journal homepage: www.elsevier.com/locate/rcim
VICUR: A human-vision-based algorithm for curve reconstruction Thanh An Nguyen, Yong Zeng Faculty of Engineering and Computer Science, Concordia Institute for Information Systems Engineering, Concordia University, 1455 de Maisonneuve Blvd. West, EV07.633, Montreal, Quebec, Canada H3G 1M8
article info
a b s t r a c t
Keywords: Curve reconstruction Human vision system Proximity Continuation Gestalt law
In this paper, a new algorithm, named VICUR, is presented for curve reconstruction problem. From a set of unorganized points, the proposed algorithm can construct curves that look natural to human vision. The VICUR algorithm is based on two connectivity criteria: proximity and good continuation from the prominent Gestalt principles of perception. Experimental results are presented to show the effectiveness of VICUR. & 2008 Elsevier Ltd. All rights reserved.
1. Introduction 1.1. Background The main problem of curve reconstruction is constructing a polygonal representation of an unknown curve from a set of sample points, as shown in Fig. 1. Curve reconstruction has many applications in manufacturing engineering, biomedical engineering, computer vision, and pattern recognition. One example can be found in the reconstruction of 3D image that models a real object. In general, the process is divided into three main steps. In the first step, technologies such as Magnetic Resonance Imaging (MRI), X-ray Computed Tomography (CT) or laser scanner takes digitized image slices from the object. Boundary points of each slice are then extracted. Fig. 2 shows the point cloud produced by a 3D digitizer. At this step, each slice is represented by a set of points, called sampled points. In the second step, the sampled points on each slice will be connected to reconstruct curves as illustrated in Fig. 3. Each slice may include multiple curves with various features such as sharp corners or boundaries. The sampled points may be distributed uniformly or non-uniformly. In the last step, constructed curves are assembled to form a 3D image. Fig. 4 shows an example where the constructed slices from Fig. 3 are put together. Because of its importance in various application domains, curve reconstruction has been attracting numerous research attentions over the last three decades. In general, existing algorithms construct only simple curves. A simple curve is a curve without self-intersection. There are mainly two kinds of simple curves: open and closed curves. A simple curve may Corresponding author. Tel.: +1 514 8482424x5801; fax: +1 514 8483171.
E-mail address: [email protected] (Y. Zeng). 0736-5845/$ - see front matter & 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.rcim.2008.03.007
contain sharp corner. A simple curve without sharp corner is called smooth curve. All these curves can be sampled either uniformly or non-uniformly. There are a few algorithms working for uniformly sampled curves such as alpha-shape [1], r-regular shape [2], and EMST [3]. There are other algorithms that work for non-uniformly sampled curves such as Crust [4,5], Nearest Neighbor [6], Conservative Crust [7], Traveling Salesman Path [8,9], and Gathan [10,11]. Non-uniform sampling allows sparser sampling at less detailed section of the curve while uniform sampling unnecessarily requires dense sampling in areas where sparse sampling should be enough. Most of the existing algorithms work for non-uniform sample requires the sampling points to be an e-sample. A point set P is called an e-sample of a curve T if any point p on T has a sample within distance gf(p) where g is a constant factor and f(p) is the local feature size at p defined as the minimum Euclidean distance from p to the medial axis [12]. In the curve reconstruction problem, the original curve is not known and the sample set is taken arbitrarily. Most of the existing algorithms provide a sampling condition with respect to geometric properties, which may result in a sample set not natural to human vision. When some connections go wrong, it is not easy or intuitive to find a sample to produce the right reconstruction. This is an inspiration to develop an algorithm that is able to connect a set of unorganized points to form a pattern that is consistent with human visual system. 1.2. Literature reviews The first provable curve reconstruction algorithm for simple close smooth curves is given by Amenta–Bern–Eppstein named Crust [4]. The paper proves that for gp0.252 the polygonal reconstruction of a curve is the crust. The crust is constructed by computing Delaunay triangulation on the set of sampling points
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
and Voronoi vertices, and choosing only Delaunay edges which have the endpoints belonging to the set of sampling points. Later, Dey–Kumar presented Nearest Neighbor (NN-Crust) algorithm [6] which is based on Crust. The algorithm constructs simple close
Fig. 1. Curve reconstruction problem.
Fig. 2. A set of 3D points.
Fig. 3. Each slice is constructed.
825
smooth curves by connecting each point to its nearest neighbor; then, for each point p that is incident to only one edge e, the algorithm connects p to its nearest neighbor in the other halfspace orthogonal to e. In fact, NN-Crust is based on Crust but with better sampling density. To deal with open curves, Dey–Mehlhorn– Ramos proposed Conservative Crust [7]. The algorithm constructs the curve by computing Delaunay triangulation on the sampling points and choosing only Delaunay edge e which has an empty ball of Voronoi vertices centering at the midpoint of e with radius l(e)/p where p is a parameter for the algorithm and l(e) is the length of edge e. Then it filters all chosen edges e that have a large enough ball centering at the midpoint of e containing a 01 or 11 vertices. Conservative Crust presents g as a constant c multiplied by parameter p. Later, Funke–Ramos used Conservative Crust with different sampling condition near corner points to guarantee construction of open non-smooth curves [13]. In 1999, Dey– Wenger proposed an algorithm, named Gathan, which can construct curves with sharp corners [10]. The algorithm requires different sampling conditions near non-smooth region and smooth region of the curve. The sampling density near nonsmooth region is dependent on the angle of the sharp corner. Nevertheless, Dey–Wenger does not guarantee the reconstruction result of their algorithm. In 2002, a guarantee version of Gathan is introduced called GathanG [11]. Another algorithm can reconstruct non-smooth curve is based on traveling salesman problem (TSP). TSP-based algorithm defines a modified cost function to set the sampling condition for every two adjacent sample points on a curve [8]. This condition results in a sparser sampling density compared to Crust, NN-Crust, and Conservative-Crust [14]. According to Althaus–Mehlhorn, the TSP algorithm also works for the same sampling condition in the algorithms Crust, NN-Crust and Conservative-Crust [14]. However, TSP only handles close single curve. Recently, a parameter-free, distance-based algorithm DISCUR proposed in Ref. [15] requires neither parameters in the algorithm nor parameters in the sampling condition. Based on a vision function defined in terms of distances, the algorithm aims to construct curves that agree with human vision system. According to proximity and continuity rules of Gestalt laws, human visual system tends to group nearest points given that they are close enough and human eyes tend to connect points to form a smooth curve [16]. Algorithm DISCUR connects the sample points based merely on proximity. This means constructing curves based on proximity property also guarantees continuity property of the curves. Guaranteed reconstruction of DISCUR algorithm is based on two theorems. The first theorem provides sampling condition for sampling interior points while the second theorem ensures that boundaries are detected correctly. DISCUR is proved to correctly reconstruct non-smooth open curves. However, since the algorithm uses only nearness to quantify human visual perception,
Fig. 4. Constructed curves and surfaces in 3D.
ARTICLE IN PRESS 826
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 5. Wrong construction by DISCUR [15]. (a) Sampling point and (b) reconstructed result by DISCUR.
An open polyline T ¼ ½p1 ; p2 ; . . . ; pm is called a-smooth if: p apffðpi1 pi piþ1 Þpp þ a;
Fig. 6. Desired result [15].
wrong construction may occur when a sample has more than one nearest neighbors. The shortcoming of DISCUR is demonstrated in Fig. 5 where there are two neighbors with equal distance, DISCUR will arbitrarily choose one of them to make a connection and wrong construction occurs as is shown in Fig. 5(b). The desired reconstruction should be as shown in Fig. 6. Another issue with DISCUR is that although it can construct non-smooth curve, a very dense sampling is required near the sharp corner area as shown in Fig. 7(c). If smoothness property is considered, sparser sampling density can be sufficient because human eyes are capable of recognizing the curve in Fig. 7(b) from the sample in Fig. 7(a). To overcome the problems implied in the algorithm DISCUR, a preliminary effort of applying the experimental method was made in the author’s research group in an attempt to include the continuity property in the vision function [17]. A group of experiments were conducted to find out the relation between the connectivity of a point to a reconstructed curve segment. The experiments were focused on factors such as point size, the number of points that should be considered from the reconstructed curve segment, and statistical properties of the concerned curve segment. The algorithm developed in Ref. [17] can generate comparable results to those from existing ones under some circumstances, which shows that it is possible to obtain the vision function through experiments. However, its advantages are not obvious. By redesigning the experiments, vision function, and the connection rules, a robust human-vision-based curve reconstruction algorithm, named VICUR, is proposed in this paper. Numerical experiments show that the algorithm VICUR’s performance conforms well to human visual system. The rest of this paper is organized as follows. Section 2 introduces the structure of the algorithm VICUR. VICUR can construct the sample points to form a pattern that looks natural to human visual system. Section 3 presents experimental results to show the effectiveness of the algorithm; limit of VICUR is also discussed in this section. Finally, conclusion is given in Section 4.
2. VICUR algorithm 2.1. Notations and definitions 2.1.1. Characteristics of curves The distance between any two points p and q in a point set is their Euclidean distance, denoted by d(p, q) ¼ ||pq||. A polyline T is a piecewise linear curve and is denoted as T ¼ ½p1 ; p2 ; . . . ; pm where p1, p2, y, pm are vertices on the polyline and pi apiþ1 for all i ¼ 1, 2, y, m1. If p16¼pm, then T is open; otherwise, T is closed.
i ¼ 2; 3; . . . ; m 1,
(1)
where ffðpi1 pi piþ1 Þ is the angle at vertex pi. How the angle at a vertex is computed is decided by the candidate angle which is defined in next section. Value a determines the smoothness of the curve. The smaller the value a is, the smoother the curve is. In this paper, we set a ¼ 451 for all the experiments. 2.1.2. Connectivity area We observe that human eyes tend to connect a point to an existing curve when the point lies within a certain area determined by the characteristics of the curve. We name this area connectivity area, which is illustrated in Fig. 8. The connectivity area at an endpoint p1, denoted as Aðp1 ; Rp1 Þ, is a set of points having the probability of connection to p1 greater than 0 and is defined as a sector of a circle centering at p1 with center angle y and radius Rp1 ¼ bd¯ where b is a parameter and d¯ is the average distance of the a-smooth curve defined in Eq. (1). All points within connectivity area are called candidate points. The segment between candidate point and curve endpoint is called a candidate segment. The line segment incident to a curve endpoint is called ending segment. Points fall outside the bounded area are considered as outliers. In Fig. 8, p is a reference point, p0 is an outlier with respect to p1, pp1 is a candidate segment, and p1p2 and p4p5 are curve-ending segments. The edge p1p2 is the bisector of +(Mp1N). The angle between candidate segment pp1 and curve ending segment p1p2 is called candidate angle and is defined as a clockwise rotation from candidate segment to curve ending segment. The choice of angle at each vertex is determined by the candidate angle. It is obvious that a curve divides the plane into two parts so the angle at each vertex is chosen to be in the part which contains the candidate angle. In Fig. 8(b), a is the candidate angle and a1 is the angle at vertex p2. 2.1.3. Vision function When there are two or more than two sampling points in a connectivity area, all these samples are candidates to be considered for connection to the corresponding curve endpoint. Thus, it is necessary to define a criterion to evaluate the possibility of the connectivity for each sample in the connectivity area. We express this criterion as a dynamic function: E½p; T p1 ¼ f ðp; VÞ,
(2)
where E½p; T p1 is the connectivity value between sample p and curve endpoint p1, T p1 is an a-smooth segment of the curve and V is a vector that comprises statistical properties of the curve segment T p1 , such as the distance mean, distance standard deviation, angle mean, and angle standard deviation. We have conducted several experiments on how human connect points to construct a curve. From the collected data, we derived five factors which have the most impact on the construction process. These factors include candidate angle, length of candidate segment, average angle, average distance of the curve, and standard deviation of the distance. Based on these elements, a concrete
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
827
Fig. 7. Human vision can virtually construct a curve from a set of points when the points are well distributed. (a)Sample, (b) constructed curve and (c) DISCUR.
Fig. 9. Relationship between candidate angle and parameter, ds ¼ d¯ and a¯ ¼ 180 .
Fig. 8. Connectivity area is a bounded area. (a) Connectivity area and (b) candidate angle, candidate segment.
form of the function obtained through observation and experiment is given as follows: " E½p; T p1 ¼ c
a
s
a¯
1
2
þ
#1 2 1c ds þ1 , 4 d¯ þ s
(3)
where as is the candidate angle, c is the user-defined parameter, a¯ is the average angle, ds is the length of candidate segment; d¯ and s are defined in Eqs. (4) and (5) as follows: d¯ ¼
Pm1 i¼1
jjpi piþ1 jj , m1
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pm1 ¯ 2 i¼1 ðjjpi piþ1 jj dÞ s¼ . m2
(4)
(5)
We assume that if p and p1 have the highest value E½p; T p1 , then they can be connected. Fig. 9 illustrates the relationship among candidate angle as, parameter c and the connectivity value. Given ds ¼ d¯ and a¯ ¼ 180 , for any parameter c connectivity value is largest when the value of candidate angle as reaches the value of average angle a¯ . The effect of candidate angle as on connectivity value increases when parameter c approaches 1. When parameter c equals to 0, connectivity value remains unchanged regardless of the value of
Fig. 10. Relationship between candidate distance and parameter, as ¼ a¯ ¼ 180 .
candidate angle because at this time connectivity value is determined by candidate distance ds only. Fig. 10 shows the effect of candidate distance ds and parameter c on the connectivity value. When parameter c approaches 1, connectivity value is determined by candidate angle as. When candidate distance ds approaches 0 which means that candidate point is very near curve endpoint, the connectivity value rises
ARTICLE IN PRESS 828
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 12. Connectivity rule for two free points.
Fig. 11. Relationship between candidate distance and candidate angle, c ¼ 0.8.
substantially. When parameters c approaches 0, the impact of candidate distance ds on the connectivity value increases. In Fig. 11, given parameter c ¼ 0.8, the largest connectivity value occurs when candidate angle as reaches the mean angle value a¯ and candidate distance ds approaches 0. This means points which are very near to the constructed curve endpoint and also form the smoothest path with the constructed curve have the high possibility to connect to the curve. When the candidate angle as deviates from the angle mean a¯ , the connectivity value decreases. Similarly, when candidate distance ds is far from curve endpoint, connectivity value decreases. In summary, when connecting a point to a curve, two factors should be considered: distance from the point to the curve endpoint and the smoothness of the curve after the point is connected. If the nearest point to the curve endpoint also forms with the constructed curve the smoothest path, connection is easily determined. However, conflict arises when the nearest point does not form with the constructed curve a smooth path and when a point connects to the constructed curve resulting in the smoothest path is not the nearest point. To solve this difficulty, parameter c is introduced. When parameter c approaches 0, the nearness property becomes more important than the smoothness property. In this case, the algorithm tries to connect the nearest neighbor rather than constructing a smooth curve. On the other hand, when parameter c approaches 1, the algorithm tries to maintain the smoothness of the curve. 2.2. Connectivity rules Curve reconstruction can be considered as the combination of three problems: connecting two points, connecting a point to a curve and connecting two curves. To deal with these three problems, the two Gestalt principles of perception are adopted: proximity and continuity. Firstly, Gestalt theory indicates that human eyes perceive close elements as group. In the context of curve reconstruction, this means human tend to connect nearest points to form a curve. However, in some cases two closest neighbors are not necessarily two adjacent points on a curve. For example, in the case of sharp corner illustrated in Fig. 12 sample p2 and p3 are the nearest neighbors to each other but human eyes do not see them being adjacent on the curve. Therefore, an attempt to connect any two nearest free points may result in wrong connection. In this case, we observe that the shortest edge p2p3 forms with p3p5 (or p3p4) an angle smaller than +(p1p3p5). Therefore, before trying to connect p2 to p3, it should be checked if there is an angle g at p3 (or p2) formed by p2p3 and other incident
edges such that g is larger than other angle at p3 (or p2) formed by edges other than p2p3. For any angle formed by two edges, we consider interior angle. However, there is a case where the edge between two free points forms with other edge a largest angle but these two points should not be connected. The situation is illustrated in Fig. 12 where +(p2p3p6) is the largest angle but p2 should not connect to p3. This leads to another observation: there is a threshold for which all the points can be considered as ‘a group’. Beyond this value, a point is seen as outlier or belongs to another group. This observation can be considered as an attempt to quantify proximity law of Gestalt theory. Based on our experiment results, we compute the threshold as the average distance of the shortest and second shortest Delaunay edges multiplied by a constant. Secondly, Gestalt law of continuity states that human eyes connect points into the smoothest path. This property of perception suggests that after a point is connected to a curve, it should not introduce abrupt change to the statistical properties of the curve [15]. To achieve this objective, a vision function introduced in Section 2.1.3 should be used. Based on the above assumptions, we propose the following rules to determine connection between two samples. Rule 1. (Point–point connectivity). For any shortest edge e ¼ [q1, q2] where q1 and q2 are both free points. Let B(q1, r) be a ball centered at q1 with radius r ¼ 1=2fðq1 qk1 þ q1 qk2 Þ where q1 qk1 and q1 qk2 are the shortest and second shortest Delaunay edge to q1. If there exists a qi such that ffðqt q1 qj Þoffðqi q1 q2 Þ for all qi ; qj ; qt 2 B and q2 efqi ; qj ; qt g where +(qtq1qj) and +(qiq1q2) are the interior angles at q1, then q1 and q2 can be connected. Rule 2. (Point–curve connectivity). For an a-smooth curve T ¼ ½p1 ; p2 ; . . . ; pm ; m41, which is partially reconstructed from a sample set S, if there exists a sampling point pi 2 Aðp1 ; Rp1 Þ (or Aðpm ; Rpm Þ) such that E½pi ; T p1 4E½pj ; T p1 (or E½pi ; T pm 4E½pj ; T pm ) for all pj 2 Aðp1 ; Rp1 Þ (or Aðpm ; Rpm Þ), then pi and p1 (or pm) can be connected. Rule 3. (Curve–curve connectivity). For two a-smooth curves T ¼ ½p1 ; p2 ; . . . ; pm ; T 0 ¼ ½q1 ; q2 ; . . . ; qn ; m; n41 which is partially reconstructed from a sample set S, if p1 (or pm) can connect to T0 by Rule 1 or q1 (or qn) can connect to T by Rule 1, then these two curves can be connected. 2.3. Algorithm Algorithm (input: sample S) Step 1: Compute Delaunay triangulation. Let D be a set of Delaunay edges and R be a set of temporarily removed edges. Initially, R’0.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
829
Step 2: Let e ¼ ½pi ; pj 2 D be a shortest Delaunay edge. There are three possibilities for e: (1) Both pi and pj are free points. Apply Rule 1 with f ¼ 1.849. If pi and pj cannot be connected, then R R [ ½pi ; pj . Otherwise, connect pi to pj. (2) Point pi is an endpoint of curve T and pj is a free point. Consider only a-smooth T pi of curve T. Construct connectivity area Aðpi ; Rpi Þ with b ¼ 1.849 and y ¼ 2701 If there does not exist a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then R R [ ½pi ; pj . If there exists only one point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then connect pi to q. If there exists more than one point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then compute the connectivity value E½q; T pi for each q. Choose the point q with the largest corresponding connectivity value to connect to pi. (3) Both pi and pj are endpoints. For each pi, pj where pi is an endpoint of curve T and pj is an 0 endpoint of curve T0, consider only the a-smooth T pi , T pi of curve T and T0, respectively. Construct connectivity area Aðpi ; Rpi Þ and Aðpj ; Rpj Þ If there does not exist a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there does not exist a point q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R then R R [ ½pi ; pj . If there exists a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there does not exist a point q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R then connect pi to q. If there exists point(s) q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there exists point(s) q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R, then compute connectivity value E½q; T pi and E½q0 ; T pj . Choose the point with the largest connectivity value to connect to corresponding endpoint.
Fig. 13. Input sample and Delaunay triangulation.
Remove [pi, pj] from D. Repeat Step 2 until D becomes empty set. Step 3: As long as D becomes empty, remove all edges in set R and put them in set D. Apply (1), (2) and (3) in Step 2. However, this time when pi is connected to pj to form a curve T00, examine T00 for potential extension by exploring all edges e 2 fD [ Rg adjacent to T00, and apply (1), (2) and (3) in Step 2 for each edge e found. The exploration stops when curve T00 can no longer be extended at both endpoints. Step 3 terminates when D becomes empty. The main idea of the algorithm above is that we first find the closest pair of sampling points. If two samples are free, apply Rule 1 in Section 2.2. If one of the samples is not free or both of the samples are not free, construct connectivity area and apply Rules 2 or 3. Currently, value b and f are set to 1.849 because of many successful experiments. Fig. 14. Construction in step two of VICUR algorithm.
3. Experimental results and discussions 3.1. Results of the algorithm In this section, procedure of VICUR algorithm will be demonstrated. Fig. 13 shows input sample and corresponding Delaunay triangulation. Fig. 14 shows that three situations may occur in Step 2 of the algorithm. The three situations are described as follows:
Fig. 14(a)) shows that p1p2 is the shortest Delaunay edge and both p1 and p2 are free points. The algorithm checks if connection between p1 and p2 may result in potentially wrong connection by drawing a ball Bðp1 ; Rp1 Þ as shown in Fig. 14(b) where Rp1 ¼ 1=2fðp1 p2 þ p1 p5 Þ. There are six samples in the
ball B : p1 ; p2 ; p3 ; p4 ; p5 ; p8 . We find that ffp4 p1 p5 4ffp2 p1 p8 so p1 and p2 fails the checking test and is temporarily removed. Fig. 14(c)) shows p7p8 is the shortest Delaunay edge where p7 is an endpoint of T p7 ¼ ½p7 ; p6 and p8 is endpoint of T p8 ¼ ½p8 ; p9 ; p10 ; p11 ; p12 ; p13 . Construct connectivity area Aðp7 ; Rp7 Þ at 0 0 p7 for T p7 and construct Aðp8 ; Rp8 Þ at p8 for T p8 because T p8 ¼ ½p8; p9 ; p10 is an a-smooth segment of curve T p8 . Fig. 14(d) shows p7 2 Aðp8 ; Rp8 Þ and p8 2 Aðp7 ; Rp7 Þ. Thus, p7 and p8 is connected and p7p8 is removed from Delaunay set. Segment p5p1 is the shortest Delaunay edge, p1 is an endpoint of T p1 ¼ ½p1 ; p4 ; p3 ; p2 and p5 is free point. We have [p1, p4] which is an edge of the a-smooth curve T p1 ¼ ½p1 ; p4 ; p3 ; p2 . Construct the connectivity area Aðp1 ; Rp1 Þ as shown in Fig. 14(f).
ARTICLE IN PRESS 830
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Vertex p2 is the only sample in Aðp1 ; Rp1 Þ available for connection so p1 and p2 are connected and removed from Delaunay set. Step 2 repeats until Delaunay set becomes empty. Fig. 15 shows Step 3 of the algorithm. In Step 3, all the temporarily removed edges in Step 2 are reconsidered because during construction process, the curve may be updated which makes connectivity value and connectivity area changed. Fig. 15 shows that all the temporarily removed edges are converted to Delaunay edges for reconsideration. Repeat Step 2 on new Delaunay edge set. The reconstructed result is depicted in Fig. 16. To highlight the effectiveness of VICUR, Fig. 17 shows an intuitive sample set including multiple single open and closed curves. Reconstructed curves from VICUR algorithm conforms to human vision. Also, constructed results from other algorithms are presented in Fig. 18.
Fig. 15. Step three of VICUR algorithm.
3.2. Sampling condition Most of the current algorithms use medial axis or local feature size to determine sampling condition. As a result, the sample sets for those algorithms are not intuitive. In contrast, VICUR algorithm aims to work for intuitive sampling condition, which implies that the samples are connected to form curves that look natural to human eyes. Intuitive sampling condition helps users to have better control over the sample set. For instance, when construction goes wrong, users just need to use their perception to adjust the sample set and construction should be correct. Fig. 19 shows an example where connection between points from the new algorithm agrees with human perception compared with other algorithms. 3.3. Boundary and sharp corner Most of the current algorithms can construct closed smooth curves correctly. To reconstruct non-smooth curve, Giesen developed TSP but the algorithm can construct only single closed curves [9]. Dey–Wenger introduced another algorithm named Gathan which can detect corner point and endpoint well in practice but with no guarantees [10]. Later, they proposed GathanG algorithm based on Gathan [11]. GathanG guarantees correct construction on closed curves but not on open curves. Fig. 20 shows a situation where GathanG fails. The algorithm DISCUR presented in Ref. [15] can also handle sharp corners but sampling is very dense near the corner compared to our new algorithm as shown in Fig. 21. Our algorithm correctly construct curve with the sample given in Fig. 21(a) while DISCUR needs a denser sampling, Fig. 21(d). Ramos–Funke also proposed another algorithm that can construct open non-smooth curves with guarantees [13]. However, due to limited resources we did not do any experiment with their algorithm. The only comparison we did is the MPI data set taken from the article as shown in Fig. 22 [13]. 3.4. Discussion
Fig. 16. Final result.
Despite the fact that VICUR can handle well many examples, we are aware of the limit of our algorithm. Firstly, VICUR is sensitive to vertex position. Fig. 23 illustrates such a situation where human eyes hardly realize a difference between 461 and 431. Construction result of Fig. 23(a) and (c) should be similar. However, given y ¼ 2701, from the algorithm perspective, p3 in Fig. 23(a) is considered within connectivity area boundary whereas in Fig. 23(c) p3 is considered out of the range. As a
Fig. 17. Reconstructed curves by VICUR. (a) Sample and (b) VICUR.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 18. Reconstructed curves by other algorithms. (a) NN-Crust, (b)Crust, (c) conservative Crust and (d) GathanG.
Fig. 19. Result from VICUR is consistent with human visual system. (a) Sample set, (b) Crust, (c) NN-Crust, (d) GathanG, (e) DISCUR and (f) VICUR.
831
ARTICLE IN PRESS 832
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 20. GathanG fails to construct open curves. (a) Sample set, (b) GathanG and (c) our algorithm.
Fig. 21. DICUR requires dense sampling around corner point. (a) Sample set, (b) our algorithm, (c) DISCUR and (d) Sample set for DISCUR.
Fig. 22. Result of MPI data set from Funke–Ramos’s article. (a) Funke–Ramos’s algorithm and (b) VICUR.
result, sample p3 cannot be connected to p1, causing construction result Fig. 23(b) and (d) to be different. Secondly, although checking between two free points prior to connection helps avoid wrong connection in case of sharp corner, sometimes it may create a problem as shown in Fig. 24(b). In this
case, VICUR detects +(p1p2p3) as a sharp corner with p2 as a corner point. Consequently, the algorithm does not connect p1 to p3 and construction result becomes unnatural to human vision. However, this problem can be fixed easily by increasing the sampling density, as is shown in Fig. 24.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Additionally, we will try to find more accurate parameters to better evaluate connectivity value and to better identify connectivity area. Currently, based on numbers of experiments y are set at 2701, b and f are set at 1.849. In the future, we aim to find optimal value that correctly constructs all sample sets. Most of the examples work best with c ¼ 0.8. However, in some cases, the
833
curve is constructed correctly with c ¼ 0.7 or c ¼ 0.9 as illustrated in Fig. 25.
4. Conclusion We proposed a new algorithm for curve reconstruction named VICUR. Foundation of VICUR algorithm is established from two laws of Gestalt theories of perceptual organization: law of proximity and law of continuity. VICUR can construct open, non-smooth curve and the result is agreeable with human perception. Our algorithm is developed based on statistical data obtained from experiments. Motivation for our algorithm is not only to provide a new approach to curve reconstruction problem but also to attempt to quantify some properties of human visual perception. Although our algorithm does not have a mathematical proof to guarantee its correctness, the complexity of our successful experiments shows the effectiveness of our algorithm. The future work will include the development of a better vision function based on more comprehensive experiments.
Acknowledgments
Fig. 23. VICUR is sensitive to vertex position. (a) +(p2p1p3) ¼ 461, (b) construction result (c) +(p2p1p3) ¼ 431 and (d) construction result.
We would like to thank Mr. Guangqing He for his discussions about the vision function during the research. Without data from his experiment, we can hardly find the appropriate parameter value for our algorithm within a short period of time. In addition, we would like to express sincere thanks to Mr. Shuren Li for building a basic curve-reconstruction application which facilitates our task for implementing and testing our algorithm.
Fig. 24. Wrong construction. (a) Sample set (b) wrong connection and (c) Sampling density increased.
Fig. 25. Different parameters yield different results. (a) Sample set (b) c ¼ 0.8 and (c) c ¼ 0.9.
ARTICLE IN PRESS 834
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
References [1] Edelsbrunner H, Kirkpatrick D, Seidel R. On the shape of a set of points in the plane. IEEE Trans Inf Theory 1983;29:71–8. [2] Attali D. r-Regular shape reconstruction from unorganized points. In: Proceedings of the 13th annual ACM symposium on computational geometry, 1997. p. 248–53. [3] Henrique de Figueiredo L, Gomes J. Computational morphology of curves. Vis Comput 1995;11:105–12. [4] Amenta N, Bern M, Eppstein D. The crust and the beta-skeleton: combinatorial curve reconstruction. Graph Models Image Process: GMIP 1998;60: 125–35. [5] Gold C. Crust and anti-crust: a one-step boundary and skeleton extraction algorithm. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 189–96. [6] Dey TK, Kumar P. A simple provable algorithm for curve reconstruction. In: Proceedings of the 10th annual ACM-SIAM symposium on discrete algorithms, 1999. p. 893–4. [7] Dey TK, Mehlhorn K, Ramos E. Curve reconstruction: connecting dots with good reason. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 197–206.
[8] Althaus E, Mehlhorn K. TSP-based curve reconstruction in polynomial time. In: Symposium on discrete algorithms, 2000. p. 686–95. [9] Giesen J. Curve reconstruction, the TSP, and Menger’s theorem on length. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 207–16. [10] Dey TK, Wenger R. Reconstructing curves with sharp corners. Comput Geom Theory Appl 2001;19:89–99. [11] Dey TK, Wenger R. Fast reconstruction of curves with sharp corners. Int J Comput Geom Appl 2002:353–400. [12] Dey TK. Curve and surface reconstruction. Cambridge University Press; 2006. [13] Funke S, Ramos EA. Reconstructing a collection of curves with corners and endpoints. In: Symposium on discrete algorithms, 2001. p. 344–53. [14] Althaus E, Mehlhorn K, Naher S, Schirra S. Experiments on curve reconstruction. In: Proceedings of the 2nd workshop algorithm eng. exper., 2000. p. 103–14. [15] Zeng Y, Nguyen TA, Yan BQ, Li SR. A distance-based parameter free algorithm for curve reconstruction. Comput Aided Des 2008, in press. [16] Mather G. Foundations of perception. Comput Aided Des 2008;40:210–22. [17] Li SR. Vision based curve reconstruction. Master thesis, Concordia University, Canada, 2007.
Contents lists available at ScienceDirect
Robotics and Computer-Integrated Manufacturing journal homepage: www.elsevier.com/locate/rcim
VICUR: A human-vision-based algorithm for curve reconstruction Thanh An Nguyen, Yong Zeng Faculty of Engineering and Computer Science, Concordia Institute for Information Systems Engineering, Concordia University, 1455 de Maisonneuve Blvd. West, EV07.633, Montreal, Quebec, Canada H3G 1M8
article info
a b s t r a c t
Keywords: Curve reconstruction Human vision system Proximity Continuation Gestalt law
In this paper, a new algorithm, named VICUR, is presented for curve reconstruction problem. From a set of unorganized points, the proposed algorithm can construct curves that look natural to human vision. The VICUR algorithm is based on two connectivity criteria: proximity and good continuation from the prominent Gestalt principles of perception. Experimental results are presented to show the effectiveness of VICUR. & 2008 Elsevier Ltd. All rights reserved.
1. Introduction 1.1. Background The main problem of curve reconstruction is constructing a polygonal representation of an unknown curve from a set of sample points, as shown in Fig. 1. Curve reconstruction has many applications in manufacturing engineering, biomedical engineering, computer vision, and pattern recognition. One example can be found in the reconstruction of 3D image that models a real object. In general, the process is divided into three main steps. In the first step, technologies such as Magnetic Resonance Imaging (MRI), X-ray Computed Tomography (CT) or laser scanner takes digitized image slices from the object. Boundary points of each slice are then extracted. Fig. 2 shows the point cloud produced by a 3D digitizer. At this step, each slice is represented by a set of points, called sampled points. In the second step, the sampled points on each slice will be connected to reconstruct curves as illustrated in Fig. 3. Each slice may include multiple curves with various features such as sharp corners or boundaries. The sampled points may be distributed uniformly or non-uniformly. In the last step, constructed curves are assembled to form a 3D image. Fig. 4 shows an example where the constructed slices from Fig. 3 are put together. Because of its importance in various application domains, curve reconstruction has been attracting numerous research attentions over the last three decades. In general, existing algorithms construct only simple curves. A simple curve is a curve without self-intersection. There are mainly two kinds of simple curves: open and closed curves. A simple curve may Corresponding author. Tel.: +1 514 8482424x5801; fax: +1 514 8483171.
E-mail address: [email protected] (Y. Zeng). 0736-5845/$ - see front matter & 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.rcim.2008.03.007
contain sharp corner. A simple curve without sharp corner is called smooth curve. All these curves can be sampled either uniformly or non-uniformly. There are a few algorithms working for uniformly sampled curves such as alpha-shape [1], r-regular shape [2], and EMST [3]. There are other algorithms that work for non-uniformly sampled curves such as Crust [4,5], Nearest Neighbor [6], Conservative Crust [7], Traveling Salesman Path [8,9], and Gathan [10,11]. Non-uniform sampling allows sparser sampling at less detailed section of the curve while uniform sampling unnecessarily requires dense sampling in areas where sparse sampling should be enough. Most of the existing algorithms work for non-uniform sample requires the sampling points to be an e-sample. A point set P is called an e-sample of a curve T if any point p on T has a sample within distance gf(p) where g is a constant factor and f(p) is the local feature size at p defined as the minimum Euclidean distance from p to the medial axis [12]. In the curve reconstruction problem, the original curve is not known and the sample set is taken arbitrarily. Most of the existing algorithms provide a sampling condition with respect to geometric properties, which may result in a sample set not natural to human vision. When some connections go wrong, it is not easy or intuitive to find a sample to produce the right reconstruction. This is an inspiration to develop an algorithm that is able to connect a set of unorganized points to form a pattern that is consistent with human visual system. 1.2. Literature reviews The first provable curve reconstruction algorithm for simple close smooth curves is given by Amenta–Bern–Eppstein named Crust [4]. The paper proves that for gp0.252 the polygonal reconstruction of a curve is the crust. The crust is constructed by computing Delaunay triangulation on the set of sampling points
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
and Voronoi vertices, and choosing only Delaunay edges which have the endpoints belonging to the set of sampling points. Later, Dey–Kumar presented Nearest Neighbor (NN-Crust) algorithm [6] which is based on Crust. The algorithm constructs simple close
Fig. 1. Curve reconstruction problem.
Fig. 2. A set of 3D points.
Fig. 3. Each slice is constructed.
825
smooth curves by connecting each point to its nearest neighbor; then, for each point p that is incident to only one edge e, the algorithm connects p to its nearest neighbor in the other halfspace orthogonal to e. In fact, NN-Crust is based on Crust but with better sampling density. To deal with open curves, Dey–Mehlhorn– Ramos proposed Conservative Crust [7]. The algorithm constructs the curve by computing Delaunay triangulation on the sampling points and choosing only Delaunay edge e which has an empty ball of Voronoi vertices centering at the midpoint of e with radius l(e)/p where p is a parameter for the algorithm and l(e) is the length of edge e. Then it filters all chosen edges e that have a large enough ball centering at the midpoint of e containing a 01 or 11 vertices. Conservative Crust presents g as a constant c multiplied by parameter p. Later, Funke–Ramos used Conservative Crust with different sampling condition near corner points to guarantee construction of open non-smooth curves [13]. In 1999, Dey– Wenger proposed an algorithm, named Gathan, which can construct curves with sharp corners [10]. The algorithm requires different sampling conditions near non-smooth region and smooth region of the curve. The sampling density near nonsmooth region is dependent on the angle of the sharp corner. Nevertheless, Dey–Wenger does not guarantee the reconstruction result of their algorithm. In 2002, a guarantee version of Gathan is introduced called GathanG [11]. Another algorithm can reconstruct non-smooth curve is based on traveling salesman problem (TSP). TSP-based algorithm defines a modified cost function to set the sampling condition for every two adjacent sample points on a curve [8]. This condition results in a sparser sampling density compared to Crust, NN-Crust, and Conservative-Crust [14]. According to Althaus–Mehlhorn, the TSP algorithm also works for the same sampling condition in the algorithms Crust, NN-Crust and Conservative-Crust [14]. However, TSP only handles close single curve. Recently, a parameter-free, distance-based algorithm DISCUR proposed in Ref. [15] requires neither parameters in the algorithm nor parameters in the sampling condition. Based on a vision function defined in terms of distances, the algorithm aims to construct curves that agree with human vision system. According to proximity and continuity rules of Gestalt laws, human visual system tends to group nearest points given that they are close enough and human eyes tend to connect points to form a smooth curve [16]. Algorithm DISCUR connects the sample points based merely on proximity. This means constructing curves based on proximity property also guarantees continuity property of the curves. Guaranteed reconstruction of DISCUR algorithm is based on two theorems. The first theorem provides sampling condition for sampling interior points while the second theorem ensures that boundaries are detected correctly. DISCUR is proved to correctly reconstruct non-smooth open curves. However, since the algorithm uses only nearness to quantify human visual perception,
Fig. 4. Constructed curves and surfaces in 3D.
ARTICLE IN PRESS 826
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 5. Wrong construction by DISCUR [15]. (a) Sampling point and (b) reconstructed result by DISCUR.
An open polyline T ¼ ½p1 ; p2 ; . . . ; pm is called a-smooth if: p apffðpi1 pi piþ1 Þpp þ a;
Fig. 6. Desired result [15].
wrong construction may occur when a sample has more than one nearest neighbors. The shortcoming of DISCUR is demonstrated in Fig. 5 where there are two neighbors with equal distance, DISCUR will arbitrarily choose one of them to make a connection and wrong construction occurs as is shown in Fig. 5(b). The desired reconstruction should be as shown in Fig. 6. Another issue with DISCUR is that although it can construct non-smooth curve, a very dense sampling is required near the sharp corner area as shown in Fig. 7(c). If smoothness property is considered, sparser sampling density can be sufficient because human eyes are capable of recognizing the curve in Fig. 7(b) from the sample in Fig. 7(a). To overcome the problems implied in the algorithm DISCUR, a preliminary effort of applying the experimental method was made in the author’s research group in an attempt to include the continuity property in the vision function [17]. A group of experiments were conducted to find out the relation between the connectivity of a point to a reconstructed curve segment. The experiments were focused on factors such as point size, the number of points that should be considered from the reconstructed curve segment, and statistical properties of the concerned curve segment. The algorithm developed in Ref. [17] can generate comparable results to those from existing ones under some circumstances, which shows that it is possible to obtain the vision function through experiments. However, its advantages are not obvious. By redesigning the experiments, vision function, and the connection rules, a robust human-vision-based curve reconstruction algorithm, named VICUR, is proposed in this paper. Numerical experiments show that the algorithm VICUR’s performance conforms well to human visual system. The rest of this paper is organized as follows. Section 2 introduces the structure of the algorithm VICUR. VICUR can construct the sample points to form a pattern that looks natural to human visual system. Section 3 presents experimental results to show the effectiveness of the algorithm; limit of VICUR is also discussed in this section. Finally, conclusion is given in Section 4.
2. VICUR algorithm 2.1. Notations and definitions 2.1.1. Characteristics of curves The distance between any two points p and q in a point set is their Euclidean distance, denoted by d(p, q) ¼ ||pq||. A polyline T is a piecewise linear curve and is denoted as T ¼ ½p1 ; p2 ; . . . ; pm where p1, p2, y, pm are vertices on the polyline and pi apiþ1 for all i ¼ 1, 2, y, m1. If p16¼pm, then T is open; otherwise, T is closed.
i ¼ 2; 3; . . . ; m 1,
(1)
where ffðpi1 pi piþ1 Þ is the angle at vertex pi. How the angle at a vertex is computed is decided by the candidate angle which is defined in next section. Value a determines the smoothness of the curve. The smaller the value a is, the smoother the curve is. In this paper, we set a ¼ 451 for all the experiments. 2.1.2. Connectivity area We observe that human eyes tend to connect a point to an existing curve when the point lies within a certain area determined by the characteristics of the curve. We name this area connectivity area, which is illustrated in Fig. 8. The connectivity area at an endpoint p1, denoted as Aðp1 ; Rp1 Þ, is a set of points having the probability of connection to p1 greater than 0 and is defined as a sector of a circle centering at p1 with center angle y and radius Rp1 ¼ bd¯ where b is a parameter and d¯ is the average distance of the a-smooth curve defined in Eq. (1). All points within connectivity area are called candidate points. The segment between candidate point and curve endpoint is called a candidate segment. The line segment incident to a curve endpoint is called ending segment. Points fall outside the bounded area are considered as outliers. In Fig. 8, p is a reference point, p0 is an outlier with respect to p1, pp1 is a candidate segment, and p1p2 and p4p5 are curve-ending segments. The edge p1p2 is the bisector of +(Mp1N). The angle between candidate segment pp1 and curve ending segment p1p2 is called candidate angle and is defined as a clockwise rotation from candidate segment to curve ending segment. The choice of angle at each vertex is determined by the candidate angle. It is obvious that a curve divides the plane into two parts so the angle at each vertex is chosen to be in the part which contains the candidate angle. In Fig. 8(b), a is the candidate angle and a1 is the angle at vertex p2. 2.1.3. Vision function When there are two or more than two sampling points in a connectivity area, all these samples are candidates to be considered for connection to the corresponding curve endpoint. Thus, it is necessary to define a criterion to evaluate the possibility of the connectivity for each sample in the connectivity area. We express this criterion as a dynamic function: E½p; T p1 ¼ f ðp; VÞ,
(2)
where E½p; T p1 is the connectivity value between sample p and curve endpoint p1, T p1 is an a-smooth segment of the curve and V is a vector that comprises statistical properties of the curve segment T p1 , such as the distance mean, distance standard deviation, angle mean, and angle standard deviation. We have conducted several experiments on how human connect points to construct a curve. From the collected data, we derived five factors which have the most impact on the construction process. These factors include candidate angle, length of candidate segment, average angle, average distance of the curve, and standard deviation of the distance. Based on these elements, a concrete
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
827
Fig. 7. Human vision can virtually construct a curve from a set of points when the points are well distributed. (a)Sample, (b) constructed curve and (c) DISCUR.
Fig. 9. Relationship between candidate angle and parameter, ds ¼ d¯ and a¯ ¼ 180 .
Fig. 8. Connectivity area is a bounded area. (a) Connectivity area and (b) candidate angle, candidate segment.
form of the function obtained through observation and experiment is given as follows: " E½p; T p1 ¼ c
a
s
a¯
1
2
þ
#1 2 1c ds þ1 , 4 d¯ þ s
(3)
where as is the candidate angle, c is the user-defined parameter, a¯ is the average angle, ds is the length of candidate segment; d¯ and s are defined in Eqs. (4) and (5) as follows: d¯ ¼
Pm1 i¼1
jjpi piþ1 jj , m1
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pm1 ¯ 2 i¼1 ðjjpi piþ1 jj dÞ s¼ . m2
(4)
(5)
We assume that if p and p1 have the highest value E½p; T p1 , then they can be connected. Fig. 9 illustrates the relationship among candidate angle as, parameter c and the connectivity value. Given ds ¼ d¯ and a¯ ¼ 180 , for any parameter c connectivity value is largest when the value of candidate angle as reaches the value of average angle a¯ . The effect of candidate angle as on connectivity value increases when parameter c approaches 1. When parameter c equals to 0, connectivity value remains unchanged regardless of the value of
Fig. 10. Relationship between candidate distance and parameter, as ¼ a¯ ¼ 180 .
candidate angle because at this time connectivity value is determined by candidate distance ds only. Fig. 10 shows the effect of candidate distance ds and parameter c on the connectivity value. When parameter c approaches 1, connectivity value is determined by candidate angle as. When candidate distance ds approaches 0 which means that candidate point is very near curve endpoint, the connectivity value rises
ARTICLE IN PRESS 828
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 12. Connectivity rule for two free points.
Fig. 11. Relationship between candidate distance and candidate angle, c ¼ 0.8.
substantially. When parameters c approaches 0, the impact of candidate distance ds on the connectivity value increases. In Fig. 11, given parameter c ¼ 0.8, the largest connectivity value occurs when candidate angle as reaches the mean angle value a¯ and candidate distance ds approaches 0. This means points which are very near to the constructed curve endpoint and also form the smoothest path with the constructed curve have the high possibility to connect to the curve. When the candidate angle as deviates from the angle mean a¯ , the connectivity value decreases. Similarly, when candidate distance ds is far from curve endpoint, connectivity value decreases. In summary, when connecting a point to a curve, two factors should be considered: distance from the point to the curve endpoint and the smoothness of the curve after the point is connected. If the nearest point to the curve endpoint also forms with the constructed curve the smoothest path, connection is easily determined. However, conflict arises when the nearest point does not form with the constructed curve a smooth path and when a point connects to the constructed curve resulting in the smoothest path is not the nearest point. To solve this difficulty, parameter c is introduced. When parameter c approaches 0, the nearness property becomes more important than the smoothness property. In this case, the algorithm tries to connect the nearest neighbor rather than constructing a smooth curve. On the other hand, when parameter c approaches 1, the algorithm tries to maintain the smoothness of the curve. 2.2. Connectivity rules Curve reconstruction can be considered as the combination of three problems: connecting two points, connecting a point to a curve and connecting two curves. To deal with these three problems, the two Gestalt principles of perception are adopted: proximity and continuity. Firstly, Gestalt theory indicates that human eyes perceive close elements as group. In the context of curve reconstruction, this means human tend to connect nearest points to form a curve. However, in some cases two closest neighbors are not necessarily two adjacent points on a curve. For example, in the case of sharp corner illustrated in Fig. 12 sample p2 and p3 are the nearest neighbors to each other but human eyes do not see them being adjacent on the curve. Therefore, an attempt to connect any two nearest free points may result in wrong connection. In this case, we observe that the shortest edge p2p3 forms with p3p5 (or p3p4) an angle smaller than +(p1p3p5). Therefore, before trying to connect p2 to p3, it should be checked if there is an angle g at p3 (or p2) formed by p2p3 and other incident
edges such that g is larger than other angle at p3 (or p2) formed by edges other than p2p3. For any angle formed by two edges, we consider interior angle. However, there is a case where the edge between two free points forms with other edge a largest angle but these two points should not be connected. The situation is illustrated in Fig. 12 where +(p2p3p6) is the largest angle but p2 should not connect to p3. This leads to another observation: there is a threshold for which all the points can be considered as ‘a group’. Beyond this value, a point is seen as outlier or belongs to another group. This observation can be considered as an attempt to quantify proximity law of Gestalt theory. Based on our experiment results, we compute the threshold as the average distance of the shortest and second shortest Delaunay edges multiplied by a constant. Secondly, Gestalt law of continuity states that human eyes connect points into the smoothest path. This property of perception suggests that after a point is connected to a curve, it should not introduce abrupt change to the statistical properties of the curve [15]. To achieve this objective, a vision function introduced in Section 2.1.3 should be used. Based on the above assumptions, we propose the following rules to determine connection between two samples. Rule 1. (Point–point connectivity). For any shortest edge e ¼ [q1, q2] where q1 and q2 are both free points. Let B(q1, r) be a ball centered at q1 with radius r ¼ 1=2fðq1 qk1 þ q1 qk2 Þ where q1 qk1 and q1 qk2 are the shortest and second shortest Delaunay edge to q1. If there exists a qi such that ffðqt q1 qj Þoffðqi q1 q2 Þ for all qi ; qj ; qt 2 B and q2 efqi ; qj ; qt g where +(qtq1qj) and +(qiq1q2) are the interior angles at q1, then q1 and q2 can be connected. Rule 2. (Point–curve connectivity). For an a-smooth curve T ¼ ½p1 ; p2 ; . . . ; pm ; m41, which is partially reconstructed from a sample set S, if there exists a sampling point pi 2 Aðp1 ; Rp1 Þ (or Aðpm ; Rpm Þ) such that E½pi ; T p1 4E½pj ; T p1 (or E½pi ; T pm 4E½pj ; T pm ) for all pj 2 Aðp1 ; Rp1 Þ (or Aðpm ; Rpm Þ), then pi and p1 (or pm) can be connected. Rule 3. (Curve–curve connectivity). For two a-smooth curves T ¼ ½p1 ; p2 ; . . . ; pm ; T 0 ¼ ½q1 ; q2 ; . . . ; qn ; m; n41 which is partially reconstructed from a sample set S, if p1 (or pm) can connect to T0 by Rule 1 or q1 (or qn) can connect to T by Rule 1, then these two curves can be connected. 2.3. Algorithm Algorithm (input: sample S) Step 1: Compute Delaunay triangulation. Let D be a set of Delaunay edges and R be a set of temporarily removed edges. Initially, R’0.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
829
Step 2: Let e ¼ ½pi ; pj 2 D be a shortest Delaunay edge. There are three possibilities for e: (1) Both pi and pj are free points. Apply Rule 1 with f ¼ 1.849. If pi and pj cannot be connected, then R R [ ½pi ; pj . Otherwise, connect pi to pj. (2) Point pi is an endpoint of curve T and pj is a free point. Consider only a-smooth T pi of curve T. Construct connectivity area Aðpi ; Rpi Þ with b ¼ 1.849 and y ¼ 2701 If there does not exist a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then R R [ ½pi ; pj . If there exists only one point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then connect pi to q. If there exists more than one point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R, then compute the connectivity value E½q; T pi for each q. Choose the point q with the largest corresponding connectivity value to connect to pi. (3) Both pi and pj are endpoints. For each pi, pj where pi is an endpoint of curve T and pj is an 0 endpoint of curve T0, consider only the a-smooth T pi , T pi of curve T and T0, respectively. Construct connectivity area Aðpi ; Rpi Þ and Aðpj ; Rpj Þ If there does not exist a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there does not exist a point q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R then R R [ ½pi ; pj . If there exists a point q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there does not exist a point q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R then connect pi to q. If there exists point(s) q 2 Aðpi ; Rpi Þ such that pi q 2 D [ R and there exists point(s) q0 2 Aðpj ; Rpj Þ such that pj q0 2 D [ R, then compute connectivity value E½q; T pi and E½q0 ; T pj . Choose the point with the largest connectivity value to connect to corresponding endpoint.
Fig. 13. Input sample and Delaunay triangulation.
Remove [pi, pj] from D. Repeat Step 2 until D becomes empty set. Step 3: As long as D becomes empty, remove all edges in set R and put them in set D. Apply (1), (2) and (3) in Step 2. However, this time when pi is connected to pj to form a curve T00, examine T00 for potential extension by exploring all edges e 2 fD [ Rg adjacent to T00, and apply (1), (2) and (3) in Step 2 for each edge e found. The exploration stops when curve T00 can no longer be extended at both endpoints. Step 3 terminates when D becomes empty. The main idea of the algorithm above is that we first find the closest pair of sampling points. If two samples are free, apply Rule 1 in Section 2.2. If one of the samples is not free or both of the samples are not free, construct connectivity area and apply Rules 2 or 3. Currently, value b and f are set to 1.849 because of many successful experiments. Fig. 14. Construction in step two of VICUR algorithm.
3. Experimental results and discussions 3.1. Results of the algorithm In this section, procedure of VICUR algorithm will be demonstrated. Fig. 13 shows input sample and corresponding Delaunay triangulation. Fig. 14 shows that three situations may occur in Step 2 of the algorithm. The three situations are described as follows:
Fig. 14(a)) shows that p1p2 is the shortest Delaunay edge and both p1 and p2 are free points. The algorithm checks if connection between p1 and p2 may result in potentially wrong connection by drawing a ball Bðp1 ; Rp1 Þ as shown in Fig. 14(b) where Rp1 ¼ 1=2fðp1 p2 þ p1 p5 Þ. There are six samples in the
ball B : p1 ; p2 ; p3 ; p4 ; p5 ; p8 . We find that ffp4 p1 p5 4ffp2 p1 p8 so p1 and p2 fails the checking test and is temporarily removed. Fig. 14(c)) shows p7p8 is the shortest Delaunay edge where p7 is an endpoint of T p7 ¼ ½p7 ; p6 and p8 is endpoint of T p8 ¼ ½p8 ; p9 ; p10 ; p11 ; p12 ; p13 . Construct connectivity area Aðp7 ; Rp7 Þ at 0 0 p7 for T p7 and construct Aðp8 ; Rp8 Þ at p8 for T p8 because T p8 ¼ ½p8; p9 ; p10 is an a-smooth segment of curve T p8 . Fig. 14(d) shows p7 2 Aðp8 ; Rp8 Þ and p8 2 Aðp7 ; Rp7 Þ. Thus, p7 and p8 is connected and p7p8 is removed from Delaunay set. Segment p5p1 is the shortest Delaunay edge, p1 is an endpoint of T p1 ¼ ½p1 ; p4 ; p3 ; p2 and p5 is free point. We have [p1, p4] which is an edge of the a-smooth curve T p1 ¼ ½p1 ; p4 ; p3 ; p2 . Construct the connectivity area Aðp1 ; Rp1 Þ as shown in Fig. 14(f).
ARTICLE IN PRESS 830
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Vertex p2 is the only sample in Aðp1 ; Rp1 Þ available for connection so p1 and p2 are connected and removed from Delaunay set. Step 2 repeats until Delaunay set becomes empty. Fig. 15 shows Step 3 of the algorithm. In Step 3, all the temporarily removed edges in Step 2 are reconsidered because during construction process, the curve may be updated which makes connectivity value and connectivity area changed. Fig. 15 shows that all the temporarily removed edges are converted to Delaunay edges for reconsideration. Repeat Step 2 on new Delaunay edge set. The reconstructed result is depicted in Fig. 16. To highlight the effectiveness of VICUR, Fig. 17 shows an intuitive sample set including multiple single open and closed curves. Reconstructed curves from VICUR algorithm conforms to human vision. Also, constructed results from other algorithms are presented in Fig. 18.
Fig. 15. Step three of VICUR algorithm.
3.2. Sampling condition Most of the current algorithms use medial axis or local feature size to determine sampling condition. As a result, the sample sets for those algorithms are not intuitive. In contrast, VICUR algorithm aims to work for intuitive sampling condition, which implies that the samples are connected to form curves that look natural to human eyes. Intuitive sampling condition helps users to have better control over the sample set. For instance, when construction goes wrong, users just need to use their perception to adjust the sample set and construction should be correct. Fig. 19 shows an example where connection between points from the new algorithm agrees with human perception compared with other algorithms. 3.3. Boundary and sharp corner Most of the current algorithms can construct closed smooth curves correctly. To reconstruct non-smooth curve, Giesen developed TSP but the algorithm can construct only single closed curves [9]. Dey–Wenger introduced another algorithm named Gathan which can detect corner point and endpoint well in practice but with no guarantees [10]. Later, they proposed GathanG algorithm based on Gathan [11]. GathanG guarantees correct construction on closed curves but not on open curves. Fig. 20 shows a situation where GathanG fails. The algorithm DISCUR presented in Ref. [15] can also handle sharp corners but sampling is very dense near the corner compared to our new algorithm as shown in Fig. 21. Our algorithm correctly construct curve with the sample given in Fig. 21(a) while DISCUR needs a denser sampling, Fig. 21(d). Ramos–Funke also proposed another algorithm that can construct open non-smooth curves with guarantees [13]. However, due to limited resources we did not do any experiment with their algorithm. The only comparison we did is the MPI data set taken from the article as shown in Fig. 22 [13]. 3.4. Discussion
Fig. 16. Final result.
Despite the fact that VICUR can handle well many examples, we are aware of the limit of our algorithm. Firstly, VICUR is sensitive to vertex position. Fig. 23 illustrates such a situation where human eyes hardly realize a difference between 461 and 431. Construction result of Fig. 23(a) and (c) should be similar. However, given y ¼ 2701, from the algorithm perspective, p3 in Fig. 23(a) is considered within connectivity area boundary whereas in Fig. 23(c) p3 is considered out of the range. As a
Fig. 17. Reconstructed curves by VICUR. (a) Sample and (b) VICUR.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 18. Reconstructed curves by other algorithms. (a) NN-Crust, (b)Crust, (c) conservative Crust and (d) GathanG.
Fig. 19. Result from VICUR is consistent with human visual system. (a) Sample set, (b) Crust, (c) NN-Crust, (d) GathanG, (e) DISCUR and (f) VICUR.
831
ARTICLE IN PRESS 832
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Fig. 20. GathanG fails to construct open curves. (a) Sample set, (b) GathanG and (c) our algorithm.
Fig. 21. DICUR requires dense sampling around corner point. (a) Sample set, (b) our algorithm, (c) DISCUR and (d) Sample set for DISCUR.
Fig. 22. Result of MPI data set from Funke–Ramos’s article. (a) Funke–Ramos’s algorithm and (b) VICUR.
result, sample p3 cannot be connected to p1, causing construction result Fig. 23(b) and (d) to be different. Secondly, although checking between two free points prior to connection helps avoid wrong connection in case of sharp corner, sometimes it may create a problem as shown in Fig. 24(b). In this
case, VICUR detects +(p1p2p3) as a sharp corner with p2 as a corner point. Consequently, the algorithm does not connect p1 to p3 and construction result becomes unnatural to human vision. However, this problem can be fixed easily by increasing the sampling density, as is shown in Fig. 24.
ARTICLE IN PRESS T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
Additionally, we will try to find more accurate parameters to better evaluate connectivity value and to better identify connectivity area. Currently, based on numbers of experiments y are set at 2701, b and f are set at 1.849. In the future, we aim to find optimal value that correctly constructs all sample sets. Most of the examples work best with c ¼ 0.8. However, in some cases, the
833
curve is constructed correctly with c ¼ 0.7 or c ¼ 0.9 as illustrated in Fig. 25.
4. Conclusion We proposed a new algorithm for curve reconstruction named VICUR. Foundation of VICUR algorithm is established from two laws of Gestalt theories of perceptual organization: law of proximity and law of continuity. VICUR can construct open, non-smooth curve and the result is agreeable with human perception. Our algorithm is developed based on statistical data obtained from experiments. Motivation for our algorithm is not only to provide a new approach to curve reconstruction problem but also to attempt to quantify some properties of human visual perception. Although our algorithm does not have a mathematical proof to guarantee its correctness, the complexity of our successful experiments shows the effectiveness of our algorithm. The future work will include the development of a better vision function based on more comprehensive experiments.
Acknowledgments
Fig. 23. VICUR is sensitive to vertex position. (a) +(p2p1p3) ¼ 461, (b) construction result (c) +(p2p1p3) ¼ 431 and (d) construction result.
We would like to thank Mr. Guangqing He for his discussions about the vision function during the research. Without data from his experiment, we can hardly find the appropriate parameter value for our algorithm within a short period of time. In addition, we would like to express sincere thanks to Mr. Shuren Li for building a basic curve-reconstruction application which facilitates our task for implementing and testing our algorithm.
Fig. 24. Wrong construction. (a) Sample set (b) wrong connection and (c) Sampling density increased.
Fig. 25. Different parameters yield different results. (a) Sample set (b) c ¼ 0.8 and (c) c ¼ 0.9.
ARTICLE IN PRESS 834
T. An Nguyen, Y. Zeng / Robotics and Computer-Integrated Manufacturing 24 (2008) 824–834
References [1] Edelsbrunner H, Kirkpatrick D, Seidel R. On the shape of a set of points in the plane. IEEE Trans Inf Theory 1983;29:71–8. [2] Attali D. r-Regular shape reconstruction from unorganized points. In: Proceedings of the 13th annual ACM symposium on computational geometry, 1997. p. 248–53. [3] Henrique de Figueiredo L, Gomes J. Computational morphology of curves. Vis Comput 1995;11:105–12. [4] Amenta N, Bern M, Eppstein D. The crust and the beta-skeleton: combinatorial curve reconstruction. Graph Models Image Process: GMIP 1998;60: 125–35. [5] Gold C. Crust and anti-crust: a one-step boundary and skeleton extraction algorithm. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 189–96. [6] Dey TK, Kumar P. A simple provable algorithm for curve reconstruction. In: Proceedings of the 10th annual ACM-SIAM symposium on discrete algorithms, 1999. p. 893–4. [7] Dey TK, Mehlhorn K, Ramos E. Curve reconstruction: connecting dots with good reason. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 197–206.
[8] Althaus E, Mehlhorn K. TSP-based curve reconstruction in polynomial time. In: Symposium on discrete algorithms, 2000. p. 686–95. [9] Giesen J. Curve reconstruction, the TSP, and Menger’s theorem on length. In: Proceedings of the 15th annual ACM symposium on computational geometry, 1999. p. 207–16. [10] Dey TK, Wenger R. Reconstructing curves with sharp corners. Comput Geom Theory Appl 2001;19:89–99. [11] Dey TK, Wenger R. Fast reconstruction of curves with sharp corners. Int J Comput Geom Appl 2002:353–400. [12] Dey TK. Curve and surface reconstruction. Cambridge University Press; 2006. [13] Funke S, Ramos EA. Reconstructing a collection of curves with corners and endpoints. In: Symposium on discrete algorithms, 2001. p. 344–53. [14] Althaus E, Mehlhorn K, Naher S, Schirra S. Experiments on curve reconstruction. In: Proceedings of the 2nd workshop algorithm eng. exper., 2000. p. 103–14. [15] Zeng Y, Nguyen TA, Yan BQ, Li SR. A distance-based parameter free algorithm for curve reconstruction. Comput Aided Des 2008, in press. [16] Mather G. Foundations of perception. Comput Aided Des 2008;40:210–22. [17] Li SR. Vision based curve reconstruction. Master thesis, Concordia University, Canada, 2007.