- Email: [email protected]
Segmentation of edges into lines and arcs
Segmentation of edges into lines and arcs Paul L Rosin and Geoff A W West*
.4 long standing problem in computer vision is the extraction of meaningfulfeatures from images. This paper describes a method of segmenting curves in images into a combination of circular arcs and straight lines. This uses a recursive algorithm that first analyses lists of connected edge points and finds a polygonal description, and then analyses this description fitting arcs to groups of connected lines. The result is a description of image edges consisting of circular arcs and lines. The algorithm uses no thresholding. Instead the best option is chosen at each decision stage. Keywords: curve segmentation, arc detection, line detection
In the field of computer vision, there is a need to extract from images, descriptions of curves based on high level descriptors. The curves considered here are produced by linking together edge pixels produced by edge detectors etc. The type of description required can be application dependent but in general will require a description of the curves based on a combination of straight line approximations and higher order curves such as arcs, splines etc. Much research has been performed into the development of algorithms that produce a representation that is compact, accurate, meaningful and suitable for further processing. Early work has concentrated on generating a straight line approximation by segmenting the curve at points of maximum deviation1-3 and discontinuities4. More recently, an algorithm has been proposed for segmenting two and three-dimensional curves5. This algorithm is an improvement on others as it uses circular arcs as well as straight lines for the description segmenting these at discontinuities detected using maxima in curvature and rate of change of curvature. This paper describes a method for finding circular arcs in images that is an extension of a method proposed by Lowe6 which analyses abitrary curves and produces a high-level straight line description. Lowe’s algorithm segments each curve by recursively splitting it at the Department of Neurology, Guy’s Hospital, London Bridge, London SEI 9RT, UK *Measurement and Instrumentation Centre, School of Electrical Engineering and Applied Physics, City University, Northampton Square, London EClV OHB. UK
0262-8856/89/020109-06 ~017 no 2 May 1989
$03.00
maximum deviation from the approximating straight line. At each level a decision is made as to whether the single straight line is better than the representation at a lower level which will consist of two or more approximating straight lines. The measure of goodness of Iit is termed the ‘significance’ and is defined as the ratio of maximum deviation from the straight line to the length of the straight line. The interesting property of this algorithm that is attractive is that no thresholds are used to control the accuracy of the resulting representation. This is controlled by the significance values that can be regarded as the error between the curve and straight line description weighted by the length of the straight line. Thus long straight lines are regarded as being a better representation even though the error can be greater. This introduces scale invariance such that a contour of different scales will have the same or similar description. The examination of Lowe’s algorithm has lead to the hypothesis that circular arcs can be detected in the straight line description using a similar algorithm to find the best Iit of arcs and straight lines to the data. The difference is that at each level of recursion, a decision is taken as to whether an arc is a better fit than the lower level description consisting of straight lines and arcs. The decision is again based on significances with the significance for an arc being the error weighted by the arc length. This is equivalent to the significance measure for straight lines so can be directly compared. The straight line description is used instead of the original data because the points of segmentation have already been found and it is simple to replace a number of straight lines by an arc while retaining connectivity. When recursively splitting the straight line description to find arcs, the split is taken at the point of maximum deviation from the best fit arc. This will not segment the curve at meaningful points, i.e. points of maximum change of curvature as used by Pridmore et al5 although it does produce reasonable results considering the simplicity of the algorithm. The complexity increases when extending the algorithm to deal with space curves since arcs, ellipses etc. need more parameters for their description than the straight line. This means that there are more degrees
@ 1989 Butterworth &Co. (Publishers) Ltd 109
--------2
-----
-
-
-u 0
Figure 1. Image showing edges as lists of connectedpixels of freedom when fitting the curve to the data. However, a circular arc fitting algorithm has been developed that determines the best fit (in a least mean squares error sense) of an arc to the data using the constraint that the arc must pass through the end points of the data (so as to connect with the adjoining arcs or lines). Instead of using thresholds (that normally have to be tuned to the particular application) the processing is organized so that at each decision stage, the best option is taken, e.g. whether an arc or a combination of straight lines and arcs is best. It should therefore be general purpose and useful for many applications7. Any connected set of points in 2D space can be segmented by this process and chain coded boundaries from binary images have also been processed by the authors. For images of zero crossings, the only constraints are that the image must be reasonably clean and real edges in the image represented by connected zero crossing points. If there is too much disconnectedness, then the algorithm produces poor results although these may be improved at a later processing stage. The algorithm has been used to determine high-level descriptions of images from industrial scenes as well as natural scenes. These are being used for further processing in the fields of object recognition and automatic inspection.
PREPROCESSING The image must be processed and lists of connected pixels extracted for the algorithm to function. The choice of edge detector is not critical and in this implementation the Marr-Hildreth edge detector8 is used, after which the zero crossings are detected, e.g. Figure 1. For this particular example, the image is 5 12 x 512 pixels and a sigma value of 1.8 was used for the edge detector. The edge pixels are stored in an image as pixels of value 255 on a background of value 0. The image is converted to a number of lists of linked edge pixels. A real edge in the image will generally be represented by one or more lists, Since the edges of the resulting image are only one pixel wide, thinning is unnecessary. Theoretically, the space curves should form closed loops. However, this is only possible where the zero crossings are detected to subpixel accuracy. In the authors’ applications, this is unnecessary so zero crossings are placed at the nearest pixel. This causes some edge pixels to be combined and others ignore.
110
To allow fast processing of the edge data, the image is scanned and each space curve is transformed into a linked list of pixel coordinates. These are either open or closed lists representing open and closed curves respectively. Open lists are detected first, followed by closed lists. In the case of open lists, the algorithm looks for a pixel on a space curve that has only one neighbour (which is therefore at the end of the curve) and then follows the curve, generating the list until it reaches the other end. During this process all pixels that are added to the lists are deleted from the image thereby removing them from the search space. After all open lists have been processed, the remaining pixels are searched for closed lists. A closed list is acquired by locating a pixel on a closed curve (one with two neighbours) and then following the curve until it returns to the start pixel. As each pixel is examined it is deleted from the image as with the open curves.
LINE DETECTION The next stage is to find a polygonal approximation of the lines. In Lowe’s work, a list of edge pixels is hypothesized as being a straight line passing through its end points. This is segmented into two lists at the point of maxims deviation and the process repeated recursively on each of the two lists. The recursive process is halted when a line segment is smaller than four pixels long or the deviation is less than three pixels. A three pixel length line is the smallest line that can contain non-zero deviations. If the deviation is zero, then this short line wili be retained which is undesirabie. The result of the recursive process is a multilevel tree where the description of the list of edge pixels at each level is a finer approximation of the level above. As the recursion unwinds, the tree is traversed back up to the root. At each level, if any of the line segments passed up from the lower level are more significant than the line segment at the current level, they are retained and passed up to the next higher level as candidate line segments. If this is not the case, the line segment at the current level is passed up. Once the recursion has completely unwound, a polygonal approximation of the space curve is obtained. The decision is based on a pseudo-psychological measure of perceptual significance. Figure 2 shows an example of line fitting for three levels of recursion. Lowe’s measure of significance is determined by the ratio of the line segment length divided by the maximum deviation of the space curve from the straight line segment. However, when all the pixels lie directly on the straight line (which occurs regularly with short lines), the significance is infinite. To avoid divide-by-zero errors, the significance measure has been redefined as the ratio of the maximum deviation divided by the line segment length i.e. the reciprocal of Lowe’s significance. Thus, the lower the significance value, the more signiftcant the line. The procedure is weighted in favour of long (i.e. highly significant) line segments. The longer the line, the greater the deviations that will be tolerated. Thus at different scales the same shape of curve will have the same significance. This algorithm produces a high quality, general purpose polygonal approximation. No arbitrary error threshoid is required. Instead, the most appropriate
image and vision computing
Figure 4. Example qf fitting 2 indicating point of maximum
an arc to data of Figure deviation
+Y
-X
(0.0)
+x
:’
:’ :.
Figure 2. Three levels in straight line,fitting
,:’
(after Lowe”)
“(xcent,ycent)
t::
-Y
Figure 5. Arcfinding
Figure 3. Result in Figure I
qf straight line approximation
of image
values are chosen dynamically throughout the procedure. For instance, curves of different sizes give similar approximations but at different scales. The use of a fixed threshold would approximate the smaller curves crudely and larger curves with unnecessary detail. Figure 3 shows the result of detecting line segments in an image of a bicycle in which the wheels are represented by polygonal approximations (vertices in the description indicated by crosses).
space
parameters as it has more degrees of freedom than a line. To fit a circular arc to a space curve requires the determination of the centre, start and end coordinates and the radius. Algorithms for general purpose arc fitting have been proposed by Landau9 and Albano’O. The process can be simplified by applying the same constraint that is used for fitting straight lines described in the section above to reduce the number of degrees of freedom. Since space curves are described by a combination of connected lines and arcs, arcs are constrained to start and finish at the two end points of the space curve of interest. Given the two end points of the arc, three parameters need to be determined, namely the radius and coordinates of the centre of the circle forming the arc. Minimizing the error for the three parameters is a non-trivia1 task but the problem can be simplified as the end points of the arc are known. As the centre point of the circle must be equidistant from the end points of the arc, it is constrained to lie on a straight line that is normal to the line joining the two end points as shown in Figure 5. Thus a method of finding the best or optimum arc is to search along this line in some intelligent way. In this algorithm, the measure of match (error,,,) between
ARC DETECTION The same approach can be applied to curve detection by extending Lowe’s work. To determine a line description, sequences of pixels are hypothesized as line segments and the hypotheses recursively relined by subdividing at the point of maximum deviation. In a similar manner, sequences of line segments, obtained from the polygonal approximation procedure, can be hypothesized as curves and relined by subdividing at the point of maximum deviation as shown in Figure 4. Note this shows the fitting of an arc to the original data. For computational reasons in this implementation, the arc is fitted to the line description. Circular arc detection is more complicated than line detection since the specification of an arc requires more
t
(xcent,
ycent)
-Y
Figure 6. Error for one vertex when.fitting approximation
arc to polygonal
the data and the arc is determined using a least squares error criterion N-
errortot =
I
C e’,
111
where there are N vertices defining the polygonal approximation. The error (e) for each vertex is the shortest distance to the circle. This is the difference between the radius and the distance from the vertex to the centre as shown in Figure 6. A recursive binary split algorithm rapidly searches for the minimum error using the gradient descent method. The algorithm has the following stages. First, transform the coordinates of the vertices so that the centre of the line joining the two outer points lies at the origin and the two outer points lie on the x-axis. This means that the centre of the circle will lie along the y-axis (Figure 5.) Second, compute the errors with the centre of the hypothesized circle centred at the origin (0,O) and either side of the x-axis (0,l) and (0, -1). This enables the gradient of the error function to be determined at the origin. The required minimum will lie on the side of the x-axis indicated by the direction of negative gradient. The rule is: if error (0,l) > error (0, -1)
then direction = - 1 else direction = 1
Third, compute the mean y coordinate of all the vertices Of-mean). This indicates which side of the x-axis the arc is. This measure is then used in combination with direction to decide whether the arc is small (the angle subtended by the arc is less than 180 degrees) or large (angle subtended by the arc is greater than or equal to 180 degrees) using the following rules: if direction = 1 and y-mean
< 0.0 then small arc
if direction = -1 and y-mean > 0.0 then small arc if direction = 1 and ymean
> 0.0 then large arc
if direction = - 1 and y-mean < 0.0 then large arc Fourth, set up the search space in which to search for the minimum error. At this point the maximum radius has to be decided. If the arc is small, then a reasonable maximum radius is 16 383 pixels. Thus for a small arc the search space is 0 to 16 383. For a large arc the centre will be somewhere between the origin and the maximum y value of the data. The maximum radius will be 256 pixels for a 512 x 512 pixel image as this is the biggest large arc that will fit in an image. Fifth, search for the minimum error. This uses a recursive binary splitting algorithm to find the minimum quickly. The maximum number of recursions required is for a small arc with a search space of 0 to 16 383 requiring 14 levels of recursion. For large arcs the maximum radius will be dependent on the image size. For example the maximum radius will be 256 for a 512 x 5 12 pixel image requiring eight levels of recursion. Sixth, compute the parameters of the arc, namely centre, radius, length of the arc and polygonal approximation. Also required is the maximum deviation of the polygonal data from the arc. This is carried out by walking around the arc computing the minimum deviation at each point between the polygonal data and the arc along a radius and saving the maximum of these. This stage is necessary to accommodate situations such as
112
Figure 7. Example of good least squares fit requiring stage six to detect poor match
those shown in Figure 7 where the hypothesized arc gives a good approximation to the polygonal data in a least squares error sense but is obviously not perceptually good. This situation, which occurs frequently, can be recognized by computing deviation at a number of points around the arc. This stage could have been incorporated into stage six as the least squares error procedure would have accounted for this situation. However, this would increase the computational burden and slow the algorithm. Seventh, compute the signicance as for line segments, i.e. the ratio of maximum deviation to feature length where in this case, the feature length is the length of the arc. The recursive procedure repeatedly subdivides arcs into subarcs, stopping when the sequence of line segments to be approximated contains fewer than three line segments. This is analogous to the three pixel limit in the polygonal line approximation. Obviously a single line segment should not be replaced by a single arc. Two line segments contain three points and would always be replaced by a perfect fit. This would prevent its replacement by higher level matches since they could never improve on the lower level match. The deviation between the hypothesized arc and the line segments was initially calculated by finding the minimum distances between the line segment endpoints and midpoints and the arc, and returning the maximum. Certain non-circular arrangements of lines returned low deviations. A more robust measure of deviation was provided by stepping around the arc, finding the minimum distance from the line segments and returning the maximum (see stage six above). Another difference between line and arc detection is that whereas all pixels should be replaced by straight line segments, in many cases lines are more appropriate than arcs. The decision to replace lines by arcs can be uniformly incorporated without the need for an arbitrary threshold. The significance of an arc is calculated in a similar manner to the significance of a line, namely the ratio of the maximum deviation divided by the length of the arc. Therefore, since their signiticances are compatible, lines are only replaced by an arc if the arc is more significant than all the lines. It is only because the previous procedure results in such good polygonal approximations that the arc signiticances are compatible. If it did not dynamically determine the most appropriate
image and vision computing
Figure 8. Result offinding
arcs in image of Figure 3
error threshold, small curves would be crudely approximated, producing arcs with large deviations and thus poor signilicances.
RESULTS
AND
DISCUSSION
The algorithm has been tried on numerous images ranging from manufactured components to outdoor scenes containing manmade artefacts e.g. vehicles. Figure 8 shows the result of processing the bicycle (compare with Figure 3) in which arcs describing the wheels are evident (again crosses mark the ends of each arc and line). To show an example of processing in more detail the operation of the algorithm on one list from the bicycle image is shown in Figure 9. Figure 9a shows the original data which is a convoluted list of connected edge pixels. Figure 9b shows the result of Lowe’s algorithm where each line is evident. Figure 9c shows the result after arc detection. Much of the line data describing the circular wheels has been replaced by arcs. The segmentation of the curves using maximum deviation has resulted in a good description of the data by a combination of arcs and lines with many points of segmentation (indicated by crosses) at meaningful segmentation points. This has been generally the case, see other examples in Rosin’. The circular arc part of the curve has not been segmented perfectly as it is now described by two arcs and four lines. This is because using maximum deviation for dividing the curve means that parts of the curve nil1 not be in the same part of the tree generated by
a
SUMMARY This paper has described an algorithm for detecting circular arcs in arbitrary space curves that result from edge detection in images. It essentially extends the line fitting algorithm proposed by Lowe6 so that the curves are described as a combination of straight lines and circular arcs. Examples were given of grey-level processing with zero crossings detected in an image after convolution with the Marr-Hildreth edge detector. The algorithm is not restricted to this image type and has been used on boundary descriptions from chain-coded binary images. A fast algorithm for determining the minimum error match of a circular arc with the data is used. The computational burden is reduced over other curve fitting methods by the constraint that the curve must be bounded by two defined endpoints. It would be rewarding to extend this work to detect
b
Figure 9a. One list of edge pixels extracted c. result of arc detection in Figure 9h
~017 no 2 May 1989
recursive subdivision. Two parts of the same arc will not then be considered for replacement by one arc. There are situations where a meaningful unique description is not found where positions of maximum deviation do not occur at points of high change in curvature. This highlights a drawback of the algorithm which is its dependence on the positions of the end points of the list of pixels. This is a drawback in Lowe’s algorithm that is propagated into the arc detection stage. The problem of uniqueness should not be a drawback as higher level processing can use the available description to further process the description for more meaningful information’. The performance of the algorithm on simulated data is shown in Figure 10 for a number of circles. Scale invariance is apparent in the line description and only arcs are present after arc detection. The representation can be seen to be approximately the same despite the varying size of the circles, both as a combination of lines and lines and arcs. Absolute accuracy can be poor as a long line with a low significance value can have a large absolute error because significances are used instead of absolute thresholds. However at this scale it is still a good description of the curve. For Figure 9b the significance values ranged from 0.022 to 0.25 and for Figure 9c from 0.005 to 0.25.
from
Figure
I; b, result qf straight
line approximation
in Figure 9a:
113
Figure I&. ~im~l~~ed circles (5fZ x 51.2 irnuge~~ b, resub of straight line ~pprox~m~tion~ c, resuIt of arc dejection
other curve types e.g. ellipses, tonics etc. from the arc description as a higher level description would be obtained, for example a combination of arcs replaced by a single ellipse. An interesting property of this algorithm is that no thresho~ding is necessary at any stage from the edge detection through line and then arc detection. At each stage of curve fitting, the ‘best’ description is chosen i.e. that description with the best significance. The use of signiticances allows a scale invariant description of a curve to be obtained. The lack of thresholding should make the algorithm general purpose.
4
5
6
7
REFERENCES Pavlidis, T and Horowitz, S L ‘Segmentation of plane curves’ IEEE Trans. Computers Vol C-23 No 8 (1974) pp 860-870 Ramer, U ‘An interative procedure for the polygonal approximation of plane curves’ Comp~~er Graphics & Image Process. Vol 1 (1972) pp 244-256 Freeman, H ‘On the encoding of arbitrary geometric
114
8
9
10
configurations’ IRE Trans. Electronic Computers Vol EC-10 (1961) pp 26@269 Freeman, H and Davis, L S ‘A corner finding algorithm for chain coded curves’ IEEE Trans. Computers Vol C-26 (1977) pp 297-303 Pridmore, A P, Porrill, J and Mayhew, J E W ‘Segmentation and description of binocularly viewed contours’ Image Vision Comput. (May 1987) pp 132-138 Lowe, D G ‘Three-dimensional object recognition from single two-dimensional images’ Artif. Zntell. Vol31 (1987) pp 355-395 Rosin, P L ‘Model driven image understanding: a frame-based approach’ PhD Thesis City University, London, UK (1988) Marr, D and Hildreth, E ‘Theory of edge detection’ Proc. Royal Society London B UK Vol 207 (1980) pp 187-217 Landau, U ‘Estimation of a circular arc centre and its radius’ Comput. Vision, Graphics & Image Process. Vol38 (1987) pp 317-326 Albano, A ‘Representation of digitised contours in terms of conic arcs and straight-line segments’ Comput. Vision, Graphics & Image Proces. Vol 3 (1974) pp 23-33
image and vision computing
.4 long standing problem in computer vision is the extraction of meaningfulfeatures from images. This paper describes a method of segmenting curves in images into a combination of circular arcs and straight lines. This uses a recursive algorithm that first analyses lists of connected edge points and finds a polygonal description, and then analyses this description fitting arcs to groups of connected lines. The result is a description of image edges consisting of circular arcs and lines. The algorithm uses no thresholding. Instead the best option is chosen at each decision stage. Keywords: curve segmentation, arc detection, line detection
In the field of computer vision, there is a need to extract from images, descriptions of curves based on high level descriptors. The curves considered here are produced by linking together edge pixels produced by edge detectors etc. The type of description required can be application dependent but in general will require a description of the curves based on a combination of straight line approximations and higher order curves such as arcs, splines etc. Much research has been performed into the development of algorithms that produce a representation that is compact, accurate, meaningful and suitable for further processing. Early work has concentrated on generating a straight line approximation by segmenting the curve at points of maximum deviation1-3 and discontinuities4. More recently, an algorithm has been proposed for segmenting two and three-dimensional curves5. This algorithm is an improvement on others as it uses circular arcs as well as straight lines for the description segmenting these at discontinuities detected using maxima in curvature and rate of change of curvature. This paper describes a method for finding circular arcs in images that is an extension of a method proposed by Lowe6 which analyses abitrary curves and produces a high-level straight line description. Lowe’s algorithm segments each curve by recursively splitting it at the Department of Neurology, Guy’s Hospital, London Bridge, London SEI 9RT, UK *Measurement and Instrumentation Centre, School of Electrical Engineering and Applied Physics, City University, Northampton Square, London EClV OHB. UK
0262-8856/89/020109-06 ~017 no 2 May 1989
$03.00
maximum deviation from the approximating straight line. At each level a decision is made as to whether the single straight line is better than the representation at a lower level which will consist of two or more approximating straight lines. The measure of goodness of Iit is termed the ‘significance’ and is defined as the ratio of maximum deviation from the straight line to the length of the straight line. The interesting property of this algorithm that is attractive is that no thresholds are used to control the accuracy of the resulting representation. This is controlled by the significance values that can be regarded as the error between the curve and straight line description weighted by the length of the straight line. Thus long straight lines are regarded as being a better representation even though the error can be greater. This introduces scale invariance such that a contour of different scales will have the same or similar description. The examination of Lowe’s algorithm has lead to the hypothesis that circular arcs can be detected in the straight line description using a similar algorithm to find the best Iit of arcs and straight lines to the data. The difference is that at each level of recursion, a decision is taken as to whether an arc is a better fit than the lower level description consisting of straight lines and arcs. The decision is again based on significances with the significance for an arc being the error weighted by the arc length. This is equivalent to the significance measure for straight lines so can be directly compared. The straight line description is used instead of the original data because the points of segmentation have already been found and it is simple to replace a number of straight lines by an arc while retaining connectivity. When recursively splitting the straight line description to find arcs, the split is taken at the point of maximum deviation from the best fit arc. This will not segment the curve at meaningful points, i.e. points of maximum change of curvature as used by Pridmore et al5 although it does produce reasonable results considering the simplicity of the algorithm. The complexity increases when extending the algorithm to deal with space curves since arcs, ellipses etc. need more parameters for their description than the straight line. This means that there are more degrees
@ 1989 Butterworth &Co. (Publishers) Ltd 109
--------2
-----
-
-
-u 0
Figure 1. Image showing edges as lists of connectedpixels of freedom when fitting the curve to the data. However, a circular arc fitting algorithm has been developed that determines the best fit (in a least mean squares error sense) of an arc to the data using the constraint that the arc must pass through the end points of the data (so as to connect with the adjoining arcs or lines). Instead of using thresholds (that normally have to be tuned to the particular application) the processing is organized so that at each decision stage, the best option is taken, e.g. whether an arc or a combination of straight lines and arcs is best. It should therefore be general purpose and useful for many applications7. Any connected set of points in 2D space can be segmented by this process and chain coded boundaries from binary images have also been processed by the authors. For images of zero crossings, the only constraints are that the image must be reasonably clean and real edges in the image represented by connected zero crossing points. If there is too much disconnectedness, then the algorithm produces poor results although these may be improved at a later processing stage. The algorithm has been used to determine high-level descriptions of images from industrial scenes as well as natural scenes. These are being used for further processing in the fields of object recognition and automatic inspection.
PREPROCESSING The image must be processed and lists of connected pixels extracted for the algorithm to function. The choice of edge detector is not critical and in this implementation the Marr-Hildreth edge detector8 is used, after which the zero crossings are detected, e.g. Figure 1. For this particular example, the image is 5 12 x 512 pixels and a sigma value of 1.8 was used for the edge detector. The edge pixels are stored in an image as pixels of value 255 on a background of value 0. The image is converted to a number of lists of linked edge pixels. A real edge in the image will generally be represented by one or more lists, Since the edges of the resulting image are only one pixel wide, thinning is unnecessary. Theoretically, the space curves should form closed loops. However, this is only possible where the zero crossings are detected to subpixel accuracy. In the authors’ applications, this is unnecessary so zero crossings are placed at the nearest pixel. This causes some edge pixels to be combined and others ignore.
110
To allow fast processing of the edge data, the image is scanned and each space curve is transformed into a linked list of pixel coordinates. These are either open or closed lists representing open and closed curves respectively. Open lists are detected first, followed by closed lists. In the case of open lists, the algorithm looks for a pixel on a space curve that has only one neighbour (which is therefore at the end of the curve) and then follows the curve, generating the list until it reaches the other end. During this process all pixels that are added to the lists are deleted from the image thereby removing them from the search space. After all open lists have been processed, the remaining pixels are searched for closed lists. A closed list is acquired by locating a pixel on a closed curve (one with two neighbours) and then following the curve until it returns to the start pixel. As each pixel is examined it is deleted from the image as with the open curves.
LINE DETECTION The next stage is to find a polygonal approximation of the lines. In Lowe’s work, a list of edge pixels is hypothesized as being a straight line passing through its end points. This is segmented into two lists at the point of maxims deviation and the process repeated recursively on each of the two lists. The recursive process is halted when a line segment is smaller than four pixels long or the deviation is less than three pixels. A three pixel length line is the smallest line that can contain non-zero deviations. If the deviation is zero, then this short line wili be retained which is undesirabie. The result of the recursive process is a multilevel tree where the description of the list of edge pixels at each level is a finer approximation of the level above. As the recursion unwinds, the tree is traversed back up to the root. At each level, if any of the line segments passed up from the lower level are more significant than the line segment at the current level, they are retained and passed up to the next higher level as candidate line segments. If this is not the case, the line segment at the current level is passed up. Once the recursion has completely unwound, a polygonal approximation of the space curve is obtained. The decision is based on a pseudo-psychological measure of perceptual significance. Figure 2 shows an example of line fitting for three levels of recursion. Lowe’s measure of significance is determined by the ratio of the line segment length divided by the maximum deviation of the space curve from the straight line segment. However, when all the pixels lie directly on the straight line (which occurs regularly with short lines), the significance is infinite. To avoid divide-by-zero errors, the significance measure has been redefined as the ratio of the maximum deviation divided by the line segment length i.e. the reciprocal of Lowe’s significance. Thus, the lower the significance value, the more signiftcant the line. The procedure is weighted in favour of long (i.e. highly significant) line segments. The longer the line, the greater the deviations that will be tolerated. Thus at different scales the same shape of curve will have the same significance. This algorithm produces a high quality, general purpose polygonal approximation. No arbitrary error threshoid is required. Instead, the most appropriate
image and vision computing
Figure 4. Example qf fitting 2 indicating point of maximum
an arc to data of Figure deviation
+Y
-X
(0.0)
+x
:’
:’ :.
Figure 2. Three levels in straight line,fitting
,:’
(after Lowe”)
“(xcent,ycent)
t::
-Y
Figure 5. Arcfinding
Figure 3. Result in Figure I
qf straight line approximation
of image
values are chosen dynamically throughout the procedure. For instance, curves of different sizes give similar approximations but at different scales. The use of a fixed threshold would approximate the smaller curves crudely and larger curves with unnecessary detail. Figure 3 shows the result of detecting line segments in an image of a bicycle in which the wheels are represented by polygonal approximations (vertices in the description indicated by crosses).
space
parameters as it has more degrees of freedom than a line. To fit a circular arc to a space curve requires the determination of the centre, start and end coordinates and the radius. Algorithms for general purpose arc fitting have been proposed by Landau9 and Albano’O. The process can be simplified by applying the same constraint that is used for fitting straight lines described in the section above to reduce the number of degrees of freedom. Since space curves are described by a combination of connected lines and arcs, arcs are constrained to start and finish at the two end points of the space curve of interest. Given the two end points of the arc, three parameters need to be determined, namely the radius and coordinates of the centre of the circle forming the arc. Minimizing the error for the three parameters is a non-trivia1 task but the problem can be simplified as the end points of the arc are known. As the centre point of the circle must be equidistant from the end points of the arc, it is constrained to lie on a straight line that is normal to the line joining the two end points as shown in Figure 5. Thus a method of finding the best or optimum arc is to search along this line in some intelligent way. In this algorithm, the measure of match (error,,,) between
ARC DETECTION The same approach can be applied to curve detection by extending Lowe’s work. To determine a line description, sequences of pixels are hypothesized as line segments and the hypotheses recursively relined by subdividing at the point of maximum deviation. In a similar manner, sequences of line segments, obtained from the polygonal approximation procedure, can be hypothesized as curves and relined by subdividing at the point of maximum deviation as shown in Figure 4. Note this shows the fitting of an arc to the original data. For computational reasons in this implementation, the arc is fitted to the line description. Circular arc detection is more complicated than line detection since the specification of an arc requires more
t
(xcent,
ycent)
-Y
Figure 6. Error for one vertex when.fitting approximation
arc to polygonal
the data and the arc is determined using a least squares error criterion N-
errortot =
I
C e’,
111
where there are N vertices defining the polygonal approximation. The error (e) for each vertex is the shortest distance to the circle. This is the difference between the radius and the distance from the vertex to the centre as shown in Figure 6. A recursive binary split algorithm rapidly searches for the minimum error using the gradient descent method. The algorithm has the following stages. First, transform the coordinates of the vertices so that the centre of the line joining the two outer points lies at the origin and the two outer points lie on the x-axis. This means that the centre of the circle will lie along the y-axis (Figure 5.) Second, compute the errors with the centre of the hypothesized circle centred at the origin (0,O) and either side of the x-axis (0,l) and (0, -1). This enables the gradient of the error function to be determined at the origin. The required minimum will lie on the side of the x-axis indicated by the direction of negative gradient. The rule is: if error (0,l) > error (0, -1)
then direction = - 1 else direction = 1
Third, compute the mean y coordinate of all the vertices Of-mean). This indicates which side of the x-axis the arc is. This measure is then used in combination with direction to decide whether the arc is small (the angle subtended by the arc is less than 180 degrees) or large (angle subtended by the arc is greater than or equal to 180 degrees) using the following rules: if direction = 1 and y-mean
< 0.0 then small arc
if direction = -1 and y-mean > 0.0 then small arc if direction = 1 and ymean
> 0.0 then large arc
if direction = - 1 and y-mean < 0.0 then large arc Fourth, set up the search space in which to search for the minimum error. At this point the maximum radius has to be decided. If the arc is small, then a reasonable maximum radius is 16 383 pixels. Thus for a small arc the search space is 0 to 16 383. For a large arc the centre will be somewhere between the origin and the maximum y value of the data. The maximum radius will be 256 pixels for a 512 x 512 pixel image as this is the biggest large arc that will fit in an image. Fifth, search for the minimum error. This uses a recursive binary splitting algorithm to find the minimum quickly. The maximum number of recursions required is for a small arc with a search space of 0 to 16 383 requiring 14 levels of recursion. For large arcs the maximum radius will be dependent on the image size. For example the maximum radius will be 256 for a 512 x 5 12 pixel image requiring eight levels of recursion. Sixth, compute the parameters of the arc, namely centre, radius, length of the arc and polygonal approximation. Also required is the maximum deviation of the polygonal data from the arc. This is carried out by walking around the arc computing the minimum deviation at each point between the polygonal data and the arc along a radius and saving the maximum of these. This stage is necessary to accommodate situations such as
112
Figure 7. Example of good least squares fit requiring stage six to detect poor match
those shown in Figure 7 where the hypothesized arc gives a good approximation to the polygonal data in a least squares error sense but is obviously not perceptually good. This situation, which occurs frequently, can be recognized by computing deviation at a number of points around the arc. This stage could have been incorporated into stage six as the least squares error procedure would have accounted for this situation. However, this would increase the computational burden and slow the algorithm. Seventh, compute the signicance as for line segments, i.e. the ratio of maximum deviation to feature length where in this case, the feature length is the length of the arc. The recursive procedure repeatedly subdivides arcs into subarcs, stopping when the sequence of line segments to be approximated contains fewer than three line segments. This is analogous to the three pixel limit in the polygonal line approximation. Obviously a single line segment should not be replaced by a single arc. Two line segments contain three points and would always be replaced by a perfect fit. This would prevent its replacement by higher level matches since they could never improve on the lower level match. The deviation between the hypothesized arc and the line segments was initially calculated by finding the minimum distances between the line segment endpoints and midpoints and the arc, and returning the maximum. Certain non-circular arrangements of lines returned low deviations. A more robust measure of deviation was provided by stepping around the arc, finding the minimum distance from the line segments and returning the maximum (see stage six above). Another difference between line and arc detection is that whereas all pixels should be replaced by straight line segments, in many cases lines are more appropriate than arcs. The decision to replace lines by arcs can be uniformly incorporated without the need for an arbitrary threshold. The significance of an arc is calculated in a similar manner to the significance of a line, namely the ratio of the maximum deviation divided by the length of the arc. Therefore, since their signiticances are compatible, lines are only replaced by an arc if the arc is more significant than all the lines. It is only because the previous procedure results in such good polygonal approximations that the arc signiticances are compatible. If it did not dynamically determine the most appropriate
image and vision computing
Figure 8. Result offinding
arcs in image of Figure 3
error threshold, small curves would be crudely approximated, producing arcs with large deviations and thus poor signilicances.
RESULTS
AND
DISCUSSION
The algorithm has been tried on numerous images ranging from manufactured components to outdoor scenes containing manmade artefacts e.g. vehicles. Figure 8 shows the result of processing the bicycle (compare with Figure 3) in which arcs describing the wheels are evident (again crosses mark the ends of each arc and line). To show an example of processing in more detail the operation of the algorithm on one list from the bicycle image is shown in Figure 9. Figure 9a shows the original data which is a convoluted list of connected edge pixels. Figure 9b shows the result of Lowe’s algorithm where each line is evident. Figure 9c shows the result after arc detection. Much of the line data describing the circular wheels has been replaced by arcs. The segmentation of the curves using maximum deviation has resulted in a good description of the data by a combination of arcs and lines with many points of segmentation (indicated by crosses) at meaningful segmentation points. This has been generally the case, see other examples in Rosin’. The circular arc part of the curve has not been segmented perfectly as it is now described by two arcs and four lines. This is because using maximum deviation for dividing the curve means that parts of the curve nil1 not be in the same part of the tree generated by
a
SUMMARY This paper has described an algorithm for detecting circular arcs in arbitrary space curves that result from edge detection in images. It essentially extends the line fitting algorithm proposed by Lowe6 so that the curves are described as a combination of straight lines and circular arcs. Examples were given of grey-level processing with zero crossings detected in an image after convolution with the Marr-Hildreth edge detector. The algorithm is not restricted to this image type and has been used on boundary descriptions from chain-coded binary images. A fast algorithm for determining the minimum error match of a circular arc with the data is used. The computational burden is reduced over other curve fitting methods by the constraint that the curve must be bounded by two defined endpoints. It would be rewarding to extend this work to detect
b
Figure 9a. One list of edge pixels extracted c. result of arc detection in Figure 9h
~017 no 2 May 1989
recursive subdivision. Two parts of the same arc will not then be considered for replacement by one arc. There are situations where a meaningful unique description is not found where positions of maximum deviation do not occur at points of high change in curvature. This highlights a drawback of the algorithm which is its dependence on the positions of the end points of the list of pixels. This is a drawback in Lowe’s algorithm that is propagated into the arc detection stage. The problem of uniqueness should not be a drawback as higher level processing can use the available description to further process the description for more meaningful information’. The performance of the algorithm on simulated data is shown in Figure 10 for a number of circles. Scale invariance is apparent in the line description and only arcs are present after arc detection. The representation can be seen to be approximately the same despite the varying size of the circles, both as a combination of lines and lines and arcs. Absolute accuracy can be poor as a long line with a low significance value can have a large absolute error because significances are used instead of absolute thresholds. However at this scale it is still a good description of the curve. For Figure 9b the significance values ranged from 0.022 to 0.25 and for Figure 9c from 0.005 to 0.25.
from
Figure
I; b, result qf straight
line approximation
in Figure 9a:
113
Figure I&. ~im~l~~ed circles (5fZ x 51.2 irnuge~~ b, resub of straight line ~pprox~m~tion~ c, resuIt of arc dejection
other curve types e.g. ellipses, tonics etc. from the arc description as a higher level description would be obtained, for example a combination of arcs replaced by a single ellipse. An interesting property of this algorithm is that no thresho~ding is necessary at any stage from the edge detection through line and then arc detection. At each stage of curve fitting, the ‘best’ description is chosen i.e. that description with the best significance. The use of signiticances allows a scale invariant description of a curve to be obtained. The lack of thresholding should make the algorithm general purpose.
4
5
6
7
REFERENCES Pavlidis, T and Horowitz, S L ‘Segmentation of plane curves’ IEEE Trans. Computers Vol C-23 No 8 (1974) pp 860-870 Ramer, U ‘An interative procedure for the polygonal approximation of plane curves’ Comp~~er Graphics & Image Process. Vol 1 (1972) pp 244-256 Freeman, H ‘On the encoding of arbitrary geometric
114
8
9
10
configurations’ IRE Trans. Electronic Computers Vol EC-10 (1961) pp 26@269 Freeman, H and Davis, L S ‘A corner finding algorithm for chain coded curves’ IEEE Trans. Computers Vol C-26 (1977) pp 297-303 Pridmore, A P, Porrill, J and Mayhew, J E W ‘Segmentation and description of binocularly viewed contours’ Image Vision Comput. (May 1987) pp 132-138 Lowe, D G ‘Three-dimensional object recognition from single two-dimensional images’ Artif. Zntell. Vol31 (1987) pp 355-395 Rosin, P L ‘Model driven image understanding: a frame-based approach’ PhD Thesis City University, London, UK (1988) Marr, D and Hildreth, E ‘Theory of edge detection’ Proc. Royal Society London B UK Vol 207 (1980) pp 187-217 Landau, U ‘Estimation of a circular arc centre and its radius’ Comput. Vision, Graphics & Image Process. Vol38 (1987) pp 317-326 Albano, A ‘Representation of digitised contours in terms of conic arcs and straight-line segments’ Comput. Vision, Graphics & Image Proces. Vol 3 (1974) pp 23-33
image and vision computing