- Email: [email protected]
A fast algorithm for corner detection using the morphologic skeleton
Pattern Recognition Letters 22 (2001) 891±900
www.elsevier.nl/locate/patrec
A fast algorithm for corner detection using the morphologic skeleton q Liu Wen-Yu *, Li Hua, Zhu Guang-Xi State Education Commission Laboratory for Image Processing and Intelligent Control, Department of Electronics and Information Engineering, Huazhong University of Science and Technology, South Building No. 1, Wuhan, Hubei 430074, People's Republic of China Received 6 April 2000; received in revised form 21 November 2000
Abstract A fast corner detection algorithm based on the morphologic skeleton is proposed in this paper. The method diers from traditional chain-code-based corner detection algorithms in that the source image is represented as a polygon, and the corners are obtained through detecting the zero radius of the maximum plate on the morphologic skeleton. The result of the corner detection is achieved by using logical hetero-OR operation between two corner sets of the source image and its complement set. In addition, the pseudo-lattice corners generated by discrete sampling can also be eliminated. Experiments show that this method leads to results of good quality and robustness to noise with a low computational cost. Ó 2001 Elsevier Science B.V. All rights reserved. Keywords: Corner detection; Mathematical morphology; Skeleton
1. Introduction Corners in digital images are important features for shape representation and analysis. They have been widely used in computing optical ¯ow, pattern recognition, image matching, and motion analysis. Considerable research has been carried out on corner detection in recent years. The traditional corner detection methods involved segmenting an image into regions followed by representing object boundaries by using a chain code to ®nd high-curvature points. The work done by Rosenfeld (1982) and Kitchen and Rosenfeld (1982) is to detect a corner using k-curvature. It is the motivation for the presented work. Asada and Brady (1986) used the Gauss smoothness technique and cubic B-splines to calculate k-curvature, detect the discontinuous points on a curve, and then ®nd the corners. Other developments based on Asada's work include those done by Langride (1982), Medioni and Yasumoto (1986) and Fischler and Bolles (1986). Many approaches work directly at the grayscale level, which are based on directional derivatives. Local measurements are used in order to obtain corner strength (Deriche and Giraudon, 1990; Noble, q The project is supported by The National Science Foundation of China (No. 69973018) and The Natural Science Foundation of Hubei Province of China (No. 99J009). * Corresponding author. E-mail address: [email protected] (L. Wen-Yu).
0167-8655/01/$ - see front matter Ó 2001 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 8 6 5 5 ( 0 1 ) 0 0 0 2 9 - 0
892
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
1988). All these methods have the same drawback: local estimation of derivatives is very sensitive to noise and, when smoothing is applied, the corner localization precision is reduced. In addition, the computational complexity of the smoothing operation, derivative estimation and corner strength computation involved in such methods can be very high. Model-based approaches proposed by Deriche and Blaszka (1993) and Rohr (1992) can allow corner localization at sub-pixel accuracy. These methods are normally CPU-intensive and only used on a preliminary corner map obtained in the previous step. Recent development has focused on better performance and cost-eective methods. Smith and Brady proposed a simple approach based on brightness comparisons. The SUSAN corner detector described in their paper is based on the computation of the area of points inside a circular region having brightness similar to the one at the center point (Smith and Brady, 1997). Laganiere (1998) proposed a corner detector derived from this approach with the help of some basic morphological tools. This kind of method is simple in computation and has good performance. There are also many other methods for grayscale image corner detection. Krishnan et al. (1989) proposed a detector which tries to ®nd an analytical expression for an optimal function whose convolution with the windows of an image has signi®cant values at corner points. They describe a corner as the production of the sine in x and an exponential in their direction in a portion of the mask and the production of two sines in x- and y-directions in the remaining portion. However, this method needs complex computation and is dicult to implement in hardware. Lai and Wu (1996) considered edge±corner detection for defective images. Tsai (1997) proposed a method for boundary-based corner detection using neural networks. Ji and Haralick (1997) presented a technique for corner detection with covariance propagation. Lee and Bien (1996) applied fuzzy logic to corner detection. In this paper, a novel method for corner detection based on the mathematical morphologic skeleton is proposed. The goal was to obtain a fast corner detector that is, at the same time, suciently accurate, robust to noise, and easy for hardware implementation. The source image is represented as a polygon, and the corners can be obtained through detecting the zero radius of the maximum plate on the morphologic skeleton. In particular, the corners of a concave vertex can be obtained using the complement set of an image. The result of corner detection is given by logical hetero-OR operation between two corner sets of the source image and its complement. Experiments show that this method can lead to results of good quality and robustness to noise with a low computational cost. 2. The method of skeleton extracted from continuous sets De®nition 1. Suppose the set X and structuring element B are two arbitrary sets of points in E2 space. Minkowski addition and subtraction can be de®ned as follows: X B fx : B~I \ X 6 ;g; X B fx : B~x X g; where x's are points of X , B~I fx
b : b 2 Bg; B~x fx b : b 2 Bg.
De®nition 2. Suppose X 2 E2 , B is a closed unit circle in E2 . If rBx (r P 0 is a real number) satis®es the following conditions: 1. rBx X , 2. there is no other structure element r0 Bx0 X ; r0 > r; r0 Bx0 rBx , 3. rBx is the maximum plate in set X , where point x is the center and r is the radius,
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
893
then point x is called the skeleton point in set X . A full set of skeleton points is called the skeleton of set X and denoted as SK X .SThe center of the maximum plate with radius r is denoted as SKr X , which can be expressed as SK X r>0 SKr X . De®nition 3. Assuming a set X 2 E2 and B to be a convex structure element (including coordinate origin), for any x 2 E2 , Supfr : rBx X g; x 2 X ; rX jb x 0; x 62 X ; is called the constructing characteristic function of set X about structure element B. rX jb xBx denotes the maximum constructing characteristic element corresponding to X . When B is a closed unit circle, the constructing function is denoted as rX x. De®nition 4. Assume a set X 2 E2 , B to be a closed unit circle. Then SK a X , fx : limt!0 d x; X rX x tB=t > ag; a P 1, is called the conditional skeleton of set X . And limt!0 d x; X rX x tB=t is called the contract ratio of skeleton point x, where d denotes the Euclid distance. Theorem 1. Let X be a polygon and one of its internal angles be 2h, h < p=2. When a < 1= sin h, SKra X X rB
X rBdrB a
1drB;
1
where structure element B is a unit plate, dr is a positive minuteness real number, drB is a minuteness plate with a radius dr, and drB is a symbol of morphological opening for drB to ; thus X rBdrB X rB drB drB: When a 1; SK 1 X X rB X rBdrB . Suppose X be a polygon with an internal angle of 2h. If a < 1= sin h; SK a X is the bisector of this internal angle; otherwise, if a P 1= sin h, it is a null set. With a given value of a; SK a X can ®lter out those bisectors crossing the internal angle 2h which satisfy the condition (sin h P 1=a). This is shown in Fig. 1. We can prove that SK 1 X SK X , the skeleton of X is the conditional skeleton of X when a 1. According to the characteristics of the conditional skeleton, the extended line of the conditional skeleton must cross the vertex of a polygon. So the conditional skeleton halves the internal angle of the polygon X , and for each vertex of X there is one extended line of the conditional skeleton crossing it. The corner of X
Fig. 1. The contract ratio of skeleton point x and conditional skeleton of polygon X .
894
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Fig. 2. The result of corner detection. The dashed line denotes the skeleton of the polygon.
must be on the extended line of the conditional skeleton, and the maximum plate of the skeleton point on the conditional skeleton leads to zero when it is close to the corner. Then the corner is denoted as SK0a X X
XdrB a
1drB;
SK01 X X
XdrB X
X drB drB SK0 X ;
when a 1:
2
Corner detection is the simplest method in this case, as shown in Fig. 2. Let X Q1 PQ2 be a polygon. Then _
X drB drB Q1 ABQ2 ; X
_
X drB drB PAABPB: _
The result of corner detection is the region composed of PA; AB; PB. The bigger the drB, the larger the regions detected. Because SK0 X SK01 X , the internal angle 2h 1 a < 1= sin h can only be detected according to the characteristics of the conditional skeleton. The internal angle should be less than p; thus operator SK0 X is only used for detection of the convex corner instead of the concave corner. In this case, the complement set X C of X needs to be calculated in order to detect the corners of X C . The corners of X and X C together compose the whole corner set of X .
3. The method of corner detection for discrete binary images Corners are identi®ed where the direction changes rapidly in a chain code in a discrete image. The principle of the corner detector based on morphology is to probe the image under study with a structuring element. Since the shape and size of the structuring element must be set according to the information that is to be extracted, drB cannot be set too small in the discrete image. Instead, it should be de®ned as an accurate plate. Therefore, Eq. (2) should be modi®ed as shown in Eq. (3) in terms of a generalized morphologic opening: SK0 X X
X B1 B2 :
3
When the radius of B2 is one pixel larger than the radius of B1 in the discrete image, X can be reconstructed completely from the skeleton. Thus, two kinds of minimum plate S4 and S8 are de®ned and shown in Fig. 3. Suppose I be an image, then I S4 i; j maxfI i 1; j; I i; j 1; I i; j; I i
1; j; I i; j
1g;
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
895
Fig. 3. Two kinds of discrete minimum plate.
I S4 i; j minfI i 1; j; I i; j 1; I i; j; I i
1; j; I i; j
1g;
I S8 i; j maxfI i 1; j 1; I i 1; j; I i 1; j I i 1; j; I i 1; j 1g;
1; I i; j 1; I i; j; I i; j
1; I i
1; j 1;
I S8 i; j minfI i 1; j 1; I i 1; j; I i 1; j
1; I i; j 1; I i; j; I i; j
1; I i
1; j 1;
I i
1; j; I i
1; j
1g:
If the radii of B2 and B1 are unequal, a more complex calculation is needed. The following equation is used as a corner detection operator: Cor1 I
I S8 S8 :
4
This operator can only detect convex corners whose internal angles are less than p. It can also remove pseudo-lattice corners. For detection of an obtuse-angle corner, B2 should be a proper subset of B1 and is rather similar to a plate in Eq. (3). Therefore, the following equation is derived as a new corner detection operator: Cor2 I
I S8 S4 :
5
This operator can be used not only for acute-angle corners but also for obtuse-angle corners where the directions change rapidly in chain code. In an extreme case, the diagonal in the p=4 direction will be detected as a continuous line because there is a little sidestep in each adjacent point on the diagonal. The complement set I C of a source image I is calculated for a detection concave corner, but the boundaries of I and I C are not the same and have one pixel discrepancy. After executing a dilation calculation for I C , I 0 I C S4 , I 0 and I have the same boundary. The Cor3 operator in the following equation (6) is used for I 0 corner detection. Cor3 I 0
I 0 S8 S4 :
6
The concave corner of I will be detected using this operator and the diagonal in the p=4 direction is detected as a continuous line that is the same as that obtained using Cor2 . As a result, all the corners are denoted as the following formula: Cor Cor2 ^ Cor3 ;
7
where ^ denotes the hetero-OR operation of two images; the pseudo-corner can be counteracted. Fig. 4 shows the ¯owchart of the corner detection method based on the morphologic skeleton.
896
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Fig. 4. The ¯owchart of the corner detection method based on the morphologic skeleton.
4. Experimental results and discussions Fig. 5 shows the result of corner detection for English letters ``HUST''. Only the concave corners and some little sidesteps in adjacent points on the edge are detected in Fig. 5(b), and only the convex corners and some little sidesteps in adjacent points on the edge are detected in Fig. 5(c). The real corners in the source image are boosted up, and some pseudo-lattice corners on the diagonal are eliminated in Fig. 5(d). The result of corner detection of concave polygons with holes is shown in Fig. 6. Only the acute-angle corners are detected in Fig. 6(e). Fig. 6(f) is the result when the supplemental of source image is operated by operator Cor1 , then hetero-OR operated with Fig. 6(e). All real corners are detected, except for the obtuseangle corners. This method has an accuracy of about one pixel locating. Considering the computational complexity, there are 1 4 8N 2 times comparing calculations for an N N pixel image by operator Cor2 , 1 4 1 4 8N 2 times by operator Cor3 , and N 2 times by operator Cor. There are only 32N 2 times comparing calculations involved in the algorithm shown in Fig. 4. The experiments show that this method produces a very high value in locating corners. Because morphological operations do not have any ¯oating point and multiplication operations, this algorithm can be executed in parallel. Hence it is cost-eective with simple hardware implementation. For noisy images the algorithm has smoothing eects on small changes of noisy edges; this advantage is attributed to morphological opening ®lter and the uniformly disk-like structuring element. The experiment results show that the larger the radii of the structuring elements B1 and B2 , the larger the corner regions and the easier the corner estimation. Thus, the precision for locating a corner is reduced, and the computational cost is increased. Laganiere's (1998) algorithm is similar to the method proposed in this paper. It is much more complex than our method because of the combination of several structuring elements; the detection result for
Fig. 5. The result of corner detection for English letters ``HUST''. (a) Source image; (b) result of corner detection using operator Cor3 ; (c) result of corner detection using operator Cor2 ; (d) result of hetero-OR operation between (b) and (c).
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
897
Fig. 6. The result of corner detection for concave polygons with holes. (a) Source image; (b) result of corner detection using operator Cor2 ; (c) result of corner detection using operator Cor3 ; (d) result of hetero-OR operating between (b) and (c); (e) result of corner detection using operator Cor1 ; (f) result when the supplemental of source image is operated by operator Cor1 and hetero-OR operator with (e).
concave points is not reported; also, it must use global thresholds to get the ®nal detection result, but the author cannot report an eective method to select appropriate threshold values. Furthermore, because the algorithm uses a Gauss ®lter for noisy images, the hardware implementations are also complex. 5. Conclusions We have presented a new algorithm for corner detection based on the morphologic skeleton. This approach gives competitive results with respect to other available techniques. The proposed method has the advantages of computational simplicity and high-speed performance over existing algorithms. It also performs well on noisy images. Stable and accurate corner detection results have been achieved using the proposed method. Future research will focus on corner detection in grayscale images. Appendix A. The proof of Theorem 1 Lemma A.1. Suppose X is a polygon, B is a closed unit circle in E2 , then SKr X X rB
X rBdrB :
Proof. rBx X () x 2 X rB;
r aBy X () aBy X rB;
where symbol () denotes if and only if.
rBx r aBy () x 2 aBy ;
898
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
As a result, x 2 SKr X () rBx is the maximum circle with center x () x is the maximum circle of X rB (radius is zero) () x 2 X rB, but x do not belong to any X rBdrB () x 2 X rB X rBdrB , then Lemma A.1 holds. Lemma A.2. SK a X SK X . Proof. Suppose x 2 X , x 2 SK a X , but x 62 SK X . As a result, rX x is the maximum circle with center x and is tangential to oX in point P (oX denotes the boundary of X ). Connect P and x to obtain a line, make a neighboring zone t of x on the line. By all appearances, when t ! 0; x t 62 SK X ; rX x t is the maximum circle with center x t and is tangential to oX in point P , then d x; X rX x tB t, so x 62 SK a X . Therefore x 2 SK X . Theorem 1. Let X be a polygon and one of its internal angles be 2h; h < p=2. When a < 1= sin h, SKra X X rB
X rBdrB a
Proof. (1) First we prove that d x; X rX x tB x : lim >a ; t!0 t
1drB:
a P 1 X rB
X rBdrB a
1drB:
Since SKra X SKr X ;
9r > 0; x 2 SKra X ) x 2 SKr X ;
denotes that rBx is a maximum circle in X and r rX x, so x 2 X rB Now we must prove that x 62 X rBdrB a 1drB. Since x 2 SK a X ; for e ax a ax
r rX x;
ax , lim
t!0
X rBdrB.
d x; X r tB > a; t
a > 0; 9d > 0, when 0 < t < d, e<
d x; X r tB < ax e; t
so d x; X r tB=t > a 8t 2 0; d; and d x; X r tB > at or x atB \ X r tB ;: Then x 62 X r tB atB X r tB t a Then x 62 X rBdrB a
1drB; dr ! 0:
1tB X rBtB a
1tB:
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Therefore d x; X rX x tB x : lim >a ; t (2) We now prove X rB
a P 1 X rB
X rBdrB a
1drB
x : lim
t!0
X rBdrB a
d x; X rX x tB >a ; t
899
1drB:
a P 1:
Because x 2 X rB
X rBdrB a
1drB X rB
X rBdrB SKr X ;
then rBx X is a maximum circle and r rX x. Since 8dr > 0;
x 2 X rB
X rBdrB a
1drB;
we have x 62 X rBdrB a
1drB X r drB a drB:
Then x adrB \ X r drB ;; d x; X r drB > a dr d x; X r drB >a dr
8dr > 0;
8dr > 0:
Then lim
t!0
d x; X r tB P a; t
and we must prove that lim d x; X r tB=t 6 a:
t!0
Since x 2 SKr X , in polygon X , d x; X r tB t= sin h, as is shown in Fig. 1. If lim d x; X r tB=t a
t!0
holds, then lim
t!0
d x; X r tB t= sin h 1 lim a; t!0 t t sin h
so we can ®nd some points x, and make d x; X r tB at. Then x atB \ X r tB 6 ;, x 2 X r tB atB. As a result, x 62 X rB X rBdrB a 1drB, which con¯icts with the supposition. So limt!0 d x; X r tB=t a does not hold. Therefore lim
t!0
d x; X r tB > a; t
a P 1:
900
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Thus
X rB
X rBdrB a
1drB
d x; X rX x tB x : lim >a ; t!0 t
by all appearances, when a P 1= sin h; SKra X is a null set.
a P 1;
References Asada, H., Brady, M., 1986. The curvature primal sketch. IEEE Trans. Pattern Anal. Machine Intell. 8 (1), 2±14. Deriche, R., Blaszka, T., 1993. Recovering and characterizing image features using an ecient model based approach. In: Proc. CVPR. pp. 530±535. Deriche, R., Giraudon, G., 1990. Accurate corner detection: an analytical study. In: Proc. Internat. Conf. Computer Vision. pp. 66±70. Fischler, M.A., Bolles, R.C., 1986. Perceptual organization and curve partitioning. IEEE Trans. Pattern Anal. Machine Intell. 8 (1), 100±105. Ji, Q., Haralick, R.M., 1997. Corner detection with covariance propagation. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition. pp. 362±367. Kitchen, L., Rosenfeld, A.K., 1982. Gray level corner detection. Pattern Recognition Letters 3 (1), 95±102. Krishnan, R., Mubarak, S., David, V.B., 1989. Optimal corner detector. CVGIP 48, 230±245. Laganiere, R., 1998. A morphological operator for corner detection. Pattern Recognition 31 (11), 1643±1652. Lai, K.K., Wu, P.S.Y., 1996. Eective edge±corner detection method for defective images. Proc. Internat. Conf. Signal Processing 2 (1), 151±154. Langride, D.J., 1982. Curve encoding and the detection of discontinuities. CVGIP 20 (1), 58±71. Lee, K.J., Bien, Z., 1996. Grey-level corner detector using fuzzy logic. Pattern Recognition Letters 17 (9), 939±950. Medioni, G., Yasumoto, Y., 1986. Corner detection and curve representation using cubic B-splines. In: Proc. 1986 IEEE Internat. Conf. Robotics and Automation, San Fransisco, April. pp. 764±769. Noble, J.A., 1988. Finding corners. Image and Vision Comput. 6, 121±128. Rohr, K., 1992. Modeling and identi®cation of characteristic intensity variations. Image and Vision Comput. 10, 66±76. Rosenfeld, A.K., 1982. Digital Picture Processing, second ed. Academic Press, New York. Smith, S.M., Brady, J.M., 1997. SUSAN ± a new approach to low-level image processing. Internat. J. Comput. Vision 23 (1), 45±78. Tsai, D.M., 1997. Boundary based corner detection using neural networks. Pattern Recognition 30 (1), 85±97.
www.elsevier.nl/locate/patrec
A fast algorithm for corner detection using the morphologic skeleton q Liu Wen-Yu *, Li Hua, Zhu Guang-Xi State Education Commission Laboratory for Image Processing and Intelligent Control, Department of Electronics and Information Engineering, Huazhong University of Science and Technology, South Building No. 1, Wuhan, Hubei 430074, People's Republic of China Received 6 April 2000; received in revised form 21 November 2000
Abstract A fast corner detection algorithm based on the morphologic skeleton is proposed in this paper. The method diers from traditional chain-code-based corner detection algorithms in that the source image is represented as a polygon, and the corners are obtained through detecting the zero radius of the maximum plate on the morphologic skeleton. The result of the corner detection is achieved by using logical hetero-OR operation between two corner sets of the source image and its complement set. In addition, the pseudo-lattice corners generated by discrete sampling can also be eliminated. Experiments show that this method leads to results of good quality and robustness to noise with a low computational cost. Ó 2001 Elsevier Science B.V. All rights reserved. Keywords: Corner detection; Mathematical morphology; Skeleton
1. Introduction Corners in digital images are important features for shape representation and analysis. They have been widely used in computing optical ¯ow, pattern recognition, image matching, and motion analysis. Considerable research has been carried out on corner detection in recent years. The traditional corner detection methods involved segmenting an image into regions followed by representing object boundaries by using a chain code to ®nd high-curvature points. The work done by Rosenfeld (1982) and Kitchen and Rosenfeld (1982) is to detect a corner using k-curvature. It is the motivation for the presented work. Asada and Brady (1986) used the Gauss smoothness technique and cubic B-splines to calculate k-curvature, detect the discontinuous points on a curve, and then ®nd the corners. Other developments based on Asada's work include those done by Langride (1982), Medioni and Yasumoto (1986) and Fischler and Bolles (1986). Many approaches work directly at the grayscale level, which are based on directional derivatives. Local measurements are used in order to obtain corner strength (Deriche and Giraudon, 1990; Noble, q The project is supported by The National Science Foundation of China (No. 69973018) and The Natural Science Foundation of Hubei Province of China (No. 99J009). * Corresponding author. E-mail address: [email protected] (L. Wen-Yu).
0167-8655/01/$ - see front matter Ó 2001 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 8 6 5 5 ( 0 1 ) 0 0 0 2 9 - 0
892
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
1988). All these methods have the same drawback: local estimation of derivatives is very sensitive to noise and, when smoothing is applied, the corner localization precision is reduced. In addition, the computational complexity of the smoothing operation, derivative estimation and corner strength computation involved in such methods can be very high. Model-based approaches proposed by Deriche and Blaszka (1993) and Rohr (1992) can allow corner localization at sub-pixel accuracy. These methods are normally CPU-intensive and only used on a preliminary corner map obtained in the previous step. Recent development has focused on better performance and cost-eective methods. Smith and Brady proposed a simple approach based on brightness comparisons. The SUSAN corner detector described in their paper is based on the computation of the area of points inside a circular region having brightness similar to the one at the center point (Smith and Brady, 1997). Laganiere (1998) proposed a corner detector derived from this approach with the help of some basic morphological tools. This kind of method is simple in computation and has good performance. There are also many other methods for grayscale image corner detection. Krishnan et al. (1989) proposed a detector which tries to ®nd an analytical expression for an optimal function whose convolution with the windows of an image has signi®cant values at corner points. They describe a corner as the production of the sine in x and an exponential in their direction in a portion of the mask and the production of two sines in x- and y-directions in the remaining portion. However, this method needs complex computation and is dicult to implement in hardware. Lai and Wu (1996) considered edge±corner detection for defective images. Tsai (1997) proposed a method for boundary-based corner detection using neural networks. Ji and Haralick (1997) presented a technique for corner detection with covariance propagation. Lee and Bien (1996) applied fuzzy logic to corner detection. In this paper, a novel method for corner detection based on the mathematical morphologic skeleton is proposed. The goal was to obtain a fast corner detector that is, at the same time, suciently accurate, robust to noise, and easy for hardware implementation. The source image is represented as a polygon, and the corners can be obtained through detecting the zero radius of the maximum plate on the morphologic skeleton. In particular, the corners of a concave vertex can be obtained using the complement set of an image. The result of corner detection is given by logical hetero-OR operation between two corner sets of the source image and its complement. Experiments show that this method can lead to results of good quality and robustness to noise with a low computational cost. 2. The method of skeleton extracted from continuous sets De®nition 1. Suppose the set X and structuring element B are two arbitrary sets of points in E2 space. Minkowski addition and subtraction can be de®ned as follows: X B fx : B~I \ X 6 ;g; X B fx : B~x X g; where x's are points of X , B~I fx
b : b 2 Bg; B~x fx b : b 2 Bg.
De®nition 2. Suppose X 2 E2 , B is a closed unit circle in E2 . If rBx (r P 0 is a real number) satis®es the following conditions: 1. rBx X , 2. there is no other structure element r0 Bx0 X ; r0 > r; r0 Bx0 rBx , 3. rBx is the maximum plate in set X , where point x is the center and r is the radius,
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
893
then point x is called the skeleton point in set X . A full set of skeleton points is called the skeleton of set X and denoted as SK X .SThe center of the maximum plate with radius r is denoted as SKr X , which can be expressed as SK X r>0 SKr X . De®nition 3. Assuming a set X 2 E2 and B to be a convex structure element (including coordinate origin), for any x 2 E2 , Supfr : rBx X g; x 2 X ; rX jb x 0; x 62 X ; is called the constructing characteristic function of set X about structure element B. rX jb xBx denotes the maximum constructing characteristic element corresponding to X . When B is a closed unit circle, the constructing function is denoted as rX x. De®nition 4. Assume a set X 2 E2 , B to be a closed unit circle. Then SK a X , fx : limt!0 d x; X rX x tB=t > ag; a P 1, is called the conditional skeleton of set X . And limt!0 d x; X rX x tB=t is called the contract ratio of skeleton point x, where d denotes the Euclid distance. Theorem 1. Let X be a polygon and one of its internal angles be 2h, h < p=2. When a < 1= sin h, SKra X X rB
X rBdrB a
1drB;
1
where structure element B is a unit plate, dr is a positive minuteness real number, drB is a minuteness plate with a radius dr, and drB is a symbol of morphological opening for drB to ; thus X rBdrB X rB drB drB: When a 1; SK 1 X X rB X rBdrB . Suppose X be a polygon with an internal angle of 2h. If a < 1= sin h; SK a X is the bisector of this internal angle; otherwise, if a P 1= sin h, it is a null set. With a given value of a; SK a X can ®lter out those bisectors crossing the internal angle 2h which satisfy the condition (sin h P 1=a). This is shown in Fig. 1. We can prove that SK 1 X SK X , the skeleton of X is the conditional skeleton of X when a 1. According to the characteristics of the conditional skeleton, the extended line of the conditional skeleton must cross the vertex of a polygon. So the conditional skeleton halves the internal angle of the polygon X , and for each vertex of X there is one extended line of the conditional skeleton crossing it. The corner of X
Fig. 1. The contract ratio of skeleton point x and conditional skeleton of polygon X .
894
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Fig. 2. The result of corner detection. The dashed line denotes the skeleton of the polygon.
must be on the extended line of the conditional skeleton, and the maximum plate of the skeleton point on the conditional skeleton leads to zero when it is close to the corner. Then the corner is denoted as SK0a X X
XdrB a
1drB;
SK01 X X
XdrB X
X drB drB SK0 X ;
when a 1:
2
Corner detection is the simplest method in this case, as shown in Fig. 2. Let X Q1 PQ2 be a polygon. Then _
X drB drB Q1 ABQ2 ; X
_
X drB drB PAABPB: _
The result of corner detection is the region composed of PA; AB; PB. The bigger the drB, the larger the regions detected. Because SK0 X SK01 X , the internal angle 2h 1 a < 1= sin h can only be detected according to the characteristics of the conditional skeleton. The internal angle should be less than p; thus operator SK0 X is only used for detection of the convex corner instead of the concave corner. In this case, the complement set X C of X needs to be calculated in order to detect the corners of X C . The corners of X and X C together compose the whole corner set of X .
3. The method of corner detection for discrete binary images Corners are identi®ed where the direction changes rapidly in a chain code in a discrete image. The principle of the corner detector based on morphology is to probe the image under study with a structuring element. Since the shape and size of the structuring element must be set according to the information that is to be extracted, drB cannot be set too small in the discrete image. Instead, it should be de®ned as an accurate plate. Therefore, Eq. (2) should be modi®ed as shown in Eq. (3) in terms of a generalized morphologic opening: SK0 X X
X B1 B2 :
3
When the radius of B2 is one pixel larger than the radius of B1 in the discrete image, X can be reconstructed completely from the skeleton. Thus, two kinds of minimum plate S4 and S8 are de®ned and shown in Fig. 3. Suppose I be an image, then I S4 i; j maxfI i 1; j; I i; j 1; I i; j; I i
1; j; I i; j
1g;
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
895
Fig. 3. Two kinds of discrete minimum plate.
I S4 i; j minfI i 1; j; I i; j 1; I i; j; I i
1; j; I i; j
1g;
I S8 i; j maxfI i 1; j 1; I i 1; j; I i 1; j I i 1; j; I i 1; j 1g;
1; I i; j 1; I i; j; I i; j
1; I i
1; j 1;
I S8 i; j minfI i 1; j 1; I i 1; j; I i 1; j
1; I i; j 1; I i; j; I i; j
1; I i
1; j 1;
I i
1; j; I i
1; j
1g:
If the radii of B2 and B1 are unequal, a more complex calculation is needed. The following equation is used as a corner detection operator: Cor1 I
I S8 S8 :
4
This operator can only detect convex corners whose internal angles are less than p. It can also remove pseudo-lattice corners. For detection of an obtuse-angle corner, B2 should be a proper subset of B1 and is rather similar to a plate in Eq. (3). Therefore, the following equation is derived as a new corner detection operator: Cor2 I
I S8 S4 :
5
This operator can be used not only for acute-angle corners but also for obtuse-angle corners where the directions change rapidly in chain code. In an extreme case, the diagonal in the p=4 direction will be detected as a continuous line because there is a little sidestep in each adjacent point on the diagonal. The complement set I C of a source image I is calculated for a detection concave corner, but the boundaries of I and I C are not the same and have one pixel discrepancy. After executing a dilation calculation for I C , I 0 I C S4 , I 0 and I have the same boundary. The Cor3 operator in the following equation (6) is used for I 0 corner detection. Cor3 I 0
I 0 S8 S4 :
6
The concave corner of I will be detected using this operator and the diagonal in the p=4 direction is detected as a continuous line that is the same as that obtained using Cor2 . As a result, all the corners are denoted as the following formula: Cor Cor2 ^ Cor3 ;
7
where ^ denotes the hetero-OR operation of two images; the pseudo-corner can be counteracted. Fig. 4 shows the ¯owchart of the corner detection method based on the morphologic skeleton.
896
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Fig. 4. The ¯owchart of the corner detection method based on the morphologic skeleton.
4. Experimental results and discussions Fig. 5 shows the result of corner detection for English letters ``HUST''. Only the concave corners and some little sidesteps in adjacent points on the edge are detected in Fig. 5(b), and only the convex corners and some little sidesteps in adjacent points on the edge are detected in Fig. 5(c). The real corners in the source image are boosted up, and some pseudo-lattice corners on the diagonal are eliminated in Fig. 5(d). The result of corner detection of concave polygons with holes is shown in Fig. 6. Only the acute-angle corners are detected in Fig. 6(e). Fig. 6(f) is the result when the supplemental of source image is operated by operator Cor1 , then hetero-OR operated with Fig. 6(e). All real corners are detected, except for the obtuseangle corners. This method has an accuracy of about one pixel locating. Considering the computational complexity, there are 1 4 8N 2 times comparing calculations for an N N pixel image by operator Cor2 , 1 4 1 4 8N 2 times by operator Cor3 , and N 2 times by operator Cor. There are only 32N 2 times comparing calculations involved in the algorithm shown in Fig. 4. The experiments show that this method produces a very high value in locating corners. Because morphological operations do not have any ¯oating point and multiplication operations, this algorithm can be executed in parallel. Hence it is cost-eective with simple hardware implementation. For noisy images the algorithm has smoothing eects on small changes of noisy edges; this advantage is attributed to morphological opening ®lter and the uniformly disk-like structuring element. The experiment results show that the larger the radii of the structuring elements B1 and B2 , the larger the corner regions and the easier the corner estimation. Thus, the precision for locating a corner is reduced, and the computational cost is increased. Laganiere's (1998) algorithm is similar to the method proposed in this paper. It is much more complex than our method because of the combination of several structuring elements; the detection result for
Fig. 5. The result of corner detection for English letters ``HUST''. (a) Source image; (b) result of corner detection using operator Cor3 ; (c) result of corner detection using operator Cor2 ; (d) result of hetero-OR operation between (b) and (c).
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
897
Fig. 6. The result of corner detection for concave polygons with holes. (a) Source image; (b) result of corner detection using operator Cor2 ; (c) result of corner detection using operator Cor3 ; (d) result of hetero-OR operating between (b) and (c); (e) result of corner detection using operator Cor1 ; (f) result when the supplemental of source image is operated by operator Cor1 and hetero-OR operator with (e).
concave points is not reported; also, it must use global thresholds to get the ®nal detection result, but the author cannot report an eective method to select appropriate threshold values. Furthermore, because the algorithm uses a Gauss ®lter for noisy images, the hardware implementations are also complex. 5. Conclusions We have presented a new algorithm for corner detection based on the morphologic skeleton. This approach gives competitive results with respect to other available techniques. The proposed method has the advantages of computational simplicity and high-speed performance over existing algorithms. It also performs well on noisy images. Stable and accurate corner detection results have been achieved using the proposed method. Future research will focus on corner detection in grayscale images. Appendix A. The proof of Theorem 1 Lemma A.1. Suppose X is a polygon, B is a closed unit circle in E2 , then SKr X X rB
X rBdrB :
Proof. rBx X () x 2 X rB;
r aBy X () aBy X rB;
where symbol () denotes if and only if.
rBx r aBy () x 2 aBy ;
898
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
As a result, x 2 SKr X () rBx is the maximum circle with center x () x is the maximum circle of X rB (radius is zero) () x 2 X rB, but x do not belong to any X rBdrB () x 2 X rB X rBdrB , then Lemma A.1 holds. Lemma A.2. SK a X SK X . Proof. Suppose x 2 X , x 2 SK a X , but x 62 SK X . As a result, rX x is the maximum circle with center x and is tangential to oX in point P (oX denotes the boundary of X ). Connect P and x to obtain a line, make a neighboring zone t of x on the line. By all appearances, when t ! 0; x t 62 SK X ; rX x t is the maximum circle with center x t and is tangential to oX in point P , then d x; X rX x tB t, so x 62 SK a X . Therefore x 2 SK X . Theorem 1. Let X be a polygon and one of its internal angles be 2h; h < p=2. When a < 1= sin h, SKra X X rB
X rBdrB a
Proof. (1) First we prove that d x; X rX x tB x : lim >a ; t!0 t
1drB:
a P 1 X rB
X rBdrB a
1drB:
Since SKra X SKr X ;
9r > 0; x 2 SKra X ) x 2 SKr X ;
denotes that rBx is a maximum circle in X and r rX x, so x 2 X rB Now we must prove that x 62 X rBdrB a 1drB. Since x 2 SK a X ; for e ax a ax
r rX x;
ax , lim
t!0
X rBdrB.
d x; X r tB > a; t
a > 0; 9d > 0, when 0 < t < d, e<
d x; X r tB < ax e; t
so d x; X r tB=t > a 8t 2 0; d; and d x; X r tB > at or x atB \ X r tB ;: Then x 62 X r tB atB X r tB t a Then x 62 X rBdrB a
1drB; dr ! 0:
1tB X rBtB a
1tB:
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Therefore d x; X rX x tB x : lim >a ; t (2) We now prove X rB
a P 1 X rB
X rBdrB a
1drB
x : lim
t!0
X rBdrB a
d x; X rX x tB >a ; t
899
1drB:
a P 1:
Because x 2 X rB
X rBdrB a
1drB X rB
X rBdrB SKr X ;
then rBx X is a maximum circle and r rX x. Since 8dr > 0;
x 2 X rB
X rBdrB a
1drB;
we have x 62 X rBdrB a
1drB X r drB a drB:
Then x adrB \ X r drB ;; d x; X r drB > a dr d x; X r drB >a dr
8dr > 0;
8dr > 0:
Then lim
t!0
d x; X r tB P a; t
and we must prove that lim d x; X r tB=t 6 a:
t!0
Since x 2 SKr X , in polygon X , d x; X r tB t= sin h, as is shown in Fig. 1. If lim d x; X r tB=t a
t!0
holds, then lim
t!0
d x; X r tB t= sin h 1 lim a; t!0 t t sin h
so we can ®nd some points x, and make d x; X r tB at. Then x atB \ X r tB 6 ;, x 2 X r tB atB. As a result, x 62 X rB X rBdrB a 1drB, which con¯icts with the supposition. So limt!0 d x; X r tB=t a does not hold. Therefore lim
t!0
d x; X r tB > a; t
a P 1:
900
L. Wen-Yu et al. / Pattern Recognition Letters 22 (2001) 891±900
Thus
X rB
X rBdrB a
1drB
d x; X rX x tB x : lim >a ; t!0 t
by all appearances, when a P 1= sin h; SKra X is a null set.
a P 1;
References Asada, H., Brady, M., 1986. The curvature primal sketch. IEEE Trans. Pattern Anal. Machine Intell. 8 (1), 2±14. Deriche, R., Blaszka, T., 1993. Recovering and characterizing image features using an ecient model based approach. In: Proc. CVPR. pp. 530±535. Deriche, R., Giraudon, G., 1990. Accurate corner detection: an analytical study. In: Proc. Internat. Conf. Computer Vision. pp. 66±70. Fischler, M.A., Bolles, R.C., 1986. Perceptual organization and curve partitioning. IEEE Trans. Pattern Anal. Machine Intell. 8 (1), 100±105. Ji, Q., Haralick, R.M., 1997. Corner detection with covariance propagation. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition. pp. 362±367. Kitchen, L., Rosenfeld, A.K., 1982. Gray level corner detection. Pattern Recognition Letters 3 (1), 95±102. Krishnan, R., Mubarak, S., David, V.B., 1989. Optimal corner detector. CVGIP 48, 230±245. Laganiere, R., 1998. A morphological operator for corner detection. Pattern Recognition 31 (11), 1643±1652. Lai, K.K., Wu, P.S.Y., 1996. Eective edge±corner detection method for defective images. Proc. Internat. Conf. Signal Processing 2 (1), 151±154. Langride, D.J., 1982. Curve encoding and the detection of discontinuities. CVGIP 20 (1), 58±71. Lee, K.J., Bien, Z., 1996. Grey-level corner detector using fuzzy logic. Pattern Recognition Letters 17 (9), 939±950. Medioni, G., Yasumoto, Y., 1986. Corner detection and curve representation using cubic B-splines. In: Proc. 1986 IEEE Internat. Conf. Robotics and Automation, San Fransisco, April. pp. 764±769. Noble, J.A., 1988. Finding corners. Image and Vision Comput. 6, 121±128. Rohr, K., 1992. Modeling and identi®cation of characteristic intensity variations. Image and Vision Comput. 10, 66±76. Rosenfeld, A.K., 1982. Digital Picture Processing, second ed. Academic Press, New York. Smith, S.M., Brady, J.M., 1997. SUSAN ± a new approach to low-level image processing. Internat. J. Comput. Vision 23 (1), 45±78. Tsai, D.M., 1997. Boundary based corner detection using neural networks. Pattern Recognition 30 (1), 85±97.