- Email: [email protected]
Contour sequence moments for the classification of closed planar shapes
0031 3203/87 $3.00+ .00 Pergamon Journals Ltd. © 1987 Pattern Recognition Society
Pattern Reco~lnition. Vol. 20. No. 3. pp. 267 272, 1987. Printed in Great Britain.
CONTOUR SEQUENCE MOMENTS FOR THE CLASSIFICATION OF CLOSED PLANAR SHAPES L. GUPTA and M. D. SRINATH Department of Electrical Engineering, Southern Methodist University, Dallas, Texas 75275, U.S.A. (Received 16 January 1986; in revised form 3 June 1986)
Abstract-- A technique for classifying closed planar shapes is described in which a shape is characterized by a one-dimensional sequence that represents the Euclidean distance between the centroid and all contour pixels of the shape. Contour sequences obtained from shapes belonging to the same class are similar and are considered to be different realizations of a stochastic process. Statistical moment functions derived from contour sequences are used to characterize the process and hence the shapes. It is shown that these moment functions are invariant to shape translation, rotation and scale transformations. Examples are presented to illustrate the performance of these moment functions in the classification of noisy shapes. Shape classification moments
Planar shape description
Contour representation
1. I N T R O D U C T I O N
The use of two dimensional geometrical moments computed from the Cartesian and polar coordinates has been discussed in the classification and analysis of shapes. "-'8) In this paper, an alternate approach is described in which a binary shape is classified through the use of one-dimensional statistical moments extracted from a sequential representation of the contour of a shape. The boundary is characterized by an ordered sequence that represents the Euclidean distance between the centroid and all contour pixels of the digitized shape. This ordered sequence provides a unique representation for each shape. In general, the contour of a closed planar digitized shape can be described by a sequence V -- {v(1), v(2), .... v(n)}. The samples v(j) may be the elements of a chain code, "9'2°) primitive elements of a syntactic representation, (21'221 segments of polygon approximations, (2~''~) or the elements of shape and angular variation profiles derived by sampling the boundaryJ 2"-'71In this paper, the boundary is described by an ordered sequence Z which is obtained as follows. Starting at a selected contour pixel, the coordinates of the N contour pixels of a digitized shape are described by an ordered set {(x(i), y(/)), i = 1, 2. . . . . N}. The Euclidean distance z(i), i ffi 1, 2 . . . . . N of the vector connecting the centroid (g, .P) and the ordered set of contour pixels forms a single valued one-dimensional functional representation of the contour. ( ~ ) Since only closed contours are considered, the resulting sequential representation is circular. That is z ( N + i) f z(i)
if
1 , 2 . . . . . N.
(1)
The sequential representation is obtained by an
Invariant
algorithm that tracks the contour in a clockwise direction. In the process of tracking~ the algorithm orders contour pixeis and computes the Euclidean distance between the centroid and the contour pixels. Examples of contour sequences of shapes A3 and A4 shown in Fig. 1 are given in Fig. 2. Contour sequences are similar to several boundary representations derived by sampling the contour at equal angles from the centroid.(24-271 In the equiangular sampling approach, the contour is represented by a sequence whose elements may be the length of the radii connecting the centroid and the sampled contour points, or the tangent at sampled contour points, or the length of the chords connecting the sampled contour pixels. The sequence obtained by sampling the contour at equal angles from the centroid has a fixed duration and is therefore a very convenient representation for techniques that require a fixed number of observations. However, there are several disadvantages in implementing this procedure. The sequences obtained are approximations of the contour and many shapes may have similar sequences if the number of radii samples are not large enough. Also, equi-angular sampling of the contour does not lead to uniform spacing between the selected pixels along the contour. Problems also occur if shapes have spiral boundaries in which the radii intersect the contour at more than one point. For such shapes, the resulting representation is a multi-valued function. Additional processing is required to obtain a single valued function, which leads to further approximationsJ ~7)The contour sequence approach overcomes the disadvantages encountered with the equi-angular sampling method. For example, since every pixel is considered in a predefined order, contour sequences are identical only
267
268
L. GUPTAand M. D. SRINATH SET
AI
A2
A
SET
A3
BI
A4
B
' B2
B3
B4
Fig. 1. Reference shapes Set A and Set B. A4
A3
°.•~
z(i)
°~
:. .:'.~
:"'" "
"'°"
z(i)
•.~'•
• "...•."
•
~.-" ::
•'•
:•
=.•.
-
,•
•r
~:
:
". :~ ".
:
o
•..
.:"•
'.°
°¢ :
.•
-.. "o.
Fig. 2. Contour sequences of shapes A3 and A4.
if they are derived from the same shape. As a result of tracking the contour, the functional representation is always single valued. Therefore, no extra processing is required for shapes having spiral or concave contours• A careful examination of contour sequences reveals that similar shapes have similar contour sequences• Therefore the similarity between shapes can be determined from the degree of similarity between measures extracted from these sequences.
2. INVARIANT FEATURES FROM CONTOUR SEQUENCES
For invariant shape classification, the measures or features extracted from contour sequences must be independent of elementary shape transformations such as translation, rotation and scaling. The selected set of features should also contain sufficient information and be robust in the presence of a reasonable amount of noise in the shape. The determination of transformation invariant features is facilitated by considering the effects of the elementary shape transformations on contour sequences. Approximations in the functional representation due to quantization are ignored in the following discussion. Translation. A shift in the position of a shape results in a new set of ordered contour pixels {(x,(/), Y,(0), i = 1, 2 . . . . . N} as well as a new centroid ( ~ 9,). The relative Euclidean distance between the ordered set of contour pixels and the new centroid remains unchanged. Therefore, the contour sequence z,(/) of a translated shape is identical to the contour z(0 of the original shape. That is, z,(/) = z(0
i = 1, 2..... N.
(2)
Rotation• Rotation in a shape about any point in the image plane corresponds to a translation operation
accompanied by rotation about the centroid. From equation (2), contour sequences are translation invariant and hence shape rotation corresponds only to a reordering of the contour sequence pixels. This results in a circular shift s in the samples of the contour sequences due to a different pixel being chosen as the start point. Therefore, the contour sequence z,(i) of a rotated shape is related to that of the original shape by z,(O=z(i+s)
s=-N,-N+I
..... 1,2 ..... N.
(3)
Scaling. If a shape is uniformly scaled by a factor ~, the perimeter is scaled by • and the area is scaled by a factor a2.oo)Therefore, the length of the boundary and the vectors connecting the centroid and the boundary are scaled by ,,. As a result, the duration as well as the amplitudes of the contour sequence are scaled proportionately• Since the overall shape of an object is unchanged by scale transformations, the contour sequence representation of a scaled shape has the same profile as that of the original shape. Summarizing, contour sequences that represent the Euclidean distance between the centroid and all contour pixels are invariant to shape translation. Rotation causes a circular shift of the sequence and scale changes result in a scaling of the amplitudes and duration• In order to achieve invariant classification, the set of features computed from the contour sequences must be invariant to rotation and scaling or must be normalized with respect to these two transformations. As noted earlier, the contour sequence representation is different for each shape, and sequences obtained from the same class of shapes are similar. lntra-class variations due to geometric distortion and random noise reflect equally as variations in the contour sequence representation. Therefore, contour sequences belonging to a particular class may be
Classification of dosed planar shapes regarded as different realizations of the stochastic process that describe~ the class. Under the assumption that the underlying process densities are distinct, statistical moments can be used to characterize the process densities and hence the shapes. Given an N-point contour sequential representation z(i), i = 1, 2 ..... N of a binary shape Z(x, y), the rth moment can be estimated as
,n,
1 s
E [z(0]'
=
(4)
iml'm
and the rth central moment can be estimated as 1
269 = --,m'° 33= __m°
moo where,
m~ = ~ ~ x, yq G(x, y)
is the (p + q)th geometric'moment of the shape function G(x, y). Similarly, (ti, fi) is the centroid of the transformed shape. Let the m,n represent the rth contour sequence moment of H(u, v). From equations (6) and (7), the normalized contour sequence moments of shape H(u, v) are l
(5) • ~=
Let the rth normalized contour sequence moment be defined as
Z
and let the corresponding normalized central moment be defined as
1 M,
22 [z(0 - m,T N ~-J
It can be shown that the normalized contour sequence moments rh, and /Q, are invariant to translation rotation and scaling by noting that the coordinates of a transformed shape H(u, v) are related to the original shape G(x, y) by a transformation of the form
n(u, v) = A G(x, y) + B.
(8)
That is, the transformed coordinate variables are given by
v,
L-sin0
,,cos0J
f [ h ( k ) - toni'
M k=|
,l,j
,.,f
(16)
[h(k) -- miniz Jlr/2
N i-t
'"')"
M
[ --~,~1
1 ~ [z(0]' ffl, .~. mr
(15)
P q
N
M, = ~ ,=,~ [z(0 - m,]'.
(14)
moo
y,
/~ and ~, are the translation variables, ~t is the scale factor, 0 is the angle through which the shape is rotated. Let the contour sequence of the original shape be G and that of the transformed shape be H. That is G = [g(l), 0(2)..... g(N)]
(10)
H = [h(1), h(2)..... h(M)]
(11)
where,
By substituting (9) in (13) and using the resulting h(k) in (16) and (17), the normalized moments r~f a n d / ~ of the transformed shape H(u, v) are expressed in terms of the original coordin.ate variables (x, y). Using the definitions of normalized contour sequence moments given in equations (6) and (7), and some algebraic manipulations, it can be shown that r~,n = r~
(18)
/~," =/Q,~.
(19)
While moments of an arbitrarily large order can be derived from the contour sequence and used as features for shape classification, due to their larger dynamic range, higher order moments are more sensitive to noise and the resulting classifier will be less tolerant to noise. Therefore, a few relatively low order moments which are stable (i.e. have smaller variances) are selected. For example, a set of good shape features based on the four lower order moments are (a) Normalized amplitude variation:
r l = (M')'------~ ~=
,-, [z(0 - ,rid' N ~1 ,.~ z(O
m,
(20)
(b) Coefficient of skewness: a(/) = [(x, -- ~)2 + (y, _ .~)2]1/2
(12)
h(k) = [(uk - fi)2 + (v~ - ~)2]1/2
(13)
(~, ~) is the centroid of the original shape and is given by
F2=
M3
± ~ [ z ( 0 - rn,] ~ N~-I
(M2)3/2
[z(i)i-i
rnl] 2
(21)
270
L. GuvT^ and M. D, SRINATH
(c) Coefficient of kurtosis:
l_ M,, N i-I F3=(M2)'---'~=[I .~'
~,
Ez(O - r n ; l "
]2"
(22)
[z(/) - m,]2
FI is a dispersion statistic that characterizes the variability of a density function and also can be regarded as a measure of the amplitude variations in a contour sequence. F I is non-negative with a value equal to zero only if the contour is a perfect circle (the corresponding contour sequence is constant over its entire range). Note that F I is a transformation invariant statistic. F2 and F3 (the third and fourth normalized contour sequence moments), are shape measures which relate to the degree of symmetry and peakedness of the density function respectively. Skew is positive when the majority of contour sequence samples have a value lower than the mean sample value and kurtosis is positive if most of the samples are concentrated around the mean. 3. CLASSIFICATION
Experimentation with the two sets of data shown in Fig. 1 shows that FI, F2, F3, and the fifth normalized contour sequence moment F4 = Ats are sufficient for reliable shape classification using an unweighted minimum distance classifier. The data sets in Fig. 1 have shapes with concave contours in which the vector projected from the centroid may intersect the contour at more than one point. In a 32 by 32 image plane, reference shapes A 1 through A4 have 129, I 17, 129, 119 contour pixels and shapes BI through B4 have 88, 92, 106, 102 contour pixels. Noisy test shapes are generated by adding various degrees of distortion to the contour of a shape. Each contour pixei is assigned a probability p of retaining its original location and a probability (1 - p) of being shifted randomly to one of its eight neighbours. Shape distortion can be increased by repeating the process for a given value ofp. Typical noisy shapes are shown in Fig. 3. Note that introducing noise in this manner alters the amplitudes, duration and the overall shape of the resulting sequential representation. Independent experiments were conducted using the two sets of shapes in Fig. 1. In the training phase, reference feature vectors F t = rFk(1), Ft(2), Fk(3), FX(4)], k = 1, 2, 3, 4 were computed for the four classes belonging to each set. In the testing phase one hundred noisy shapes were generated for each class using a particular value ofp. A
variety of decision rules may be used to determine which reference feature vector best matches the unknown test feature vector. One of the simplest and most practical to use is the minimum distance rule, in which the unknown shape, represented by a feature vector T, is assigned to the class k*, given by k* = arg rain [O k]
k = 1, 2, 3, 4
(23)
D~ is the Euclidean distance between the test feature vector Tand the kth reference feature vector F ~and is given by Dk = l I T - Fkll---- [ ( T - F ~ r t r - Fk)] 'n k = 1, 2, 3, 4.
(24)
Classification results are shown in the form of confusion matrices in Table I. The diagonal terms in the matrices correspond to correct classifications and the off-diagonal terms represent misclassifications. For example in Table 1, out of the hundred noisy test shapes belonging to class AI, eighty eight were classified correctly as A 1 and twelve were erroneously classified as A2. The ratio of the sum of the off-diagonal elements to the sum of all the elements gives the probability of error, which is used as a measure of performance. This technique was compared with the method of geometrical moment invariants which uses a set of seven features that are derived from the second and third order geometrical moments of the shape.(L 4.6,7.m... ~3)The moment invariants of a binary shape G(x, y) are given by ~1 = ('/,o + '/02)
(25)
~b2 = ('/,o - qo2)2 + 4'/~1
(26)
~a = ('/ao - '/1,) 2 + (3'/21 + '/03)2
(27)
~a = (r/30 + '/12) 2 + ('/21 + '/03) 2
(28)
#5 = ('/3o- 3'/~2)('/3o+ '/~)[('/~o + '/,92_ 3('/2~ + t/03)2] +(3'/a~ -'/o3)(~'~ +'/03)[3('/30 +'/~,)2-('/2~ +'Io3)2] (29) #6 = ('ho - '/02)[('/~o+ '/~z)~- ('7:, + '/03)2] + 4'/11('/3o+r/12) ('/21+qo3) (30) ~ = (3rh2 -'/30) ('/30 +'/~2) [0/30 +'/12)2 - 3('/21 + '/03)2] +(3rl21 --'/03)('/21 +'/03)[3('/30 +'/12)2--('/21 +'/03) 2]
(31)
where '/~ = ~-~
PffiO.8
pzo.7
Fig. 3. Typical noisy test shapes.
(32)
Classification of closed planar shapes
271
Table 1. Classification using contour sequence moments. (a) Probability of error = 0.10. (b) Probability of error = 0.13
AI A2 A3 A4
A1
A2
A3
A4
B1
88 12
4
0
0
92
0
0
B1 B2
80 20
0
4
88
8
B3
0
0
0
12
92
B4
0
(a) Set A, p = 0.7
B2
B3
IM
4
0
0
80 16 0
8 88 4
0 0
100
(b) Set B, p = 0.8
Table 2. Classification using moment invariants. (a) Probability of error = 0.18. (b) Probability of error = 0.21
AI A2 A3 A4
A1
A2
A3
A4
80 0 0 20
0 76 24 0
8 16 76 0
0 0 4 96
(a) Set A, p -- 0.7 with
and
#~ = ~ , ~ , ( x - Y , ) P ( Y - Y ) q G( x, Y). l~ q = 0 , 1. . . . (34) x
y
T w o sets of m o m e n t invariants ~1 through ~ are computed from G(x, y) to form a total of fourteen features. The first set is computed from the pixel coordinates of the entire shape and the second set from the coordinates of the contour pixds. Results using geometrical m o m e n t invariants are shown in Table 2. The confusion matrices in Tables 1 and 2 describe typical classification results obtained with several sets of data similar to the shapes shown in Fig. 1. Experimentation with various classes of shapes reveals that both techniques are data dependent. In particular, geometrical m o m e n t invariants are highly dependent on the symmetry of the shapes. For example in equation (34), moments involving odd powers of p (q) are zero for shapes that are symmetrical about the vertical (horizontal) axis. Any m o m e n t containing an odd power of either p and q is zero for shapes that are symmetric both in the vertical and horizontal directions. F o r such shapes that are both vertically and horizontally symmetric, moment invariants 4~3 through 4~7 are the same (zero), and thus provide no information for class separ~ition. In general, it was observed that lower probabilities of error were obtained using contour sequence moments. REFERENCES
1. M. K. Hu, Visual pattern recognition by moment invariants, IRE Trans. Inf. Theory, 179-187 (1962). 2. F.L. Air, Digital pattern recognition by moments, Optical Character Recognition, G. L. Fischer et al., eds, pp. i 53 - 179. Spartan, Washington D.C. (1962).
B1 B2 B3 134
BI
B2
B3
IM
80 20 0 0
4 92 4 0
4 12 64 20
0 16 4 80
(b) Set B, p = 0.8
3. J. W. Butler et al. Automatic classification of chromosomes, Data Acquisition and Processing in Biology and Medicine, Vol. 3, K. Enshen, ed. Pergamon Press, New York (1964). 4. S. Dudani et al., Aircraft Identification by Moment Invariants, IEEE Trans. Comput. C-26, 39-45 (1977). 5. F. W. Smith and M. H. Wright, Automatic ship photo interpretation by the method of moments, IEEE Trans. Comput. C-20, 1089-1094 (1971). 6. R. C. Gonzalez and'P. Wintz, Digital Image Processing, pp. 354-358. Addison-Wesley, Reading, MA (1977). 7. F. A. Sadjadi and E. L. Hall, Numerical computation of moment invariants for scene analysis, IEEE Conf. Pattern Recognition and Image Process. pp. 181-187, Chicago (1978). 8. R.Y. Wong and E. L. Hall, Scene matching with invariant moments, Computer Graphics Image Process. 8, 16-24 (1978). 9. E. L. Hall, Computer Image Processing and Recognition, pp. 287-290, Academic Press, New York (1979). 10. S. Maitra, Moment invariants, Proc. IEEE 67, 697-699 (1979). 11. D. Casasent and D. Psaltis, Optical pattern recognition using normalized invariant moments, SPIE 201, Optical Pattern Recognition, pp. 107-114 (1979). 12. M. R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70, 920-930 (1980). 13. S. S. Reddi, Radial and angular moment invariants for image identification, IEEE Trans. Pattern Anal. Mach. lntell. 3, 240-242 (1981). 14. D. Casasent, et al., Optical system to compute intensity moments, Appl. Opt. 21, 3292-3298 (1982). 15. D. Lucas, Moment techniques in picture analysis, Proc. IEEE Computer Society on Computer Vision and Pattern Recognition, Washington D.C., pp. 178-187 (1983). 16. Y. S. Mostafa and D. Psaltis, Recognition aspects of moment invariants, IEEE Trans. Pattern Anal. Mach. lntell. 6, 698-706 (1984). 17. C. H. Teh and R. T. Chin, On digital approximation of moment invariants, Proc. IEEE Computer Society on Computer Vision and Pattern Recognition, California, pp. 640-642 (1985). 18. Y. S. Mostafa and D. Psaltis, Image normalization by complex moments, IEEE Trans. Pattern Anal. Mach. lntell. 7, 46-55 (1985). 19. H. Freeman, Shape description via the use of critical points, Proc. IEEE Computer Society Conf. Pattern Recognition and Image Processing, pp. 168-197 (1977).
272
L. GUPTAand M. D. SPJ~ATH
20. H. Freeman and A. Saghri, Generalized chain codes for planar curves, Proc. 4th Int. Joint Conf. Pattern Recognition, Kyoto, Japan, pp. 701-703 (1978). 21. T. Pavlidis and F. Ali, A hierarchical syntactic shape analyser, IEEE Trans. Pattern Anal. Mack lntell. 1, 307-309 (1979). 22. K.S. Fu, Syntactic Pattern Recognition and Applications. Prentice Hall, New Jersey (1982). 23. T. Pavlidis and F. Ali, Computer recognition of handwritten numerals by polygon approximation, IEEE Trans. Syst. Man. Cybernet. SMC-5, 610-614 (1975). 24. J. J. Hwang and E. L. Hall, Scene representation using adjacency matrices and sampled shapes of regions, Proc. IEEE Conf. Pattern Recognition lmaoe Process, Washington D.C., pp. 250-261 (1978). 25. R. L. Kashyap and R. Chellappa, Stochastic models for closed boundary analysis, IEEE Trans. Inf. Theory,
IT-27, 627-637 (1981). 26. P. F. Singer and R. Chellapa, Classification ofboundaries on the plane using stochastic models, Proc. IEEE Conf. Computer Vision and Pattern Recognition, Washington D.C., pp. 146-147 (1983). 27. S. R. Dubois and F. H. Glanz, An autoregressive model approach to two-dimensional shape classification, IEEE Trans. Pattern Anal. Mach. lnteU. 8, 55-66 (1986). 28. L. Gupta and M. D. Srinath, Non-linear alignment of contours for shape classification, Twenty Third Allerton Conf. Communications, Control and Computing (1985). 29. L. Gupta and M. D. Srinath, Contour classification using invariant moments, Fifth Annual Phoenix Conf. on Computers, Communications and Control (I 986). 30. A. Rosenfeld and A. V. Kak, Digital Imaoe Processing, Vol. 2. Academic Press, New York (1982).
About the Author-- L^LIT GUPTAreceived the B.E. (Hons) degree in Electrical Engineering from the Birla Institute of Technology and Science, Pilani, India (1976), the M.S. degree in digital systems from Brunel University, Middlesex, England (1981) and the Ph.D. degree in Electrical Engineering from Southern Methodist University, Dallas, Texas (1986)• He is currently an Assistant Professor with the Department of Electrical Engineering, Southern Illinois University at Carbondale. His research interests include computer vision, pattern recognition and signal processing. About the Author--M. D. SRXNATHreceived the B.Sc. degree from the University of Mysore, India, in 1954, the diploma in Electrical Technology from the Indian Institute of Science, Bangalore, in 1957, and the M.S. and Ph.D. degrees in Electrical Engineering from the University of Illinois, Urbana, in 1959 and 1962, respectively. He has been with the Department of Electrical Engineering at both the University of Kansas, Lawrence, and the Indian Institute of Science, Bangaiore, India. Since 1967 he has been with the Southern Methodist University, Dallas, Texas, where he is currently a Professor of Electrical Engineering. He served as Interim Chairman of the Department of Electrical Engineering from September 1980 to August 1982. His research interests include estimation theory, digital signal processing, and system identification. He has published numerous papers and is principal author of the text, Introduction to Statistical Signal Processing with Applications (Wiley-1nterscience).
Pattern Reco~lnition. Vol. 20. No. 3. pp. 267 272, 1987. Printed in Great Britain.
CONTOUR SEQUENCE MOMENTS FOR THE CLASSIFICATION OF CLOSED PLANAR SHAPES L. GUPTA and M. D. SRINATH Department of Electrical Engineering, Southern Methodist University, Dallas, Texas 75275, U.S.A. (Received 16 January 1986; in revised form 3 June 1986)
Abstract-- A technique for classifying closed planar shapes is described in which a shape is characterized by a one-dimensional sequence that represents the Euclidean distance between the centroid and all contour pixels of the shape. Contour sequences obtained from shapes belonging to the same class are similar and are considered to be different realizations of a stochastic process. Statistical moment functions derived from contour sequences are used to characterize the process and hence the shapes. It is shown that these moment functions are invariant to shape translation, rotation and scale transformations. Examples are presented to illustrate the performance of these moment functions in the classification of noisy shapes. Shape classification moments
Planar shape description
Contour representation
1. I N T R O D U C T I O N
The use of two dimensional geometrical moments computed from the Cartesian and polar coordinates has been discussed in the classification and analysis of shapes. "-'8) In this paper, an alternate approach is described in which a binary shape is classified through the use of one-dimensional statistical moments extracted from a sequential representation of the contour of a shape. The boundary is characterized by an ordered sequence that represents the Euclidean distance between the centroid and all contour pixels of the digitized shape. This ordered sequence provides a unique representation for each shape. In general, the contour of a closed planar digitized shape can be described by a sequence V -- {v(1), v(2), .... v(n)}. The samples v(j) may be the elements of a chain code, "9'2°) primitive elements of a syntactic representation, (21'221 segments of polygon approximations, (2~''~) or the elements of shape and angular variation profiles derived by sampling the boundaryJ 2"-'71In this paper, the boundary is described by an ordered sequence Z which is obtained as follows. Starting at a selected contour pixel, the coordinates of the N contour pixels of a digitized shape are described by an ordered set {(x(i), y(/)), i = 1, 2. . . . . N}. The Euclidean distance z(i), i ffi 1, 2 . . . . . N of the vector connecting the centroid (g, .P) and the ordered set of contour pixels forms a single valued one-dimensional functional representation of the contour. ( ~ ) Since only closed contours are considered, the resulting sequential representation is circular. That is z ( N + i) f z(i)
if
1 , 2 . . . . . N.
(1)
The sequential representation is obtained by an
Invariant
algorithm that tracks the contour in a clockwise direction. In the process of tracking~ the algorithm orders contour pixeis and computes the Euclidean distance between the centroid and the contour pixels. Examples of contour sequences of shapes A3 and A4 shown in Fig. 1 are given in Fig. 2. Contour sequences are similar to several boundary representations derived by sampling the contour at equal angles from the centroid.(24-271 In the equiangular sampling approach, the contour is represented by a sequence whose elements may be the length of the radii connecting the centroid and the sampled contour points, or the tangent at sampled contour points, or the length of the chords connecting the sampled contour pixels. The sequence obtained by sampling the contour at equal angles from the centroid has a fixed duration and is therefore a very convenient representation for techniques that require a fixed number of observations. However, there are several disadvantages in implementing this procedure. The sequences obtained are approximations of the contour and many shapes may have similar sequences if the number of radii samples are not large enough. Also, equi-angular sampling of the contour does not lead to uniform spacing between the selected pixels along the contour. Problems also occur if shapes have spiral boundaries in which the radii intersect the contour at more than one point. For such shapes, the resulting representation is a multi-valued function. Additional processing is required to obtain a single valued function, which leads to further approximationsJ ~7)The contour sequence approach overcomes the disadvantages encountered with the equi-angular sampling method. For example, since every pixel is considered in a predefined order, contour sequences are identical only
267
268
L. GUPTAand M. D. SRINATH SET
AI
A2
A
SET
A3
BI
A4
B
' B2
B3
B4
Fig. 1. Reference shapes Set A and Set B. A4
A3
°.•~
z(i)
°~
:. .:'.~
:"'" "
"'°"
z(i)
•.~'•
• "...•."
•
~.-" ::
•'•
:•
=.•.
-
,•
•r
~:
:
". :~ ".
:
o
•..
.:"•
'.°
°¢ :
.•
-.. "o.
Fig. 2. Contour sequences of shapes A3 and A4.
if they are derived from the same shape. As a result of tracking the contour, the functional representation is always single valued. Therefore, no extra processing is required for shapes having spiral or concave contours• A careful examination of contour sequences reveals that similar shapes have similar contour sequences• Therefore the similarity between shapes can be determined from the degree of similarity between measures extracted from these sequences.
2. INVARIANT FEATURES FROM CONTOUR SEQUENCES
For invariant shape classification, the measures or features extracted from contour sequences must be independent of elementary shape transformations such as translation, rotation and scaling. The selected set of features should also contain sufficient information and be robust in the presence of a reasonable amount of noise in the shape. The determination of transformation invariant features is facilitated by considering the effects of the elementary shape transformations on contour sequences. Approximations in the functional representation due to quantization are ignored in the following discussion. Translation. A shift in the position of a shape results in a new set of ordered contour pixels {(x,(/), Y,(0), i = 1, 2 . . . . . N} as well as a new centroid ( ~ 9,). The relative Euclidean distance between the ordered set of contour pixels and the new centroid remains unchanged. Therefore, the contour sequence z,(/) of a translated shape is identical to the contour z(0 of the original shape. That is, z,(/) = z(0
i = 1, 2..... N.
(2)
Rotation• Rotation in a shape about any point in the image plane corresponds to a translation operation
accompanied by rotation about the centroid. From equation (2), contour sequences are translation invariant and hence shape rotation corresponds only to a reordering of the contour sequence pixels. This results in a circular shift s in the samples of the contour sequences due to a different pixel being chosen as the start point. Therefore, the contour sequence z,(i) of a rotated shape is related to that of the original shape by z,(O=z(i+s)
s=-N,-N+I
..... 1,2 ..... N.
(3)
Scaling. If a shape is uniformly scaled by a factor ~, the perimeter is scaled by • and the area is scaled by a factor a2.oo)Therefore, the length of the boundary and the vectors connecting the centroid and the boundary are scaled by ,,. As a result, the duration as well as the amplitudes of the contour sequence are scaled proportionately• Since the overall shape of an object is unchanged by scale transformations, the contour sequence representation of a scaled shape has the same profile as that of the original shape. Summarizing, contour sequences that represent the Euclidean distance between the centroid and all contour pixels are invariant to shape translation. Rotation causes a circular shift of the sequence and scale changes result in a scaling of the amplitudes and duration• In order to achieve invariant classification, the set of features computed from the contour sequences must be invariant to rotation and scaling or must be normalized with respect to these two transformations. As noted earlier, the contour sequence representation is different for each shape, and sequences obtained from the same class of shapes are similar. lntra-class variations due to geometric distortion and random noise reflect equally as variations in the contour sequence representation. Therefore, contour sequences belonging to a particular class may be
Classification of dosed planar shapes regarded as different realizations of the stochastic process that describe~ the class. Under the assumption that the underlying process densities are distinct, statistical moments can be used to characterize the process densities and hence the shapes. Given an N-point contour sequential representation z(i), i = 1, 2 ..... N of a binary shape Z(x, y), the rth moment can be estimated as
,n,
1 s
E [z(0]'
=
(4)
iml'm
and the rth central moment can be estimated as 1
269 = --,m'° 33= __m°
moo where,
m~ = ~ ~ x, yq G(x, y)
is the (p + q)th geometric'moment of the shape function G(x, y). Similarly, (ti, fi) is the centroid of the transformed shape. Let the m,n represent the rth contour sequence moment of H(u, v). From equations (6) and (7), the normalized contour sequence moments of shape H(u, v) are l
(5) • ~=
Let the rth normalized contour sequence moment be defined as
Z
and let the corresponding normalized central moment be defined as
1 M,
22 [z(0 - m,T N ~-J
It can be shown that the normalized contour sequence moments rh, and /Q, are invariant to translation rotation and scaling by noting that the coordinates of a transformed shape H(u, v) are related to the original shape G(x, y) by a transformation of the form
n(u, v) = A G(x, y) + B.
(8)
That is, the transformed coordinate variables are given by
v,
L-sin0
,,cos0J
f [ h ( k ) - toni'
M k=|
,l,j
,.,f
(16)
[h(k) -- miniz Jlr/2
N i-t
'"')"
M
[ --~,~1
1 ~ [z(0]' ffl, .~. mr
(15)
P q
N
M, = ~ ,=,~ [z(0 - m,]'.
(14)
moo
y,
/~ and ~, are the translation variables, ~t is the scale factor, 0 is the angle through which the shape is rotated. Let the contour sequence of the original shape be G and that of the transformed shape be H. That is G = [g(l), 0(2)..... g(N)]
(10)
H = [h(1), h(2)..... h(M)]
(11)
where,
By substituting (9) in (13) and using the resulting h(k) in (16) and (17), the normalized moments r~f a n d / ~ of the transformed shape H(u, v) are expressed in terms of the original coordin.ate variables (x, y). Using the definitions of normalized contour sequence moments given in equations (6) and (7), and some algebraic manipulations, it can be shown that r~,n = r~
(18)
/~," =/Q,~.
(19)
While moments of an arbitrarily large order can be derived from the contour sequence and used as features for shape classification, due to their larger dynamic range, higher order moments are more sensitive to noise and the resulting classifier will be less tolerant to noise. Therefore, a few relatively low order moments which are stable (i.e. have smaller variances) are selected. For example, a set of good shape features based on the four lower order moments are (a) Normalized amplitude variation:
r l = (M')'------~ ~=
,-, [z(0 - ,rid' N ~1 ,.~ z(O
m,
(20)
(b) Coefficient of skewness: a(/) = [(x, -- ~)2 + (y, _ .~)2]1/2
(12)
h(k) = [(uk - fi)2 + (v~ - ~)2]1/2
(13)
(~, ~) is the centroid of the original shape and is given by
F2=
M3
± ~ [ z ( 0 - rn,] ~ N~-I
(M2)3/2
[z(i)i-i
rnl] 2
(21)
270
L. GuvT^ and M. D, SRINATH
(c) Coefficient of kurtosis:
l_ M,, N i-I F3=(M2)'---'~=[I .~'
~,
Ez(O - r n ; l "
]2"
(22)
[z(/) - m,]2
FI is a dispersion statistic that characterizes the variability of a density function and also can be regarded as a measure of the amplitude variations in a contour sequence. F I is non-negative with a value equal to zero only if the contour is a perfect circle (the corresponding contour sequence is constant over its entire range). Note that F I is a transformation invariant statistic. F2 and F3 (the third and fourth normalized contour sequence moments), are shape measures which relate to the degree of symmetry and peakedness of the density function respectively. Skew is positive when the majority of contour sequence samples have a value lower than the mean sample value and kurtosis is positive if most of the samples are concentrated around the mean. 3. CLASSIFICATION
Experimentation with the two sets of data shown in Fig. 1 shows that FI, F2, F3, and the fifth normalized contour sequence moment F4 = Ats are sufficient for reliable shape classification using an unweighted minimum distance classifier. The data sets in Fig. 1 have shapes with concave contours in which the vector projected from the centroid may intersect the contour at more than one point. In a 32 by 32 image plane, reference shapes A 1 through A4 have 129, I 17, 129, 119 contour pixels and shapes BI through B4 have 88, 92, 106, 102 contour pixels. Noisy test shapes are generated by adding various degrees of distortion to the contour of a shape. Each contour pixei is assigned a probability p of retaining its original location and a probability (1 - p) of being shifted randomly to one of its eight neighbours. Shape distortion can be increased by repeating the process for a given value ofp. Typical noisy shapes are shown in Fig. 3. Note that introducing noise in this manner alters the amplitudes, duration and the overall shape of the resulting sequential representation. Independent experiments were conducted using the two sets of shapes in Fig. 1. In the training phase, reference feature vectors F t = rFk(1), Ft(2), Fk(3), FX(4)], k = 1, 2, 3, 4 were computed for the four classes belonging to each set. In the testing phase one hundred noisy shapes were generated for each class using a particular value ofp. A
variety of decision rules may be used to determine which reference feature vector best matches the unknown test feature vector. One of the simplest and most practical to use is the minimum distance rule, in which the unknown shape, represented by a feature vector T, is assigned to the class k*, given by k* = arg rain [O k]
k = 1, 2, 3, 4
(23)
D~ is the Euclidean distance between the test feature vector Tand the kth reference feature vector F ~and is given by Dk = l I T - Fkll---- [ ( T - F ~ r t r - Fk)] 'n k = 1, 2, 3, 4.
(24)
Classification results are shown in the form of confusion matrices in Table I. The diagonal terms in the matrices correspond to correct classifications and the off-diagonal terms represent misclassifications. For example in Table 1, out of the hundred noisy test shapes belonging to class AI, eighty eight were classified correctly as A 1 and twelve were erroneously classified as A2. The ratio of the sum of the off-diagonal elements to the sum of all the elements gives the probability of error, which is used as a measure of performance. This technique was compared with the method of geometrical moment invariants which uses a set of seven features that are derived from the second and third order geometrical moments of the shape.(L 4.6,7.m... ~3)The moment invariants of a binary shape G(x, y) are given by ~1 = ('/,o + '/02)
(25)
~b2 = ('/,o - qo2)2 + 4'/~1
(26)
~a = ('/ao - '/1,) 2 + (3'/21 + '/03)2
(27)
~a = (r/30 + '/12) 2 + ('/21 + '/03) 2
(28)
#5 = ('/3o- 3'/~2)('/3o+ '/~)[('/~o + '/,92_ 3('/2~ + t/03)2] +(3'/a~ -'/o3)(~'~ +'/03)[3('/30 +'/~,)2-('/2~ +'Io3)2] (29) #6 = ('ho - '/02)[('/~o+ '/~z)~- ('7:, + '/03)2] + 4'/11('/3o+r/12) ('/21+qo3) (30) ~ = (3rh2 -'/30) ('/30 +'/~2) [0/30 +'/12)2 - 3('/21 + '/03)2] +(3rl21 --'/03)('/21 +'/03)[3('/30 +'/12)2--('/21 +'/03) 2]
(31)
where '/~ = ~-~
PffiO.8
pzo.7
Fig. 3. Typical noisy test shapes.
(32)
Classification of closed planar shapes
271
Table 1. Classification using contour sequence moments. (a) Probability of error = 0.10. (b) Probability of error = 0.13
AI A2 A3 A4
A1
A2
A3
A4
B1
88 12
4
0
0
92
0
0
B1 B2
80 20
0
4
88
8
B3
0
0
0
12
92
B4
0
(a) Set A, p = 0.7
B2
B3
IM
4
0
0
80 16 0
8 88 4
0 0
100
(b) Set B, p = 0.8
Table 2. Classification using moment invariants. (a) Probability of error = 0.18. (b) Probability of error = 0.21
AI A2 A3 A4
A1
A2
A3
A4
80 0 0 20
0 76 24 0
8 16 76 0
0 0 4 96
(a) Set A, p -- 0.7 with
and
#~ = ~ , ~ , ( x - Y , ) P ( Y - Y ) q G( x, Y). l~ q = 0 , 1. . . . (34) x
y
T w o sets of m o m e n t invariants ~1 through ~ are computed from G(x, y) to form a total of fourteen features. The first set is computed from the pixel coordinates of the entire shape and the second set from the coordinates of the contour pixds. Results using geometrical m o m e n t invariants are shown in Table 2. The confusion matrices in Tables 1 and 2 describe typical classification results obtained with several sets of data similar to the shapes shown in Fig. 1. Experimentation with various classes of shapes reveals that both techniques are data dependent. In particular, geometrical m o m e n t invariants are highly dependent on the symmetry of the shapes. For example in equation (34), moments involving odd powers of p (q) are zero for shapes that are symmetrical about the vertical (horizontal) axis. Any m o m e n t containing an odd power of either p and q is zero for shapes that are symmetric both in the vertical and horizontal directions. F o r such shapes that are both vertically and horizontally symmetric, moment invariants 4~3 through 4~7 are the same (zero), and thus provide no information for class separ~ition. In general, it was observed that lower probabilities of error were obtained using contour sequence moments. REFERENCES
1. M. K. Hu, Visual pattern recognition by moment invariants, IRE Trans. Inf. Theory, 179-187 (1962). 2. F.L. Air, Digital pattern recognition by moments, Optical Character Recognition, G. L. Fischer et al., eds, pp. i 53 - 179. Spartan, Washington D.C. (1962).
B1 B2 B3 134
BI
B2
B3
IM
80 20 0 0
4 92 4 0
4 12 64 20
0 16 4 80
(b) Set B, p = 0.8
3. J. W. Butler et al. Automatic classification of chromosomes, Data Acquisition and Processing in Biology and Medicine, Vol. 3, K. Enshen, ed. Pergamon Press, New York (1964). 4. S. Dudani et al., Aircraft Identification by Moment Invariants, IEEE Trans. Comput. C-26, 39-45 (1977). 5. F. W. Smith and M. H. Wright, Automatic ship photo interpretation by the method of moments, IEEE Trans. Comput. C-20, 1089-1094 (1971). 6. R. C. Gonzalez and'P. Wintz, Digital Image Processing, pp. 354-358. Addison-Wesley, Reading, MA (1977). 7. F. A. Sadjadi and E. L. Hall, Numerical computation of moment invariants for scene analysis, IEEE Conf. Pattern Recognition and Image Process. pp. 181-187, Chicago (1978). 8. R.Y. Wong and E. L. Hall, Scene matching with invariant moments, Computer Graphics Image Process. 8, 16-24 (1978). 9. E. L. Hall, Computer Image Processing and Recognition, pp. 287-290, Academic Press, New York (1979). 10. S. Maitra, Moment invariants, Proc. IEEE 67, 697-699 (1979). 11. D. Casasent and D. Psaltis, Optical pattern recognition using normalized invariant moments, SPIE 201, Optical Pattern Recognition, pp. 107-114 (1979). 12. M. R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70, 920-930 (1980). 13. S. S. Reddi, Radial and angular moment invariants for image identification, IEEE Trans. Pattern Anal. Mach. lntell. 3, 240-242 (1981). 14. D. Casasent, et al., Optical system to compute intensity moments, Appl. Opt. 21, 3292-3298 (1982). 15. D. Lucas, Moment techniques in picture analysis, Proc. IEEE Computer Society on Computer Vision and Pattern Recognition, Washington D.C., pp. 178-187 (1983). 16. Y. S. Mostafa and D. Psaltis, Recognition aspects of moment invariants, IEEE Trans. Pattern Anal. Mach. lntell. 6, 698-706 (1984). 17. C. H. Teh and R. T. Chin, On digital approximation of moment invariants, Proc. IEEE Computer Society on Computer Vision and Pattern Recognition, California, pp. 640-642 (1985). 18. Y. S. Mostafa and D. Psaltis, Image normalization by complex moments, IEEE Trans. Pattern Anal. Mach. lntell. 7, 46-55 (1985). 19. H. Freeman, Shape description via the use of critical points, Proc. IEEE Computer Society Conf. Pattern Recognition and Image Processing, pp. 168-197 (1977).
272
L. GUPTAand M. D. SPJ~ATH
20. H. Freeman and A. Saghri, Generalized chain codes for planar curves, Proc. 4th Int. Joint Conf. Pattern Recognition, Kyoto, Japan, pp. 701-703 (1978). 21. T. Pavlidis and F. Ali, A hierarchical syntactic shape analyser, IEEE Trans. Pattern Anal. Mack lntell. 1, 307-309 (1979). 22. K.S. Fu, Syntactic Pattern Recognition and Applications. Prentice Hall, New Jersey (1982). 23. T. Pavlidis and F. Ali, Computer recognition of handwritten numerals by polygon approximation, IEEE Trans. Syst. Man. Cybernet. SMC-5, 610-614 (1975). 24. J. J. Hwang and E. L. Hall, Scene representation using adjacency matrices and sampled shapes of regions, Proc. IEEE Conf. Pattern Recognition lmaoe Process, Washington D.C., pp. 250-261 (1978). 25. R. L. Kashyap and R. Chellappa, Stochastic models for closed boundary analysis, IEEE Trans. Inf. Theory,
IT-27, 627-637 (1981). 26. P. F. Singer and R. Chellapa, Classification ofboundaries on the plane using stochastic models, Proc. IEEE Conf. Computer Vision and Pattern Recognition, Washington D.C., pp. 146-147 (1983). 27. S. R. Dubois and F. H. Glanz, An autoregressive model approach to two-dimensional shape classification, IEEE Trans. Pattern Anal. Mach. lnteU. 8, 55-66 (1986). 28. L. Gupta and M. D. Srinath, Non-linear alignment of contours for shape classification, Twenty Third Allerton Conf. Communications, Control and Computing (1985). 29. L. Gupta and M. D. Srinath, Contour classification using invariant moments, Fifth Annual Phoenix Conf. on Computers, Communications and Control (I 986). 30. A. Rosenfeld and A. V. Kak, Digital Imaoe Processing, Vol. 2. Academic Press, New York (1982).
About the Author-- L^LIT GUPTAreceived the B.E. (Hons) degree in Electrical Engineering from the Birla Institute of Technology and Science, Pilani, India (1976), the M.S. degree in digital systems from Brunel University, Middlesex, England (1981) and the Ph.D. degree in Electrical Engineering from Southern Methodist University, Dallas, Texas (1986)• He is currently an Assistant Professor with the Department of Electrical Engineering, Southern Illinois University at Carbondale. His research interests include computer vision, pattern recognition and signal processing. About the Author--M. D. SRXNATHreceived the B.Sc. degree from the University of Mysore, India, in 1954, the diploma in Electrical Technology from the Indian Institute of Science, Bangalore, in 1957, and the M.S. and Ph.D. degrees in Electrical Engineering from the University of Illinois, Urbana, in 1959 and 1962, respectively. He has been with the Department of Electrical Engineering at both the University of Kansas, Lawrence, and the Indian Institute of Science, Bangaiore, India. Since 1967 he has been with the Southern Methodist University, Dallas, Texas, where he is currently a Professor of Electrical Engineering. He served as Interim Chairman of the Department of Electrical Engineering from September 1980 to August 1982. His research interests include estimation theory, digital signal processing, and system identification. He has published numerous papers and is principal author of the text, Introduction to Statistical Signal Processing with Applications (Wiley-1nterscience).