Symmetry operators in computer vision

Symmetry operators in computer vision

Vol. 40, No. 4, pp. 461468, 1996 Copyright @ 1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0083-6656/96 $32.00 + 0.00 Vista...

546KB Sizes 14 Downloads 187 Views

Vol. 40, No. 4, pp. 461468, 1996 Copyright @ 1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0083-6656/96 $32.00 + 0.00

Vistas in Astronomy

PII: SOOS3-6656(96)00030-X

SYMMETRYOPERATORSINCOMPUTER VISION VITO DI GES6 Dipartimento

* and CESARE

VALENTI

+

di Matematica e Applicazioni, Universita di Palermo, via Archirafi 34, I-90 123 Palermo, Italy

Symmetry plays a remarkable role in perception problems. For example, peaks of brain activity are measured in correspondence with visual patterns showing symmetry. Relevance of symmetry in vision was already noted by Killer in 1929. Here, properties of a symmetry operator are reported and a new algorithm to measure local symmetries is proposed. Its performance is tested on segmentation of complex visual patterns and the classification of sparse images. Copyright @ 1996 Elsevier Science Ltd.

Abstract-

1. DEFINITION OF SYMMETRY An object is said to exhibit symmetry if the application of certain isometries, called symmetry operators, leaves it unchanged while parts are permuted. For example the letter A, remains unchanged under reflection while the circle has a circular symmetry around its centre. Moreover, an object in a 20 space exhibits a symmetry with respect to an axis, if it divides the object in two mirror-like components. The definition of symmetry can be extended to objects in a 30 space, by including planes and axis of symmetry. The search for symmetry is more difficult whenever real 20 and 30 noisy images are considered. In these conditions, due to surface roughness or bad illumination, global symmetry may be destroyed if robust algorithms are not used. In Marola (1989) this problem is addressed and fast solutions to retrieve axis of symmetry of a planar image are proposed. Shape symmetry is also useful to develop intermediate and high level vision algorithms; for example a weighted version of the Symmetric Axis Transform (SAT) algorithm has been developed in Blum (1978) to shape description. In Reisfeld et al. (1995) a symmetry measure is introduced to detect points of interest in a scene. An application of a circular symmetry operator to locate eyes in human faces is shown. The algorithm to detect points of symmetry is developed in two phases: compute the gradient of the input image; for each point of the gradient image compute the symmetry *E-mail: t E-mail:

a* 40:4-e

[email protected]. [email protected]. 461

462

V Di Gesli and C. Valenti

value. Here a new measure of symmetry, based on the definition of axial moments, is introduced. The algorithm is implemented as a product of local filtering operators. In Section 2 formal definitions of digital objects and symmetry are given. Section 2.1 describes a new Discrete Symmetry Trasform (DST). Its performance in two computer vision applications is given in Section 3. Section 4 is dedicated to final remarks and future work.

2. MATHEMATICAL

BACKGROUND

In the following we consider digital objects defined in 20 and 3D digital spaces, elements of which are named pixel and voxel, respectively. A digital image is a mapping: I : nD - F, withn = 2, 3. The set F is said feature space; in the simplest case it represents the luminosity of an element. In general, an ordered s-tuple of features can be assigned to each pixel (voxel): (vi, vz, .... v.,). Therefore elements of F are features vector, f. In the following, each point of the nD space (il, i2, . . . . i,,) will be denoted by p. The notion of symmetry can be now stated as follows: Definition

1: Given an image I(i), and a bijective geometric transformation with respect to T @l(p) = Z(T(p)).

T defined on nD,

(i.e. T2(i) = i), then I is symmetric

T may represent the transformation that returns the symmetric of a pixel i with respect to a given axis or an analytical curve. The study of symmetry becomes more complex whenever graylevels objects, perturbed by noise, are considered. In this case digital geometry must be combined with probabilistic or fuzzy methods in order to consider the uncertainty inherent to the data.

2.1. Measure of symmetry The definition depends on the problem to be addressed. For example the evaluation of the spatial orientation of an object, is of relevance in the analysis of movement; while the use of the Symmetry Transform (ST) for shape recognition is useful in segmentation and texture analysis. 2.1.1. The moment measure The symmetry measure, here peoposed, holds for graylevels images, and it is based on the classical definition of symmetry of a continuous rigid object 0 around its barycentre b. In the case of an object defined in a 20 continuous Euclidean space Q the moment of order n is: HO(b),

0, n) =

O( p)b”(p, r(b, 8)) dp

where 8 is the direction of the symmetry axis, r, through the barycenter, and 6 is the distance between the point p and the axis r. The previous definition is kernel size dependent and this may create problems in classification tasks. Normalized Moments (NM) may be defined as follows to avoid such a dependence:

Symmetry Operators in Computer Vision

463

Fig. 1. Kernels used for the computation of the DST.

NM( O(b), 19,n) = ‘(o(b)’

@ n,

Clmax

where prnax is the maximum value of p(O(b), 0, n) for 0 I 0 < r-r. It is easy to see that 0 I NM I 1. In the following, NM(O(b), 8,2) is also named symmetry transform (ST) of an object 0. In case of digital images I, the ST can be implemented in a discrete way, and is named Discrete Symmetry Transform (DST). In the following the images are considered 20 arrays of size K. The graylevel in the pixel (i, j) is denoted by gi,j E {0.. G - 1). The algorithm to compute the DST, in case of circular symmetry, is implemented in two phases: l

Selection of not uniform zones. In this phase the following filter is computed, for a given radius r and n axial moments with slope c with k = 0, 1, . ... n - 1: E(i,j)

=

c

I&,?l - g/J,4I

u,~nEC,,(fAq)EC,+I

where C, and Cr+r are digital circle centered in (i, j), moreover the pixels must be 4-connected ((I - p)2 + (m - q)2 = 1). l

Computation of the DST During this phase the DST of the image I is computed where E(i,j) > 0: DST(i, j) = E(i, j) X T(i, j) with T(i,j)

= 1-

Ck(fi(i, j))2 _ n ( J

Ck(Tk(t j)) 2 n

>

where Tk(i,j)=

C l(i-Z) U,WEC,

sin(?)

- (j - m) cos(F)

X gl,,JL,

where Tmaxis the normalizing factor, computed as in the previous section.

464

V Di Gesti and C. Valenti

Fig. 2. (a) Line illusion;

(b) DST of the image; (c) regions

separation

In Fig. 1 the meaning of the kernels, used to compute the DST, is shown. Note that the kernel used to compute the filter E is a circular ring. It is easy to show that the DST is invariant for image size and rotation, and normalized in the interval [0, 11. The choice of the kernel radius depends on the area of the objects in I. This information can be retrieved from a preliminary segmentation procedure, which selects relevant components of I, or chosen on the basis of a priori information on the data. The selection of non uniform zones is performed by the filter E. The main difference between the gradient operator is that in this case the computation of the DST is performed on each pixel of the input image by using uniformly defined kernels on I. This feature allows an easy parallelization of the DST algorithm. Note that, for a given a kernel size, the computation returns not zero values only on not uniform zones, this rule corresponds to the search of maximal circle in the input image. The DST can be applyied to search relevant symmetry zones by using the following rule:

TRESH(DST(‘), ~) =

1 if DST(1) > + 0

othenvise

with 4 > 0. For 4 = p + no the previous function is named the gold rule, where I.Iand u are the mean value and the standard deviation of DSTU).

3. APPLICATIONS

TO COMPUTER

VISION

Symmetry operators have been included in vision systems to perform different visual tasks. In Kelly and Levine (1994) a set of annular operators is used to identify enclosed symmetry points, and then a grouping algorithm is applied to represent and to describe object-parts. The directional proprierties of the symmetry axis has been applied to image segmentation (Gauch and Pizer, 1993). 3.1. Symmetry

and image segmentation

Segmentation is one of the main step of a visual system. Grouping techniques are often used to perform a partition of pixels in homogeneous classes. The performance of the segmentation depends on the proper choice of the feature space. It is well known that

Symmetry Operators in Computer Vision

Fig. 3. The comet SL9.

histogram information is not useful if the image has regular textures, which interferes at a perceptual level with graylevels discrimination. Two principles ofproximity and continuity have been used by psycologists of the Gelstat to explain such perceptual behaviour. Both of them are used to explain how data are grouped. These principles have been succesfully used to design clustering algorithms to analyse dotted images Zahn (1971). In Fig. 2(a) two sets of straight lines at different densities require a line to separate the image in two different regions, in Fig. 2(b) the DST’s of the image in Fig. 2(a) is shown. Figure 2(c) reports the result of the segmentation. The agreement with the expected segmentation is quite good. The approach here followed was to perform the segmentation on the trasformed image, by using a relaxation labeling technique. Segmentation phase of very noisy images, containing objects embedded in a structured background is hard. In Fig. 3 an example of such kind of images is given. It represents the comet Shoemaker-Levy 9 as detected by the Hubble Space Telescope. In Fig. 4 the segmentation of the image SL9 is obtained by convolving it with TRESH(DST(SL9), Jo + a). The thresholding parameter is obtained from the graylevels histogram. A quick look at this result indicates that the transform has enhanced the useful content of the input image. In fact, relevant features reside in the analysis of local symmetries found with a 5 x 5 kernel. This size has been determined empirically in order to detect smallest objects in the plate. 3.2. Symmetry and classiJication Symmetry has been used to classifyfuzzy objects (Di Gesu and Maccarone, 1984) characterized by a spread set of points. Examples of such kind of objects are stars and galaxies. Last ones are usually classified in spiral, circular, and elliptique (see Fig. 5). This classifi-

466

V Di Gesti and C. Valenti

Fig. 4. Segmentation of the SL9 plate

Fig. 5. Examples of fuzzy images: (a) circle; (b) ellipse; (c) spiral.

cation is rough and it has been further refined by astronomers. Automated classification of galaxies plays a central role in the analysis of astronomical plates (Thonnat, 1988; Ossorio and Kurtz, 1988). A fuzzy object can be represented by a tuple (~0, ~1, .... Us) elements of which are Normalized Axial Moments (NAM) computed by the DST. This representation is very sensitive to the spatial distribution of the points around the object barycenter, and it has been used to design a supervised linear classifier to discriminate the following classes: circles, ellipsis, and spirals. For each class k (in our example k = 1, 2, 3) a training set is used to evaluate the mean value of the NAM corresponding to the class k. At the end of the training phase a decision rule is implemented by using a distance function, p, between an unknown fuzzy object X and the prototype Pk = (p’f, ~5, . ., pk), both of them represented in the NAM space: x E

pk

-

p(x,

pk)

=

,$$p(x,

&)I

Symmetry Operators in Computer Vision

467

0 CIRCLE *

ELLIPSIS

0 SPIRAL

Q4

Q

04

0

10

l

4

20

30

k Fig. 6. The NAM of fuzzy images in Fig. 4 Table 1. Circle Ellipsis Spiral

Circle 0.85 0.1 0.05

Ellipsis 0.1 0.75 0.15

Spiral 0.05 0.15 0.75

In the experiments the distance function was the Normalized Fuzzy Entropy: NFE(X,&)

= -l/r~l&,ilog5k,i+

(1 -&,i)log(l

-i&i)1

i=l

where ?& = jpf - /.$ I. It is easy to see that 0 5 NFE I 1, in fact from the definition of NAM follows that 0 I &i I 1. In this case 1 - NFE(X, Pk) can be interpreted as the belonging degree of X to the fuzzy set Pk. In Fig. 6 the NAM’s of the fuzzy objects in Fig. 5 are given, for m = 30. Experiments have been performed on 300 fuzzy objects equally distributed in the classes above considered. Ellipses have been generated with y = 0.3. In Table 1 the confusion matrix is reported. The results indicate the applicability of the method proposed to classification problems.

4. CONCLUSION The paper reviews some current techniques and describes a new algorithm to measure local symmetries in graylevels images. The introduced DST shows interesting properties of size and rotation invariance; moreover it can be expressed as a local operator, given by the product of two filters. This allows us to implement fast serial and parallel algorithms. The applicability of our algorithm to several classes of vision problem is also shown. The results

468

I! Di Gesti and C. Valenti

are encouraging, and have been compared with existing ones. Further work will be done in order to extend the DST to 30 objects analysis, and to integrate our method in a vision system to classify fuzzy images like astronomical ones.

5. DISCUSSION Reza Ansari: Is it necessary to compute the DST for each direction? It seems to me that, for example, in 30 space 3 eigenvectors are enough. Vito Di Gesri: Your remark is correct, however, in our case the computation of each axial moment is less time expensive. Albert Bijaoui: Is there an advantage of using symmetry operators in a multi-resolution

approach? Vito Di Gesti: Yes, we have performed some experiments on they indicate that hierarchies of symmetries can be discovered. speed up the algorithm by using lower resolution for both the by performing higher resolution computation only on selected

multi-resolution pyramids; Moreover, it is possible to image and the kernels, and zones.

References Blum H. and Nagel R. N. (1978) Shape description using weighted symmetric axis features. Pattern Recognition 10, 167-180. Di Gesh V. and Maccarone M. C. (1984) A method to classify spread shapes based on the possibility theory. Proc. 7th Int. Conj Pattern Recognition, IEEE Comp. Sot., Montreal, Canada. Gauch J. M. and Pizer S. M. (1993) The intensity axis of symmetry application to image segmentation. IEEE Trans. PAMI 15, 753-770. Kelly M. F. and Levine M. D. (1994) From symmetry to representation. Technical Report, TR-CIM-94-12, Center for Intelligent Machines. McGill Univ., Montreal, Canada. Marola G. (1989) On the detection of the axes of symmetry of symmetric and almost symmetric planar images. IEEE Trans. PAMZ 11, 104-108. Ossorio P G. and Kurtz M. J. (1988) Classification of resolved galaxies. In Data Analysis in Astronomy (Edited by Di Gesh V, Scarsi L. et al.), Vol. III, pp. 121-128. Plenum Press, Ettore Majorana International Science Series. Reisfeld D., Wolfson H. and Yeshurun Y (1995) Context free attentional operators: the generalized symmetry transform. Znt. J Computer Vision 14, 119-l 30. Thonnat M. (1988) Toward an automated classification of galaxies. In Le monde des galaxies, PHYSICS. Springer, New York. Zahn C. T. (197 1) Graph-theoretical methods for detecting and describing Gestalt cluster. IEEE Trans. Comp. C-20,68-86.