Distorted shape recognition using attributed grammars and error-correcting techniques

COMPUTER

GRAPHICS

Distorted

AND

IMAGE

(1980)

Shape Recognition Using Attributed and Error-Correcting Techniques I<.

School

13, 1-16

PROCESSING

of Electrical

c.

Engineering,

Received

l-out

Purdue

November

AND

K. 5. Fu

University,

30, 1978;

Grammars

accepted

West

Lafayette,

June

Indiana

47907

14, 1979

Using attributed grammars, the syntactic method can be applied to describe and recognize picture shapes. The parsing of attributed grammars involves both symbolic and numerical manipulations. A distinguished capability of the syntactic method in pattern recognition is the recognition of partially distorted patterns using error-correcting techniques. In this paper, the error-correcting technique is combined with the primitive-extraction-embedding parser to recognize partially distorted shapes. A modified Earley’s parsing algorithm is developed, and the recognition of several distorted airplane shapes is demonstrated.

1. INTRODUCTION

To recognize an object in a two-dimensional image has been an interesting research topic for years. In the conventional approach of statistical pattern recognition, the recognition performance can hardly be reliable without successful selection of features. A feature set that is sensitive to the differences between pattern classes and yet insensitive to the noise and distortion of the pattern is difficult to select and varies from one application to another. The shape of an object, represented by its outer contour, usually contains important information for classification. Therefore, many authors have tried to select features which can extract information about shape for successful classification. Fourier descriptors, moment invariants, and other geometrical or topological features have been studied for shape description. Person and Fu [l] have used Fourier descriptors to classify numerals and machine parts. Dudani et al. [2] have used moment invariants as features for airplane identification. It has been shown that the geometrical feature (Perimeter)2/Area is effective in classifying biological images [S]. These features describe the global structure of shape in terms of numerical measurements, since the shape structure is usually a deciding factor in recognition. But if the features can only describe the global structure, they are often not very effective for recognition of partially distorted shapes. * This work was supported t K. C. You is presently

by NSF with Life

Grant ENG 78-16970 and DARPA Grant Systems, Inc., Cleveland, Ohio 44122.

MDA

903-77-G-1.

0146-664X/80/050001-16$02.00/0 Copyright @ 1980 by Academic Press. Inc. All rights of reproduction in any form reserved.

2

YOU AND

FU

Recently, many studies of syntactic pattern recognition have been reported. In the syntactic approach [4,5], a pattern is decomposed into simpler subpatterns. Production rules are used to describe the relationships among the subpatterns. Recent results about error-correcting (EC) techniques [G-S] have rendered this approach capable of recognizing distorted shapes. iMany existing syntactic shape recognition methods [4, 51 were developed for particular classes of shapes. You and Fu [l] have proposed a syntactic method for general shape description and recognition using attributed grammars. The technique of primitive-extraction-embedding (PEE) [9] was developed to combine primitive extraction and structural parsing into one step so that primitive extraction errors can be reduced by utilizing the contextual information. In this paper, a parsing algorithm using EC and PEE techniques is developed for the recognition of partially distorted shapes. Experimental results on aircraft shape recognition using the proposed algorithm are also presented. 2. ATTRIBUTED

SHAPE

GRAMMARS

AND

PEE PARSING

To obtain a shape grammar, a shape is first decomposed into subshapes. +‘or each decomposition, a production rule is constructed to describe the concatenating relationship of the subpatterns. The subshapes can further be decomposed until they are all very simple curve segments. Each curve segment is assigned a curve primitive and described by a set, of attributes. The four attributes of 3 curve segment are the vector C, its total curve length L, its total angle A, and a symmetry measure X. These four attributes constitute a C-descriptor for a curve segment p. A and S can be obtained from the curvature function f(b) of p. They are written as follows : D(P) = cc, L, 4 81,

FIG. 1. A simple airplane shape: N,, nose; N,, left wing; Na, right wing; N, fuselage aud tail; Ng, tail.

DISTORTED

where C = zz

SHAPE

RECOGNITION

(x1 and x2 are the two ends of

3

p),

L

L =

dl, / 0 L

A= and

f (04

s0 L

t

f(Z)dZ - -;dt.

S=

s 0 (J

0

The concatenation relationship of two connected curve segments is represented by an angle primitive a and described by its A-descriptor which has one attribute, that is, the concatenating angle. The C-descriptor can describe a nonterminal curve segment or a curve primitive. If a curve segment N is broken into segments N1 and Nz with connection angle primitive a, the production rule LV -+ NlaN2 is established. The descriptor of IV can be obtained from those of N1, Np, and a, and the relationship D(N) = D(N,) @a D(Nz). The additivity 0. is associative. The curvature function f(Z) in the discrete case is a summation of pulse functions. In fact, the C-descriptors in the discrete case can be computed by additions and multiplications instead of integrations [7]. A C-descriptor can be transformed into a transformed descriptor which has four scalar attributes and is invariant with respect to translation, scaling, and rotation of the object under noise-free conditions [9]. The recognition of curve srgment(s is based on the comparison of transformed descriptors. A general attributed shape grammar can be illustrated by the following example. EXAMPLE 1. A simple airplane shape (see Fig. 1) can be decomposed into four parts: nose, left wing, right wing, and fuselage and tail, denoted X1, X?, Na, and Nq, respectively. N, is further decomposed into three parts: left side, tail, and right side, denoted pg, Ns, and plo, respectively. The nonterminals -VI, N,, iVs, and N, can be further decomposed into primitives. For each decomposition, there is a symbolic production rule and a set of attribute rules. The angle primitives, a,‘s, describe the connection angles. cylabels the pattern. The shape grammar is as follows : G = (V, T, P, A%), v = f&, N/cl1 5 k 5 51, T = (ai’s, pj’s j 1 5 i 5 7, 1 _< j 5 12}, P: S, -+ N1a2NzasN~aaN~a, {Answer,),

c + a,

N I+

p2alpb

D(NJ

+-

N 2--+

p3a3p5a4p7,

DW2)

+D(p3)

@

D(p5)

@

D(P~),

N 3+

p3a4p6a3p4,

D(N3)

+D(ps)

ii

D(ps)

iii

DC&,

Db2)

2 D(PI),

N4 --, pga&5amo,

D(N4) +- D(pg) ii D(Ni%

N 5 -+

DWJ

pmp~3a7p12,

+

Wpi&

D(p13)';

D(plo), D(p12).

YOU

4

AND

FU

A general form of the attributed shape grammar is GL = (V,, TI, PL, Slj, where S1 is the starting symbol with a special attribute I, which is the label of i&e pattern. VL = {S,, N’s), TZ = (F’s, A’s 1F : curve primitive, PI : Sl h N+

(XA)*XA

{Answer,)

(XA)*z;

D(N)

A : angle primitive), ;

c +- 1,

+ (D(X)

9 )*0(x),

where X E {N’s, F’s). For each S-production rule, (Answer,] and c t- 1 mean that if the parsing is successfui, the shape pattern is recognized as being in the class labeled by c. The idea of (Answer,}, an action symbol, is borrowed from translational grammar [3, 41. The attribute rules for each N-production rule can be obtained easily from the symbolic rule. The PEE parser combines the primitive extraction and the conventional parsing. It is known that primitives are sometimes difficult, to extract, without contextual information which is explicitly described by production rules and used in parsing. The PEE parser can reduce a great many primitive extra&on cmors. It accepts a vector string which describes the shape instead of a primitive string. The PEE parser extracts the primit,ives based on t,he transformed descriptors and performs the parsing with the extracted primitives. If t#herc are many candidates for a primitive, they are all accepted and will be selected later during parsing. For a better understanding, the PEE parsing can be formulated as follows. Vi,i denotes vector subchain vi. . ‘vi. R(X, V) = I implies that vector subchain V is recognized as X. (a) An S-Production

Rule

SI *

(Y = X1AzXa..

.XZn+1Azn+2(Answerc.,),

Xk E {F’s, N’s} is selected, and c t if and only if the following

conditions

(1) For

any Ak in LY, R(Ak, Vik,jk) = 1. (2) For any Xk in LY, R(Xkj Vik,jk) = 1. (3) il = 1, jk = i(lc + 1) unknown shape has m vectors (b) An N-Production

1

are satisfied :

thcrc

is a corresponding

subchain

Vlk,ll; and

there

is a corresponding

subchain

V!,,,,

for 1 5 k 5 2n + 1 and j(2n in the boundary chain.

+ 2)

=

m

+

and I.

Thr

Rule

to a vector ND(N) + P = X,AzXa. . .Xzn+~, Xk E (F’s, :V’s) corresponding subchain Vi,j is selected and D(N) + D(X,) @D(,Q) D(X,). . . @D(A~,,) ~(XZ.,.& if and only if the following conditions are satisfied : (1) For any Ak R(Ak, Vik.jk) = 1.

in

&

there

is a corresponding

subchain

Vik,jk and

DISTORTED

SHAPE

5

RECOGNITION

(2) For any Xk in 0, there is a corresponding subchain R(Xkj Vik,jk) = 1. (3) il = i, .jk = i(k + 1) for 1 5 k 5 2n and j(2n + 1) = j. For nonterminal (4) R(N, V,,j) =

recognition,

we could add one more condition,

Vik,jk and

that is,

1.

In fact, condition (4) is implicitly included in (2). In [9], two PEE parsing algorithms are described. They are the PEE Earley’s algorithm for context-free shape grammars (CFSG) and the PEE finite automaton for finite-state shape grammars (FSSG). These two algorithms demonstrate the feasibility of PEE parsing. In this paper,.only the modified Earley’s parser will be referred to and further extended to include an error-correcting technique for the recognition of distorted shapes. 3. AN

ERROR-CORRECTING

EARLEY’S

PARSING

ALGORITHM

In 1972, Aho and Peterson [S] added an error-correcting capability to Earley’s parsing algorithm by adding error productions to the production rule set, and modifying the algorithm to track the error productions used. Fu and Lu [S] further introduced the idea of weighted error for each error production. If the error weights of all the error production rules are equal to 1, the total error weight will be the number of error productions used. If some particular errors are impossible or not allowed, the corresponding error productions will not be added to the production rule set. Therefore, the advantage of using error productions is that it controls the allowable errors, but the drawbacks are the large storage and processing time required, especially when the number of primitives is large. If deletion, substitution, or insertion errors are possible or allowed to occur for all primitives at any place in the string, we do not need to add the error productions to the production rule set in the actual implementation. But we do need to modify Earley’s algorithm to take care of all three types of errors. In this section, we show the feasibility of this idea for processing symbolic patterns. No PEE mechanism is involved. In the following algorithms, we assume that error weights for primitives are independent and additive. @‘( ‘) denotes the error-weighting functions with subscripts D, I, and S indicating deletion, insertion, and substitution errors. X denotes the primitive. For example, w,(X, bj) means the error weight of bj substituting X. Step (2.~) in algorithm 1 takes care of the deletion errors, while step (3.b) takes care of the insertion and substitution errors. The corresponding flowchart is in Fig. 2. An item [A -+ cy./3, i, w] is in 1j for input string bl. . . b,, 0 5 i = j 1 n, if and only if * (1) a;;;: uL+l-- . ak. (2) The difference between al+l. . .ak and bi+l- . .bj is ( adl, . . . , adD, bil, . . . , bir, (~1, bad, . . ., (a,~, bed 1, where ~1. . .adD are the primitives deleted, bil- . .bir are the primitives inserted, and (a,l, b,l), . . . , (ads, b,s) indicates the substitution errors

YOU

AND

FU

Q START

(a)

If then

(b)

If

add

is

If

for

all

add

rB

is

El

Ij

to

and

y,4,w23

is ,

I.

in jtl

P

in

Ii

to

Ij

to

Ii

I

< n, -

and

X # b

add

CA + aX*E,i,o+$(X)l

j+l

in

Ij

EA * y,k,w,+w21

*

B * T is

I., I

in

+ 6-A

CA + a'Xfi,i,wl

then

in

CB + * y,j,O7

TA + a ',i,w,l

then

Cc)

i,wl

CA * o*RB,

Find in

rA

+ a*XE,i,wl

all

(a)

If

X = b i then

add

CA *

aX'E,i,w'l

(b)

If

X # bj

add

TA *

a'X6,i,w+Sji(bj)l

then and

in

I

For

In

CS *

D *,O,w3

uith

minimum

w

j-l to

CA + oX*6,i,w+cS(X,bj)l

I.

I

to

1. I

I

FIG.

Algorithm

2. The

Flowchart

of the modified

1: ModiJied Error-Correcting

error-correcting

l&&y’s

algorithm.

Earley’s Parser

Input: An ordinary context-free grammar G = (V, T, P, X), error-weighting functions, r( .)‘s, and an unknown input string, bt. . ‘b,. Output : Error-weight, w.

DISTORTED

SHAPE

7

RECOGNITION

Method : (1) Add[S-+~a,O,O]toloforallS-tainP j = 0 (2) (a) If[A-+a.Bfi,i,w]isinIjand@-+risinP then add [B -+ ‘y, j, 0] to Ij (b) If[A-+a.,i,&JisinIj, then for all [B -+ P.Ay, k, u2] in Ii add [B-+pA.r, k, ~1 +wz] to Ij (c) If [A + a.XP, i, w] is in 1j and X # bj+l, j + 1 5 n, then add [A -+ aX*p, i, w + lT’~(x)]

to

Ij

(3)j=j+l,ifj>ngoto(5) For all [A * a -X0, i, w] in 1j-1 (a) If X = bj then add [A -+ CYX.fl, i, w] to Ij (b) If X # bj then add [A -+ cr.Xfl, i, w + RI(bj)] and [A -+ CZX.~, i, w + Ws(X, bj)]

to

Ij

(4) Go to (2) (5) Find [S-+cy.,

0, w] in I, with minimum

W.

In the above algorithm, all the possible errors are considered. If we know beforehand that a shape of total error weight higher than a threshold will not be accepted, we can threshold the error when an item is added to the item lists to eliminate a large number of possibilities. For instance, if we do not accept a shape of error higher than w. = 0.4, then we only need to consider items of the form [A -+ (Y./3, i, w], in which w 5 w0. In such a case, step (5) may end up with no [S -+ ff., 0, W] in I,, for w 5 wo. That means the input pattern is rejected by the grammar because no derivation with error smaller than w. can be found. We may use the proportional length of a primitive to the whole shape as the error weight for each curve primitive in our shape recognition process. In this case, an item [S --+ ff., 0, w] in I, implies that w X 100% of the shape is badly distorted. The selection of the threshold w. may affect the computational time and recognition result. The smaller w. is, the shorter the computation time. But a shape with distortion greater than w. X 100% of what the shape is supposed to be will be rejected. 4. ECPEE

PARSING

AND

ECPEE

EARLEY’S

ALGORITHM

An ECPEE parser using both an error-correcting (EC) technique and primitiveextraction-embedding (PEE) is proposed. As mentioned before, if the recognition function R(X, V) = 1, the vector subchain V is recognized as X, if R(X, V) = 0, V is rejected as X. In the latter case, there could be a substitution error or a deletion error. The insertion errors may not occur if the boundary vector chain is correctly segmented by the PEE mechanism. EXAMPLE 2. Figure 3 shows the top view of a simple airplane distorted by a

8

YOU AND

FU

FIG. 3. Top view of a simple airplane with a distorted

right wing.

cloud on the right wing. The true shape can be found in Fig. 1. The cloud on the right side creates an extra area attached to the airplane. The boundary of this extra area is very noisy. Although some filtering techniques may be used to smooth out the zigzags of the boundary, the shape of this extra area is unpredictable because it changes with the cloud and the shape of a cloud is unpredictable. In such a case, the primitive extraction on the boundary of this extra area becomes a difficult problem. One way of solving this difficulty is to use an extra error primitive to represent the noisy curve segments which cannot be recognized as any legal primitive. Then, we add the error production rules for this error primitive in all possible cases. But the length of this extra error primitive is still undefined, since the boundary length of the extra area of the cloud is unpredictable. A primitive without any specification on its attributes will cause confusion in extractions. For instance, the noisy boundary of that extra area may be considered as one or several consecutive noisy curve segments or error primitives. If the distorted boundary (X,X,) is extracted as one error primitive, the erroneous symbolic pattern can be accepted by an EC parser which allows only substitution errors. But there is no existing method which can systematically achieve such an extraction without using contextual information. If the distorted boundary is extracted as more than one error primitive, the erroneous symbolic pattern has to be accepted by an EC parser which allows substitution and insertion errors. If X1X2 is very long, as shown in Fig. 3, due to distortion, it may be extracted to be k error primitives where k may be 3 or more. The number of errors counted by a EC parser may change with k. But the length of X1X2 or the noisy condition of X~XZ should not change the error count, which may affect the classification. Since a human can recognize Fig. 3 by the unaltered portion of the shape, the similarity between Figs. 1 and 3 should closely approximate the percentage of the unaltered portion of the shape. The PEE parser can segment the shape in Fig. 3 at the same positions as the true shape in Fig. 1, but the distorted portion, X1X2, will be rejected and can be taken care of by an ECPEE parser as a substitution error with a proper error weight. The total weight of the whole shape will remain

DISTORTED

Add

rS

1,01

* *a,

SHAPE

to

I,

for

i,w7

is

in

9

RECOGNITION

all

S + (1 in

P

j=l

(a)

If

ITN + a-BE,

then (b)

If then

Cc)

If

CR + ‘y,

add CN

*

j O? to

a*,

i,w,3

for

all

Cl3 + B*Ny,k,

add

CB + BN r*y,k,w,+w23

r

is

CN * a’XB,i,wl

then

I.

add

I.

in

B * y in

1

w 23 in

I

to

Ij

i

X E CF's,A's>

i,w+vDtX)l

CN + axo'6,

P

Ij

Ij,

in

and

1

to

I.

A

1

i =

j

Find

j+l

CS

with

For

all

(a)

If then

CN + a*XB,i,

6 = A, add

P = R(X,Bk

(b)

If

6 = A,

Cc)

If

6 = A, w > 0,

and then (d)

If

r2

(e)

If

= R(NaX,V

add

add

I

j'

2

r.

1 <

x

P 0

r = R(X,Vk

I.

I

= R(NaX,Vi,j)

2

r.

r.

ipl ,

to

j'

z

,

to j'

The Flowchart

Ij

r. Ij

< r.

TN + aXo'!3,i,u+~S(X,Vk

4.

r2

I

j)‘J

l,w3

s and

XE CF'S,A*S>

to

and

l-1 = R(X,Vk,j)

i,j

j,

,

P = R(X,Vk

and

k 5

w+riS(X , V k j’l

CN + aXpm8,i,w3

add

FIG.

15

high

< r.

CNp2 + aXr,*,

6 = A, then

j)

i,

= R(X,Vk

6 = X and then

I

TN + aXn', rl

Ik,

WY in

e a*,

in

S

e

to

of the ECPEE

Ij

Earley’s algorithm.

low

Im+, w.

10

YOU AND

FU

unchanged, no matter how badly XIXZ is distorted. In other words, the ECPEE parser can fully use the contextual information to extract the primitives, so that, a distorted shape like Fig. 3 can be handled by only one substitution error. Besides, the associated attribute information can be used to obtain a proper error weight which is close to human perception. In Algorithm 2, step (2.~) takes care of deletion errors and step (3.a) takes cw,re of substitution errors. Steps (3.b) and (3.d) take care of the successful primitive and nonterminal recognitions. Step (3.~) implies that if the recognition of primitive X is successful, that of nonterminal N is not successful and w = 0, the recognition of X will not be accepted. That is, if all the successors of ,V are successfully recognized, the recognition of N must succeed. The recognition of a primitive or nonterminal is successful if and only if the recognition function R(., V) = 1. An item [N -+ (Y.@,i, w] in 1.i for input vector chain v1I urn, 1 5 i’ < 1 _< m + 1, implies that * (1) iff cr # X, than a;;;: al,, ‘ak A vl. . ‘Vj but { ~1’ . .u~D/ alee dt>leted and { k&l, Vil,jl), * * *, (u,s, Vis,js) ) are substitution + ZL K3(uszr Viz.id (2) if (Y = X, then i = ,j

The flowchart

errors and w = Cz=1 E~I(Q~~~)

of this ECPEE Earley’s algorithm

is shown in Fig;. 1.

Algorithm 2: The ECPEE Eurley’s Algorithm Input : (W( .)‘s), Output: errors Method

A CFSG G, recognition functions (R’s), error-weighting function and an unknown chain of m vectors VI,,. Error-weight w = summation of weights of deletion and substitution :

(1) Add [S -+ . a, 1, 0] to II for all S 3 a in P j=l (2) (a) If[IY--tcu.Bp,i,w]isinIjandB--,rinP then add LB -+ . y, j, 0] to If (b) If [N + (Y., i, WI] is in 1, then for all [B -+ p-N-y, k, wz] in Ii add [B -+ pN.7, k, ~1 + wz] to Ii (c) If [N -+ cr.X& i, w] in Ii, X f IF’s, A’sj then add [N -+ cuX.p, i, w + V’D (X)] to Ij (3) j = j + 1

if j > m + 1 to (5) Forall[N--tar~X~,i,]w]inIk,l~liF:,~,X~(F’s,A’s] (a) If /3 = X, R(X, = 0 then add [N 4 CUX., i, w + W,(X, vk,j)

V/c,j)]

to

Ij

DISTORTED

FIG.

SHAPE

5. A B52

RECOGNITION

11

in a cloudy sky.

(b) If p = y, R(X, Vk,j) = 1 and R(N, Vk,i) = 1 then add [N -+ ax., i, w] to Ij (C) Ifp=X,w>O,R(X,Vk,j)=l and R(N, Vi,j) = 0 then add [N -+ CYX., i, w] to Ij (d) If fl = X, and R(X, Vk,j) = 1 then add [N ---$ax./?, i, co] to Ii (e) If p = A, and R(X, Vk,j) = 0 then add [N -+ (rX-@, i, w + ms(X, V,,j)] to Ij (4) go to (2) (5) Find [S-a!.,

1, w] in Im+1 with low W. 5. EXPERIMENTAL

RESULTS

When a human recognizes a partially distorted shape, he/she relies on the unaltered parts of the shape. The larger the distorted part is, the more difficult the recognition will be. In other words the less confidence the recognition will have. If the recognition of a part of a shape is unsuccessful, it does not matter

FIG.

6.

The shape of the distorted

B52 after preprocessing.

12

YOU

AND

FU

FIG. 7. An F86, T without

how bad the distortion follows : WD(X)

is. Therefore, = W,(X,

nose.

we defined the error-weighting

V,,j) = ,“- (X) II

funct.ion as

for X E ilr”s),

(L/Lo)(X) is a variable in the transformed descriptor of curve primitive X. (L/L,)(X) represents the bad boundary length in proportion to Lo, the total length of a good shape. Of course, L and Lo refer to the primitives defined in t$rt> grammar, so that they were known. But the distortion of the unknown input pattern may change the total boundary length considerably and unpredictably. In our experiment, we assume that the total boundary length can be obtained by other means. This assumption is reasonable because the distance from the object to the camera can usually be estimated accurately. For instance, tht: distance between a flying airplane and an observer can be estimated by radar. Distance under water can be estimated by sonar. Once the distance of a given

FIG. 8. The shape obtained

after preprocessing

FIG. 7.

DISTORTED

SHAPE

FIG. 9. An F86,

T with

13

RECOGNITION

a broken

right

wing.

object is known, the total boundary length of a shape can be calculated using a proportionality relation. Therefore, we assume that the total boundary length is known and use it as a size normalization factor. Since ideally the angle primitive has no length, we defined VD(A) = W,(A, V,,j) = 0. But a curve primitive is recognized only if one of its two adjacent angle primitives shown in the production rule is recognized. Hence the angle primitive recognition itself has no error weight, but it is used as a condition for adjacent curve primitives in our implementation. With these error-weighting functions, the final error weight w of [S --f (1~., 1, w] in Im+l implies that w X 100% of the shape is badly distorted. EXPERIMENT 1. Figure 5 is a picture of a B52 in a cloudy sky. It is obtained by adding a picture of clouds to a picture of an arbitrary-angle view of the B52. Beca,use the cloud on the right side is smoothly connected to and is as dark as the airplane, there is no way to delineate the boundary between the cloud and the airplane without knowing the model and viewing angle of the airplane first. Through the procedures of threshold finding, boundary following, and smoothing, we obtained the shape shown in Fig. 6. The vector chain in Fig. 6 has 120 vectors. The vector chain was fed into the ECPEE Earley’s parser with maximum error weight ~0 = 0.10 and the shape grammar Gz52. It was recognized as B52 with error weight 0.09 in 140 see (CDC 6500).

FIG. 10. The shape

obtained

after

preprocessing

Fig.

9.

14

YOU AND FU TABLE I Experimental Results of Processing Distorted F86,T’s with ECPEE Earley’s Parser against GIW,T and GMIG.U,U with Different Error Threshold wO’s Error weight and processing time Grammar

Shape

GF86.T WI3= 0.15 GIW.T

00 = 0.20 GFB.S,T

wn = 0.25 GMIG-1a.u wo = 0.15 GMIG-15.11

wo = 0.20 GMIG-15.u wo = 0.25

Figure 7 : F86,T with a broken nose

(65 vectors)

Figure 9 : F86,T with a broken wing (72 vectors)

w = 0.10 143 set

0 = 0.14 117 see

w = 0.10

375 see

0 = 0.14 370 set

Exceeds the parsing table limit

Exceeds the parsing table limit

Reject

Reject

52 set

64 set

Reject 126 set

Reject 165 set

Reject

Reject

346 set

503 see

EXPERIMENT 2. This experiment demonstrates the recognition capability for another type of distortion. When some parts of the shape are missing, we may need to allow both deletion and substitution errors. Figures 7 and 9 are two F86’s with a broken nose and a broken right wing, respectively. After the preprocessing, Fig. 8 having 65 vectors and Fig. 10 having 72 vectors are obtained. The starting points of the shapes are the leftmost points in the figures. The vector chains describing Figs. 8 and 10 are fed into the ECPEE Earley’s parser with w,, = 0.15 and shape grammars GrS6 and GMrG--II. The F86 and MIG-15 are very similar. Their top view differs only in the width at the end of the fuselage. Figures 8 and 10 are accepted by GrS6 with error weights 0.10 and 0.14, respectively, and are both rejected by GMIG--~~.The processing times for GFaaaccepting Fig. 8 (having 65 vectors) and Fig. 10 (having 72 vectors) are 143 set and 117 set (CDC 6500), respectively. The times for GhlIo--15 rejecting Fig. 8 and Fig. 10 are 52 set and 64 set, respectively. We have tried the experiment with different error thresholds (wis). The result can be found in Table I. The experimental results show that the selection of w. affects the computation time, but does not affect the recognition results at all. of course, a very small wg may reject some recognizable shapes and a very large wg may take a long computing time and a large memory for the parsing table to recognize or reject a shape. The details of these experiments are described in [12].

DISTORTED

SHAPE

RECOGNITION

15

6. DISCUSSION

As mentioned before, none of the existing methods can recognize shapes which are partially distorted. McKee and Aggarwal [13] developed an automatic system to learn and recognize entire shapes and partial shapes. By partial shapes, we mean that only part of the shape is visible. McKee and Aggarwal utilized a library to store the, edges, approximated and described by straight lines, of each shape and then recognized the unknown shape by matching the edges in the library. This matching method can recognize the partial shape only if the set of edges belonging to the object is known. As a matter of fact, this kind of partial shape can be considered as a special case of a distorted shape. A very straightforward extension of our distorted shape recognition method involves identification of touching or overlapping objects. In biomedical images, touching objects such as blood cells are very difficult to separate. If two touching objects are in different classes, the whole shape should be recognized by the two corresponding grammars with two error weights. If both error weights are lower than a threshold, the touching assumption is correct. The same argument is also true for more than two touching objects, all of which are different. If twotou thing objects are in the same class, then we need to do further processing to make sure there are two objects touching each other. This further processing involves the second part of parsing, parse construction from the parsing table. In constructing a parse from the parsing table, we can find all the possible segmentations of the vector chain. If there are two segmentations *I and \k2, 91 = (811, . . ., ~1~) and \kZ = (sZl, ..., SZ~,), \kjl and \kjo are sets of subchains Sji’s which are recognized and rejected, respectively. So, \Ej = 9j, U \kjl and 9jo r\ *jl = 4, the empty set, forj = 1, 2. If there are no overlapping subchains between *ll and *21, *l and +Z should be two objects of the same class. If the two error weights corresponding to *I and q’z are both lower than the error threshold, the whole vector chain should contain portions of the boundaries where the two objects touch each other. This idea can be extended to more than two touching objects. Another extension of distorted shape recognition is the reconstruction of distorted parts, REFERENCES 1. E. Persoon and K. S. Fu, Sequential Decision Procedures with Prespecified Error Probabilities and Their Applications, Purdue University, TR-EE 74-30, August 1974. 2. S. A. Dudani, K. J. Breeding, and R. B. McGhee, Aircraft identification by moment invariants, IEEE Trans. Computers C-26, No. 1, Jan. 1977,39-45. 3. J. W. Bacus and E. E. Gose, Leukocycte pattern recognition, IEEE Trans. on Systems Man Cyhnetics, No. 6, Sept. 1972. 4. K. S. Fu, Syntactic Methods in Pattern Recognition, Academic Press, New York/London, 1974. 5. K. S. Fu (Ed.), Syntactic Pattern Recognition Applications, Springer-Verlag, Berlin/New York, 1977. 6. A. V. Aho and T. G. Peterson, A minimum distance error-correcting parser for context-free languages, SIAM J. Comput. 1, No. 4, December 1972. 7. L. W. Fung and K. S. Fu, Syntactic Decoding for Computer Communication and Pattern Recognition, Purdue University, TR-EE 7447, December 1974. 8. S. Y. Lu and K. S. Fu, Error-Correcting Parsing for Syntactic Pattern Recognition, Purdue University, TR-EE 77-31, August 1977.

16

YOU AND

FU

9. K. C. You and K. S. Fu, Syntactic shape recognition using attributed grammars, in Proc. of Eighth Annual Conj. on Automatic Imagery Pa&m Recognition, Gaithersburg, Md, April 1978. 10. A. Rosenfeld and A. C. Kak, Digital Picture Processin,g, Academic Press, New York/London, 1976. 11. 0. R. Mitchell, E. J. Delp, and P. L. Chen, Filtering to remove cloud cover in satellite imagery, IEEE Trans. Geosc. Electron. GC-15, No. 3, July 1977, 137-141. 12. K. C. You and K. S. Fu, Syntactic Shape Recognition Using Attributed Grammars, Purdue University, TR-EE 78-38, September 1978. 13. J. W. McKee and J. K. Aggarwal, Computer recognition of partial views of curved objects, IEEE Trans. Computers C-26, No. 8, August 1977,7QO-800.