Locating straight-line edge segments on outdoor scenes

Pattern Recotjnition, Vol. 10, pp. 145 157.

0031 3203/78/0601 0145 $02.00/0

Pergamon Press Ltd. 1978. Printed in Great Britain. © Pattern Recognition Society.

LOCATING STRAIGHT-LINE EDGE SEGMENTS ON O U T D O O R SCENES*+ SAHIBSINGH A. DUDANI~ and ANTHONY L. LUK Hughes Research Laboratories, 3011 Malibu Canyon Road, Malibu, CA 90265, U.S.A. (Received 22 July 1977; in revised form 29 November 1977; received fi)r publication 3 January 1978) Abstract - Straight-line edge segments are often used as key features in building scene models. This paper presents techniques for extracting edge segments from a digital picture. First, an edge detection is performed on the entire picture to obtain a list of edge elements. Second, these edge elements are parametrically transformed. Third, using this parametric representation, the edge elements are divided into several groups. Each group consists of edge elements which lie on a contiguous portion of a straight line. The line segments extracted are represented by the positions of their two end points. Results of line extraction on a few outdoor scenes are also presented.

Edge detection Scene matching

Edge filtering Edge segment extraction Hough transformation

I. INTRODUCTION

Vertex models

Straight-line fitting

3 present the Hueckel edge-detection process and a filtering operation, respectively. Section 4 discusses the process of extracting line segments from the edge elements located. Finally, Section 5 presents some representative results from applying our process to a few outdoor scenes. The scenes considered consisted of houses and some natural background (such as bushes and trees). The least-squares scheme used to fit a straight line to the grouped edge elements is derived in the Appendix.

Our work in scene analysis has long been aimed at developing methods of reducing the immense quantity of picture information to derive suitable robust features. Straight-line edge-segment features of a picture have often been used to represent the scene structure. As an example, straight-line edge segments have been used as the key features in representing scenes through vertex-based models, m This paper discusses the process of extracting line segments from a digital picture. Several methods have been used to recover straight lines from an original scene. Perkins ~21 describes a boundary-following scheme. A line-growth method is described by Jarvis/3t A more recent paper, by O ' G o r m a n , describes a process to find edge-segments through collinearity of feature points/4) O ' G o r m a n uses a parametric representation of a straight line known as the H o u g h transformation. All these investigations were primarily carried out on indoor scenes consisting of blocks and polyhedra. Our process of extracting straight-line edge segments has been developed with outdoor scenes in mind. The two main steps in locating line segments are (1) edge detection and filtering and (2) extraction of line segment s from the edge elements found. Sections 2 and

2. EDGE DETECTION Many edge-detection methods described in the literature apply a local operator to points in a picture to obtain short edge elements. Some of these operators are relatively simple, such as Roberts and Sobel, ~5~ while others are more complex and allow eliminating most of the local noise. Hueckel ~6) has developed an edge operator that performs quite well on a large variety of pictures. The edge model used by the Hueckel operator is a step function F in a circular disc. Let F(x,y,c,s,p,b,d)

* Copyright © 1977 by The Institute of Electrical and Electronics Engineers, Inc. Reprinted, with permission, from Proceedings of the IEEE Computer Society Conference on Pattern Recognition and Image Processing, 6-8 June 1977, p. 366. f This work was supported by the Defense Advanced Research Projects Agency, Contract F30602-77-C-01M9, monitored by the Rome Air Development Center, Griffiss Air Force Base, New York. :~ S. A. Dudani was with Hughes Research Laboratories, Malibu, CA. He is now a consultant interested in imageanalysis-related problems. 145

= { bb

+d

c x + s v <" pcx+sy>p,

where the origin of the x - y coordinate system is at the center of the circular region. The intensity levels on the two sides of the step edge are b and b + d. The equation of the straight-line edge is given by cx + sy = p. Figure 1 shows a circular Hueckel disc with a step edge. Let E ( x , y ) be the given intensity function in the circular disc. The intensity function E is approximated with an ideal step edge F so that the distance between E and F is minimum, i.e. we minimize ~ f [E(x, y) - F(x, y , c , s , p , b , d ) ] z d x d y .

146

S.A. DUDANIand A. L. LUK (Xe, Ye) cx

~

b

j(xc,

sv

y

b+d

Fig. 1. Hueckel edge model consisting of a step edge in a circular disc. The minimization procedure returns both the best edge position (xe, Ye) and its orientation. There are two parameters associated with this operator: (1) D I F F , which is related to the step edge size, and (2) CONF, which is a threshold value for the edge confidence measure. Our experience using this operator has shown that values of 100 and 0.85 are suitable for D I F F and CONF, respectively, for 8-bit grey-level pictures. The edges found are relatively less sensitive to the exact values of these parameters. This is a major advantage of using the Hueckel operator. Also, this operator is to a high degree insensitive to local noise. The disadvantages of this operator are that it is complex and that it uses an ideal step edge model. In an outdoor scene, an ideal step edge model often misses some important edge elements. To avoid the problem of obtaining multiple edge elements with the Hueckel operator, an additional requirement is imposed on accepting edge elements. An edge element (xe, y~) is accepted only if Ixe - xcl < 1 lYe - Ycl < 1, where (x,, Yc) is the center of the Hueckel disc. An edge element satisfying this condition is referred to as a central edge. An example of imposing this additional requirement on edge detection is shown in Fig. 2. In this figure, note that multiple edge points were present before the additional condition was imposed. The processing for extracting edge segments becomes simpler if only the central edges are retained. The result of

edge detection on a given scene is a list of edge elements that are defined by their x and y coordinates and by their orientation. 3. E D G E F I L T E R I N G

The edge-detection process on a given scene produces a list of edge elements specified by their position and orientation. The edge element is a basic unit for the extraction of straight-line edge segments. The success or failure of locating line segments depends on the information derived from these edge elements. The edge elements obtained may be grouped into two separate classes: undesired edges and desired edges. The undesired edges may be produced by sensor noise, use of an improper edge model, and the curved structures in the scene. Edge elements resulting from sensor noise are distributed randomly throughout the scene. As an example of undesired edges due to the use of an improper model, the Hueckel operator gives false edges at corners or intersections of straight-line segments. The bulk of edge elements of the undesired and the desired classes comes from the curved and the straight-line scene structures. An edge segment is formed by grouping edge elements that form a contiguous part of a straight line. Because we are only interested in edge elements that form straight-line segments, a filtering technique is used to eliminate edge elements that do not contribute towards the formation of straight-line edge segments. There are two main payoffs from edge filtering of this type : (1) data reduction for further processing and (2)

I

J

i !

PICTURE

MULTIPLE EDGE ELEMENTS

CENTRAL EDGE ELEMENTS

Fig. 2. Multiple and central Hueckel edge elements for a simple picture.

Locating straight-line edge segments

X

Ik

147

segment, is quite simple. The final result is a list of line segments represented by the x and y coordinates of the two end points ofeach segment. Figure 4 illustrates the steps involved in dividing the entire edge element list into several groups and the subsequent extraction of line segments. These steps are described below.

A parametric representation of straight lines A straight line may be represented parametrically (see Fig. 5) by the following relationship(71: p = xcos 0 + ysin0,

EDGEPICTURE

Fig. 3. Filter window centered around an edge element. increased chances of finding short line segments by working with cleaner data. It is important that the filtering process retains as many edge elements belonging to short edge segments as possible. A straight-line edge filter, as we define it, retains an edge element (x, y, 0) if and only ifa minimum total ofn edge elements with angles in the range (0 + 15 °) are found within a rectangular window of width 1 and length 1~ pixels centered around (x,y) and in the direction 0, as shown in Fig. 3. Each element found in the edge-detection process is checked to see if it satisfies this edge filter requirement ; if it does not, the edge is rejected. A detailed description of this edge filter is presented in Ref. 8.

where 0 is the angle of the intensity vector to the given line, and p is its displacement from the origin. The intensity vector is defined as the normal to the given line directed from the low grey-level to the high greylevel side. The value of p is considered positive when the intensity vector points away from the origin and negative when it points toward the origin. The O-p representation has the advantage over the conventional slope-intercept (y = m x + c) line representation that the values of 0 and p lie within closed intervals : 0:_<0_<360

I,OI _< (X2max "Jr Y2max)l;2" where Xmax and Ymax are picture width and height, respectively. The techniques described here for extracting line segments use this O-p representation for straight lines.

LOWLE~LEVEL 4. STRAIGHT-LINE EDGE SEGMENT EXTRACTION

/~NTE

The list of edge elements {(x~,yi, Oi), i= 1, N} obtained by edge detection and filtering is used in extracting straight-line edge segments. The main problem is how to fit straight lines to the edge elements located. To do this requires dividing the entire edge element list into several groups, each representing a single contiguous line segment. Once this is done, the next step, which is to fit a single straight line to each group and to determine the two end points of the line

NSITYVECTOR

// J4 /

GIVEN LINE ~,

Fig. 5. A parametric representation of straight-line edge segments.

STRAIGHT LINE FIT ~ ?

ix,, v,,

r- - - - - -'1

FC2 2,

II~II

01,P 1)

IL=~=JI i Ic;-=-,-11i I

O2,p 2)

e ±i --

i ~ ~ I xyI~ -" -11 GROUPING IJ--"-]IGROUPING lIT --' .--,I GROUPING I L~.JiI " IPt EZZ3"it

lJ__ LI I

×N, VN,

9N.p N

EDGE ELEMENTS

(8i, p i,

Pli, P2i) i=l,M

I[~ZZ],

I, [ ~ [ ~ ', I

L'-----'---~J

Fig. 4. Extracting straight-line segments from a list of edge elements.

LINE SEGMENTS

148

S.A. DUDANIand A. L. LUK

0o

100

100° I

I

-400

-400 -

-200

-2oo

0

0

200

200

400

400

o 200 ° i I

300 ° 360 °

I

I

i Oi

Z

< et-

PrO u. Z

~ . . . .

HOUGH PICTURE IS AN ACCUMULATIVE A R R A Y OF THE O R I G I N A L EDGE POINTS BEING DETECTED

0 p. :D m, Ito

I

ct.

I

,0:tA/ 0

100

EN 200

300

~8 360

8 DISTRIBUTION

Fig. 6. One-dimensional local minima search on the Hough picture. The Hough transform, therefore, maps a single edge element point in the image space to a point (0, p) in the Hough space. Furthermore, edge elements lying on the same straight line will be mapped to the same point (0, p). Thus, the line-finding task can be mechanized using a two-dimensional accumulative array known as the Hough picture, t4'7~ A high count in the Hough array indicates that a reasonably strong straight-line segment exists in the original scene. A twodimensional clustering technique is usually applied on the Hough picture to locate high-count elements and to extract the corresponding straight-line parameters (0~,p3's. These straight-line edge segments are subsequently recovered from the image space by a tracing algorithm. However, this line-finding process often misses short edge segments. It also requires enormous

computation in the two-dimensional clustering and line tracing techniques. Besides, a substantial core memory is needed to generate the Hough picture. Our line-finding approach uses three grouping steps known as 0-, p-, and xy-grouping. The first two steps constitute a one-dimensional local minima search on their respective distributions. Figure 6 gives an intuitive insight into 0- and p-grouping techniques from the perspective of a two-dimensional clustering algorithm on the Hough picture. Therefore, in the three grouping steps, 0-grouping gathers all the edge elements that have intensity vectors within a particular angular range, say 01 to 02. On the other hand, pgrouping separates the edge elements under the same 0-group according to the displacements of their intensity vectors. The next step, called xy-grouping, goes

i

SCENE

EDGE

POINTS

Fig. 7. Hueckel edge detection on a scene containing simple block structures.

Locating straight-line edge segments back to the image space and extracts line segments from the 0 and p groupings using a distance threshold or gap tolerance. The final result is a list of line segments represented by the coordinates of end points obtained by a least-squares fitting method.

z

LU

149

120

ILl

J

LM

90

C3 60

_L;.

O-grouping Let {(xi, yi, Oi), i = 1, N} be the position and direction of the edge elements found. The 0-distribution of the edge elements, which represents the number of edge elements with the angle 0, is computed from the edge element list. A scene containing some simple block structures is used here to illustrate the method of dividing edge elements into line segment groups. Figure 7 shows the original scene and the edge elements found. Because of the simplicity of the scene being considered, no edge filtering is necessary. The 0distribution for this example is shown in Fig. 8(a). The 0-distribution is a closed graph in the sense that 0 ~'and 36ff represent the same edge element direction. As described earlier, the main problem in finding line segments is the division of the edge element list into several groups, each representing a single line segment. As a first step, the edge elements are broken into groups by dividing the complete 0-range (0-360 ' ) into several groups. This is done in two steps. First, a smooth 0-distribution is obtained by local averaging the original distribution with a moving window. Let the width of the moving window be w. To have a symmetrically centered window around a point, the value selected for w must be odd. Note that a value of 1 for w has no smoothing effect, since the original distribution is unaltered. The value ofw is chosen such that a reasonably smooth distribution is obtained. For most outdoor scenes, a value of 11 for w appears to give acceptable results. Figure 8(b) shows the averaged 0disttibution for w = 11. Second, the 0 range is partitioned where local minima exist in the averaged 0 distribution. A point in the distribution is considered to be a local minimum only if it is less than all other points in a local neighborhood. If there is a local plateau present in the distribution, the local minimum point is considered to be at the middle point of the plateau. A moving window is used to define the local neighborhood for locating minima. As a rule of thumb, the size of the local minimum window is chosen to be of the same size as the averaging window used for obtaining a smooth 0 distribution. The partitioning of the 0 range is done at the local minimum points so that the likelihood of assigning edge elements belonging to a single line segment into two different groups is small. Each 0 group obtained in this manner includes edge elements belonging to line segments having approximately the same direction. For the example under consideration, a total of nine 0 groups were obtained through this procedure. The 0 range and the number of edge elements for the groups found are given in Table 1. Figure 8(c) shows edge elements belonging to 0 group No. 8, which has a range 249 ° 309 °.

N

0 50

Z

100

150

ANGLE

200

250

300 350

IN de9

(a) 8-DISTRIBUTION

{/3

50

LIJ

40

LtJ tad Lg C3 UJ LL

1

3

0

50

5

7

9

2O

© rr

== Z

100 150 200 250

300

350

ANGLE IN deg (b) GROUPING ON AVERAGEDS-DISTRIBUTION (e) LOCAL MINIMUM POINT

f J I

(c) E D G E E L E M E N T S

FOR O GROUP NO. 8 (249 ° TO 309 ° )

Fig. 8. 0-grouping on scene of Fig. 7.

p-grouping This step considers each 0 group separately and attempts to divide it further into smaller groups based on the p parameters of the edge elements in the 0 group. For each edge element in the group, p is computed through the relationship p = x c o s 0 + ysin0.

150

S.A. DUDANIand A. L. Lug Table 1. 0 groups for the block scene of Fig. 7

o~ I80

z

0 Group No.

0 range (deg)

Number of edge elements

1 2 3 4 5 6 7 8 9

1 to 6, 341 to 360 7 to 46 47 to 60 61 to 87 88 to 140 141 to 195 196 to 248 249 to 309 310 to 340

99 139 84 115 377 206 76 358 85

For each 0 group, a/9 distribution is first obtained. This distribution represents the n u m b e r of edge elements having the parameter/9. The partitioning of the /9 range is also performed in a m a n n e r similar to that for the 0 range. Figure 9 shows the original and averaged/9-distribution for the 0 group No. 8 shown in Fig. 8(c). In this case, the averaging on the/9 distribution was performed using a window 11 pixels wide. This value has generally provided good results on outdoor scenes. This procedure is not sensitive to the exact width of the averaging window. All that is needed is a reasonably smooth distribution adequate for locating local minima. Next, local minima are detected on the averaged/9 distribution with a local window width of 11 pixels. In this example, seven local m i n i m u m points are obtained, which gives rise to six different/9 groups. The/9 range and the n u m b e r of edge elements for the groups obtained are given in Table 2. Figure 9(c) shows the edge elements belonging to/9 group No. 5, which has a range of 3 to 56 pixels.

I.H

.a 60

iii

40

a: 20 D

z

0 -400

-200

,lnL 0

200

400

/9 IN pixels (a)

IZ uJ Lu .-I LU O UJ U© EE

Z

19 - D I S T R I B U T I O N

1 3 5 ~ 2 r ~ 4 r-~ 6 ~,~ ~--,

10

8 6 4 2 0 -4O0

-200

0 200 p IN pixels

400

(b) GROUPING ON A V E R A G E D p - D I S T R I B U T I O N (e) LOCAL M I N I M U M POINT

xy-grouping

At the end of/9 grouping, the original list of edge elements for the entire scene is divided into several subgroups, the edge elements of which have approximately the same 0 and/9 values. An example of such a subgroup was shown in Fig. 9(c). This step considers edge elements in each 0 and /9 group spatially (x-y space) and examines whether further grouping is possible. The idea is that groups of edge elements are to be obtained to which a single contiguous line may be fit. This procedure, which is similar to what is commonly referred to as point clustering in x - y space, involves the following steps: Table 2. p groups for the 0 group No. 8 belonging to the block scene of Fig. 7 p Group No.

p range (pixels)

Number of edge elements

1 2 3 4 5 6

- 1 8 1 t o -101 -100to -61 - 6 0 to - 3 2 - 3 1 to +2 3 to 56 57 to 181

3 36 116 54 105 44

f

J

(c) EDGE ELEMENTS F O R p

GROUP NO. 5 ( 3 T O 5 6 )

Fig. 9. p-grouping on 0 group No. 8 corresponding to Fig. 8. The first edge element in the group is considered to form subgroup (cluster) No. 1. The rest of the edge elements in the group a r e considered one by one. Each is checked to see ifit lies within a preselected threshold distance (usually ca. 5 pixels) from the elements already considered. If the element n u m b e r under consideration does lie within the threshold distance from another neighboring element, it is included in the subgroup to which that neighbor belongs. Note that group merging is performed when an edge element is within the threshold from two separate groups.

Locating straight-line edge segments If, however, the edge element under consideration does not lie within the threshold distance from any previously considered edge elements, then it is considered to form a new subgroup or cluster. Any subgroups containing fewer than a preselected threshold number of edge elements are eliminated at this step. This may be considered to be a filtering operation to eliminate short edge segments. A simple example of xy-grouping is shown in Fig. 10(a), where O - p group is further divided into two subgroups, each representing a contiguous line segment.

151

accurately and therefore simple and less accurate edge detection and orientation techniques may be used. The criteria used for determining the best-fit straight line is to minimize the total normal distance from edge points to the straight line. The normal distance from a point (xi,Yi) to the straight line (x cos 0 + y sin 0 - p) is given by: d i = p - x i c o s 0 - y i s i n 0.

The total normal distance D for the N points is given by : N

D2 = ~

Straight-line fitting

The three grouping steps - 0, p and x y - produce several subgroups, each of which contains edge elements representing a single line segment. Next a straight line is fit to the edge elements in each of the subgroups, and the two end points of the line segments are determined. In determining the best-fit straight line, only the x - y position information of the edge elements is used. This is advantageous in a way as it is not necessary to estimate edge element directions very

25 50 75 100125150175200225250 25 50

(P - x i c o s O - y i s i n O ) 2.

i=1

The best-fit straight line (p, 0) is determined by solving the following simultaneous equations minimizing D 2 ~D z -0 ~D z ~p

--0.

A solution to these equations is presented in the Appendix, which also includes the relationships used for determining the end points of the line segment. There are two measures of goodness of fit that may be used for selecting or filtering line segments. The first is the root-mean-square (rms) distance from the grouped elements to the line given by

75 100

d =

150

200 225 250

(a) x y - GROUPING ( 0 - p - G R O U P : 249 ° - 3 0 9 ° , 3 - 5 6 p i x e l s )

25 50 75 100125150175200225250

50 75 100

SEGMENT1: 8 = 2 7 8 °, p = 4 2 , P1 = (37,182), P2 = (91, 174) SEGMENT 2: 0 = 2 7 8 ° , p = 2 9 , P1 = (144, 155), P2 = (187, 148)

125 150 175 200

x i cos 0 -

Yi s i n 0 ) 2.

SEGMENT 2 SEGMENT I

175

25

(p -

i=1

125

J

®

®

225 250

(b) FITTINGSTRAIGHT LINES Fig. 10. xy-grouping on edge elements of Fig. 9.

A value of zero indicates an ideal fit. The acceptance value of this measure depends on the resolution of the scene being considered. In our work on outdoor scenes, we have chosen 2 pixels as the acceptance value. The second is the number of points constituting the edge segment divided by the length of the line : r1 = N / l ' c ,

where ~ is the length of the line segment and c is a compensating factor based on the slope of the line. The compensating factor c is given by: c=cos0,

if

]tan0]< 1

=sin0,

if

[ t a n 0 [ > i.

The point length ratio t/provides a density measure of the grouped edge elements for the best estimated line. This measure is an indication of the thickness of a line segment. If the value of q is too small (generally below 0.5) or too large (generally above 3.0), the corresponding edge segment is rejected. The compensating factor c in the denominator is explained as follows : when 0 = 90 °, c = sin 90 ° = 1 and r / = 1 ; that is, the number of points N along the x direction approximately gives the length IL Figure 10(b) shows the straight-line fit to the linesegment subgroups of Fig. 10(a). The results of line

152

S.A. DUDANIand A. L. LUK

segment extraction on the complete block scene are illustrated in the next section.

5. RESULTS The sequential, one-pass, straight-line edgesegment-extraction technique described in this paper has been tested on both indoor block pictures and on outdoor scenes containing man-made structures and natural background. Figure 11 shows edge elements, line segments, and the vertex model for the block scene being considered. The formation of the vertex model from the extracted line segments and the use of these segments in scene matching is presented. (1~

It is more difficult to extract straight-line edge segments from outdoor than from indoor scenes. This is so primarily because more edge elements are detected from outdoor scenes. Some preliminary results are presented here for outdoor scenes containing houses and natural background (such as trees and bushes). Figure 12 shows the original scene, edge elements, line segments, and the resulting vertex model. The edge elements shown were obtained after the filtering operation. The amount of the edge filtering in this case was great enough to cause some line segments to be missed. However, most of the prominent line segments essential for representing the original scene were indeed found. Figures 13 and 14 show similar results obtained on house pictures.

SCENE

EDGE POINTS

29 24 2

35

18

LINE SEGMENTS

~

34

25

12

T7

3 ~ 15 14

4

30

~0

8

VERTEX MODEL

Fig. 11. Edge points, line segments, and vertex model for a scene containing some block structures.

Locating straight-line edge segments

153

°~

,MF'

I!

°

SCENE

EDGE

ELEMENTS

3 4 24 5 4o / / ~ 42

f

_

~ 21 18 2. ~ 1L~--.~9 3 ~ 32 33

1

~

~

17 9 29 9~ lo 31 ~ 2 1 f39 F

I 15

11 1__2 16

LINE SEGMENTS

VERTEX

_ _ 13

MODEL

Fig. 12. Vertex model for a scene containing a house with a background of bushes and trees.

!

154

S.A. DUBANI and A. L. LUK

I.

¢

"k -...-~._.~-.~,.~....~..,. •- ' " ~

...-,-~

Ct'

",

N

-t,~y"

.~'

"

.,.-~x

Ij

l:l ,•

m

.,,,=~&

I't •

e

----

' i,i

_ ~ _ _ 2

i

SCENE

EDGE ELEMENTS

\

f ~

-1

I--I

4~ 15"~,~46

47

43

' 1

22

23

-X

24 35

1

2

8

2

2

8

48 29

50-

I' 10 I,-..----~ 30

LINE SEGMENTS

VERTEX MODEL

Fig. 13. Vertex model for a scene containing a house with a background of bushes and trees.

51

--49

Locating straight-line edge segments

155

't'

'

~

,.,..-.

/V = m

SCENE

. . m w

p , ,. - t, ~ , ~ ~

'

i,

~

!

,i i-It

"b,-""~'.''=-"

• "#

"=

EDGE ELEMENTS

\

30

/y

35 17

I 111 I -

j

29

2618

28

2

36

42

19

20

38.~J22 39

LINE SEGMENTS

VERTEX MODEL

Fig. 14. Vertex model for a scene containing a house with a background of bushes and trees.

PR 10/3

C

6

156

S.A. DUDAN! and A. L. LUK 6. SUMMARY

Y

Straight-line edge segments are often used as key features for representing scenes through models. This paper discusses a procedure for extracting such features from a digital picture. Edge detection is the first step in the process. This step yields a list of the edge points, or edge elements, present in the scene. Edge filtering then eliminates those edge elements that do not contribute to forming straight-line edge segments. Next, the problem is how to fit straight lines to the edge elements located. This problem is not trivial since neither the number of straight lines to be fitted nor even the different edge element groups to which they are to be fitted is known. Our technique divides the edge element list into several subgroups, each containing edge elements representing a contiguous straightline segment. The edge elements are grouped by a three-step process, called the 0-, p-, and xy-grouping. 0 and p are the parameters used in the Hough transformation. Once the edge elements are grouped, straight lines are fitted to individual groups to determine the exact locations of edge segments. The line segments extracted are represented by the coordinates of their end points. The results shown represent our most recent work on outdoor scenes. We feel that we have a sound and robust procedure of extracting lines. The overall performance of the complete process can, however, be improved upon through further work on individual process steps.

{X[~ xi'Yi)

x

Fig. A-1. Straight-line fit to N given points.

OOz -= 0. Op

(5)

Substituting the value of D 2 into equations (4) and (5) yields OD 2

O0 = 2{sin 0cos O [ Z y 2 - Ex 2] + p sin 0 Ex - p cos 0 Ey + X x y [ c o s Z O - sin20]} = 0

(6)

OD 2

ep

= 2 N p - 2cos 0 E x - 2 sin 0 Z y = 0.

(7)

Thus, from equation (7) we have Ex Xv p = cos 0 - - + sin 0 ~ . N N

APPENDIX

(8)

Fitting a straight line to N given points

Substituting p into equation (6) yields This appendix derives the equations used for determining a best-fit straight line to N given points. The criterion used for determining such a straight line is to minimize the total normal distance from given points to the straight line. Let the N given points be {(x, y3, i = 1, N} and let the straight line be represented by the following relation: p -- x c o s 0 + , y sin 0.

(2)

Let D be the total normal distance from the N given points to the straight line; it is given by Dz=

~

(p-x~cos0-yisin0)

(9)

where A = Ey 2 - -

EX 2

+

(Xx) 2 - (Xy) 2 N

Zx~y B = Exy - - N

(1)

The normal distance of a point (x~,y 3 to a straight line (Fig. A-l) is given by d i = p - x i cos 0 - y~ sin 0.

A sin 0 cos 0 + B cos 20 - B sin 20 = 0,

Equation (9) is further simplified as shown below for two different cases based on whether or not sin 0 cos 0 equals zero. Case 1. sin 0 cos 0 = 0 (i.e. either 0 = 0 ° or 0 = 90°). In this, the value o f ( E x y - ~ E x T £ y ) \

.

,

is small (if this value is

not

/

below a certain small quantity, then (0, p) is computed as discussed in Case 2). First compute the variances in x and y directions.

2

i=l

= N p 2 + c o s 2 0 Z x z + sin20Ey 2

~

-- 2p cos 0 Zx - 2p sin 0 Ey + 2 sin 0 cos 0 E x y .

-

(3)

OD 2

- 0

Xx

(to)

N

The parameters (0, p) are obtained through solving the two simultaneous equations that minimize the square of the total distance :

O0

= lxx~

(4)

( 1 )

ey2=--N 5~y2-

2

NY~Y "

(11)

If e 2 > a y 2, then 0 = 9 0 ° ; if a ~ > o ~ , then 0 = 0 °. This is intuitively obvious since a horizontal line will have a very small variance in the y direction, but a relatively large variance in the x direction.

Locating straight-line edge segments

Case 2. sin 0 cos 0 # 0 (i.e. Y x y

\

Ex Xy ~ 0']. Dividing

N

/

equation (9) by sin 0 cos 0 and letting C = tan 0, we have BC 2 - A C - B = 0.

A + v/A 2 - 4B 2

-

2B

(13)

2B

if I t a n 0 1 < l if

Itan0 I > 1 .

Acknowledgements - The authors wish to thank Ms. J.

pl = cos 01 Xx ~ - + sin 01 ~Xy -

(15)

02 = cos 02 Ex - ~ + sin 02 ~Zy- .

(16)

Thus, the two possible solutions are (01, P 0 and (02,P2). The solution which gives the minimum distance as computed in equation (3) is chosen as the best-fit straight line to the given points. The parameters 0 and p give the position and direction of the best-fit straight line. At this point, one does not know the two end points of the line in x-y space. The two end points can be determined from the given list of N points. Each point (xl,y 0 is projected on the best-fit line. The projection point (x'/,y'0 and the projection distance pl as shown in Fig. A-I are given by x'i = x~ + p c o s 0 - x / c o s 2 0 - YlsinOcosO

(17)

3'/= Yi + P sin 0 - xl cos 0 sin 0 - yl sin20

(18)

pl = --x/sin 0 + yicosO.

(19)

The projection distances (if/s) are sorted to find their minimum and maximum values. The (x'i,y'i) points corresponding to the two extreme projection distances are considered to be the two end points. There are two measures of goodness of fit that may be used : The root-mean-square (rms) distance from a point to the line: this is given by N i=1 (p - x ~ c o s 0 - ylsin0) 2.

c=cos0

(14)

Substituting 01 and 02 into equation (8) gives the two values of p as follows :

d=

rl = N/f~c,

where ~ is the distance between the two end-points and c is a compensating factor based on the slope of the line :

=sin0

A - x/A 2 - 4B 2 C 2 = tan 02 -

A value of zero indicates an ideal fit. The number of points constituting the edge segment divided by the length of the line

(12)

The two solutions for C from this quadratic equation are C t = tan01 - -

157

Stafsudd for her contributions in implementing the edgefiltering and the vertex formation techniques discussed in this paper. Thanks are also due to Mr. Gary Klein, who derived the relationships for the least-squares straight-line fit.

REFERENCES

1. S.A. Dudani, A. L. Luk, J. P. Stafsudd, C. S. Clark and B. L. Bullock, Model-based scene matching, Hughes Research Laboratories, Research Report No. 509, May (1977). 2. W. A. Perkins and T. O. Binford, A corner finder for visual feedback, Computer Graphics and Image Processing 2, 355-376, December (1974). 3. R. A. Jarvis, Image segmentation by interactively combining line, region, and semantic structure, Proc. Conf. on Computer Graphics, Pattern Recognition, and Data Structure, May (1975).

4. R. O'Gorman and M. B. Clowes, Finding picture edges collinearity of feature points, I E E E Trans. Comput. 25, 449-456, April (1976). 5. R. O. Duda and P. E. Hart, Pattern Class{[ication and Scene Analysis. Wiley, New York (1973). 6. M.H. Hueckel, A local visual operator which recognizes edges and lines, J. A C M 20, 634-647, October (1973). 7. R. O. Duda and P. E. Hart, Use of the Hough transformation to detect lines and curves in pictures, Comm. A C M 15, 11-15, January (1972). 8. S. A. Dudani and J. P. Stafsudd, Straight line edge filter, in preparation.

About the Author SAHIBSINGHA. DUDANI was born in Bikaner, India, on 8 September 1949. He received the B.Tech. degree from the Indian Institute of Technology, Bombay, India, in 1970 and the M.S. and Ph.D. degrees in Electrical Engineering from Ohio State University, Columbus, Ohio, in 1971 and 1973, respectively. During the fall of 1973, he was a visiting assistant professor of electrical engineering at Ohio State University. In 1974 he joined RBM Associates, a consulting firm based in Columbus, Ohio, as a research scientist. From 1974 to 1977, he did applied research work in the fields of scene analysis and pattern recognition at Hughes Research Laboratories, Malibu, California, where he was the head of the Image Analysis Group. At present, he is working with image-analysis-related problems as a consultant. His areas of research interest include pattern recognition, image processing, scene analysis, and computer engineering.

About the Author- L. ANTHONYL. LUK was born in Hong Kong on 9 November 1948. He received a B.S. and an M.S. in physics and an M.S. in computer science from the University of California, Los Angeles, in 1971, 1973, and 1976, respectively. From 1974 to 1976, he was a system programmer with the Laboratory of Nuclear Medicine and Radiation Biology at the University of California, Los Angeles. Presently, he is a member of the Technical Staff at Hughes Research Laboratories, Malibu, California, working in the areas of pattern recognition and image analysis.