- Email: [email protected]
Symbolic pixel labeling for curvilinear feature detection
Pattern Recognition Letters 8 (1988) 299 310 North-Holland
December 1988
Symbolic pixel labelingfor curvilinearfeature detection John C A N N I N G , J. John KIM, Nathan N E T A N Y A H U and Azriel R O S E N F E L D ('enler l~r Automation Research, University (~['Maryland, College Park, MD 20742, USA Received 2 March 1988
Abstract." This paper describes a method of detecting thin curvilinear features in an image based on a detailed analysis of the local gray level patterns at each pixel. This allows operations such as thinning and gap filling to be based on more accurate information.
Key words. line detection, curve detection, pixel labeling.
Many types of images contain thin curvilinear features: for example, aerial photographs contain numerous features such as roads and streams. This paper describes a method of detecting thin curvilinear features in an image based on a detailed analysis of the local gray level patterns at each pixel. As we shall see, this allows operations such as thinning and gap filling to be based on more accurate information. The traditional approach [1] to extracting thin curvilinear features from an image is usually along the following lines: (a) Apply local line detectors to the image. (b) Link the detected line pixels into connected components. (c) Thin the components using standard binaryimage thinning techniques. (d) Fill gaps in the components using 'good continuation' interpolation. A serious weakness of this approach is that at step (a) it decides, at each pixel, whether a line passes through it, and if so, in what direction, based on
the absolute and relative strengths of a set of line detector responses. This decision is made independently for each pixel. Thereafter, in steps (b
The support of the Defense Mapping Agency under Contract D M A 85 C 0007 is gratefully acknowledged, as is the help of Sandy German in preparing this paper. 0167-8655/88/$3.50 © 1988, Elsevier Science Publishers B.V. (North-Holland)
299
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
X Junction
December 1988
~I~EI~I~
T dunctlon
~
Rcute Rn9]e
Y Junction
L L~ne
R~9ht An9le
End Point
Ed9e
Ii
Ii
Ooro
Figure I. Local edge and line patterns.
These patterns are found by examining all possible thresholdings of the 3 x 3 neighborhood and selecting those of the resulting binary patterns ('masks') that are line- or edge-like. These patterns are shown in Figure 1. Each pixel now has associated with it a set (possibly empty) of line- or edge-like masks. Note that there can be more than one of these at a given pixel, since two different thresholds can both yield edge(or line-)like binary patterns, as we see if we apply the thresholds i and 3 to the 3 x 3 neighborhood 224 224 0O2 However, there cannot be very m a n y of them; if the gray levels in the neighborhood are z 1..... Zg, where 0 = z 0 _ < z l < _ z 2 _ < .... < z g < Z x o = 2 5 5 , the patterns produced by any threshold between zi and zi+ 1 are all the same, and some of these (e.g., for i = 0 or 9) are not line- or edge-like. Now that we have associated masks with the pixels, we can use the mask information in performing steps analogous to ( b ~ ) rather than simply working with sets of pixels. As we shall now see, this leads to improved results. We begin by examining the masks at pairs of adjacent pixels, and checking pairs of these masks for compatibility with the presence of a line (or edge). Figure 2 shows sets of pairs that are compatible 300
with the presence of a thin line (the 'extend' and 'maybe-extend' pairs) or with a line two pixels thick (the ~broaden' and 'maybe-broaden' pairs). Figure 3 shows sets of pairs that are compatible with the presence of edges in various orientations. By taking the transitive closure of mask compatibility, we obtain sets of masks that represent connected line (or edge) fragments. These are analogous to the connected components of step (b), but they are now components of masks, not of pixels, and a pixel may belong to more than one component. We can now illustrate the advantage of keeping all the masks at each pixei rather than just the 'strongest' one. [The examples used in this and the following paragraphs are all fragments of roads on an aerial photograph; they are all taken from Figure 4.] Figure 5a shows an 8 x 8 portion of the image; the numbers in the centers of the squares are the pixel gray levels. In each square we show the masks that describe the 3 x 3 neighborhood of that pixel; the number under each mask is a measure of its contrast. The lines joining the masks represent compatibilities (solid lines for 'extend' compatibilities; dashed lines for 'broaden' compatibilities). We see that this piece of image contains a branching thin region. If we keep only the highest-contrast mask at each pixel, we obtain Figure 5b, in which nearly all the compatibilities are gone! Things are somewhat better, but still far from perfect, if we
Volume 8, N u m b e r 5
P A T T E R N R E C O G N I T I O N LETTERS
I)ecember 1988
EXTEND EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
ELi3
ELE
ELE
EL[
ELE
EkE
ELE
EtE
ELE
ELE
ELE
[LE
EH
EH
EH
EH
EH
[H
EY
EY
[Y
EY
[Y
EL
EL
EL
EL
[L
EL
EL
ELC: ELC
[LC
EL(::
EL(::
[LC
[LC:
ELE
ELE
[T
ET
[T
[T
ET
[T
ET
ET
ET
ET
EY
[Y
EY
EY
[Y
[Y
EY
EY
EY
EY
ECHH ECHH ECHK EDTY EDTY [DTY [DTY EDTY EDTY EDTY [DTY EDTY EDTY ELi
ELi
ELI
EL1
ELi
ELi
EL"1 EL1
EWL
[WL
[WL
[HL
EY
EL
EL
EY"13 EY13 EYO
MAYBE -EXTEND
ELi
EL"1
EL "1
EL"1
EL "1
ETE
[TE
ETE
ETE
ETE
[TE
EWL
EWL
EWL
EWL
EWL
[WL
EWL
ENL
EWL
EWL
EWL
EWL
EWL
[WL
EWL
EWL
EWL
EWL
EWL
EWL
EWL EWL[ [WLE ENLE EWL[ ENLE
EWL[ EWT
ENT
[WT
[WT
EWT
[WT
[WT
ENT
[WT
[WT
[WT
[WT
[WT
EWTE EWT[ [WTE ENT[ EHTE EWTE [NTE
EHE
EHE
EHE
EHE
EVE
EVE
EYE
BT[
BY
EWY EWY EWY EWY ENY
EWY EWY
EVE
EVE
[WL
EVE
BROADEN BE
BE
BL
BL
BL
BL
BL
BL
BL
BT
BT
BTE
MAYBE -BROADEN MBE
MBE
MBL
MBL
I"IBL MBL
MBL
PIBL MBL
MBL
MBL
MSL
MBL
MBT
PIBT MBWL MBWL PIBWL MBWL HSWL MBI,.IT
MBWT MBWT M B W H MBWY M B N Y
None CHK
CHK
CHK
CHK
OHH
CHK DURL DUAL DURL DUAL DURL DUAL DUAL EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE
EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EL2
EL2
[TO
ETC
[TO
EWLT HOLE HOLE HOLE MEWL NEWL NEWL MEHL MEHL MEWL MEWL MEWL MEWL MEWL HEWT
MEWT MEWT MEWT NEWT MEWT MEWT NEWT MEWT MEWT MEHT MEWT MEWT MEHT ME~T MEWH HEWH MEHH MEHH MEWY MEHY HEWY
MEWY WIGL
Figure 2. Consistent pairs of local edge or line patterns.
301
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
December 1988
HORIZONTAL -EDGE HE HE HE HE HE HE HE HE HE HE
V E R T I C A L -EDGE DE DE DE DE DE DE DE DE DE DE
D I A G O N A L -EDGE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE
B A C K - D I A G O N A L -EDGE BDF" BDIr BDE BD£ BD[ BDIr BD~" BD[: BDIr BDr BDIr 5Dr BD£ BD£ BD( B[)£ BD£ BDIr BD(
H O R I Z O N T A L -BOUNDARY HB HB HB HB HB HB lib
HB HB HB HB HB HB 14B HB HB HB HB HB HB liB
lib
HB HB lib
HB HB
V E R T I C A L -BOUNDARY
D I A G O N A L -BOUNDARY DB DB DB DB DB DB DB DB
B A C K - D I A G O N A L -BOUNDARY BDB BDB BDB BDB BDB BDB BDB BDB
None
m m m ~ ~ l n ~ k ~
s~nh NM~I
m mmmmemelm ~ ~ / l ~ g m l ~ m
~M~
Figure 3. Consistent pairs of local edge patterns.
302
~Bll~ 4~Nll S ~H~
~~
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
December 1988
Figure 4. Roads.
keep the two highest-contrast masks at each pixel (Figure 5c). We next illustrate the use of the mask in 'thinning', by analyzing the mask patterns to determine which pixels lie on or closest to the midline of the curvilinear feature. In Figure 6, (a) shows a two pixel thick linear feature; (c~t) show standard thinning results (depending on which side we thin from first); and (b) shows the result when the masks are used. The details of how this result was obtained are shown in Figure 7a, where the numbers under the masks are now component numbers; component 949 is the one shown in Figure 6. The masks show that the pixels in column 93 better represent the midline in rows 70 73, while those in column 92 better represent it in rows 74-77. In Figure 7b the masks not on the midline have been dropped (though their links to other masks have been kept). Finally, we illustrate the use of the masks to fill small gaps. Figure 8 shows the midlines of three linear feature fragments, two denoted by black squares and one by white circles. If we regarded the fragments as sets of pixets, and filled the gap on geometric grounds, we would join the white fragment to the lower black one. Inspection of the masks, however, which are shown (for the boxed
center part of Figure 8) in Figure 9, indicates that we should in fact join the white fragment (No. 240) to the upper black one (No. 647), not to the lower one (No. 626). A more detailed discussion of the mask-based approach to curvilinear feature detection can be found in [2~4], where additional examples are also given. We believe that the mask-based approach can be useful in the precise delineation of thin linear features. It may also be of value in precisely delineating the edges of thick objects; here we would use only the edge-like masks. It should be pointed out that the mask-based approach to extracting edge or curve fragments is likely to be more computationally costly than conventional approaches. However, when massively parallel S I M D systems are available at low cost, which can be expected to happen during the next decade, the extra computational cost of detailed pixel-neighborhood analysis will no longer be a major factor. We have therefore concentrated on demonstrating the advantages of such analysis, in terms of the accuracy of the resulting linear feature delineations, on the assumption that in the relatively near future, it will no longer be necessary to rule out this approach on grounds of cost. 303
Volume 8, Number 5
7
1 97
[] []
1
1
1 98
I~1 []
Z
PATTERN RECOGNITION LETTERS
1
1
76
Z
Z
[] [] 5
1
97
1
8/I•
/
[]
1
- .
=
[]
1
1
1
7Z
[] 1 []
-
79
Z
75
[]
December 1988
7q
NN 1
[] 2 7Z
[]
1
1
~'
78
69
68
68
73
7z
~
69
8Z
78
qez
9s
[]
NN
1
81~90,,
86
e
e
91
9Z
109
92
[] 98
',,
109 3
92:
\
418
\ lZ 117
1OZ
[] " 13~ ' ~8i 8 100
~10 t 9Z
~11 i
6
15
188
8q
7 76
[]
[]
[]
[]
7
5
7
1
:3
7
10
3
6 77
[]
[]
[]
7
1
1
1 67
[]
Figure 5a. An 8 x 8 image showing masks and compatibilities.
304
3 80
81
1 7LI'
6
1
96
65
66
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
[]
[]
[]
[]
[]
[]
18 .O
16.6
15,8
9.15
6,6q.
5,55
98
g7
63
75
[]
[]
[]
17.5
19,3
17,?.
97
79
9~
95
[]
[]
15.5
1.u,.6 96
[]
I rill L
3.33
3.71
75
7Z
7°,
72
78
69
68
68
[] 15.5 6~
:u,5
[]
[]
[]
Z~4/4
17.3
9,8
lot4
lOq
December 1988
73
88
[] 6,5 7::'
71
\
[] 1Z .5
7.38 9e
[]
[]
[]
[]
16.1
26.1
Z 7,8
29.7 6g
109
91
[]
[] 30,3 82
78
--IN
10,~
92
85
92:
&
[]
[]
1~,9
16.7
Ze~.997
ZO/41Ot4
&
[] 117
I
\
~3
27,8
Z6,3
109
92
[] 17.9
[]
[]
[]
[]
[]
33.3
3e,7
28,7
25.3
2%2
100
le5
92:
?.9.7
84
35,2 108
le2
29,7
118
38,5 I02
11Z
95
\
Z3 .Zl. 117
23.3
1
78
[]
[]
39,1
25.3
96
[] 21,9
81
80
77
67
55
56
[] 13,8 7q.
Figure 5b. Results of keeping only the strongest mask at each pixel.
305
Volume 8, N u m b e r 5
P A T T E R N R E C O G N I T I O N LETTERS
[]
[]
[]
15,9
7.5
9,15
97
98
[]
[]
16,1
17,5
~W 6.6q.
76
6.5
79
q.,93
75
[] [] 1~6,8
17,2:
December 1988
5.55
[]
lind 3,Z5
3,33
3,71
3.35
7Z
7t4
7Z
69
68
68
[]
15,5
13,8 78
97 / .t
9@
I~11,7
[] 1Z,.5
le.
96
%
7,38
~ t "8~
15,1 91
9@
16,1 1@9
[] []
9,17
9,8
73
ZZ,5
[]
[] []
[] []
Z7,8
Z7,7
Z9,7
89
11Z
18.~
Z6.3
8Z
Z@,q Zq'.9
Z7.8
117
9Z '%
Z9,7
17,9
117
15.7 23.3
ZZ,7
Z8,9
102
1@5
Z9.7
3Z,3
[] []
[] []
[] []
[] []
3Z .2:
33.3
[email protected] Z8,2:
Z8,7
Z5,3
10@
92
84
78
35.Z Z9.3 96
108
Z~.@
Z9,~
81
[] []
[]
[email protected] 2:5,4?. Z5,3 Z1,5 Z1,9 77 8@
[] []
18,@
Zl,2 74
13.8 67
Figure 5c. Results of keeping only the two strongest mask at each pixel.
306
35,5 95
,~,
[]
30.7
35,3 78
Z8.6 1QZ
[][] Z3.q
[]
Z7,3
8 30,5
\
19.5
6,5 71
7Z
Z7.@
[]
6,zk3
86
92
9~
Z8,1
[]
65
66
Volume 8. Number 5
PATTERN RECOGNITION LETTERS
)~
89 98 91 92 93 94 95 96
,~
89 98 91 92 93 94 95 96
mm
•
mm
•
mm mm mm mm • •
,3
~4 ~ ,~ ,~
December 1988
• • mm • • •
a.
b.
89 99 91 92 93 94 95 '96
89 98 91 92 93 94 95 96
?g
•
~ ,~
•
,~
•
,. ,~ ~, ~
•
72
•
73
• • • ••o .
,4
•
71
•
74
•
75
• ••
76
,,I
d,
Figure 6. (a) Fragment: (b) midline found using masks (see Figure 7), (c d) results of conventional thinning (frum the right and left, respectively).
54 5 5 56 57 58 59 60 61 62 69 6,4 6 5 6 6 6~ 6 8 69 70 71 72
,6 "" '47
.9
|
so
51
|
52
|
i m-
m
m~
,.
_
_
_
59
61 62
~
63
~
64 Figure 8. Three fragments that could be linked.
307
Z
c~
°i
!
[]
[] []
[]
[]
[]
[]
f-rl
~
[]
m
[]
[]
[] []
[] [] []
[]
ID
cJ~
Z
j~
8g
December 1988
PATTERN R E C O G N I T I O N LETTERS
Volume 8, Number 5
g0
gl
g2
g3
g4
g5
g6
70
71
72
go~"
.,y,
73
74
75
1186 76
77
Figure 7b. Masks not on the midline have been deleted (though the links to them are still shown).
309
Volume 8, Number 5
December 1988
PATTERN RECOGNITION LETTERS
5g
60
54
52
63
64
65
56
[]
[] 71
74
53
[]
74
85
92
[]
[] 74
75
54
74
682
76
g7
647i
84.
87
1583
[] 82
55
[] 55
650 7g~ 649
90",,
\\
94
85
[] 87
57 fl
~ 9 '7
9/11"t
[] 58
94
84.
84
8g
[] 78
__"~"
73
[] 5g
240 240 104.
87
72
[] [] 50
~aN
[]
240 g3
72
/
/
88
51g
[] m~4o
[]
519 '5..19
51g
~IQ
~4Q
[]
[] 73
[] 74
Figure 9, Masks for the center part of Figure 8.
References
[1] Rosenfeld, A. and A.C. Kak (1982). Digital Picture Processing, second edition, Sections 10.2 4. [2] Netanyahu, N. and A. Rosenfeld (1987). Mask matching for linear feature detection, TR-1759, Center for Automation Research, University of Maryland, College Park. January 1987. [3] Canning, J., J.J. Kim, and A. Rosenfeld (1987). Symbolic 310
pixcl labeling for linear feature detection. TR-1761, Center for Automation Research, University of Maryland, College Park, January 1987. [4] Kim, J.J. and A. Roscnfeld (1987). Feature detection based on pairwisc consistent labeling, TR-I792, Center for Automation Research. University of Maryland, College Park, January 1987.
December 1988
Symbolic pixel labelingfor curvilinearfeature detection John C A N N I N G , J. John KIM, Nathan N E T A N Y A H U and Azriel R O S E N F E L D ('enler l~r Automation Research, University (~['Maryland, College Park, MD 20742, USA Received 2 March 1988
Abstract." This paper describes a method of detecting thin curvilinear features in an image based on a detailed analysis of the local gray level patterns at each pixel. This allows operations such as thinning and gap filling to be based on more accurate information.
Key words. line detection, curve detection, pixel labeling.
Many types of images contain thin curvilinear features: for example, aerial photographs contain numerous features such as roads and streams. This paper describes a method of detecting thin curvilinear features in an image based on a detailed analysis of the local gray level patterns at each pixel. As we shall see, this allows operations such as thinning and gap filling to be based on more accurate information. The traditional approach [1] to extracting thin curvilinear features from an image is usually along the following lines: (a) Apply local line detectors to the image. (b) Link the detected line pixels into connected components. (c) Thin the components using standard binaryimage thinning techniques. (d) Fill gaps in the components using 'good continuation' interpolation. A serious weakness of this approach is that at step (a) it decides, at each pixel, whether a line passes through it, and if so, in what direction, based on
the absolute and relative strengths of a set of line detector responses. This decision is made independently for each pixel. Thereafter, in steps (b
The support of the Defense Mapping Agency under Contract D M A 85 C 0007 is gratefully acknowledged, as is the help of Sandy German in preparing this paper. 0167-8655/88/$3.50 © 1988, Elsevier Science Publishers B.V. (North-Holland)
299
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
X Junction
December 1988
~I~EI~I~
T dunctlon
~
Rcute Rn9]e
Y Junction
L L~ne
R~9ht An9le
End Point
Ed9e
Ii
Ii
Ooro
Figure I. Local edge and line patterns.
These patterns are found by examining all possible thresholdings of the 3 x 3 neighborhood and selecting those of the resulting binary patterns ('masks') that are line- or edge-like. These patterns are shown in Figure 1. Each pixel now has associated with it a set (possibly empty) of line- or edge-like masks. Note that there can be more than one of these at a given pixel, since two different thresholds can both yield edge(or line-)like binary patterns, as we see if we apply the thresholds i and 3 to the 3 x 3 neighborhood 224 224 0O2 However, there cannot be very m a n y of them; if the gray levels in the neighborhood are z 1..... Zg, where 0 = z 0 _ < z l < _ z 2 _ < .... < z g < Z x o = 2 5 5 , the patterns produced by any threshold between zi and zi+ 1 are all the same, and some of these (e.g., for i = 0 or 9) are not line- or edge-like. Now that we have associated masks with the pixels, we can use the mask information in performing steps analogous to ( b ~ ) rather than simply working with sets of pixels. As we shall now see, this leads to improved results. We begin by examining the masks at pairs of adjacent pixels, and checking pairs of these masks for compatibility with the presence of a line (or edge). Figure 2 shows sets of pairs that are compatible 300
with the presence of a thin line (the 'extend' and 'maybe-extend' pairs) or with a line two pixels thick (the ~broaden' and 'maybe-broaden' pairs). Figure 3 shows sets of pairs that are compatible with the presence of edges in various orientations. By taking the transitive closure of mask compatibility, we obtain sets of masks that represent connected line (or edge) fragments. These are analogous to the connected components of step (b), but they are now components of masks, not of pixels, and a pixel may belong to more than one component. We can now illustrate the advantage of keeping all the masks at each pixei rather than just the 'strongest' one. [The examples used in this and the following paragraphs are all fragments of roads on an aerial photograph; they are all taken from Figure 4.] Figure 5a shows an 8 x 8 portion of the image; the numbers in the centers of the squares are the pixel gray levels. In each square we show the masks that describe the 3 x 3 neighborhood of that pixel; the number under each mask is a measure of its contrast. The lines joining the masks represent compatibilities (solid lines for 'extend' compatibilities; dashed lines for 'broaden' compatibilities). We see that this piece of image contains a branching thin region. If we keep only the highest-contrast mask at each pixel, we obtain Figure 5b, in which nearly all the compatibilities are gone! Things are somewhat better, but still far from perfect, if we
Volume 8, N u m b e r 5
P A T T E R N R E C O G N I T I O N LETTERS
I)ecember 1988
EXTEND EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
EL
ELi3
ELE
ELE
EL[
ELE
EkE
ELE
EtE
ELE
ELE
ELE
[LE
EH
EH
EH
EH
EH
[H
EY
EY
[Y
EY
[Y
EL
EL
EL
EL
[L
EL
EL
ELC: ELC
[LC
EL(::
EL(::
[LC
[LC:
ELE
ELE
[T
ET
[T
[T
ET
[T
ET
ET
ET
ET
EY
[Y
EY
EY
[Y
[Y
EY
EY
EY
EY
ECHH ECHH ECHK EDTY EDTY [DTY [DTY EDTY EDTY EDTY [DTY EDTY EDTY ELi
ELi
ELI
EL1
ELi
ELi
EL"1 EL1
EWL
[WL
[WL
[HL
EY
EL
EL
EY"13 EY13 EYO
MAYBE -EXTEND
ELi
EL"1
EL "1
EL"1
EL "1
ETE
[TE
ETE
ETE
ETE
[TE
EWL
EWL
EWL
EWL
EWL
[WL
EWL
ENL
EWL
EWL
EWL
EWL
EWL
[WL
EWL
EWL
EWL
EWL
EWL
EWL
EWL EWL[ [WLE ENLE EWL[ ENLE
EWL[ EWT
ENT
[WT
[WT
EWT
[WT
[WT
ENT
[WT
[WT
[WT
[WT
[WT
EWTE EWT[ [WTE ENT[ EHTE EWTE [NTE
EHE
EHE
EHE
EHE
EVE
EVE
EYE
BT[
BY
EWY EWY EWY EWY ENY
EWY EWY
EVE
EVE
[WL
EVE
BROADEN BE
BE
BL
BL
BL
BL
BL
BL
BL
BT
BT
BTE
MAYBE -BROADEN MBE
MBE
MBL
MBL
I"IBL MBL
MBL
PIBL MBL
MBL
MBL
MSL
MBL
MBT
PIBT MBWL MBWL PIBWL MBWL HSWL MBI,.IT
MBWT MBWT M B W H MBWY M B N Y
None CHK
CHK
CHK
CHK
OHH
CHK DURL DUAL DURL DUAL DURL DUAL DUAL EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE
EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EDGE EL2
EL2
[TO
ETC
[TO
EWLT HOLE HOLE HOLE MEWL NEWL NEWL MEHL MEHL MEWL MEWL MEWL MEWL MEWL HEWT
MEWT MEWT MEWT NEWT MEWT MEWT NEWT MEWT MEWT MEHT MEWT MEWT MEHT ME~T MEWH HEWH MEHH MEHH MEWY MEHY HEWY
MEWY WIGL
Figure 2. Consistent pairs of local edge or line patterns.
301
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
December 1988
HORIZONTAL -EDGE HE HE HE HE HE HE HE HE HE HE
V E R T I C A L -EDGE DE DE DE DE DE DE DE DE DE DE
D I A G O N A L -EDGE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE DE
B A C K - D I A G O N A L -EDGE BDF" BDIr BDE BD£ BD[ BDIr BD~" BD[: BDIr BDr BDIr 5Dr BD£ BD£ BD( B[)£ BD£ BDIr BD(
H O R I Z O N T A L -BOUNDARY HB HB HB HB HB HB lib
HB HB HB HB HB HB 14B HB HB HB HB HB HB liB
lib
HB HB lib
HB HB
V E R T I C A L -BOUNDARY
D I A G O N A L -BOUNDARY DB DB DB DB DB DB DB DB
B A C K - D I A G O N A L -BOUNDARY BDB BDB BDB BDB BDB BDB BDB BDB
None
m m m ~ ~ l n ~ k ~
s~nh NM~I
m mmmmemelm ~ ~ / l ~ g m l ~ m
~M~
Figure 3. Consistent pairs of local edge patterns.
302
~Bll~ 4~Nll S ~H~
~~
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
December 1988
Figure 4. Roads.
keep the two highest-contrast masks at each pixel (Figure 5c). We next illustrate the use of the mask in 'thinning', by analyzing the mask patterns to determine which pixels lie on or closest to the midline of the curvilinear feature. In Figure 6, (a) shows a two pixel thick linear feature; (c~t) show standard thinning results (depending on which side we thin from first); and (b) shows the result when the masks are used. The details of how this result was obtained are shown in Figure 7a, where the numbers under the masks are now component numbers; component 949 is the one shown in Figure 6. The masks show that the pixels in column 93 better represent the midline in rows 70 73, while those in column 92 better represent it in rows 74-77. In Figure 7b the masks not on the midline have been dropped (though their links to other masks have been kept). Finally, we illustrate the use of the masks to fill small gaps. Figure 8 shows the midlines of three linear feature fragments, two denoted by black squares and one by white circles. If we regarded the fragments as sets of pixets, and filled the gap on geometric grounds, we would join the white fragment to the lower black one. Inspection of the masks, however, which are shown (for the boxed
center part of Figure 8) in Figure 9, indicates that we should in fact join the white fragment (No. 240) to the upper black one (No. 647), not to the lower one (No. 626). A more detailed discussion of the mask-based approach to curvilinear feature detection can be found in [2~4], where additional examples are also given. We believe that the mask-based approach can be useful in the precise delineation of thin linear features. It may also be of value in precisely delineating the edges of thick objects; here we would use only the edge-like masks. It should be pointed out that the mask-based approach to extracting edge or curve fragments is likely to be more computationally costly than conventional approaches. However, when massively parallel S I M D systems are available at low cost, which can be expected to happen during the next decade, the extra computational cost of detailed pixel-neighborhood analysis will no longer be a major factor. We have therefore concentrated on demonstrating the advantages of such analysis, in terms of the accuracy of the resulting linear feature delineations, on the assumption that in the relatively near future, it will no longer be necessary to rule out this approach on grounds of cost. 303
Volume 8, Number 5
7
1 97
[] []
1
1
1 98
I~1 []
Z
PATTERN RECOGNITION LETTERS
1
1
76
Z
Z
[] [] 5
1
97
1
8/I•
/
[]
1
- .
=
[]
1
1
1
7Z
[] 1 []
-
79
Z
75
[]
December 1988
7q
NN 1
[] 2 7Z
[]
1
1
~'
78
69
68
68
73
7z
~
69
8Z
78
qez
9s
[]
NN
1
81~90,,
86
e
e
91
9Z
109
92
[] 98
',,
109 3
92:
\
418
\ lZ 117
1OZ
[] " 13~ ' ~8i 8 100
~10 t 9Z
~11 i
6
15
188
8q
7 76
[]
[]
[]
[]
7
5
7
1
:3
7
10
3
6 77
[]
[]
[]
7
1
1
1 67
[]
Figure 5a. An 8 x 8 image showing masks and compatibilities.
304
3 80
81
1 7LI'
6
1
96
65
66
Volume 8, Number 5
PATTERN RECOGNITION LETTERS
[]
[]
[]
[]
[]
[]
18 .O
16.6
15,8
9.15
6,6q.
5,55
98
g7
63
75
[]
[]
[]
17.5
19,3
17,?.
97
79
9~
95
[]
[]
15.5
1.u,.6 96
[]
I rill L
3.33
3.71
75
7Z
7°,
72
78
69
68
68
[] 15.5 6~
:u,5
[]
[]
[]
Z~4/4
17.3
9,8
lot4
lOq
December 1988
73
88
[] 6,5 7::'
71
\
[] 1Z .5
7.38 9e
[]
[]
[]
[]
16.1
26.1
Z 7,8
29.7 6g
109
91
[]
[] 30,3 82
78
--IN
10,~
92
85
92:
&
[]
[]
1~,9
16.7
Ze~.997
ZO/41Ot4
&
[] 117
I
\
~3
27,8
Z6,3
109
92
[] 17.9
[]
[]
[]
[]
[]
33.3
3e,7
28,7
25.3
2%2
100
le5
92:
?.9.7
84
35,2 108
le2
29,7
118
38,5 I02
11Z
95
\
Z3 .Zl. 117
23.3
1
78
[]
[]
39,1
25.3
96
[] 21,9
81
80
77
67
55
56
[] 13,8 7q.
Figure 5b. Results of keeping only the strongest mask at each pixel.
305
Volume 8, N u m b e r 5
P A T T E R N R E C O G N I T I O N LETTERS
[]
[]
[]
15,9
7.5
9,15
97
98
[]
[]
16,1
17,5
~W 6.6q.
76
6.5
79
q.,93
75
[] [] 1~6,8
17,2:
December 1988
5.55
[]
lind 3,Z5
3,33
3,71
3.35
7Z
7t4
7Z
69
68
68
[]
15,5
13,8 78
97 / .t
9@
I~11,7
[] 1Z,.5
le.
96
%
7,38
~ t "8~
15,1 91
9@
16,1 1@9
[] []
9,17
9,8
73
ZZ,5
[]
[] []
[] []
Z7,8
Z7,7
Z9,7
89
11Z
18.~
Z6.3
8Z
Z@,q Zq'.9
Z7.8
117
9Z '%
Z9,7
17,9
117
15.7 23.3
ZZ,7
Z8,9
102
1@5
Z9.7
3Z,3
[] []
[] []
[] []
[] []
3Z .2:
33.3
[email protected] Z8,2:
Z8,7
Z5,3
10@
92
84
78
35.Z Z9.3 96
108
Z~.@
Z9,~
81
[] []
[]
[email protected] 2:5,4?. Z5,3 Z1,5 Z1,9 77 8@
[] []
18,@
Zl,2 74
13.8 67
Figure 5c. Results of keeping only the two strongest mask at each pixel.
306
35,5 95
,~,
[]
30.7
35,3 78
Z8.6 1QZ
[][] Z3.q
[]
Z7,3
8 30,5
\
19.5
6,5 71
7Z
Z7.@
[]
6,zk3
86
92
9~
Z8,1
[]
65
66
Volume 8. Number 5
PATTERN RECOGNITION LETTERS
)~
89 98 91 92 93 94 95 96
,~
89 98 91 92 93 94 95 96
mm
•
mm
•
mm mm mm mm • •
,3
~4 ~ ,~ ,~
December 1988
• • mm • • •
a.
b.
89 99 91 92 93 94 95 '96
89 98 91 92 93 94 95 96
?g
•
~ ,~
•
,~
•
,. ,~ ~, ~
•
72
•
73
• • • ••o .
,4
•
71
•
74
•
75
• ••
76
,,I
d,
Figure 6. (a) Fragment: (b) midline found using masks (see Figure 7), (c d) results of conventional thinning (frum the right and left, respectively).
54 5 5 56 57 58 59 60 61 62 69 6,4 6 5 6 6 6~ 6 8 69 70 71 72
,6 "" '47
.9
|
so
51
|
52
|
i m-
m
m~
,.
_
_
_
59
61 62
~
63
~
64 Figure 8. Three fragments that could be linked.
307
Z
c~
°i
!
[]
[] []
[]
[]
[]
[]
f-rl
~
[]
m
[]
[]
[] []
[] [] []
[]
ID
cJ~
Z
j~
8g
December 1988
PATTERN R E C O G N I T I O N LETTERS
Volume 8, Number 5
g0
gl
g2
g3
g4
g5
g6
70
71
72
go~"
.,y,
73
74
75
1186 76
77
Figure 7b. Masks not on the midline have been deleted (though the links to them are still shown).
309
Volume 8, Number 5
December 1988
PATTERN RECOGNITION LETTERS
5g
60
54
52
63
64
65
56
[]
[] 71
74
53
[]
74
85
92
[]
[] 74
75
54
74
682
76
g7
647i
84.
87
1583
[] 82
55
[] 55
650 7g~ 649
90",,
\\
94
85
[] 87
57 fl
~ 9 '7
9/11"t
[] 58
94
84.
84
8g
[] 78
__"~"
73
[] 5g
240 240 104.
87
72
[] [] 50
~aN
[]
240 g3
72
/
/
88
51g
[] m~4o
[]
519 '5..19
51g
~IQ
~4Q
[]
[] 73
[] 74
Figure 9, Masks for the center part of Figure 8.
References
[1] Rosenfeld, A. and A.C. Kak (1982). Digital Picture Processing, second edition, Sections 10.2 4. [2] Netanyahu, N. and A. Rosenfeld (1987). Mask matching for linear feature detection, TR-1759, Center for Automation Research, University of Maryland, College Park. January 1987. [3] Canning, J., J.J. Kim, and A. Rosenfeld (1987). Symbolic 310
pixcl labeling for linear feature detection. TR-1761, Center for Automation Research, University of Maryland, College Park, January 1987. [4] Kim, J.J. and A. Roscnfeld (1987). Feature detection based on pairwisc consistent labeling, TR-I792, Center for Automation Research. University of Maryland, College Park, January 1987.