A fast parallel algorithm for dot linking in Glass patterns

Pattern Recognition Letters 3 (1985) 263-270 North-Holland

July 1985

A fast parallel algorithm for dot linking in Glass patterns Ari G R O S S , R a l p h H A R T L E Y

a n d Azriel R O S E N F E L D

Center for Automation Research, University of Maryland, College Park, MD 20742, USA Received 5 November 1984

Abstract: The h u m a n visual system is very good at perceiving structure in complex patterns. 'Glass patterns', constructed from displaced copies of random dot patterns, are examples of this in which the only cues are pairs of matching dots. Stevens has proposed a model that accounts well for h u m a n performance at this task. A modified version of this model is proposed here which has a fast implementation on parallel 'pyramid' hardware. The existence of such implementations is highly desirable because h u m a n perception takes place very rapidly. The hardware assumed is a stack of cellular arrays each of which covers the image with a different sized grid. This architecture allows fast c o m m u n i c a t i o n between distant parts of the image as long as the a m o u n t of information that needs to be sent over long distances is limited. The modified model was found to behave quite similarly to Stevens r model.

Key words: Glass patterns, dot patterns, structure perception, pyramid linking.

1. Introduction

A 'Glass pattern' (Glass, 1969) is a Moire pattern generated by superimposing a random pattern of dots over a copy o f itself that has been displaced, e.g., shifted, rotated, or rescaled. If the displacement between the two patterns is not too large relative to the density of the dots, the viewer is capable of matching corresponding dots and perceiving the displacement. Some examples of Glass patterns are shown in Figure 1. A model for this process, proposed in (Stevens, 1978), appears to account well for human performance. Stevens' algorithm consists of two stages. The first stage finds possible pairs of corresponding dots (i.e. pairs that may be the two images of the same dot before and after displacement). In the second stage the directions determined by the dot pairs in the neighborhood of each dot are locally The support of the National Science Foundation under Grant MCS-82-18408 is gratefully acknowledged, as is the help of Janet Salzman in preparing this paper.

histogrammed and the most frequent direction is found. This is likely to be the direction of the displacement in the neighborhood of that dot, because the directions of pairs of dots which do not correspond are random. We will be concerned here mainly with the first part of Stevens' model (further details on this model are given in Section 2). The second part of the problem, that of determining the local and global flow, will be discussed in a separate paper. The human visual system is able to carry out complex perceptual tasks, such as the detection of Glass displacement patterns, very rapidly - often in not more than a few hundred times the cycle time o f a single neuron. This suggests that the brain uses a great degree of parallelism. It is thus highly desirable to find models that have very fast parallel implementations. An algorithm will be presented here that is similar to Stevens' and that has such a fast implementation. Our algorithm uses a computational structure called a pyramid, which consists of a stack of arrays each of which has half the linear dimensions

0167-8655/85/$3.30 © 1985, Elsevier Science Publishers B.V. (North-Holland)

263

Volume 3, Number 4

PATTERN RECOGNITION LETTERS

July 1985

features of the entire image or of large areas of the image to be computed in not much more time than purely local features. This property o f pyramids is used here to speed up the search for corresponding dots in a Glass pattern. "

:

:

":

-

:

:'i .

-':

"

:

'

.

:

:

2. Algorithm

• .

.

..

(a)

'

'

"

i

'.

...'.':'...

"'

:. .'."

."

,

,:'i

i ..'

.-'

"

..

'

'

. . . .

.:

-

.

.

-

..

•

...

'-.-

'.'

....

-.

.

".

:

-

..

.

..

. .. . .

•

.

:

.

..

.

..

.

...

:i:

.-

"

.

..

i i i:: ..-"

.

. -

.. :.

.

,

.. .

.

•

.-" .

'. - .

'~."..

: . . .

• .. .

"i'"

...'

•. •

.

.

-.

-

''.

" '

i .......

.-

:..

..-

"

ii . ..

.

'

.

": .

'

. . . .

Ib) Figure 1. Two Glass patterns; (a) a pure translation; (b) a rotation.

o f the array below it. The elements of the arrays, which we shall call cells, are independent processors. Each level in the stack represents the same visual field at a different level of detail. The bottom level initially contains the input image. Each cell is connected to a small set of corresponding cells on the adjacent levels. The number of cells in a pyramid is not much larger than that in an an ordinary cellular array, i.e., in the base alone; in fact, if the arrays decrease in size by a factor o f two between two levels, the pyramid has only -]- more cells. Each cell is connected to a bounded number o f other cells, but every pixel (i.e. every cell in the base) is now connected to every other pixel by a path only O(Iog(D)) long, where D is the distance between the two pixeis as measured in the base. This allows 264

Stevens' algorithm is as follows. For each point P find its nearest neighbor P ' . Let D be the distance from P to P ' . Link the point P with all points that are w~ithin rD of P, where r is a parameter. Note that the links produced by this algorithm are not symmetric since P can link to Q even if Q does not link to P. Stevens has determined that the algorithm performs comparably to humans when r is approximately 1.3. The time required by Stevens' algorithm depends on the distances between the dots; the area that must be searched in the neighborhood o f each dot goes up as the square of the nearest neighbor distance. This dependency is undesirable in a model o f h u m a n perception, where performance does not depend significantly on dot separation over a wide range. An algorithm will be presented here which yields approximately the same links as Stevens' but which requires only O(log(D)) time to find the links. The algorithm also does not involve a parameter like r, it automatically yields results that correspond approximately to r = 1.4 in Stevens' algorithm. Our algorithm uses a pyramid in which the dimensions of each level are half those of the level below. Each cell in the pyramid represents the same portion of the image as a 2 by 2 array of cells on the level below. Each cell communicates with these cells on the level below and also with their immediate neighbors, i.e. with a n by n array of cells (Figure 2), which are called the 'sons' of the given cell. The sons in the central 2 by 2 subarray will be referred to as the central sons. Each cell also communicates with an n / 2 by n/2 array o f ' f a t h e r ' cells on the level above. Stevens' algorithm could be implemented in this structure in the following way. Each cell in the pyramid contains a list of the dots in the region of the original image which it represents, together with their positions. This list can be easily corn-

V o l u m e 3, N u m b e r -1.

PATTERN

// / /

tb

/ Ii II

I I

\ ,,, ,,,

I I

I\

\\

I \ \

,\,

\ \ \ ; \ \ \I .i \

\

II I

RECOGNITION

I \X

\ I

\;',q \

,,

I

\

\

,

\',

Figure 2. Each cell is connected to n 2 'sons' on the level below, four o f which c o r r e s p o n d to the same area o f the i m a g e as the cell. ( I l l u s t r a t e d with n = 6.)

puted by initializing the lists o f the cells on the bott o m level (each such cell contains either one dot or none), and c o m p u t i n g the list o f each cell on a higher level by taking the union o f the lists o f the central sons o f the cell. Because a cell is a central son o f exactly one cell on the level above, every dot is in the list o f exactly one cell on each level. For each level L let

E(L)= n - 2 2L_l 2r For any dot P there is a lowest level such that the circle centered at P with radius E(L) contains at least one dot other than P. The cell on this level containing P in one o f its central sons can find the dots that P should link to by examining its sons. This is illustrated in Figure 3.

'k,,

j I)

F i g u r e 3. For any d o t in one o f the central sons, if the closest d o t is inside the inner circle (with r a d i u s E ( L ) = ( n - 2 ) / 2 r . 2 L - I ), then all dots that link to the given dot are inside the o u t e r circle (with radius r E ( L ) = ( n - 2 ) / 2 . 2 L - I ) and are t h e r e f o r e visible f r o m this cell.

LETTERS

July 1985

Once P has linked to its neighbors, it is no longer necessary to pass i n f o r m a t i o n about P to higher levels o f the p y r a m i d , but only to tell these levels that a cluster o f dots is present. Moreover, we can limit the n u m b e r o f dots that must be handled by a cell by specifying that P links to any individual dot within rD that is not o f a cluster, but only to the nearest dot o f any cluster that is within rD. A cluster is represented within a cell by the position o f its centroid and by the n u m b e r o f dots it represents, and is treated in much the same way as a dot. For each dot or cluster found in a central son o f a cell, its n e i g h b o r h o o d is checked to see if it should link to any other dots or clusters. If not, the dot or cluster is passed to the cells' fathers. If a dot or cluster links with one or more dots or clusters, then it is replaced by a single cluster representing it and all the dots and clusters to which it links. W h e n a dot links with a dot, nor further action is taken. W h e n a dot needs to be linked to a cluster, the dot is linked to the nearest dot in the cluster; the m e t h o d o f doing this is described in the next p a r a g r a p h . If a cluster is f o u n d in the n e i g h b o r h o o d o f a cluster, no dots need to be linked because all the necessary linking was already d o n e when the clusters were formed. If n is large e n o u g h , a dot will always link to all the dots in the central sons o f the cell to which it belongs, so that a cell can never contain m o r e than one dot or cluster. A second pass t h r o u g h the pyramid which proceeds f r o m top to b o t t o m is required to link each d o t that has been linked to a cluster to the closest d o t in that cluster. Each cell passes the dots that linked to a cluster to the son cell that contains the cluster. W h e n a dot is passed to a cell that contains a cluster the dot is passed to the son that contains the closest sub-cluster to the dot. This continues until the dot is passed to a cell that contains a single dot at which point the dots are linked. The n u m b e r o f dots that can be passed to a cell in this way is b o u n d e d because if too m a n y dots are close e n o u g h to a cluster to link to it, they will be either close e n o u g h to the cluster to link to it on a lower level, or close e n o u g h to each other to f o r m clusters on a lower level and not link to the cluster (see Figure 4). 265

Volume 3, Number 4

PATTERN R E C O G N I T I O N LEq-I'ERS

July 1985

Figure 5. Any dot in one of the central sons links to everything in a 3 by 3 array of sons.

Figure 4. Given a cluster and a dot, anything in the shaded region will link to the dot and not to the cluster. For a given cluster only a bounded number of dots can be placed so that no dot is in the shaded region corresponding to another. Therefore only a bounded number of dots can link to a cluster.

The parameter r can be eliminated from the algorithm. Instead of searching a circle centered at a dot for its nearest neighbor and linking to everything within rD of the dot, we link each dot to all the dots (or clusters) within ( n / 2 - 1 ) 2 L - I . This gives the same result as using a value o f r between 1 and 2, because the distance to the nearest dot is between ( n / 2 - 1 ) 2 L-I and ( n / 2 - 1 ) 2 L-2 (otherwise it would have linked on a lower level). In practice the effective value of r is approximately 1.4. In order to simplify the implementation in our experiments, the circle was replaced by a square. This is equivalent to using a chessboard metric, instead of the Euclidean metric, to determine the distance between a pair o f points. A 3 by 3 square was used to find the neighbors o f a dot or a cluster. The square was centered on the central son that contains the dot, as shown in Figure 5. Because the square covers all the central sons o f the cell there can never be more than one dot or cluster in a cell. The square is completely contained in the 4 by 4 array o f nodes directly below the cell so the im266

plementation uses n = 4 . Note that centering the square on the son that contains the dot instead of on the dot itself is equivalent to assuming that all dots and clusters are located at the center of a cell and discarding their actual coordinates. On a cellular array of processors Stevens' algorithm requires O(D) time where D is the maxi m u m distance between nearest neighbors. Our algorithm running on the appropriate pyramid hardware requires only O(Iog(D)).

3. Experimental results Both our algorithm and Stevens' were implemented and applied to Glass patterns generated by superimposing two copies of a uniformly distributed, pseudo-random array of dots. Figure 6 shows the links produced by Stevens' algorithm and by ours when applied to the patterns in Figure I. Note that our algorithm most closely resemble Stevens' with r = 1.4. Figure 7 shows the performance o f both algorithms on several images with the same value o f the dot density ~) and different values of the magnitude o f the displacement JD:J. The graph compares two measures of performance: the false dismissal rate, which is the percentage of corresponding dots that were not linked, and the false alarm rate, which is the percentage of links that did not connect corresponding dots. The performance o f our algorithm was close to that of Stevens' with r = 1.4. For each image our algorithm had a lower false

Volume 3, Number 4

PATTERN RECOGNITION

LETTERS

July 1985

I I

I

!

% ~,

,L 'i

:

i:

I tl

''

"1,'

I

i

'

I

I

l

I

i

, ~'-,

I

I

,I I~ I

i

I

I

I

I

i I

i

~1

I

I I

I

I

i [

I

i

I ii

I

I

I I

I

I

I

I

i

i

~I

I

I

t

I

I

i

I

'

'/

tj

I

l

-h i

n .

b bl

i

~

iu

,

I

n

I

.

t

L

r

i ~

FI I

I I

t

I

'

l

~

II

i

l

I

i 4"" i

I i

i

I

I

I

~t.':

i I

I

I

ilL)

/,,

I I

lI h

I

I I I

l.iJ , i i

I

I

i

Figure 6(d). Stevens' algorithm with r = 1.4 applied to l(a).

i

I

i *

I

I

i

:

I

[1~

I

I

q

'*1 'I

t

I

I

la)

i

i

I I i

I

Figure 6. (a-I). Results o f applying algorithms to Figure 1. (a) Pyramid algorithm applied to I(a).

I I

i

I

I

lJlll

I I

I

I

I

I

'[' I

,

,Ir

U

II I

I

I

I

i,I

I

i I

k

I

i II

r

I

I

ii

i

i I

I

I u I

I

~','

I

i

l

I

i

4

I

I I

II

I

I'

I

I

i~ll

I I

I

~

rl r I I

I

I

n

i

I

I l.I

I I

I I i I

I

I1~'1

I

1

i~l

I

I

I '..... ' I

I

i i

I

I

t

iI

:

I

I

I

I r'It

t

;-, 6

I

ii

'

t" 1 r

n

L

:

n

I

I

i F

II

t

I n

,1

n '

I

I

I

i

I I

r

i

I

ii

I

i

dip

f

,,' 1

I

I

t

¢

1

l

I

I

i

i

r

i

i

[

II

; Il

~

;

i

:

p ~

i

I I ~ hi J I

i

I

, ~

i

;

L

i

'

I

t

I

Figure 6(e). Stevens' algorithm with r = 1.5 applied to I(a).

"Jl

I

I

,

i

'

l

t~

'

i

' '~ ' "'a':z

~ ,

I

J

I

I i

(

IbI

LI ' a

L~1

, i '~r,,~ i

fll

-

L

I

r i -I

j

t~

i

,,.,' h

'' ",-J " ":"~ ' '-" "~:' , L~!" I ' ' l , 0 ,' , I '

'

I

,

I

i I.

L'l l

i

I

I1.1

,

I

L ;

~

I

i

'

~

it

I

]

i'

I

i q

Ii

i I

I

I~4

i"

L- I

I

i I

i I

I

I

I

u

,

i

I

,

* I"v

v

T

I

I I

'

r"

I I

i j

I

i ;

i

g

,p [

(e)

['

, I

l

[-

n

;

i

b

gl I I i

i

I

J

tJ I

I

I

¢'1

~

,

I

I

I

r

'

I I l;

Ill

~'I,

II

.I I

i

I

r r-"~

I

I

I

T

i

a

I

I L

,

I

(b)

i I

I

i

i

Figure 6(b). Stevens' algorithm with r= 1.0 applied to I(a).

q I

CI

i'-,

iI

II

'Jl

I

it I

I II

I I

I

I

, r

I

i

i

I

, f

I I

I

t I

i

q

I

L,

I I

ii I

i t

L}

I

I

I

(,I

I

I 'r

(c) Figure 6(c). Stevens' algorithm with r= 1.3 applied to I(a).

i '

I

i

(f) Figure 6(f). Stevens' algorithm with r = 2.0 applied to I(a). 267

Volume 3, Number 4

PATTERN

..... . ~.. ~'~'~'~ ', -. ..-' .:',s.

.~]

.

,.,~"N

-,g~,,. .,..,-. ~

~

', ',

J u l y 1985

~,..

7-

.._._

"'"

.-"

.

I~x~

..-._

,

~.-- ~' .'~-;'

-I.

'-t~!.

~, . ,

,~.s.,

LETTERS

,'

'

~',"

.'

~':~ ,:.~

RECOGNITION

,, '.

i

' ~'l,~"~ -~.

"

.'£~.

'

.,

L_

i

I

'

,,

'1

'..,~,,,+

..

.

-

~

. ,'~

.~...

.-~_/

==

-:.

:. -

''

~ . .

,.1 '-'.~.~

,,

.-

,

.._--." "

"-~" . ,,.. ~

- -'-.*-

~\~ . .r...: ......,.~ .~. 2-

(9 Figure 6{g). Pyramid algorithm applied to l(b).

.." ~- ,U"

.;. ....

.:

,l~l

( .o

"---

~

~

~,

~

"

, I v

)

Figure 6(j). Stevens' algorithm with r = 1.4 applied to I(b).

(

..=.

."

..

.

-

~.

=" ~

"

~

..

~-~

,,~

~ ~!,

",'"

•

",.,

,1' I

I I

I

~,

II

I

. i

i

.a.'~. I

I ~i

-,

"--,

~

'

,...

,

¢-

~ -

,

* ~. ~

,;"

I i i II

,,

~,,

,,.~,

:

(k)

Figure 6(h). Stevens' algorithm with

l

I

I

I

I

_-~

1.0 applied to l(b).

r=

.~

I

i

,

'.

{hi

,-.~...

~

"%'

~,

7r-

,, ~

..

'~

..,

Figure 6(k). Stevens' algorithm with r = 1.5 applied to l(b).

'.

i

'

~

,

''

1

,~,i

~'

° •

~

,

_~ --. d.~,

•, ~ . . .

-. ~

~'~

-...-';L"

c_ :--::

./~,~. ....

,~

li)

Figure 60). Stevens' algorithm with r = 1.3 applied to I(b).

268

•

,

~

1

[

",.

(I) Figure 6(I). Stevens' algorithm with r = 2.0 applied to l(b).

Volume 3, Number 4

PATTERN RECOGNITION LETTERS

July 1985

100% 90~-

- - "-

80%"

/

~

~

,,'

70% 0~ '~ t~ I...t

,

:/

.'/

.

X

/

#

-'/"

,"'

~

...../ / -

/ ~ ~ " , .. / .1 / ..,"

t

60%

-

~ " 51' .. 5.-~' / " .....

I"

i

-

/

/"

"#/

i

~_¢

..-

/

,..

r,. 30~-

~~/.../../ d / /"

20%"

(2,2) (4,4)

.. dpyramidd = (8, (6,6) 8) ............... Stevens' ....... k = 1.3 k = 1.4 k = 1.5 ................ k = 2.0

10Yo"

0%

md = • d =

/

• .

0%

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

k = 1.0

.

16~ 2~% 36% 46% 56% 6~% 7.0~ 8'o% 9~% ioo~ False dismissal

rate

Figure 7. False alarm and false dismissal errors for each algorithm for Glass patterns with various displacements.

100% 90% 80%

False dismissal

rate

70% 60%

o

50%40%30%.

20%"

lO% 0%

Density Displacement

0.008 4

0.00512 5

0.00356 6

0.0026 7

0.0020 8

Figure 8. How the performance of our algorithm changes when the displacement ]d It (in pixels) and the density o in dots/cell vary while ~d 2 ]t/.o remains fixed. (The difference between the results for ] d [ = 4 and ] d ] = 8 are a result of small sample size when low densities are used. 269

Volume 3, Number 4

PATTERN RECOGNITION LETTERS

alarm rate for a given false dismissal rate. This is due to the way in which we handle clusters of dots. Because there is only one dot corresponding to a given dot, linking to every dot in a cluster produces m a n y more false alarms than correct links. This advantage of our algorithm would disappear if the dots in the pattern from which the Glass pattern was generated were negatively correlated instead of being independent. If the dots were regularly spaced then clusters of dots would not occur and our algorithm would give results closer to those of Stevens'. Stevens' algorithm is invariant with respect to scale so its performance does not depend on the actual values of Q and D but only on the ratio ]]D]]2/.o. Its performance can therefore be measured by fixing ~o and measuring the error rates for different values of D. Our algorithm is only guaranteed to give identical performance for a change of scale by a power of two. Figure 8 shows the performance of our algorithm, on a fixed image, as a function of scaling by factors less than two.

270

July 1985

4. Conclusion An algorithm for finding pairs o f (possibly) corresponding dots in a Glass pattern has been presented. This algorithm gives results similar to Stevens' algorithm, and thus performs c o m p a r a b l y to humans. The new algorithm can be implemented in pyramid hardware so that it can process an image very quickly even if the dots are widely separated. This is a m a j o r desideratum for a biologically plausible model.

References Stevens, K. (1978). Computation of locally parallel structure. Biological Cybernet. 29, 19-28. Glass (1969). Moire effect from random dots. Nature 243, 578-580.