- Email: [email protected]
Geometrical transformations on pictures represented by leafcodes
COMPUTER
VISION,
GRAPHICS,
AND IMAGE
PROCESSING
33, 81-98 (1986)
Geometrical Transformations on Pictures Represented by Leafcodes MARLOES L. P. VAN LIEROP* Depurtment
of Mathematics
and Computing Science, Eindhoven 5600 MB Eindhoven, The Netherlunds
University
of Technoloa,
Received January 22,1985; revised June 18,1985 Quadtrees that represent pictures composed of 2” X 2” pixels, may be economically implemented using linear tree representations. An example of such a representation is the leafcode, which is an ordered sequence of the encoded leaves of the quadtree. This paper describes how the leafcodes of geometrical transformed pictures can be derived from the original leafcodes. In addition, we present some execution times. 0 19X6 Academic Press. Inc. 1. INTRODUCTION
A quadtree is a tree of which each node is either a leaf or an internal node with four children. A quadtree may be associated with a picture that is composed of 2” x 2” pixels in the following way. If all pixels of the picture have the same color, the quadtree consists of one node only; the color of the node is the color of the pixels. If the pixels do not have the same color, the quadtree consists of one node with its four subtrees, where the subtrees are the quadtrees of the four subpictures, obtained by partitioning the former picture into its four quadrants. The quadtree that results, is called the quadtree of the picture. Figure 1 shows a black-and-white picture composed of 2’ x 22 pixels, together with a representation of its quadtree. Quadtrees were introduced as a form of raster picture encoding that could reduce storage space requirements, while exploiting two-dimensional coherence [1, 21. For a comprehensive study of the history and use of quadtrees in picture encoding, we refer to Samet [3]. Some operations on quadtrees are easy to perform, like superposition and intersection, others are difficult. Troublesome operations are geometrical transformations, because the resulting quadtree may differ enormously from the original one. Quadtrees may be implemented as pointer structures or as linear structures, where pointer structures, in general, require more space but provide faster access than linear structures. Various papers [4, 5, 61 have been written on geometrical transformations of pictures that are encoded as pointer-structured quadtrees. Since quadtrees originally were introduced to reduce storage space requirements, we are interested in space-efficient implementations of quadtrees. Therefore we concentrate on linear structures, and look for efficient algorithms that perform geometrical transformations on pictures represented by linear quadtrees. In [7], Oliver and Wiseman describe a linear representation of the quadtree called the treecode, which is based on the depth-first traversal of the quadtree. For black-and-white pictures, Gargantini introduced in [8] a linear representation of the quadtree called the leafcode, which is an ordered sequence of encoded BLACK *These investigations were supported by the Netherlands TechnicaI Foundation (SW). 81 0734-189X/86
$3.00
Copyright 0 1986 by Academic Press. Inc. All rights of reproduction in any form reserved.
82
h4ARLOES L. P. VAN LIEROP
(b)
0 . 3
internal node black leaf white leaf
FIG. 1. A picture and its quadtree.
leaves. She claims that her way of encoding quadtrees reduces the storage space to one-third of the space required by pointer structures. Superposition and intersection of pictures encoded by leafcodes turn out to be simple merging operations on ordered sequences. Geometrical transformations, however, remain troublesome. In the algorithm proposed by Gargantini [9], which is restricted to translations of the picture over an integral number of pixels and rotations over 90 and 180 degrees, the new code of each black PIXEL (rather than leaf) in the input tree is computed, and afterwards a sorting and compacting step is performed. In no way does she use the fact that the input leafcode is an ordered sequence. In this paper an algorithm to perform geometrical transformations on pictures represented by leafcodes is described that works the other way round: the leafcode of the output tree is constructed leaf by leaf in the correct order, thus avoiding a sorting step, while exploiting the fact that the input leafcode is an ordered sequence. The algorithm supports all non-singular affine transformations, i.e., all invertible linear transformations, translations, and combinations thereof. The basic strategy of the algorithm resembles the algorithm for translations of octree encoded 3D objects, described by Meagher in [lo]. In Section 2 the way we notate leafcodes is described; Section 3 discusses the construction of the reverse leafcode, and is meant as an introduction to Section 4, which describes how the leafcode of a transformed picture is constructed out of the leafcode of the original picture. In Section 5, the algorithms of Section 4 are modified in order to increase efficiency. Section 6 discusses some details of the algorithms, and Section 7 contains some final remarks, and tables with execution times of implementations of both Gargantini’s algorithm and the one described in this paper. 2. THE NOTATION
OF LEAFCODES
The pictures we consider, are composed of 2” x 2” pixels, where each pixel is either black or white; n is called the re&&onof the pie&~. The levet of a n&e is defined by - the level of the root is 0; -
the level of a child node equals 1 + the level of ifs parent node.
The definition of the quadtree of a picture implies that an internal node is associated with a non-uniformly colored quadrant. A tree that has uniformly colored quadrants partitioned as well, is called a se4&
TRANSFORMATIONS
ON
83
LEAFCODES
(4 lb) (c) (d) FIG. 2.
All semi-quadtrees
of Fig. la.
quadtree of a picture is also a semi-quadtree. Several semi-quadtrees may be associated with one picture. In Fig. 2, all semi-quadtrees associated with the picture of Fig. 1 are shown. The conversion of a semi-quadtree into the quadtree is called compaction. The leafcode as proposed by Gargantini is an ordered sequence of all encoded black leaves. The code of a pixel is a sequence of quaternary digits reflecting successive quadrant subdivisions, according to the following convention: 0, 1,2,3 for the NW, NE, SW, SE subquadrants respectively, i.e.,
The most significant digit of the code refers to the first subdivision of the picture, the least significant digit to the last subdivision. Since a pixel is the result of n subdivisions, n digits are required for its code. The code of a quadrant of size 2k x 2k, hence, at level n - k, where k: 0 5 k I n, consists of n - k quaternary digits followed by k markers, denoted by “X.” A pixel is a quadrant at level n. With resolution 2, for example, the codes of all quadrants at all levels are as shown below.
xx n level 0
level 2 /
84
MARLOES
L. P. VAN
LIEROP
For a quadrant q, the NW, NE, SW, SE quadrants of q will be denoted by NW( q ), NE(q), SW(q), and SE(q), respectively. If q and r are different quadrants that intersect, then either q encloses r, denoted by q 3 Y, or r encloses q. Examples of this relation are 2x220 xX12x xx3 21, where the quadrants are represented by their codes. Note that “ 1 ” is a partial order on the quadrants. If q and r are quadrants that do NOT intersect, their codes may be ordered lexicographically; this lexicographical order of the codes of two disjoint quadrants is denoted with the symbol “ -C “. Note that “ < ” is a partial order on the quadrant codes. Examples of this relation are 21 < 23 21 < 31 ox<2x 01 -c 2x. “ < ” induces a partial order on the quadrants themselves, which will also be denoted by “ < .” For two quadrants q and r, q is said to precede r, iff q < r. The algorithms described in this paper are based on the following properties of “ < Y’ and L‘ , 99:
holds:
For any two quadrants p and q, exactly
one of the following
relations
P = 4lP < 4lP ’ 4lP = 4lP 3 4 As a consequence, for any pixel p and any quadrant q, exactly one of the following relations holds: P = 4lP < 4lP ’ 4/P = 4. -
For any quadrant p,
holds. Example: 20 < 21 < 22 -c 23. - For any quadrants p, q, and r, where p < q, both q 1 r implies p < r
and p 3 r implies r < q
TRANSFORMATIONS
85
ON LEAFCODES
hold. Example: if p = 1X and q = 2X then p -C q, and for r = 20, q I r and p < r, for r = 11, p 13 r and r < q. - For any quadrants p and q, where q 3 p, exactly one of the following relations holds: NW(q)
< pINW(d
’ plNW(d
= P.
Similarly, exactly one of the following holds: SW
’ Prwd
’ PIWd
= P.
Example: if q = XX and p = 31 then q 1 p, NW(q) = OX, and OX < p. Also, SE(q) = 3X, and 3X I p. Since the leaves of a semi-quadtree represent mutually disjoint quadrants, induces a TOTAL order on these quadrants. We define the black leafcode semi-quadtree as the ascending sequence of the codes of all black leaves. The leafcode of a semi-quadtree is defined in a similar way. The black and leafcodes of all semi-quadtrees in Fig. 2 are shown in the table below. Fig. 2
Black leafcode
“ < ” of a white
white
White leafcode
f:
11,13,2X, 31,32,33
Ii
11,13,20,21,22,23,31,32,33
ox, 10,12,30 OO,Ol, 02,03,10,12,30 ox, 10,12,30 OO,Ol, 02,03,10,12,30
One may wonder why the leafcode is defined as an ascending sequence instead of an arbitrary sequence. The reason is that sorting is required whenever compaction is needed; hence we might as well prescribe the leafcode to be ascending. Besides, by this prescription, superposition and intersection of leafcodes turn out to be simple merge operations [8]. Whenever we speak of the leafcode of a semi-quadtree, without mentioning its color, the remark concerns both the black and white leafcode. When speaking of the leafcode of a picture, the leafcode of the quadtree of the picture is meant. Hence, the black leafcode of the picture of Fig. la is 11,13,2X,
31,32,33,
and its white leafcode is ox, 10,12,30.
86
MARLOES
L. P. VAN
LIEROP
Note that each node of a semi-quadtree corresponds to a (possibly empty) contiguous subsequence of a leafcode of the semi-quadtree. In the quadtree of Fig. 1, for instance, the root of the tree corresponds with the entire sequences of the black and white leafcodes; NW(root) with an empty sequence in the black leafcode, and the sequence OX in the white leafcode; NE( root) corresponds with the sequence 11.13 in the black leafcode, and with the sequence lo,12 in the white leafcode, and so forth. 3. THE
REVERSING
ALGORITHM
A black-and-white picture may be represented by its leafcode. Given a leafcode, the reverse leafcade is the leafcode of the complementary color; hence, the reverse leafcode of the black one is the white leafcode, and the reverse leafcode of the white one is the black one. If a quadtree has more black than white leaves, it is more compactly represented by its white leafcode than by its black one. In this section an algorithm is presented that generates the reverse leafcode of a given leafcode; it is also meant as an introduction to the algorithm that generates the leafcode of a geometrically transformed picture from the leafcode of the original picture. Given a leafcode of a picture, we want to derive the reverse leafcode. The algorithm presented below is based on the following observations: - For any leafcode. the subsequence corresponding to a node, is the concatenation of the four subsequences corresponding to its children in NW, NE, SW, SE-order. Hence, a recursive procedure will be used. - To compute the subsequence corresponding to a given node of the reverse leafcode, only that part of the input leafcode corresponding to that node. needs to be inspected. This implies that the input leafcode needs to be scanned only once. The recursive procedure reoert (qd, I, i/f) scans that part of the input leafcode that corresponds to quadrant qd (whose level is I), while the subsequence corresponding to qd of the reverse leafcode is generated. The var parameter iff is used to scan the input leafcode: at calling revert ( ), ilf indicates the beginning of the part to be scanned, at leaving, iif indicates the end of it. More precisely, at calling, ilf is the FIRST leaf of the input leafcode that equals qd or is enclosed by qd, or (in case that the part of the input leafcode associated with qd is empty) that is preceded by qd, i.e., (ilf c qd or ilf > qd) and any other leaf ff of the input leafcode that satisfies y c qd or ,f > qd is preceded by i/f. On return, ilf is the FIRST leaf of the input leafcode that is preceded by qd, i.e., ilf > qd and any other leaf If of the input leafcode that satisfies lf > qd, is preceded by i/f. If, at calling, i/f = qd, then quadrant qd is an element of that input leafcode, and hence not of the reverse leafcode. If, however, irf > qd, the part of the input leafcode associated with qd is empty, hence qd is part of the reverse leafcode. If, however, iff c qd, then qd is no leaf at all, but at least one of its descendants is part of the input leafcode (e.g., irf)and revert ( ) is called for the four children of qd in NW, NE, SW, SE-order. We assume that the input leafcode is terminated by a special leaf called EOF, and that every quadrant precedes EOF. Furthermore, for convenience, we assume that codes of quadrants are represented by arrays of length n, where index 0 refers to the most significant digit of the code, and where each array value is an element of (0, 1,2,3, X}. The notation used for the programs is Pascal.
TRANSFORMATIONS
ON
87
LEAFCODES
type val = (zero, one, two, three, X); quadrant = array[O..“n - l”] of ual; var root, leaf: quadrant; k: integer; procedure reuert( qd: quadrant; I: integeryarilf: quadrant); var i: val; begin if “qd c iif ” then { qd is not a leaf of the input leafcode} “send qd to the output ” else if “qd = ilf ” then { qd is not a leaf of the reverse leafcode} ilf :=“next leaf of the input leafcode” else {“qd I ilf ” and qd is an internal node of the quadtree
of the
picture} for i := zero to three do begin qd[l] := i; revert (qd, I + 1, it’f) end end
The above procedure is invoked in the following way: leaf := “J;rst leaf of he input leafcode”; for k := 0 to n - 1 do root[k] := X, revert (root, 0, leaf );
The above quadtree. In Note that input is the compaction,
algorithm requires O(N) time, where N is the number of nodes of the Section 7, some execution times of procedure revert ( ) are shown. the above algorithm generates the leafcode of a quadtree even if the leafcode of a semi-quadtree. Hence, this algorithm may be used for by applying it twice. 4. GEOMETRICAL
TRANSFORMATION
ON
LEAFCODES
Given a leafcode representing a picture, and a non-singular affine transformation, the leafcode of the transformed picture is to be be generated. We want to take advantage of the fact that the input leafcode is an ordered sequence, and we want to avoid a sorting step for the output leafcode. Parts of the transformed picture that fall outside the 2” x 2” pixel area are clipped. For convenience, we will assume from now on, that the input and output leafcodes are black leafcodes. Furthermore, when speaking about input leaves, the elements of the input leafcode are meant, i.e., the black leaves of the input quadtree. The only way to avoid a sorting step is to generate the leaves of the output leafcode in the correct order. This is achieved by constructing the output quadtree in nearly the same way as we have done in the reversing algorithm. The only difference concerns the second observation for the reversing algorithm: the quadtree of a transformed picture will usually differ from the quadtree of the input picture; hence, to compute the subsequence corresponding to a given node of the output quadtree, it will not be sufficient to inspect only the subsequence corresponding to the same node of the input leafcode. Several parts of the input leafcode will have to be inspected, and therefore the input leafcode needs to be stored.
88
MARLOES
L. P. VAN
(4
LIEROP
lb)
(cl FIG.
3.
A picture
before
(a) and after
(b) a translation,
together
with
the latter
semi-quadtree.
A sufficient condition for a node of the output quadtree to be a black leaf, is the existence of a black leaf of the input quadtree whose transformed quadrant encloses the quadrant of the investigated output node. This is the same as the existence of an input leaf that encloses the inversely transformed quadrant of the investigated output node. If q is a quadrant, then the quadrangle that results when the inverse transformation is applied to q, is referred to as inv(q). That the above formulated condition is not a necessary condition, is illustrated by Figs. 3a and b, where the output quadtree has one black leaf (viz., 1X), but where its inverse is not enclosed by any of the input leaves (which are 03, 12, 21, 30). We will use the above sufficient condition to construct the output tree, which will be a semi-quadtree of the picture. Thus, when investigating a node, say q, of the output tree, the input leaves are examined to enclose inv(q): - If one of the input leaves encloses inv(q), q is a black leaf of the output semi-quadtree. - If none of the input leaves intersects inv(q), q is a white leaf of the output semi-quadtree. - If an input leaf intersects inv(q), but does not enclose it, q may have black descendants, and therefore its four children are recursively investigated. The tree constructed in this way need not be the quadtree of the transformed picture, because a node may have four black children. This is illustrated by the tree in Fig. 3c, which is the output semi-quadtree of the transformed picture of Fig. 3a. Thus, compaction is needed to obtain the leafcode of the picture; it is discussed in Section 6. An additional criterion is needed for the investigation of output pixels, because these quadrants do not have children. We therefore have established the following criterion for pixels: a leaf of the output tree associated with a pixel is black iff its center is enclosed by a transformed input leaf, or, in other words, iff an input leaf encloses the inversely transformed center of the pixel. For this purpose the boolean function sear&pixel(q) is designed, which reports whether or not pixel q is a black leaf of the output semi-quadtree. For quadrants other than pixels, the procedure seurch(q, ins, inters) has been constructed, which assigns the vaIue true to the boolean var parameter ins if inv( q) is enclosed by an input leaf, or otherwise, assigns
TRANSFORMATIONS
ON
LEAFCODES
89
the value true to the booleanvar parameterinters if an input leaf intersectsinv(q). Hence,the postconditionP of search(q, ins, inters) is P:
(ins and inv(q) is enclosed by an input leaf) or (-,ins and inters and inv(q) intersects an input leaf, but is not enclosed by it) or (7 ins and -, inters and inv( q) intersects none of the input leaves)
Search( ) and seurchpixel( ) are used in the recursive procedure buildtree(q, I), which generates the subsequence, corresponding to the quadrant q of the output leafcode, where I is the level of q in the semi-quadtree. The overall structure of buildtree( ) is given below. procedure buildtree( q: quadrant; 1:integer); var i: val;
inside, intersect: boolean; begin if
I = n then
{q is a pixel} begin if searchpixel
then
{the inversely transformed center of q is enclosed by an input leaf} “send q to the output” end
else { 1 < n, hence q is not a pixel} begin search( q, inside, intersect); { Because of P: ((inside and inv(q) is enclosed by an input leaf) or (,inside and intersect and inv(q) intersects an input leaf, but is not enclosed by it) or (-, inside and 7 intersect aud inv( q) intersects none of the input leaves)) 1 if inside then “send q to the output ” else if intersect then for i := zero to three do begin q[1] := i; buildtree(q, 1 + 1) end end end
The above procedure is invoked in the following way: buildtree ( root, 0) , where root has the same value as in the reversing algorithm. For discussion of seurch( ) and seurchpixel( ), we assume that the input leafcode is stored in a global array called inp. As mentioned above, the input leafcode is an ascending sequence. This property is exploited in search( ) and searchpixef( ), where a binary search is used to find a particular input leaf. Actually, in searchpixel( q) the
90
MARLOES
L. P. VAN
LIEROP
code of the pixel that encloses the inversely transformed center of q is computed, and then a binary search is performed on inp in the following way: var inp: array[O..“N - l”] of quadrants; {N is the number of input leaves} function searchpixel( q: quadrant): boolean; var cntrpix: quadrant; I, m, r: integer; begin if “the inversely transformed center of q is enclosed by the 2” X 2” pixel area ” then begin cntrpix :=“code of the pixel enclosing the inversely transformed ten ter of q “; 1 := 0; r := N; 1*) m := (1 + r)div 2; {l inp[i]“) } while (“cntrpix < inp[m]” or “cntrpix > inp[m]“) do begin if “cntrpix < inp[m]” then r := m {Q} else {“cntrpix > inp[m]“} l:= m + 1; {Q} m := (I + r)div 2 (I I m I r) end; (“cntrpix c inp[m]” or I = r} searchpixel := “cntrpix c inp[ ml”
and (I < r )
end else searchpixel := false end
Similarly, in seurch(q, ins, inters) the code of the smallest quadrant that encloses inv(q), if any, is computed, followed by a binary search: procedure seurch( q: qwdrunt;var ins, inters: boolean); var sm: quadrant; 1, m, r: integer; begin if “inv(q) is enclosed by the 2” x 2” pixel urea ” then begin sm :=“code of the smallest quadrant that encloses inv(q)“;
1 := 0; r := N; m := (I + r)div 2; {ls m
where Q: (for all i:O 2 i -c 1 or r 5 i < N: “sm < inp[m]”
or “sm > inp[m]“)
1 while (“sm < inp[m]” or “sm > inp[m]“) begin if “sm < in&m]” then r := m {Q}
a4 (1 < r) do
TRANSFORMATIONS
ON
91
LEAFCODES
else {“sm > inp[ ml”} I := m + 1; {Q} m:=(l+r)div2{1~m
end; { “sm c inp[ ml” or “sm I inp[ ml” or 1 = r } if “sm c inp[m j” then { inv( q) is enclosed by an input leaf} ins := true
else if “sm I inp[m]” then begin ins := false; . . . end else { inv( q) intersects none of the input leaves} begin ins := false; inters := false end end else if “inv(q) does not intersect the 2” x 2” pixel area” then begin ins := false; inters := false end else {inv(q) is partly enclosed by the 2” x 2” pixel area} begin ins := false; . - * end end
(i)
(ii)
The situation in line (i) and (ii) is such that we know that inv(q) is not enclosed by any input leaf, hence ins := false. However, we do not yet know whether inv(q) intersects an input leaf. In Fig. 4, for example, a configuration is shown satisfying the condition in (i), but where inv(q) does not intersect inp[m]. A correct action would be to investigate whether inp[m] and inv(q) do intersect. Because the solution of this problem requires many computations, we have chosen for another strategy: instead of determining whether inp[m] and inv(q) do intersect, we split up q into its four subquadrants, and subsequently investigate each of them. This means that buiIdtree( ) has to be invoked for each subquadrant of q, and this is achieved by assigning true to the boolean inters in line (i). The same strategy is used in line (ii): instead of determining whether inv(q) intersects an input leaf, q is partitioned and its subquadrants are subsequently investigated. As a consequence of this strategy, the postcondition P of search(q, ins, inters) has to be changed into Pl: Pl: ((ins and inv( q) is enclosed by an input leaf) or (7 ins and inters and inv( q) MAY intersect an input leaf, but is not enclosed by it) or (-, ins and 7 inters and inv( q) intersects none of the input leaves))
FIG. 4.
inp[m]
is enclosed
cl
sm
•I
inv(q)
El
inpcml
by sm, but does not intersect
inv(q).
92
MARLOES
L. P. VAN
LIEROP
The ellipses in both line (i) and (ii) are to be replaced by inters := true.
Note that, because of this strategy, the output tree may become more branched unnecessarily, resulting in more compaction time. 5. REFINEMENTS
An important property mentioned in Section 2 is, that for two quadrants p and q. where p precedes q, all descendants of p precede q as well. This observation is useful in reducing the length of the array part on which the binary search is applied: if the values of 1 and r in search( ) are stored and carried over to buifdtree( ), the part of the array on which the binary search has to be applied for the sons of q may be reduced. For this purpose the local variables I and r of seurch( ) are entered in the parameter list of search( ); they become var parameters that define, at calling, the array part on which the binary search is to be performed. In order to carry over these values to the recursive calls of buildtree( ), the parameter list of build&ree( ) is extended with two value parameters left and right. The local variables 1 and r of searchpixel( ) are entered in its parameter list as value parameters; they now define the array part on which the binary search in searchpixel( ) has to be performed. Thus, a more sophisticated version of buildtree( ), search( ), and searchpixef( ) might look like this: procedure buildtree(q: quadrant; l,ieft, right: integer); var i:ual; inside, intersect: boolean; It, rt: integer; begin if I = n then begin if searchpixel(q,left, right) then “send q to the output” end else begin It := left; rt := right; search(q, inside, intersect, It, rt ); {Pl and It I rt and (for all i: 0 5 i < It or rt < i < N: inp[i] does not intersect inv(q)) > if inside then “send q to the output” else if intersect then for i := zero to three do begin q[l] := i; buildtree(q, 1 + 1, it, rt) end end end procedure seurch( q: quadrant; var ins, inters: boolean; var 1, r: integer); var sm : quadrant; m : integer ;
TRANSFORMATIONS
ON
93
LEAFCODES
begin ins := false; inters := false; { 1~ r and Q, where Q: (for all i:O I i < I or r < i < N: inp[i] does not intersect inv(q)) > if “inv(q) is enclosed by the 2” X 2” pixel region ” then begin sm :=“‘code of the smallest quadrant that encloses inv( 4)“; m := (I + r)div 2; while (“sm -C inp[m]” or “sm > inp[m]“) and / < r do begin if “sm < inp[ ml” then r := m {Q} else {“sm > inp[m]“} f := m + 1; (Q} m := (f + r)div 2 end; {“sm c inp[ ml” or “sm I inp[m]” or 1 = r } if “sm c inp[ ml” then ins := true else if “sm IJ inp[m]” then inters := true end else if “inv(q) intersects the 2” X 2” pixel region ” then inters := true {Pl and Q and I I r } end The changes in searchpixel( ) concern the parameter list, which becomes function searchpixel( q: quadrant; I, r: integer), and line (*), which is omitted. The first call of buildtree( buildtree(root, 0, 0, N), where root has the same value as before. 6. SOME
FURTHER
) becomes
DETAILS
In this section we discuss the compaction, how the code of the pixel enclosing a given point is computed, and how the code of the smallest quadrant that encloses an inverse quadrant is computed. Before explaining how the code of the pixel enclosing a given point is computed, we describe how the 2” x 2” pixels are embedded in a coordinate system. We assume that the lower left comer of the lower left pixel has coordinates (O,O), and the upper right comer of the upper right pixel has coordinates (2”, 2”). Thus, the pixel width is 1. In procedure searchpixef( ), the following line appears: cntrpix := “code of the pixel enclosing the inversely transformed center of q “, where q and cntrpix are quadrants. Actually, the coordinates, which are reals, of the inversely transformed center of q are computed, and these coordinates are truncated
94
MARLOES
L. P. VAN
LIEROP
to become the coordinates of the lower left corner of the pixel enclosing the inversely transformed center of q. Subsequently, these integer coordinates are passed to procedure encode ( x, y : integer ;var cntrpix : quadrant
j,
which assigns to cntrpix the code of the pixel whose lower left corner is (x, y). In the procedure search( ), the following line appears: sm := “code of the smallest quadrant that encloses inv( 4)“.
For this purpose, the procedure smallest( q: quadrant;var sm: quadrant) is designed. which assigns to sm the code of the smallest quadrant that encloses inv(q). We have defined inv(q) as the quadrangle that results when the inverse transformation is applied to quadrant q. This quadrangle is specified by its four vertices, which are the inversely transformed corners of q: type coord = record x, y: real end; quadrangle = array[O..3] of coord; square = record x, y, d:integer end;
{a square is specified by the coordinates of its lower left corner, and its width} procedure smallest( q: quadrant;var sm: quadrant); var smaller: boolean; pol: quadrangle; sq: square; begin “pof := inv( q)“;smaffer := true; sq.x
:=
0;
sq.y
:=
0;
,q.d:="2"";
while smaller and (sq.d > 1) do begin if “pol is not enclosed by any of the subquadrants of sq ” then smaller := false else sq := “subsquare of sq that encloses ~01”; end; sm := “code of sq ” end
A quadrangle is enclosed by a square iff all the vertices of the quadrangle are enclosed by the square, which is easy to find out. As said before, the result of the transforming algorithm is the leafcode of a semi-quadtree, and hence a compaction step is required. Instead of using the reversing algorithm to perform compaction, we have created a filter whose input is a leafcode of a semi-quadtree, and whose output is the leafcode of the associated quadtree. This filter uses a procedure called compact(q: quadrant), which manages a global queue in which the black leaves that are candidates for compaction are stored. A black leaf is a can&Me for compact& if its preceding siblings are black too, i.e., if they are in the queue. Note that a leaf whose least significant digit is zero (a NW leaf), is always a candidate for compaction, because such leaf is the first Gb&ng. A
TRANSFORMATIONS
ON
LEAFCODES
95
black leaf q is a candidate for clearing the queue, if the queue is not empty and if q is not a proper successor of the last queue element, i.e., if there exists a quadrant p such that q > p > last queue element. The strategy used in compact( ) is as follows: - if the queue is empty, and q is a candidate for compaction, q is placed in the queue; - if the queue is empty, and q is not a candidate for compaction, q is sent to the output; - if the queue is not empty, and q is a candidate for clearing the queue, the queue is cleared (i.e., the elements of the queue are sent to the output in the correct order), and q is put in the queue or sent to the output, depending respectively on q being a candidate for compaction or not; - if the queue is not empty, and q is not a candidate for clearing (hence, q is a proper successor of the last queue element), q is put in the queue (if q is not a fourth sibling, i.e., not a SE leaf), or the last three elements of the queue are removed, and compact( ) is called for the father of q (if q is a fourth sibling). Note that the maximum number of queue elements is 3n, namely 3 siblings for each non-zero level. If only the greatest sibling that is a candidate for compaction is stored, the maximum number of queue elements even reduces to n. Hence, the internal storage requirements of compact( ) are very modest. The time complexity of procedure compact( ) is O(1). Since it is called for each node of the semi-quadtree, the compaction process requires O(M) time, where M is the number of nodes of the semi-quadtree to be compacted. 7. CONCLUDING
REMARKS
The algorithms described above have been implemented in the language C, and executed on a local microcomputer based on the M68000. The quadrant codes were not implemented as arrays, but, in a more compact manner, as integers (32 bits), where the four least significant bits were used to specify the level of the quadrant, and the 28 most significant bits were used pairwise to represent the quaternary digits. This way it is possible to deal with resolutions up to 14. We have also implemented the algorithm to perform transformations on leafcodes as described by Gargantini [8], using heapsort for the sorting step, and the same compaction algorithm as described above. The Gargantini algorithm requires 0( Ni + N2 x log( N,)) time, where Ni is the number of pixels enclosed by the input leaves, and N2 is the number of pixels enclosed by the output leaves. We will derive a time complexity for our algorithm. Let M be the number of nodes of the output semi-quadtree, i.e., the semi-quadtree that has not been compacted yet. The procedure buildtree( ) is called once for each node of the output semi-quadtree; within the procedure buildtree( ), either search( ) or searchpixef( ) is called. Within the procedures searchpixel( ) and search( ), a binary search is performed on an array part of maximum length N, where N is the number of input leaves, and thus the worst-case time complexity of the binary search is O(log(N)). Furthermore, in search( ) the procedure smaller( ) is called, which has worst-time complexity 0(n). Hence, the procedure search( ) has worst-case time complexity @log(N) + n), and searchpixel( ) O(log(N)). H ence, the worst-case time complex-
96
MARLOES
L. P. VAN LIEROP TABLE
# Input pixels
Time Gargantini
1024
Time van Lierop
# Nodes in output semi-quadtree
15.1
1.8
16384 32768 49152 64512"
1
43.1 91.6 136.3 144.5
75093 87381 87381 87381 75093
112.5 111.7 126.5 102.6
“(# input leaves is 9216) Note. # input leaves = # input pixels, # output pixels - # input pixels, times in seconds.
ity of the transforming algorithm is O(M X (log(N) + n)), where n is the resolution of the picture, N the number of input leaves, and M the number of nodes of the output semi-quadtree. Since the compaction requires O(M) time, as we have seen in the previous section, the combination of both algorithms is O(M x (n + log(N)) + M) = O(M x (n + log(N))). As far as storage requirements are concerned, the algorithm described in this paper compares favourably with Gargantini’s algorithm: we need the internal storage of all input leaves, where Gargantini needs the storage of all PIXELS enclosed by the output leaves. Remember that storage space reduction was the very reason to introduce leafcodes. See Tables 1 and 2 for the execution times of both the algorithm of Gargantini and the one described in this paper. These execution times include the time for compaction. The resolution of the picture transformed is 8, where the pictures are chessboard-like. The transformation performed was in all cases a translation of one pixel both to the right and to the south. In Table 1, the input leafcodes are such that each leaf is a pixel, i.e., the number of input leaves equals the number of input pixels. In Table 2, the number -of input leaves is as small as possible. Clearly, the execution times of the Gargantini algorithm depend on the number of black input and output pixels, whiie the execution times of our algorithm are heavily dominated by the number of nodes in the output semi-quadtree. Comparing the execution times of both algorithms, we may conclude that the Gargantini algorithm has the best
TABLE 2 # Input pixels
Time Gargantini
Time van Lierop
# Nodes in output semi-quadtree
# Input leaves
1024 16384 32768 49152 64512
1.5 33.1 69.5 107.8 143.2
4.7 4.2 5.5 4.9 6.0
5329 3957 3957 3957 5329
1 1
Note.
# output pixels - # input pixels, times in seconds.
2 3 9
TRANSFORMATIONS
ON TABLE
3
# Input leaves
# Input pixels
Time revert
1024 1024 32768 3072 960
1024 16384 32768 49152 65412
1.6 0.6 10.0 0.7 0.1
Note. Times
in seconds.
TABLE # Input leaves
4
Time Gargantini
32768 8192 2048 512 128 32 8 2 Note. # - # input
97
LEAFCODES
91.6 85.3 73.3 71.0 70.5 70.0 69.9 69.5 input pixels pixels, times
= 32768, # in seconds.
Time van Lierop 111.7 99.7 19.7 50.7 28.9 14.4 7.1 5.1 output
pixels
performance for input pictures consisting of a rather small number of black pixels. For input pictures consisting of a very large number of black pixels, the algorithm of Gargantini may be used for the reverse leafcode of the picture. The execution times of our procedure reuert( ) are shown in Table 3. Our algorithm has the best performance for pictures consisting of a considerable nut&g& black pixels, and a relatively small number of leaves. In Table 4, the execution times are shown for pictures consisting all of the same number of black pixels, but varying in the number of leaves. Finally, we like to emphasize that our algorithm is applicable for all transformations that map squares onto convex quadrangles, while the Gargantini algorithm can be used only for translations over an integral number of pixels, and rotations of 90 or 180 degrees. ACKNOWLEDGMENTS
The reversing algorithm was designed by Tom Verhoeff, who has also implemented the Gargantini algorithm. REFERENCES 1. A.
Klinger and C. R. Dyer, Experiments in picture representation using regular decomposition, Comput. Gruphics Image Process. 5, 1916, 68-105. 2. A. Klinger, Patterns and search statistics, in Optimizing Methods in Stutistics (J. S. Rustagi, Ed.), pp. 303-337, Academic Press, New York, 1971. 3. H. Samet, The quadtree and related hierarchical datastructures. ACM Comput. Suruq~s 16, 1984, 187-260.
MARLOES
98
L. P. VAN LIEROP
4. G. M. Hunter and K. Steiglitz, Linear transformation of pictures represented by quad trees, Compur. Gruphics
Image
Process.
10, 1979,
289-296.
5. M. Schneier, Calculations of geometric properties using quadtrees, Cornput. Graphics Imuge Process. 16, 1981, 296-302. 6. F. J. Peters, An Algorithm for Transformations of Pictures Represented by Quadtrees, internal report of the Dept. of Mathematics and Computing Science, Eindhoven University of Technology, 19X4. 7. M. A. Oliver and N. E. Wiseman, Operations on quadtree-encoded images, Conrpur. J. 26, 1983, 83-91. 8. 1. Gargantini, An effective way to represent quadtrees, Comm. ACM 25, 1982, 905-910. 9. I. Gargantini, Translation rotation, and superposition of linear quadtrees, Int/.-J. Mutt-Mach. Stud 18, 1983, 253-263. 10. D. Meagher, Geometric modeling using octree encoding, Conlpur. Gruphics Imuge Process. 19, 1982, 129-147.
VISION,
GRAPHICS,
AND IMAGE
PROCESSING
33, 81-98 (1986)
Geometrical Transformations on Pictures Represented by Leafcodes MARLOES L. P. VAN LIEROP* Depurtment
of Mathematics
and Computing Science, Eindhoven 5600 MB Eindhoven, The Netherlunds
University
of Technoloa,
Received January 22,1985; revised June 18,1985 Quadtrees that represent pictures composed of 2” X 2” pixels, may be economically implemented using linear tree representations. An example of such a representation is the leafcode, which is an ordered sequence of the encoded leaves of the quadtree. This paper describes how the leafcodes of geometrical transformed pictures can be derived from the original leafcodes. In addition, we present some execution times. 0 19X6 Academic Press. Inc. 1. INTRODUCTION
A quadtree is a tree of which each node is either a leaf or an internal node with four children. A quadtree may be associated with a picture that is composed of 2” x 2” pixels in the following way. If all pixels of the picture have the same color, the quadtree consists of one node only; the color of the node is the color of the pixels. If the pixels do not have the same color, the quadtree consists of one node with its four subtrees, where the subtrees are the quadtrees of the four subpictures, obtained by partitioning the former picture into its four quadrants. The quadtree that results, is called the quadtree of the picture. Figure 1 shows a black-and-white picture composed of 2’ x 22 pixels, together with a representation of its quadtree. Quadtrees were introduced as a form of raster picture encoding that could reduce storage space requirements, while exploiting two-dimensional coherence [1, 21. For a comprehensive study of the history and use of quadtrees in picture encoding, we refer to Samet [3]. Some operations on quadtrees are easy to perform, like superposition and intersection, others are difficult. Troublesome operations are geometrical transformations, because the resulting quadtree may differ enormously from the original one. Quadtrees may be implemented as pointer structures or as linear structures, where pointer structures, in general, require more space but provide faster access than linear structures. Various papers [4, 5, 61 have been written on geometrical transformations of pictures that are encoded as pointer-structured quadtrees. Since quadtrees originally were introduced to reduce storage space requirements, we are interested in space-efficient implementations of quadtrees. Therefore we concentrate on linear structures, and look for efficient algorithms that perform geometrical transformations on pictures represented by linear quadtrees. In [7], Oliver and Wiseman describe a linear representation of the quadtree called the treecode, which is based on the depth-first traversal of the quadtree. For black-and-white pictures, Gargantini introduced in [8] a linear representation of the quadtree called the leafcode, which is an ordered sequence of encoded BLACK *These investigations were supported by the Netherlands TechnicaI Foundation (SW). 81 0734-189X/86
$3.00
Copyright 0 1986 by Academic Press. Inc. All rights of reproduction in any form reserved.
82
h4ARLOES L. P. VAN LIEROP
(b)
0 . 3
internal node black leaf white leaf
FIG. 1. A picture and its quadtree.
leaves. She claims that her way of encoding quadtrees reduces the storage space to one-third of the space required by pointer structures. Superposition and intersection of pictures encoded by leafcodes turn out to be simple merging operations on ordered sequences. Geometrical transformations, however, remain troublesome. In the algorithm proposed by Gargantini [9], which is restricted to translations of the picture over an integral number of pixels and rotations over 90 and 180 degrees, the new code of each black PIXEL (rather than leaf) in the input tree is computed, and afterwards a sorting and compacting step is performed. In no way does she use the fact that the input leafcode is an ordered sequence. In this paper an algorithm to perform geometrical transformations on pictures represented by leafcodes is described that works the other way round: the leafcode of the output tree is constructed leaf by leaf in the correct order, thus avoiding a sorting step, while exploiting the fact that the input leafcode is an ordered sequence. The algorithm supports all non-singular affine transformations, i.e., all invertible linear transformations, translations, and combinations thereof. The basic strategy of the algorithm resembles the algorithm for translations of octree encoded 3D objects, described by Meagher in [lo]. In Section 2 the way we notate leafcodes is described; Section 3 discusses the construction of the reverse leafcode, and is meant as an introduction to Section 4, which describes how the leafcode of a transformed picture is constructed out of the leafcode of the original picture. In Section 5, the algorithms of Section 4 are modified in order to increase efficiency. Section 6 discusses some details of the algorithms, and Section 7 contains some final remarks, and tables with execution times of implementations of both Gargantini’s algorithm and the one described in this paper. 2. THE NOTATION
OF LEAFCODES
The pictures we consider, are composed of 2” x 2” pixels, where each pixel is either black or white; n is called the re&&onof the pie&~. The levet of a n&e is defined by - the level of the root is 0; -
the level of a child node equals 1 + the level of ifs parent node.
The definition of the quadtree of a picture implies that an internal node is associated with a non-uniformly colored quadrant. A tree that has uniformly colored quadrants partitioned as well, is called a se4&
TRANSFORMATIONS
ON
83
LEAFCODES
(4 lb) (c) (d) FIG. 2.
All semi-quadtrees
of Fig. la.
quadtree of a picture is also a semi-quadtree. Several semi-quadtrees may be associated with one picture. In Fig. 2, all semi-quadtrees associated with the picture of Fig. 1 are shown. The conversion of a semi-quadtree into the quadtree is called compaction. The leafcode as proposed by Gargantini is an ordered sequence of all encoded black leaves. The code of a pixel is a sequence of quaternary digits reflecting successive quadrant subdivisions, according to the following convention: 0, 1,2,3 for the NW, NE, SW, SE subquadrants respectively, i.e.,
The most significant digit of the code refers to the first subdivision of the picture, the least significant digit to the last subdivision. Since a pixel is the result of n subdivisions, n digits are required for its code. The code of a quadrant of size 2k x 2k, hence, at level n - k, where k: 0 5 k I n, consists of n - k quaternary digits followed by k markers, denoted by “X.” A pixel is a quadrant at level n. With resolution 2, for example, the codes of all quadrants at all levels are as shown below.
xx n level 0
level 2 /
84
MARLOES
L. P. VAN
LIEROP
For a quadrant q, the NW, NE, SW, SE quadrants of q will be denoted by NW( q ), NE(q), SW(q), and SE(q), respectively. If q and r are different quadrants that intersect, then either q encloses r, denoted by q 3 Y, or r encloses q. Examples of this relation are 2x220 xX12x xx3 21, where the quadrants are represented by their codes. Note that “ 1 ” is a partial order on the quadrants. If q and r are quadrants that do NOT intersect, their codes may be ordered lexicographically; this lexicographical order of the codes of two disjoint quadrants is denoted with the symbol “ -C “. Note that “ < ” is a partial order on the quadrant codes. Examples of this relation are 21 < 23 21 < 31 ox<2x 01 -c 2x. “ < ” induces a partial order on the quadrants themselves, which will also be denoted by “ < .” For two quadrants q and r, q is said to precede r, iff q < r. The algorithms described in this paper are based on the following properties of “ < Y’ and L‘ , 99:
holds:
For any two quadrants p and q, exactly
one of the following
relations
P = 4lP < 4lP ’ 4lP = 4lP 3 4 As a consequence, for any pixel p and any quadrant q, exactly one of the following relations holds: P = 4lP < 4lP ’ 4/P = 4. -
For any quadrant p,
holds. Example: 20 < 21 < 22 -c 23. - For any quadrants p, q, and r, where p < q, both q 1 r implies p < r
and p 3 r implies r < q
TRANSFORMATIONS
85
ON LEAFCODES
hold. Example: if p = 1X and q = 2X then p -C q, and for r = 20, q I r and p < r, for r = 11, p 13 r and r < q. - For any quadrants p and q, where q 3 p, exactly one of the following relations holds: NW(q)
< pINW(d
’ plNW(d
= P.
Similarly, exactly one of the following holds: SW
’ Prwd
’ PIWd
= P.
Example: if q = XX and p = 31 then q 1 p, NW(q) = OX, and OX < p. Also, SE(q) = 3X, and 3X I p. Since the leaves of a semi-quadtree represent mutually disjoint quadrants, induces a TOTAL order on these quadrants. We define the black leafcode semi-quadtree as the ascending sequence of the codes of all black leaves. The leafcode of a semi-quadtree is defined in a similar way. The black and leafcodes of all semi-quadtrees in Fig. 2 are shown in the table below. Fig. 2
Black leafcode
“ < ” of a white
white
White leafcode
f:
11,13,2X, 31,32,33
Ii
11,13,20,21,22,23,31,32,33
ox, 10,12,30 OO,Ol, 02,03,10,12,30 ox, 10,12,30 OO,Ol, 02,03,10,12,30
One may wonder why the leafcode is defined as an ascending sequence instead of an arbitrary sequence. The reason is that sorting is required whenever compaction is needed; hence we might as well prescribe the leafcode to be ascending. Besides, by this prescription, superposition and intersection of leafcodes turn out to be simple merge operations [8]. Whenever we speak of the leafcode of a semi-quadtree, without mentioning its color, the remark concerns both the black and white leafcode. When speaking of the leafcode of a picture, the leafcode of the quadtree of the picture is meant. Hence, the black leafcode of the picture of Fig. la is 11,13,2X,
31,32,33,
and its white leafcode is ox, 10,12,30.
86
MARLOES
L. P. VAN
LIEROP
Note that each node of a semi-quadtree corresponds to a (possibly empty) contiguous subsequence of a leafcode of the semi-quadtree. In the quadtree of Fig. 1, for instance, the root of the tree corresponds with the entire sequences of the black and white leafcodes; NW(root) with an empty sequence in the black leafcode, and the sequence OX in the white leafcode; NE( root) corresponds with the sequence 11.13 in the black leafcode, and with the sequence lo,12 in the white leafcode, and so forth. 3. THE
REVERSING
ALGORITHM
A black-and-white picture may be represented by its leafcode. Given a leafcode, the reverse leafcade is the leafcode of the complementary color; hence, the reverse leafcode of the black one is the white leafcode, and the reverse leafcode of the white one is the black one. If a quadtree has more black than white leaves, it is more compactly represented by its white leafcode than by its black one. In this section an algorithm is presented that generates the reverse leafcode of a given leafcode; it is also meant as an introduction to the algorithm that generates the leafcode of a geometrically transformed picture from the leafcode of the original picture. Given a leafcode of a picture, we want to derive the reverse leafcode. The algorithm presented below is based on the following observations: - For any leafcode. the subsequence corresponding to a node, is the concatenation of the four subsequences corresponding to its children in NW, NE, SW, SE-order. Hence, a recursive procedure will be used. - To compute the subsequence corresponding to a given node of the reverse leafcode, only that part of the input leafcode corresponding to that node. needs to be inspected. This implies that the input leafcode needs to be scanned only once. The recursive procedure reoert (qd, I, i/f) scans that part of the input leafcode that corresponds to quadrant qd (whose level is I), while the subsequence corresponding to qd of the reverse leafcode is generated. The var parameter iff is used to scan the input leafcode: at calling revert ( ), ilf indicates the beginning of the part to be scanned, at leaving, iif indicates the end of it. More precisely, at calling, ilf is the FIRST leaf of the input leafcode that equals qd or is enclosed by qd, or (in case that the part of the input leafcode associated with qd is empty) that is preceded by qd, i.e., (ilf c qd or ilf > qd) and any other leaf ff of the input leafcode that satisfies y c qd or ,f > qd is preceded by i/f. On return, ilf is the FIRST leaf of the input leafcode that is preceded by qd, i.e., ilf > qd and any other leaf If of the input leafcode that satisfies lf > qd, is preceded by i/f. If, at calling, i/f = qd, then quadrant qd is an element of that input leafcode, and hence not of the reverse leafcode. If, however, irf > qd, the part of the input leafcode associated with qd is empty, hence qd is part of the reverse leafcode. If, however, iff c qd, then qd is no leaf at all, but at least one of its descendants is part of the input leafcode (e.g., irf)and revert ( ) is called for the four children of qd in NW, NE, SW, SE-order. We assume that the input leafcode is terminated by a special leaf called EOF, and that every quadrant precedes EOF. Furthermore, for convenience, we assume that codes of quadrants are represented by arrays of length n, where index 0 refers to the most significant digit of the code, and where each array value is an element of (0, 1,2,3, X}. The notation used for the programs is Pascal.
TRANSFORMATIONS
ON
87
LEAFCODES
type val = (zero, one, two, three, X); quadrant = array[O..“n - l”] of ual; var root, leaf: quadrant; k: integer; procedure reuert( qd: quadrant; I: integeryarilf: quadrant); var i: val; begin if “qd c iif ” then { qd is not a leaf of the input leafcode} “send qd to the output ” else if “qd = ilf ” then { qd is not a leaf of the reverse leafcode} ilf :=“next leaf of the input leafcode” else {“qd I ilf ” and qd is an internal node of the quadtree
of the
picture} for i := zero to three do begin qd[l] := i; revert (qd, I + 1, it’f) end end
The above procedure is invoked in the following way: leaf := “J;rst leaf of he input leafcode”; for k := 0 to n - 1 do root[k] := X, revert (root, 0, leaf );
The above quadtree. In Note that input is the compaction,
algorithm requires O(N) time, where N is the number of nodes of the Section 7, some execution times of procedure revert ( ) are shown. the above algorithm generates the leafcode of a quadtree even if the leafcode of a semi-quadtree. Hence, this algorithm may be used for by applying it twice. 4. GEOMETRICAL
TRANSFORMATION
ON
LEAFCODES
Given a leafcode representing a picture, and a non-singular affine transformation, the leafcode of the transformed picture is to be be generated. We want to take advantage of the fact that the input leafcode is an ordered sequence, and we want to avoid a sorting step for the output leafcode. Parts of the transformed picture that fall outside the 2” x 2” pixel area are clipped. For convenience, we will assume from now on, that the input and output leafcodes are black leafcodes. Furthermore, when speaking about input leaves, the elements of the input leafcode are meant, i.e., the black leaves of the input quadtree. The only way to avoid a sorting step is to generate the leaves of the output leafcode in the correct order. This is achieved by constructing the output quadtree in nearly the same way as we have done in the reversing algorithm. The only difference concerns the second observation for the reversing algorithm: the quadtree of a transformed picture will usually differ from the quadtree of the input picture; hence, to compute the subsequence corresponding to a given node of the output quadtree, it will not be sufficient to inspect only the subsequence corresponding to the same node of the input leafcode. Several parts of the input leafcode will have to be inspected, and therefore the input leafcode needs to be stored.
88
MARLOES
L. P. VAN
(4
LIEROP
lb)
(cl FIG.
3.
A picture
before
(a) and after
(b) a translation,
together
with
the latter
semi-quadtree.
A sufficient condition for a node of the output quadtree to be a black leaf, is the existence of a black leaf of the input quadtree whose transformed quadrant encloses the quadrant of the investigated output node. This is the same as the existence of an input leaf that encloses the inversely transformed quadrant of the investigated output node. If q is a quadrant, then the quadrangle that results when the inverse transformation is applied to q, is referred to as inv(q). That the above formulated condition is not a necessary condition, is illustrated by Figs. 3a and b, where the output quadtree has one black leaf (viz., 1X), but where its inverse is not enclosed by any of the input leaves (which are 03, 12, 21, 30). We will use the above sufficient condition to construct the output tree, which will be a semi-quadtree of the picture. Thus, when investigating a node, say q, of the output tree, the input leaves are examined to enclose inv(q): - If one of the input leaves encloses inv(q), q is a black leaf of the output semi-quadtree. - If none of the input leaves intersects inv(q), q is a white leaf of the output semi-quadtree. - If an input leaf intersects inv(q), but does not enclose it, q may have black descendants, and therefore its four children are recursively investigated. The tree constructed in this way need not be the quadtree of the transformed picture, because a node may have four black children. This is illustrated by the tree in Fig. 3c, which is the output semi-quadtree of the transformed picture of Fig. 3a. Thus, compaction is needed to obtain the leafcode of the picture; it is discussed in Section 6. An additional criterion is needed for the investigation of output pixels, because these quadrants do not have children. We therefore have established the following criterion for pixels: a leaf of the output tree associated with a pixel is black iff its center is enclosed by a transformed input leaf, or, in other words, iff an input leaf encloses the inversely transformed center of the pixel. For this purpose the boolean function sear&pixel(q) is designed, which reports whether or not pixel q is a black leaf of the output semi-quadtree. For quadrants other than pixels, the procedure seurch(q, ins, inters) has been constructed, which assigns the vaIue true to the boolean var parameter ins if inv( q) is enclosed by an input leaf, or otherwise, assigns
TRANSFORMATIONS
ON
LEAFCODES
89
the value true to the booleanvar parameterinters if an input leaf intersectsinv(q). Hence,the postconditionP of search(q, ins, inters) is P:
(ins and inv(q) is enclosed by an input leaf) or (-,ins and inters and inv(q) intersects an input leaf, but is not enclosed by it) or (7 ins and -, inters and inv( q) intersects none of the input leaves)
Search( ) and seurchpixel( ) are used in the recursive procedure buildtree(q, I), which generates the subsequence, corresponding to the quadrant q of the output leafcode, where I is the level of q in the semi-quadtree. The overall structure of buildtree( ) is given below. procedure buildtree( q: quadrant; 1:integer); var i: val;
inside, intersect: boolean; begin if
I = n then
{q is a pixel} begin if searchpixel
then
{the inversely transformed center of q is enclosed by an input leaf} “send q to the output” end
else { 1 < n, hence q is not a pixel} begin search( q, inside, intersect); { Because of P: ((inside and inv(q) is enclosed by an input leaf) or (,inside and intersect and inv(q) intersects an input leaf, but is not enclosed by it) or (-, inside and 7 intersect aud inv( q) intersects none of the input leaves)) 1 if inside then “send q to the output ” else if intersect then for i := zero to three do begin q[1] := i; buildtree(q, 1 + 1) end end end
The above procedure is invoked in the following way: buildtree ( root, 0) , where root has the same value as in the reversing algorithm. For discussion of seurch( ) and seurchpixel( ), we assume that the input leafcode is stored in a global array called inp. As mentioned above, the input leafcode is an ascending sequence. This property is exploited in search( ) and searchpixef( ), where a binary search is used to find a particular input leaf. Actually, in searchpixel( q) the
90
MARLOES
L. P. VAN
LIEROP
code of the pixel that encloses the inversely transformed center of q is computed, and then a binary search is performed on inp in the following way: var inp: array[O..“N - l”] of quadrants; {N is the number of input leaves} function searchpixel( q: quadrant): boolean; var cntrpix: quadrant; I, m, r: integer; begin if “the inversely transformed center of q is enclosed by the 2” X 2” pixel area ” then begin cntrpix :=“code of the pixel enclosing the inversely transformed ten ter of q “; 1 := 0; r := N; 1*) m := (1 + r)div 2; {l
and (I < r )
end else searchpixel := false end
Similarly, in seurch(q, ins, inters) the code of the smallest quadrant that encloses inv(q), if any, is computed, followed by a binary search: procedure seurch( q: qwdrunt;var ins, inters: boolean); var sm: quadrant; 1, m, r: integer; begin if “inv(q) is enclosed by the 2” x 2” pixel urea ” then begin sm :=“code of the smallest quadrant that encloses inv(q)“;
1 := 0; r := N; m := (I + r)div 2; {ls m
where Q: (for all i:O 2 i -c 1 or r 5 i < N: “sm < inp[m]”
or “sm > inp[m]“)
1 while (“sm < inp[m]” or “sm > inp[m]“) begin if “sm < in&m]” then r := m {Q}
a4 (1 < r) do
TRANSFORMATIONS
ON
91
LEAFCODES
else {“sm > inp[ ml”} I := m + 1; {Q} m:=(l+r)div2{1~m
end; { “sm c inp[ ml” or “sm I inp[ ml” or 1 = r } if “sm c inp[m j” then { inv( q) is enclosed by an input leaf} ins := true
else if “sm I inp[m]” then begin ins := false; . . . end else { inv( q) intersects none of the input leaves} begin ins := false; inters := false end end else if “inv(q) does not intersect the 2” x 2” pixel area” then begin ins := false; inters := false end else {inv(q) is partly enclosed by the 2” x 2” pixel area} begin ins := false; . - * end end
(i)
(ii)
The situation in line (i) and (ii) is such that we know that inv(q) is not enclosed by any input leaf, hence ins := false. However, we do not yet know whether inv(q) intersects an input leaf. In Fig. 4, for example, a configuration is shown satisfying the condition in (i), but where inv(q) does not intersect inp[m]. A correct action would be to investigate whether inp[m] and inv(q) do intersect. Because the solution of this problem requires many computations, we have chosen for another strategy: instead of determining whether inp[m] and inv(q) do intersect, we split up q into its four subquadrants, and subsequently investigate each of them. This means that buiIdtree( ) has to be invoked for each subquadrant of q, and this is achieved by assigning true to the boolean inters in line (i). The same strategy is used in line (ii): instead of determining whether inv(q) intersects an input leaf, q is partitioned and its subquadrants are subsequently investigated. As a consequence of this strategy, the postcondition P of search(q, ins, inters) has to be changed into Pl: Pl: ((ins and inv( q) is enclosed by an input leaf) or (7 ins and inters and inv( q) MAY intersect an input leaf, but is not enclosed by it) or (-, ins and 7 inters and inv( q) intersects none of the input leaves))
FIG. 4.
inp[m]
is enclosed
cl
sm
•I
inv(q)
El
inpcml
by sm, but does not intersect
inv(q).
92
MARLOES
L. P. VAN
LIEROP
The ellipses in both line (i) and (ii) are to be replaced by inters := true.
Note that, because of this strategy, the output tree may become more branched unnecessarily, resulting in more compaction time. 5. REFINEMENTS
An important property mentioned in Section 2 is, that for two quadrants p and q. where p precedes q, all descendants of p precede q as well. This observation is useful in reducing the length of the array part on which the binary search is applied: if the values of 1 and r in search( ) are stored and carried over to buifdtree( ), the part of the array on which the binary search has to be applied for the sons of q may be reduced. For this purpose the local variables I and r of seurch( ) are entered in the parameter list of search( ); they become var parameters that define, at calling, the array part on which the binary search is to be performed. In order to carry over these values to the recursive calls of buildtree( ), the parameter list of build&ree( ) is extended with two value parameters left and right. The local variables 1 and r of searchpixel( ) are entered in its parameter list as value parameters; they now define the array part on which the binary search in searchpixel( ) has to be performed. Thus, a more sophisticated version of buildtree( ), search( ), and searchpixef( ) might look like this: procedure buildtree(q: quadrant; l,ieft, right: integer); var i:ual; inside, intersect: boolean; It, rt: integer; begin if I = n then begin if searchpixel(q,left, right) then “send q to the output” end else begin It := left; rt := right; search(q, inside, intersect, It, rt ); {Pl and It I rt and (for all i: 0 5 i < It or rt < i < N: inp[i] does not intersect inv(q)) > if inside then “send q to the output” else if intersect then for i := zero to three do begin q[l] := i; buildtree(q, 1 + 1, it, rt) end end end procedure seurch( q: quadrant; var ins, inters: boolean; var 1, r: integer); var sm : quadrant; m : integer ;
TRANSFORMATIONS
ON
93
LEAFCODES
begin ins := false; inters := false; { 1~ r and Q, where Q: (for all i:O I i < I or r < i < N: inp[i] does not intersect inv(q)) > if “inv(q) is enclosed by the 2” X 2” pixel region ” then begin sm :=“‘code of the smallest quadrant that encloses inv( 4)“; m := (I + r)div 2; while (“sm -C inp[m]” or “sm > inp[m]“) and / < r do begin if “sm < inp[ ml” then r := m {Q} else {“sm > inp[m]“} f := m + 1; (Q} m := (f + r)div 2 end; {“sm c inp[ ml” or “sm I inp[m]” or 1 = r } if “sm c inp[ ml” then ins := true else if “sm IJ inp[m]” then inters := true end else if “inv(q) intersects the 2” X 2” pixel region ” then inters := true {Pl and Q and I I r } end The changes in searchpixel( ) concern the parameter list, which becomes function searchpixel( q: quadrant; I, r: integer), and line (*), which is omitted. The first call of buildtree( buildtree(root, 0, 0, N), where root has the same value as before. 6. SOME
FURTHER
) becomes
DETAILS
In this section we discuss the compaction, how the code of the pixel enclosing a given point is computed, and how the code of the smallest quadrant that encloses an inverse quadrant is computed. Before explaining how the code of the pixel enclosing a given point is computed, we describe how the 2” x 2” pixels are embedded in a coordinate system. We assume that the lower left comer of the lower left pixel has coordinates (O,O), and the upper right comer of the upper right pixel has coordinates (2”, 2”). Thus, the pixel width is 1. In procedure searchpixef( ), the following line appears: cntrpix := “code of the pixel enclosing the inversely transformed center of q “, where q and cntrpix are quadrants. Actually, the coordinates, which are reals, of the inversely transformed center of q are computed, and these coordinates are truncated
94
MARLOES
L. P. VAN
LIEROP
to become the coordinates of the lower left corner of the pixel enclosing the inversely transformed center of q. Subsequently, these integer coordinates are passed to procedure encode ( x, y : integer ;var cntrpix : quadrant
j,
which assigns to cntrpix the code of the pixel whose lower left corner is (x, y). In the procedure search( ), the following line appears: sm := “code of the smallest quadrant that encloses inv( 4)“.
For this purpose, the procedure smallest( q: quadrant;var sm: quadrant) is designed. which assigns to sm the code of the smallest quadrant that encloses inv(q). We have defined inv(q) as the quadrangle that results when the inverse transformation is applied to quadrant q. This quadrangle is specified by its four vertices, which are the inversely transformed corners of q: type coord = record x, y: real end; quadrangle = array[O..3] of coord; square = record x, y, d:integer end;
{a square is specified by the coordinates of its lower left corner, and its width} procedure smallest( q: quadrant;var sm: quadrant); var smaller: boolean; pol: quadrangle; sq: square; begin “pof := inv( q)“;smaffer := true; sq.x
:=
0;
sq.y
:=
0;
,q.d:="2"";
while smaller and (sq.d > 1) do begin if “pol is not enclosed by any of the subquadrants of sq ” then smaller := false else sq := “subsquare of sq that encloses ~01”; end; sm := “code of sq ” end
A quadrangle is enclosed by a square iff all the vertices of the quadrangle are enclosed by the square, which is easy to find out. As said before, the result of the transforming algorithm is the leafcode of a semi-quadtree, and hence a compaction step is required. Instead of using the reversing algorithm to perform compaction, we have created a filter whose input is a leafcode of a semi-quadtree, and whose output is the leafcode of the associated quadtree. This filter uses a procedure called compact(q: quadrant), which manages a global queue in which the black leaves that are candidates for compaction are stored. A black leaf is a can&Me for compact& if its preceding siblings are black too, i.e., if they are in the queue. Note that a leaf whose least significant digit is zero (a NW leaf), is always a candidate for compaction, because such leaf is the first Gb&ng. A
TRANSFORMATIONS
ON
LEAFCODES
95
black leaf q is a candidate for clearing the queue, if the queue is not empty and if q is not a proper successor of the last queue element, i.e., if there exists a quadrant p such that q > p > last queue element. The strategy used in compact( ) is as follows: - if the queue is empty, and q is a candidate for compaction, q is placed in the queue; - if the queue is empty, and q is not a candidate for compaction, q is sent to the output; - if the queue is not empty, and q is a candidate for clearing the queue, the queue is cleared (i.e., the elements of the queue are sent to the output in the correct order), and q is put in the queue or sent to the output, depending respectively on q being a candidate for compaction or not; - if the queue is not empty, and q is not a candidate for clearing (hence, q is a proper successor of the last queue element), q is put in the queue (if q is not a fourth sibling, i.e., not a SE leaf), or the last three elements of the queue are removed, and compact( ) is called for the father of q (if q is a fourth sibling). Note that the maximum number of queue elements is 3n, namely 3 siblings for each non-zero level. If only the greatest sibling that is a candidate for compaction is stored, the maximum number of queue elements even reduces to n. Hence, the internal storage requirements of compact( ) are very modest. The time complexity of procedure compact( ) is O(1). Since it is called for each node of the semi-quadtree, the compaction process requires O(M) time, where M is the number of nodes of the semi-quadtree to be compacted. 7. CONCLUDING
REMARKS
The algorithms described above have been implemented in the language C, and executed on a local microcomputer based on the M68000. The quadrant codes were not implemented as arrays, but, in a more compact manner, as integers (32 bits), where the four least significant bits were used to specify the level of the quadrant, and the 28 most significant bits were used pairwise to represent the quaternary digits. This way it is possible to deal with resolutions up to 14. We have also implemented the algorithm to perform transformations on leafcodes as described by Gargantini [8], using heapsort for the sorting step, and the same compaction algorithm as described above. The Gargantini algorithm requires 0( Ni + N2 x log( N,)) time, where Ni is the number of pixels enclosed by the input leaves, and N2 is the number of pixels enclosed by the output leaves. We will derive a time complexity for our algorithm. Let M be the number of nodes of the output semi-quadtree, i.e., the semi-quadtree that has not been compacted yet. The procedure buildtree( ) is called once for each node of the output semi-quadtree; within the procedure buildtree( ), either search( ) or searchpixef( ) is called. Within the procedures searchpixel( ) and search( ), a binary search is performed on an array part of maximum length N, where N is the number of input leaves, and thus the worst-case time complexity of the binary search is O(log(N)). Furthermore, in search( ) the procedure smaller( ) is called, which has worst-time complexity 0(n). Hence, the procedure search( ) has worst-case time complexity @log(N) + n), and searchpixel( ) O(log(N)). H ence, the worst-case time complex-
96
MARLOES
L. P. VAN LIEROP TABLE
# Input pixels
Time Gargantini
1024
Time van Lierop
# Nodes in output semi-quadtree
15.1
1.8
16384 32768 49152 64512"
1
43.1 91.6 136.3 144.5
75093 87381 87381 87381 75093
112.5 111.7 126.5 102.6
“(# input leaves is 9216) Note. # input leaves = # input pixels, # output pixels - # input pixels, times in seconds.
ity of the transforming algorithm is O(M X (log(N) + n)), where n is the resolution of the picture, N the number of input leaves, and M the number of nodes of the output semi-quadtree. Since the compaction requires O(M) time, as we have seen in the previous section, the combination of both algorithms is O(M x (n + log(N)) + M) = O(M x (n + log(N))). As far as storage requirements are concerned, the algorithm described in this paper compares favourably with Gargantini’s algorithm: we need the internal storage of all input leaves, where Gargantini needs the storage of all PIXELS enclosed by the output leaves. Remember that storage space reduction was the very reason to introduce leafcodes. See Tables 1 and 2 for the execution times of both the algorithm of Gargantini and the one described in this paper. These execution times include the time for compaction. The resolution of the picture transformed is 8, where the pictures are chessboard-like. The transformation performed was in all cases a translation of one pixel both to the right and to the south. In Table 1, the input leafcodes are such that each leaf is a pixel, i.e., the number of input leaves equals the number of input pixels. In Table 2, the number -of input leaves is as small as possible. Clearly, the execution times of the Gargantini algorithm depend on the number of black input and output pixels, whiie the execution times of our algorithm are heavily dominated by the number of nodes in the output semi-quadtree. Comparing the execution times of both algorithms, we may conclude that the Gargantini algorithm has the best
TABLE 2 # Input pixels
Time Gargantini
Time van Lierop
# Nodes in output semi-quadtree
# Input leaves
1024 16384 32768 49152 64512
1.5 33.1 69.5 107.8 143.2
4.7 4.2 5.5 4.9 6.0
5329 3957 3957 3957 5329
1 1
Note.
# output pixels - # input pixels, times in seconds.
2 3 9
TRANSFORMATIONS
ON TABLE
3
# Input leaves
# Input pixels
Time revert
1024 1024 32768 3072 960
1024 16384 32768 49152 65412
1.6 0.6 10.0 0.7 0.1
Note. Times
in seconds.
TABLE # Input leaves
4
Time Gargantini
32768 8192 2048 512 128 32 8 2 Note. # - # input
97
LEAFCODES
91.6 85.3 73.3 71.0 70.5 70.0 69.9 69.5 input pixels pixels, times
= 32768, # in seconds.
Time van Lierop 111.7 99.7 19.7 50.7 28.9 14.4 7.1 5.1 output
pixels
performance for input pictures consisting of a rather small number of black pixels. For input pictures consisting of a very large number of black pixels, the algorithm of Gargantini may be used for the reverse leafcode of the picture. The execution times of our procedure reuert( ) are shown in Table 3. Our algorithm has the best performance for pictures consisting of a considerable nut&g& black pixels, and a relatively small number of leaves. In Table 4, the execution times are shown for pictures consisting all of the same number of black pixels, but varying in the number of leaves. Finally, we like to emphasize that our algorithm is applicable for all transformations that map squares onto convex quadrangles, while the Gargantini algorithm can be used only for translations over an integral number of pixels, and rotations of 90 or 180 degrees. ACKNOWLEDGMENTS
The reversing algorithm was designed by Tom Verhoeff, who has also implemented the Gargantini algorithm. REFERENCES 1. A.
Klinger and C. R. Dyer, Experiments in picture representation using regular decomposition, Comput. Gruphics Image Process. 5, 1916, 68-105. 2. A. Klinger, Patterns and search statistics, in Optimizing Methods in Stutistics (J. S. Rustagi, Ed.), pp. 303-337, Academic Press, New York, 1971. 3. H. Samet, The quadtree and related hierarchical datastructures. ACM Comput. Suruq~s 16, 1984, 187-260.
MARLOES
98
L. P. VAN LIEROP
4. G. M. Hunter and K. Steiglitz, Linear transformation of pictures represented by quad trees, Compur. Gruphics
Image
Process.
10, 1979,
289-296.
5. M. Schneier, Calculations of geometric properties using quadtrees, Cornput. Graphics Imuge Process. 16, 1981, 296-302. 6. F. J. Peters, An Algorithm for Transformations of Pictures Represented by Quadtrees, internal report of the Dept. of Mathematics and Computing Science, Eindhoven University of Technology, 19X4. 7. M. A. Oliver and N. E. Wiseman, Operations on quadtree-encoded images, Conrpur. J. 26, 1983, 83-91. 8. 1. Gargantini, An effective way to represent quadtrees, Comm. ACM 25, 1982, 905-910. 9. I. Gargantini, Translation rotation, and superposition of linear quadtrees, Int/.-J. Mutt-Mach. Stud 18, 1983, 253-263. 10. D. Meagher, Geometric modeling using octree encoding, Conlpur. Gruphics Imuge Process. 19, 1982, 129-147.