- Email: [email protected]
A formal system for texture languages
Pattern Recognition
Pergamon Press 1972. Vol. 4, lap. 53-72. Printed in Great Britain
A Formal System for Texture Languages* LUIGIA CARLUCCI Istituto di Elaborazione della Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy
(Received 27 January 1971)
A b s t r a c t - - A formal system for defining abstract languages for textures is given. A texture is represented as a tree, namely a tree m a d e of subtrees representing the occurrences o f the unit pattern. The unit pattern and the placement rule determine the texture. Languages are defined by imposing restrictions on the occurrences o f the unit pattern; such restrictions correspond to the placement rule.
I. P R E L I M I N A R I E S
1.1. Textures THIS work is concerned with the problem of developing a theory for describing texture, more precisely a description of texture is given by means of an abstract language. A scientific definition of texture is not available in the literature even though it is quite clear what is meant by texture (see PICKETT(2)). Nevertheless the importance of texture in the visual perception and in the problems of pattern recognition is often emphasized. The perception of texture was first recognized by Metzger (1930, see reference in GIBSONIt)), as a requisite for detecting the presence of a surface. The additional role that texture plays in mediating the perception of space was argued by GmsoN. ~t) In empirical studies Gibson has shown that texture cues can provide a sense of depth, though not necessarily a compelling or precise one. The lack of a precise definition of the term texture has been meaningfully explained by Muerle, who says (see: "Some thoughts in texture discrimination by computer" in LIPKIN (9)) :
"'Visual texture is a property of pattern scenes which has escaped precise definition so far. Part of the problem is that a very precise definition has not yet been required.".. "'Everyone 'knows' what texture is, since we live in a world rich in texture. In pattern recognition, texture is a major problem since much of what we see has texture when viewed from an appropriate distance." Texture has been viewed in many ways : As inhomogeneities in the gray scale, as statistical local properties, as the repetitive arrangement of a unit pattern, as a finite matrix of signs with some periodicity requirement on rows or columns. Many authors have investigated texture from the viewpoint of the automated image processing, i.e. texture synthesis, texture analysis and texture discrimination. A wide coverage of work in this area may be found in L I P K I N . ~9) The main results obtained are * This paper is part of a thesis to be submitted to the Scuola Normale Superiore (Pisa, Italy) in partial fulfilment of the requirements of the Corso di Perfezionamento, Classe di Scienze. 53
54
LUIGIA CARLUCCI
some procedures for texture synthesis based on the subpattern placement model and some statistical methods for texture discrimination (see also PUZIN, iS) CONROY,t6) ROSENFELDtS)). In this work textures are examined from a linguistic point of view. A method is described for defining texture languages, namely languages whose sentences are descriptions of textures. In other words the sentences of the languages are not pictures, but abstract descriptions of pictures. F o r this reason the languages have been called "abstract". Textures may be studied at least on two levels : statistical and structural. On the statistical level the texture of a complex pictorial pattern can be regarded as a set of statistical properties extracted from a large set of local measurements made on the pattern. On the structural level, a texture is considered to be defined by subpatterns which occur repeatedly, according to well defined placement rules, within the overall pattern. The subpattern itself may be made of structured elements or, more generally, of subpatterns which may themselves be made of still smaller elements, etc. Usually, for a given texture at a given scale, there is a subpattern which is of unit size in the sense that it is the smallest unit which one would refer to in describing the texture, even though it may itself have microstructure. F o r simplicity, only one level of this hierarchy is here considered, namely the subpatterns and their placement. The author's point of view is that the structural linguistic approach to the study of textures is the best one due to the highly structured nature of textures. In one-dimensional languages (i.e. string languages) it is necessary to distinguish between natural and artificial languages, in order to give a formal definition; in the same way in the case of picture languages it is necessary to distinguish between natural and artificial pictures. A natural picture has to be reduced to a simplified one in order to make an abstract structural description possible. In what follows texture means the regular arrangement over a given area of many occurrences of a unit pattern. A texture is then defined by the unit pattern and the law that determines the arrangement. In natural textures it is often difficult to determine the unit pattern and the placement rule, even though the regularity of the arrangement is evident (for example sand, pressed cork, pebbles); sometimes, on the other hand, the unit pattern and the placement rule are evident (for example wire nets, herring bone tweed). The law which determines the arrangement may also imply a deformation or variation of the unit pattern. F o r instance, two different occurrences of the same pattern may be not identical in the shape and/or in the size. The only requirement is that the unit pattern together with the placement rule determines in a unique way the structure of the texture. In other words it has to be possible to find a particular occurrence of the unit pattern P, and a function f depending on some parameter p, such that all the other occurrences P' of the unit pattern are obtained as images of P b y f f o r different values of p. For instance p could be the distance between P and P'. A texture may be made up of the arrangement of more than one unit pattern. Figure l(a) is an example of such a situation. In Fig. l(a) the texture has two unit patterns, namely a square and a triangle. In the schematization given here this case is reduced to the case of a unique pattern by considering a compound unit pattern, see Fig. l(b). In fact, the texture appearing in Fig. l(a) may be considered as generated from a quadrangular unit pattern, see Fig. l(c), which is structured. Its structure can be described as a square tangent to an empty square, tangent to a triangle, aligned. However, this microstructure needs not to be considered.
A formal system for texture languages
55
E]LOL
I-] ½ {-I ½ (o)
E]F (b)
(c)
FIG. 1. Texture made of a structured unit pattern. In order to clarify how the p a p e r has been divided, a m o r e detailed description of how textures have been schematized is here given. Textures considered here are m a d e of unit patterns that are polygonals. A first distinction is m a d e on the basis of the polygonals: segments, open polygonals, polygons. Further restrictions imposed on unit patterns---for example open polygonals and polygons must be not self-intersecting--are described in sections 2, 3 and 4. It has to be noted that, for the sake of simplicity, the various occurrences of the unit pattern composing a texture can be tangent but not intersecting. Intersection is only allowed in the case of segments. Some examples of textured scenes are shown as a further explanation (see Figs. 2-7).
/
/ FIG. 2. Texture made of translated segments.
FIG. 3. Texture made of rotated segments.
56
LUIGIA CARLUCCI
FIG. 4. Texture with collinear segments.
FIG. 5. Texture made of translated polygonals.
FIG. 6. Texture made of translated segment-likepolygonals.
FIG. 7. Texture made of translated tangent squares.
1.2. Pattern languages; texture languages Terminology and techniques used for string languages can, in many cases, be applied also to pattern languages. The linguistic approach to pattern recognition takes advantage from the recent improvements in the field of natural and artificial languages (see MILLER (4) and ROSEN~LO(7)). The syntax of string languages is often defined as follows : first of all a finite non empty set of symbols is given, i.e. an alphabet A such that any string in the language can be uniquely decomposed into its component symbols which belong to A. For any such A there exists an infinite set U, the set of all strings of symbols belonging to A. U is called the linguistic universe generated by A. Any specification that selects a subset L of U is called a formal syntax of the language L.
A formal system for texture languages
57
In an analogous way, in the two-dimensional case a set A' of primitives and a set of connecting laws that substitute the one dimension juxtaposition are defined. The linguistic universe U' is obtained from the primitives by means of these laws. Any subset L' of U' is a pattern language. A mean of specifying L' is called a formal syntax for the pattern language L'. There are many ways of defining such classes of patterns or pattern languages L': by specifying how patterns of the language can be generated, or by defining a procedure for deciding whether or not an arbitrary pattern belongs or not to the language. The first method relates to pattern generation and synthesis, the second one relates to pattern recognition and analysis. The formal system defined here belongs to the first kind of language definition. A method is defined for the generation of textures, namely a metagrammar which, whenever some parameters are given (unit pattern and placement rule) makes it possible to define all the textures belonging to the class determined by these parameters. Obviously a language can be defined also by union and intersection of a finite number of these classes. There are many ways to define such a metagrammar. The method used here allows to define a parsing procedure for textures, based on the syntax. This topic may be the subject for successive work : to devise a procedure which, given a texture, determines the parameters of the texture, i.e. the unit pattern and the placement rule, on the basis of the syntactic rules given here. This kind of analysis may be useful for various purposes : to "mend" a texture, to find the edges between two texturized regions, to determine surface slant. These results can then be useful, for instance, in the detection of solid objects in a three-dimensional scene. The method used here has also been chosen because the tree structured representation of objects may be interpreted as the data structure for the representation of textures in a computer memory. The formal system has been defined in the following way : two sets of symbols (vocabulary) are defined, namely a denumerable set of names (elementary selectors, i.e. names of tree branches) and the set of real numbers (elementary objects, i.e. measures of length, of arcs, cartesian coordinates in the plane, etc.). Starting from these symbols, tree structured objects are defined. In the following, tree structured objects will be named "trees" or "objects". The notation used here for the representation of trees has already been used for the formal definition of programming languages. A detailed exposition may be found in LUCAS.~3~A few concepts will be introduced here. An extended exposition may be found in the appendix. Compound trees are finite constructs consisting in a finite set of branches. Each of them points to a node which is either an elementary object or a compound tree having the same structure and which is called "immediate component" of the tree. A tree is uniquely defined by the list of the names of its branches and by the objects selected by these branches. The order in which branches appear in the formation of a tree is not relevant. Therefore two trees differing only for the order in which two branches appear are considered identical. Tree structured objects represent points, segments, open polygonals, polygons and textures. A point is represented by a tree which has two branches. Each of them selects an elementary object: One represents the x-coordinate, the other one represents the ycoordinate. A segment is represented by a tree with two immediate components, which are the two extrema of the segment. A polygonal is represented by a tree with n + 1 immediate components; n of them are points, namely the vertices of the polygonal, while the other immediate component is a number which specifies how many vertices the polygonal has.
58
LUIGIA CARLUCCI
The linguistic universe is defined as the set of all the tree structured objects having n + 1 immediate components, namely n segments, open polygonals or polygons and an immediate component which is an elementary object, i.e. the number n. In other words the linguistic universe contains all compound trees whose immediate components are a number n and n trees representing occurrences of some unit pattern in a texture. Classes of textures will be defined by imposing some restriction on the segments, open polygonals, or polygons composing the object. Namely some predicate is introduced which has to be satisfied by the immediate components for the object being considered a texture belonging to a certain class. These predicates are the formalization of the placement rule that, together with the unit pattern, determines the texture. Before going into a detailed description of the formal system only few remarks are pertinent. While in the other two-dimensional grammars, generally the symbols of the vocabulary are arcs, segments, arrows, here the symbols are abstract entities; therefore the phrases of the languages are not scenes but structural descriptions of scenes. Trees handled here are commutative. Therefore, in describing textures, the order in which the occurrences of the unit pattern are described is not relevant. In one-dimensional languages the vocabulary is finite and the linguistic universe, which is the set of all finite sequences of symbols from the vocabulary, is denumerable. Here the vocabulary, which is the union of two sets : a denumerable set of names ES and the set of all real numbers R, is not denumerable. The linguistic universe is a set of finite constructs, namely trees having a finite number of branches (a finite subset of ES) and a finite set of terminal nodes (a finite subset of R). The linguistic universe is not denumerable. Trees which are equal except for the values of the real numbers attached to their terminal nodes are said to belong to the same class then the linguistic universe may be considered as the union of a denumerable set of classes, each class being constituted of a nondenumerable set of trees. 2. T E X T U R E L A N G U A G E S . T E X T U R E S M A D E O F S E G M E N T S 2.1. The formal system The formal system here defined is based on a vocabulary which is the union of two sets : ES and EO. ES, the set of elementary selectors, is a denumerable set of names :
ES = {s-x, s-y, s-h, s-t, s-p, s-l} ~
{s-o,, s-colt, = I, 2, 3. . . . }.
In the above formula s-x and s-y are the selectors of the coordinates of some particular point. Selectors s-h and s-t indicate the head and tail extremum in some segment, while s-p selects the number of occurrences of a unit pattern in a texture, and s-l selects the number of vertices in a polygonal. Finally s-v, is the selector of the i-th vertex in a polygonal and s-co~ is the selector of the i-th occurrence of the unit pattern in a texture. EO, the set of elementary objects, is equal to R, the set of real numbers. Both EO and R will be used in the following for denoting the set of elementary objects. On R the operations of the ordinary arithmetic and the natural ordering are defined.
A formal system for texture languages
59
Starting from this vocabulary, namely ES ~ EO, finite constructs having tree structure are defined according to the rules explained in Appendix A2. 2.1.1. Points. First of all the definition of tree structured objects representing points in the plane is given. Starting from ES and EO this set is defined as follows : P = {/Zo((S-X: r l ) ; (s-y : r 2 ) ) l r I E EO A r 2 ~. EO}. The elements of P are called "objects representing points", or, shortly "points". They will be denoted as p, and their graphic representation is shown in Fig. 8.
/
$-x
f~
$-y
\ f2
FIo. 8. Graphic representation of a point.
They are defined by the composition operator #o which uses two selectors : s-x pointing to the object rl and s-y pointing to r 2. On the set P a two-place function D is defined corresponding to the euclidean distance in the plane. Let p and p' be elements of P D(p, p') = [(s-x(p) - s-x(p')) 2 + (s-y(p)-- s-y(p'))2] 1/2. Two predicates are defined on the elements of P. The first one, is-sl(pl, P2, P3), has three arguments and is satisfied if the three corresponding elements of P represent points on the same straight line. It is defined by a conditional expression (see Appendix A1). L e t p i e P , i = 1,2,3 is-sl(Pl, P2, P3) = (Pl = P2) ~ T;
(:12) [2 ~ R A S-X(p3) = 2" S-X(p 1) q- (1 -- 2)" S'x(p2) A s-y(p3)= 2. s-y(p~)+(1--2), s - y ( p 2 ) ] - - * T; T~F. Note that if the first and second points coincide the predicate is true, otherwise a linear relation must be satisfied. The second predicate, analogous to the previous one is true if and only if the third argument is a point belonging to the segment determined by the first two arguments. Let p i e P , i = 1,2,3 is'sg(Pl,P2,Pa) = -"~(P~ = P2) A (:12 such that 2 e R A 0 < 2 < 1
^ s-x(p3) = 2. s-x(pO + (1 - 2)- s-x(P2) A s-y(p3) = 2. s-y(p~)+(1 -~.). s-y(p2) ] ~ T; T~F.
LUIGIA CARLUCCI
60
2.1.2. Segments. The set of tree structured objects representing oriented segments in the plane is defined as follows : S O = {#o((s-h :p) ; (s-t :P'))IP ~ P A p' ~ P:.. Elements of S O will be denoted as seg. If points p and p' are equal the segment degenerates in a point. The graphic representation of a segment is shown in Fig. 9(a) or, more explicitly, in Fig. 9(b) where rl, r2, ra and r4 are real numbers.
/
s - x
s-h
s-f
s-h
s-f
s -y
s - x
/\1\
fl
\
f~
s
f3
-¥
f4
(b) FIG.9. Graphic representations of a segment.
In the following segments are not considered oriented. F o r this reason an equivalence relation is defined on S O which identifies segments having the same extrema. First of all the operator ITH(seg) is defined on the elements of S °. Given a segment seg ~ S °. ITH(seg) is the segment seg' obtained interchanging the head and the tail of seg. Let seg E S O ITH(seg) = #o((s-h :s-t(seg)) ; (s-t :s-h(seg))). If seg' = ITH(seg), it is obvious that seg = ITH(seg'). The equivalence relation is defined as follows : given seg and seg' belonging to S °, seg "-, seg' if and only if either seg = seg' or seg = ITH(seg'). Being reflexive, symmetric and transitive this relation is an equivalence relation. Let S = S ° / " ; S will be the set of all unoriented segments. Nevertheless, its elements will be denoted as the elements of S O. On the elements of S two real valued functions are defined representing the length and the slope of segments. L : S --* R ÷ L(seg) = D(s-h(seg), s-t(seg)).
A formal system for texture languages
61
The second one is defined by a conditional expression:
"0:S--,
-
~, 7~
O(seg) = s-x- s-h(seg) = s-x . s-t(seg) ~ ~" . r s - y . s - h ( s e g ) - s - y .s-t(seg)~ T ---, arctg ~ - x - ~ . s-t(seg)_]" Note that the symbol "" • " denotes composition of selectors, namely paths on the trees. A three-argument real valued function is now defined on P. This function, given three elements of P, namely p, p' and p" yields the value of the angle determined by the segments whose extrema are p and p', p' and p" respectively. The angle is defined as follows A(p,p',p") = (p = p' V p' = p") ~ O; T ~ [0(#o((s-h :p'): (s-t :p)))-O(#o((s-h:p'): (s-t :p")))/mod n]. The range of A is then [0, rt[. It is obvious that A(p, p', p") = A(p", p', p). The translation operator Tis now defined. Given a segment seg ~ S and an ordered pair of real number (a, b) the translation operator yields the segment obtained translating seg by the vector (a, b). T~.b(seg ) = /~o( ( S-X . s-h :s-x . s-h(seg) + a ) ; ( s-x . s-t: s-x . s-t(seg) + a ) ; ( s - y . s-h :s-y. s-h(seg)+b) ; ( s - y . s-t: s-y. s-t(seg)+b) ). If seg' = Ta,b(seg) then seg = T_a _h(seg'). A two-place predicate, namely is-gt(seg, seg') is now defined on S. This predicate, corresponding to a length-modifying translation, is satisfied if and only if, given an ordered pair of real numbers (a, b) and two segments seg and seg', seg' lies on the straight line determined translating seg by the vector (a, b). By means of a conditional expression : is-gt~.b(seg, seg') = [O(seg) = O(seg') A is-sl(s-h( T~.b(seg) ), s-t( T,.b(seg) ), s-h(seg') )] --* T; T~F. Namely, the two segments must have the same slope and the head of seg' must be collinear with the extrema of Ta,b(seg). Given seg and (a, b) there are infinitely many seg' satisfying the predicate is.gt(seg, seg'). Obviously is-gta,b(seg, seg') is true if and only if is-gt_a,_~(seg', seg) is true. The two-place predicate is-coil is now defined on S; it is true if and only if the two arguments are segments on the same straight line. is-coll(seg, seg') = O(seg) = O(seg') A is-sl( s-h(seg), s-t(seg), s-h(seg' )) --* T; T~F. is-coll(seg, seg') is true whenever is-coll(seg', seg) is true.
62
LUIGIA CARLUCCI
A two-place partial operator J is defined on the elements of S in the following way : Given two segments seg and seg', if they are collinear, d(seg, seg') is defined as the segment obtained by connecting seg and seg', otherwise it is undefined. Let seg and seg' be in S; let X = {s-h(seg), s-t(seg), s-h(seg'), s-t(seg')} and let p and p' be elements of X such that D(p, p') = maxpt,p2~ x D(pl, P2). Then by means of a conditional expression : J(seg, seg') = is-c611(seg, seg') ~ # o ( ( s - h : p ) ; (s-t:p')): T --+ undefined
Note that the segment which is the result of the J operator is uniquely determined, since segments are unoriented. Note that J(seg, seg') = J(seg', seg) in the sense that, whenever they are defined, they have the same value. 2.2. Textures made of segments Starting from the set S of all segments, the linguistic universe generated by them is now defined. Let Us ° = {bto((S-p :n) ; {(s-co i :segi)li = 1 , . . . , n})[n e N A (Vi)(seg i ~ S)~ where N is the set of natural numbers without 0, N = { 1, 2 . . . . } ~ EO. The elements of Us ° will be denoted by ts. Remember that s-coi (selector of the i-th component of the texture) points to a segment which is the i-th occurrence of the unit pattern in the texture, while s-p (selector of the number of parts) points to a terminal node which specifies the number of immediate components pointed by s-col occurring in the object. It has to be noted that the order in which the segments are numbered is not relevant in the specification of a texture. F o r this reason an equivalence relation is defined on Us °. Two elements of Us ° are equivalent, ts "~ ts', ifand only ifthey differ at most for the order in which the segments appear. Let Us denote Us°/~-. The elements of Us will be denoted as ts, that is in the same way as the elements of Us °. A texture language, that is a subset of Us, is defined by imposing some restrictions on the segments occurring in the generic ts belonging to it. Many cases have been distinguished : (1) ts in which collinear segments appear; (2) ts in which no segments are collinear. Furthermore : (a) ts made of parallel segments; (b) ts made of rotated segments. The case (2)~a) is now discussed. Let Tsa be the subset of Us defined in the following way : Tsa = {ts = lZo((s-p :n) ; {(s-co s:segl)[i = 1. . . . . n}) e Usl[Vi = I . . . . . n O{segi) = a] A [Vi, j = 1. . . . . n if i ~ j then ---n is-coll(segi, segi)]}. In words, the elements of Ts° are objects whose immediate components are parallel, not collinear segments, a is a real number such that 7Z
---~
7~
A formal system for texture languages
63
F o r each a, subsets of Tsa may be defined by means of the previously described functions and predicates. For example, the following formula defines, by means of the L function, the language of all the textures whose unit pattern is a segment and whose placement rule is the following : all the segments are parallel, no segments are collinear, all the segments have the same length.
Tsla = { t s e Tsa[ 31e R such that V i = I . . . . . s-p(ts)
L(s-codts)) = l}.
Other languages can be defined by imposing that all the segments appearing in an object have an extremum (or both) on a line (or one two lines). Classes of textures may be also defined by imposing that all the occurrences of a segment in a texture are equidistant or that the distance between the occurrences varies according to an assigned law (see Fig. 2). Let ~, be the set of all one-to-one functions ~o:{0 . . . . ,n} ~ R such that ¢p(O) = 0
Tst, = { t s e Ts,[ ]q~, ~ e e#s.vt,s ~ such that Vi = 1. . . . . s-pits) s-co~ts) = To,_ l~.*,- ,~( s'c°l(ts) )} ~0 and ~ determine the variation of distance between two segments in the texture. All the segments are equidistant in the case that ¢#(i) = i(p(1) and ~k(i) = i~k(1) for i = 1 , . . . , s - p i t s ) - 1. Using the predicate is-gt., b it is possible to define a broader class by removing all restrictions on the length of the segments occurring in a texture. The case of textures made up of segments having a varying orientation is now sketched (see Fig. 3). In this case segments belonging to straight lines which pass through the same point are considered. Let
Tp = {ts = # o ( ( s - p : n ) ; {(s-coi:segi)[i = 1. . . . . n})[ ]peP such that Vi=
1. . . . . n
is-sl(s-h(segi), s-t(segi), p) /~ Vi, j = 1 , . . . , n ifiq:j then is-coll(segl, seg 3)} . Languages may be defined as subsets of Tp by imposing restrictions on (1) the length of segments (2) the position of head and/or tail of segments (3) the angles between segments. For example, let • , be the set of all functions f:{1 . . . . . n}
- 2'21~
-2'2
64
LUIGIA CARLUCCI
that are one-to-one with respect to the first argument, i.e. for i :~ j .
f ( i , a) ~ f ( j , a) Tr = { ts ~ Tpl 3 f ~ ~ _ p , s )
such that
V i = 2 . . . . . s-p(ts) O(s-coi(ts)) = f ( i , O ( s - c o t ( t s ) ) ) } .
The set T r is the set of all the textures in which the angle between two consecutive segments varies according to a certain lawf. F o r what concerns textures in which collinear segments appear they may be studied from two points of view. Only the case of parallel segments is now discussed, for the case of rotated segments is analogous. The hypothesis is made that two collinear segments are not overlapping. The "'hole'" between two collinear segments m a y be considered either irrelevant (random) or relevant (determining the structure of the texture). In the first case a language L for textures with holes is specified by associating it to a language L' for textures without holes. L' is obtained from L by connecting all the collinear segments of the textures in L by means of the J operator. In the second case texture languages are defined in a more complex way since the presence of the holes determines the structure. For example: let Tsha be the following set Tsha = {ts = # o ( ( s - P :n) ; { ( s - c o i :segi)[i = 1 . . . . .
n~)l(Vi = 1 . . . . .
n O(segi) = a)
A (:lkl,k2~N
such that kl, k2 > 1 A k t • k 2 = n A V r = 1. . . . .
kl-1
A Vj = 0 .....
k2-1
is-coll(seg,+ ~.g~ , seg,+ 1 + j.g~) A --7 is-over(seg,+ j.k~ , segr+ 1 + j.g,)~
where is-over is a two-place predicate on S defined as follows is-over(seg, seg') = is-sg( s-h(seg), s- t(seg), s-h(seg') ) V is-sg(s-h(seg), s.t(seg), s-t(seg')) ~ T; T~F.
Two languages may be the following: H = {ts ~ TshalVj = 1 . . . . , k 2 - i A V m = 1 , . . . ,
kl
L ( s - c o m + j,k,(ts)) = f ( j ) L ( s - c o m ( t s ) ) }
w h e r e f i s a function from {1 . . . . . n]. into R . and: Hs = (ts~TshalVi=
2 . . . . . k 2 - 1 A Vl = 1. . . . . k l
is-sl(s-h . s-co~(ts), s-h . s-cog, +l(ts), s-h . s-cot + i.g,(ts)) A is-sl(s-t, s-cot(ts), s-t.s-COk, + t(ts), S - t ' s - c o I + i.g,(ts)) 1.
A formal system for texture languages
65
H N Hs is also an interesting language ; an example of texture belonging to it is the one shown in Fig. 4 : the length of corresponding segments varies according to a certain law and the extrema of corresponding segments belong to two straight lines. 3. T E X T U R E S M A D E O F O P E N P O L Y G O N A L S In this section textures in which the unit pattern is an open polygonal are introduced. O p e n polygonals considered here are nonselfintersecting and their occurrences in a texture are also nonselfintersecting. They are divided into two kinds: (1) generic open polygonals and (2) segment-like open polygonals. As an example for case (1) see Fig. 5, for case (2) see Fig. 6. O p e n polygonals of the second kind are those which may be inscribed in a rectangle whose sides are in a small ratio. This case is treated by analogy with the case of textures made of segments since a texture whose unit pattern is an open polygonal of this kind m a y be considered as the " r a n d o m d e f o r m a t i o n " of a texture m a d e of segments. O p e n polygonals are defined as an ordered sequence of n vertices, n > 2. The case of two vertices is excluded, since in this case the polygonal is reduced to a segment. Let be, for every integer n > 3 :
OP ° = {op = #o((s-l:n) ; {(s-v,:p,~li = 1. . . . . n})[is-opn(op)~. OP ° is the set of all polygonals having n vertices and it is defined by the predicate is-opn. which is the following : is-op,(op) = op = #o((s-l:n) ; {(s-vi:pi)[i = 1. . . . . n}) A ['v'i = 1. . . . . n p i e P ] A [Vi, j = 1. . . . . n - 1 ; V p e P if
i ~ j A p~ {p,,p,+l,pj, pj+ll then -'-7 [is-sg(1)i, Pi+ 1, P) A isosg(p j, p j+ 1, P)]]
A [Vi, j = 1. . . . . n i f i ~ j t h e n p , # p j ] A IV, = 1. . . . . n-2---nis-sg(pi, Pi+l,Pi+2) ] --* T; T~F. R e m e m b e r that s-l (selector of length) is a selector that points to a terminal node: the integer n u m b e r attached to it specifies how m a n y vertices the polygonal has. On the contrary, s-v, (selector of vertex) is the selector pointing to the i-th vertex of the polygonal. The predicate is-op~ says that for an object to be an open polygonal all the vertices must be points. T w o segments belonging to the polygonal cannot have c o m m o n points, except at m o s t one vertex. Vertices are different from each other (in particular the polygonal is open) and three consecutive vertices are not aligned. Let now define the union
o P° = L ) o e ° . n>_3
An open polygonal may be described in two ways according to the order of the sequence of
66
LUIGIA CARLUCCI
vertices. An equivalence relation = is defined on O P ° for identifying the two representations of the same open polygonal. As usual, the quotient set : OP = OP°/~
will denote the set of all the unoriented open polygonals. On the elements of OP operators and predicates may be defined: for instance, an operator that defines the translation of a polygonal, one that defines the rotation of a polygonal around a fixed point, a predicate that is true for two polygonals if and only if one of them is a translated part of the other one. Let Uop ° = { top = # o ( ( s ' P : n ) ; { (s-coi: opi)li = 1. . . . . n }In ~ S V i = 1. . . . . n opi ~ O P A is-nsi(tpo)}
be the set of all objects whose immediate components are an integer n and n polygonals. Uop ° is the linguistic universe generated by the polygonals. The predicate is-nsi(top) Specifies that polygonals occurring in an object are not self intersecting. Namely : is-nsi(top) = Vi, j = 1. . . . . n V p ~ P
Vl= 1,...,nj_ t
Vk
=
1 ....
, hi-
1
ifi~j
then
[is'sg(Pik, Pik+ 1, P) A is-sg(pil, Pit+ l, P)] "-* T; T-,
F.
where n, nl have an obvious meaning. As in the case of segments on the linguistic universe an equivalence relation ~- is defined : two elements are equivalent if they differ at most for the order of their immediate components selected by s-coi. T h e quotient set will be then defined : Uop = U o p ° / ~ - .
Example of languages of textures whose unit patterns are polygonals may be obtained by imposing that all the occurrences of the polygonals are translated or rotated from an assigned one, that all the polygonals have an assigned number of vertices, that corresponding vertices lie on straight lines, etc. The definition of such languages is quite long due to the complexity of predicate and function specifications. The reader interested in further details and examples about this section and the following one may s e e CARLUCCIJ10) As mentioned before elements of U o p made of segment-like polygonals are reduced to elements of Us in the following way. A segment-like polygonal may be inscribed, in some reasonable way, in a rectangle whose height h and basis b are in a small ratio. A segment seg whose length is b is then associated to the polygonal (if changes to it may be a segment which intersects the polygonal and is parallel to one of the basis of the inscribing rectangle, at a distance hi2 from it). Given top ~ Uop, if d is the maximal height among all the polygonals occurring in it, then top is said to be d-represented by ts = lao((S-p :s-p(top)) ; { ( s - c o l ; segi)li = 1. . . . . s-p(top)})
A formal system for texture languages
67
where ts ~ Us. F o r the elements of U o p made of segment-like polygonals a language L is defined by imposing that an element top belongs to L if and only if its d-repres, ntant in U s belongs to a certain language in Us. By varying these restrictions, further interesting languages may be defined : for example restrictions on the variation of h~ may be imposed. 4. T E X T U R E S M A D E O F P O L Y G O N S Textures whose unit patterns are polygons are now considered. First ofall the definition of the set of polygons is given : a polygon is an ordered sequence of n vertices (n > 4), the first vertex being equal to the last one. Three consecutive vertices are not aligned, and two sides of the polygon are nonselfinters, cting. More precisely let pol = #o((S-I: n ) ; { (s-vi:pi)[i = 1. . . . . n}).
The set of all polygons is defined as follows Pol ° = ~.) {pol [ is-pol.(pol)}. n~4
F o r all integers n _> 4 the predicate is-pol,~pol) is defined in an analogous way to the predicate is-op.(op). is-pol.(pol) = pol = #o((s-l :n) ; { (s-vi:pi)[i = 1. . . . . n}) A [Vi = 1 . . . . . n p i ~ P
A Pl = P~]
A [Vi, j = 1 , . . . , n - 1
if i ~ j
A [Vi = 1. . . . . n - 2 A[Vi, j=
i ..... n-1
then Pi ~ Pj]
---lis-sl(pi,pi+l,Pi÷2) ] Vp~P
if(/~jAp¢{p,,p,+l,pj,p~+l})
then---~[is-sg(pi, pi+ l , p ) A is-sg(pj, pj+ l , p ) ] ] ~ T; T~F.
Two polygons are considered equal if they differ only for the order of vertices. The sense in which they are numbered must be the same (clockwise or counterclockwise) but the first vertex may change. In order to define the equivalence relation on Pol o a mapping from (1 . . . . , n} onto ( 1. . . . , n} is defined as follows. Let X = { 1 , . . . , n} and let k be an element of X. By means of a conditional expression, the mapping 7rk is defined as follows : ~k:X ~ X •k(i) = 1 <__ i <__ k-'* n - k + i ; k<_i<_ n~i-k+l.
The equivalence relation is then defined : let pol, pol' ~ Pol o pol ~- pol' if and only if s-l(pol) = s-l(pol') -- n and there exists k e {1 . . . . . n} such that V i = 1 . . . . . n s-v,~pol) = s-v,k(o(pol'). As usual, Pol = P o l ° / ' , .
LUIGIA CARLUCCI
68
On the elements of Pol operators and predicates m a y be defined : for instance, a predicate which is true if and only if two polygons have a c o m m o n part (i.e. if they are tangent), or a predicate which is true when they have the same shape. Operators that determine the translation, the rotation, the scale change of a polygon, the inversion of the sense in which vertices are numbered, etc. The linguistic universe is defined as:
Upo ° = I tpo = ~o(( S-p :n) {(s-co i :poli~li = 1. . . . . n})l(Vi = 1. . . . . n poli ~ Pol) A is-tpo(tpo)}. Here is-tpo(tpo) is a predicate which specifies that two polygons occurring in a texture must have either (1) no points in c o m m o n or (2) one point in c o m m o n or 13) if they have two points in c o m m o n , the entire part between them in common. Therefore they cannot intersect. F o r this predicate definition (which is quite long) see CARLUCCI.I lo~ On Upo °, which is the linguistic universe generated by polygons, the equivalence relation ~ is defined that identifies two objects differing at most for the order of the s-co i components. As usual, let Upo = U p o ° / ~ - . As an example, the language is here presented of all the textures whose unit pattern is a square and whose placement rule allows translated and tangent occurrences. Thus the final result is a square lattice. An instance of texture belonging to this language is shown in Fig. 7. A predicate and an operator on pol ~ Pol are introduced first.
is-square(pol) = s-l(pol) = 5 A [:ld > 0 such that Vi = 1. . . . . 4 D(s-v,{pol), s-vi+ l(pol)) = d] A A(s-vl(pol), s-v2(pol), s-v3(pol) ) = - ~r~T ; 2 T~F. Ta.b(pol) = pol' ~ Pol where
pol' = #o( ( S-I : s-l(pol) ) ; { ( s-v~ : #o( ( S-X "s-x . s-v~4pol) + a ~ ; (s-y :s-y. s-v(pol)+ b)))li = 1. . . . . s-l(pol)})
is-square allows polygons with 4 equal sides and a right angle, while Ta, b is the translation operator. Then the language is' Tst = {tpo = #o((S'p :n) ; {(s-co i :poll)li -- 1. . . . . n}) e Upot ~ i = 1. . . . . n is-square(poli)]
A []k,k';k,k'
> 1 A kk' = n such that
Vj=0 ..... k-l,
Vl = 0 . . . . . k ' - I
pol~.g+~+ 1 = Tj.dj.a(PO/1)]} where
d = D(S-Vl(pOl O, s-v2(polt)).
A formal system for texture languages
69
C O N C L U D I N G REMARKS A formal system for describing languages based on the tree representation of objects has been presented. As examples of the application of this method, languages for textures whose unit patterns are segments, open polygonals or polygons have been discussed. It has to be noted that the description of languages given here is independent of the particular system of cartesian axes chosen, for all predicates on textures are based on absolute measurements. Note that, as an obvious extension, this method can be used for defining languages for textures made of a more complex unit pattern (for instance a blob) or textures made of more than one unit pattern. This formal system, based on the tree representation of objects is quite general. It has been used first for defining the abstract syntax of programming languages and an abstract machine which interprets languages. This definition technique looks appropriate not only for textures but it is very promising for the entire linguistic approach to picture processing. In fact this formal system may also be used for the definition of other kinds of picture languages by means of a similar vocabulary and an appropriate definition of the tree structured compound objects for representing pictures. The author's feeling is that picture language definitions will not be in general substantially more complicated than the definitions given here for texture languages. Acknowledgements--My gratitude to Professor A. GRASSELLIand to my colleagues for the conversations on the subject of this work. In particular I would like to thank U. MONTANAmfor his useful suggestions in writing the paper.
REFERENCES 1. J. J. GlaSON, The Senses Considered as Perceptual System. Houghton Mifflin, Boston (1966). 2. R. M. PICKETT,Perceiving visual texture: a literature survey with remarks on the theory and application of man as a pattern receiver, Ph.D. Thesis, Harvard University, School of Public Health (1966). 3. P. LUCAS,P. LAUERand H. S~GL~r~ER, Method and notations for the formal definition of Programming Languages, IBM Laboratory, Vienna, Tech. Rep. TR 25087 (1968). 4. W. F. MILLERand A. C. SHAW, Linguistic methods in picture processing: a survey, AFIPS, FJCC33, part l, 279, 290 (1968). 5. M. PUZIN, Simulation of cellular patterns by computer graphics, M.Sc. Thesis, University of Toronto, Department of Computer Science, Tech. Rep. 10 (1969). 6. P. CONROY,Simulation of texture by computer graphics, M.SC. Thesis, University of Toronto, Department of Computer Science, Tech. Rep. 14 (1969). 7. A. ROSinED, Picture Processing by Computer, Chap. 10. Academic Press, New York (1969). 8. A. ROSENFI~.Dand E. B. TROY, Visual texture analysis, University of Maryland, Computer Science Center, Tech. Rep. 70-116 (1970). 9. B. S. LIPKIN and A. ROSENFELD(Eds.), Picture Processing and Psychopictorics. Academic Press, New York (1970). 10. L. CARLUCCI,Texture languages, l.E.l., Tech. Rep. B71-4, Pisa (1971).
APPENDIX In paragraph A 1 a summary of the notational conventions used in this work is given. In A2, an informal illustration of the method of definition of tree structured objects is exposed.
70
LUIGIA CARLUCCI
A1. Notational conventions The notation used in this work has been taken mostly from predicate calculus and set theory. Some notations are taken from LISP, for example conditional expressions. (In the system, many predicates and functions are defined by means of conditional expressions.) A conditional expression has the following form : Pl ~ el ;
Pn --* en
where p ~ , . . . , Pn are truth values and e t , . . . , e, are values. The conditional expression above evaluates ei where i is the smallest integer for which pi is true and Pi is false for allj such that 1 < j < i. If such an i does not exist the expression has no value. A2. Definition of tree structured objects What follows has been written with the aim to clarify the method which has been used for the definition of the system. In LUCAS(3) it is possible to find the complete and detailed exposition of what here is only sketched. A2.1. Objects. An object will in general be composed of components which are themselves objects. The components of an object are uniquely named so that, using the names, one can refer to the components. The names used to name components of objects are called selectors and it is assumed that there is a finite or countably infinite set S of symbols defined for that purpose. In the following the symbols s, st, s2 . . . . stand for selectors, the symbols A, A1, A 2 , . . . stand for arbitrary objects. For selecting components of given objects an operation is introduced. Given an object A the result of the application of a selector s to A is the component of A whose name in the formation of A is s, if any. By analogy with functional application the operation is represented by :
s(A) and reads "s applied to A". The application of s to A is said to yield the s-component of A. Since objects will be identified with linguistic expressions, i.e. finite constructs, the only interesting class of objects are those having a finite number of components and a finite depth of nesting. The latter means that the successive application of selectors to a given object will, after a finite number ofsteps, result in an object which has no further component, i.e. no selector can be meaningfully applied to it. Objects of this kinds are called elementary objects. It is assumed that there is a set of symbols EO to represent elementary objects. The symbols, eo, eot, eo2 • • • will in the sequel stand for elementary objects ; objects that are not elementary are called compound objects. A named object is a pair (s :A) where s is a selector and A is an elementary or compound object.
A formal system for texture languages
71
A compound object can now be uniquely described by a finite non empty set of named objects, either compound or elementary, where the names must be mutually different, i.e. a compound object is described by the set:
{(s~ :Ax>,...} where n _> 1 and si ~ sj for i ~ j. The objects Ai, for i = 1. . . . . n are called the immediate components of the object described. A special form of expression has been chosen to represent compound objects. An object described by the above set of named objects may be represented by :
#o((Sl :A1);... ;
(2) A compound object described by/Zo((S 1 :Ax) . . . . . (sn:An)) is represented by the tree shown in Fig. 10
&
$1
// \
®
s2
®
s.
®
FIG. 10. Graphic representation of a compound object.
where for (~), ( ~ , . . . ,
( ~ the respective tree representations have to be inserted.
A2.3. Composite selectors. By analogy with the functional composition, an operation for selectors is introduced in the following way : sl
. s~
. . . . . s.(A)
=
s~(sd
. . .
(s.(A))...)).
s l . s 2 - . . . . s, is called a composite selector. In the following X, gt, Z2 . . . . stand for composite selectors. It is possible to introduce the identity relation on objects by means of composite selectors in the following way : A1 = A2 if and only if either A1 = A2 = eo
72
LUIGIA CARLUCCI
or Z(A 1) = z(A2) for all selectors ~. Note that the direction bottom to top in the tree corresponds to the direction left to right in the composition of selectors. Example : see Fig. 11.
/
s3
$4
\ 003
O0 2
FIG. 11, Graphic representation of a compound object. Composition of selectors.
A2.4. Extension of the meaning o f # o. Some functions that operate on objects may be defined, namely #, of which/~o is a particular case, and 6. In the definition of texture languages the/a and 6 functions have never been used, so it has been preferred to omit their definition and consequently, the definition of Po as a particular case of/~ (see LucAs~31). In the following an extension of the meaning of Po, as defined in A2.1, is given./~o has been defined for finite lists of pairs, namely for finite lists of named objects: (~l :A1) . . . . . ( ~ , : A . ) . Here the extension is given for the case that the arguments are not a list, but an unordered finite set specified by some predicate. Let P(X, A) be a predicate defined on pairs : (selector:object) which is true for a finite non empty set of pairs. Then
~o({<~ :A>Ip~x, A)}) = go( ;... ; ) where {(x:A)Ip(x, A)} = {(Z! :A~) . . . . . ( x . : A . ) } . The meaning of
i.to({lp,(z,A)} ; {Ip2(x,A)}) and of ~o(; is obvious.
{Ip(z,A)})
Pergamon Press 1972. Vol. 4, lap. 53-72. Printed in Great Britain
A Formal System for Texture Languages* LUIGIA CARLUCCI Istituto di Elaborazione della Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy
(Received 27 January 1971)
A b s t r a c t - - A formal system for defining abstract languages for textures is given. A texture is represented as a tree, namely a tree m a d e of subtrees representing the occurrences o f the unit pattern. The unit pattern and the placement rule determine the texture. Languages are defined by imposing restrictions on the occurrences o f the unit pattern; such restrictions correspond to the placement rule.
I. P R E L I M I N A R I E S
1.1. Textures THIS work is concerned with the problem of developing a theory for describing texture, more precisely a description of texture is given by means of an abstract language. A scientific definition of texture is not available in the literature even though it is quite clear what is meant by texture (see PICKETT(2)). Nevertheless the importance of texture in the visual perception and in the problems of pattern recognition is often emphasized. The perception of texture was first recognized by Metzger (1930, see reference in GIBSONIt)), as a requisite for detecting the presence of a surface. The additional role that texture plays in mediating the perception of space was argued by GmsoN. ~t) In empirical studies Gibson has shown that texture cues can provide a sense of depth, though not necessarily a compelling or precise one. The lack of a precise definition of the term texture has been meaningfully explained by Muerle, who says (see: "Some thoughts in texture discrimination by computer" in LIPKIN (9)) :
"'Visual texture is a property of pattern scenes which has escaped precise definition so far. Part of the problem is that a very precise definition has not yet been required.".. "'Everyone 'knows' what texture is, since we live in a world rich in texture. In pattern recognition, texture is a major problem since much of what we see has texture when viewed from an appropriate distance." Texture has been viewed in many ways : As inhomogeneities in the gray scale, as statistical local properties, as the repetitive arrangement of a unit pattern, as a finite matrix of signs with some periodicity requirement on rows or columns. Many authors have investigated texture from the viewpoint of the automated image processing, i.e. texture synthesis, texture analysis and texture discrimination. A wide coverage of work in this area may be found in L I P K I N . ~9) The main results obtained are * This paper is part of a thesis to be submitted to the Scuola Normale Superiore (Pisa, Italy) in partial fulfilment of the requirements of the Corso di Perfezionamento, Classe di Scienze. 53
54
LUIGIA CARLUCCI
some procedures for texture synthesis based on the subpattern placement model and some statistical methods for texture discrimination (see also PUZIN, iS) CONROY,t6) ROSENFELDtS)). In this work textures are examined from a linguistic point of view. A method is described for defining texture languages, namely languages whose sentences are descriptions of textures. In other words the sentences of the languages are not pictures, but abstract descriptions of pictures. F o r this reason the languages have been called "abstract". Textures may be studied at least on two levels : statistical and structural. On the statistical level the texture of a complex pictorial pattern can be regarded as a set of statistical properties extracted from a large set of local measurements made on the pattern. On the structural level, a texture is considered to be defined by subpatterns which occur repeatedly, according to well defined placement rules, within the overall pattern. The subpattern itself may be made of structured elements or, more generally, of subpatterns which may themselves be made of still smaller elements, etc. Usually, for a given texture at a given scale, there is a subpattern which is of unit size in the sense that it is the smallest unit which one would refer to in describing the texture, even though it may itself have microstructure. F o r simplicity, only one level of this hierarchy is here considered, namely the subpatterns and their placement. The author's point of view is that the structural linguistic approach to the study of textures is the best one due to the highly structured nature of textures. In one-dimensional languages (i.e. string languages) it is necessary to distinguish between natural and artificial languages, in order to give a formal definition; in the same way in the case of picture languages it is necessary to distinguish between natural and artificial pictures. A natural picture has to be reduced to a simplified one in order to make an abstract structural description possible. In what follows texture means the regular arrangement over a given area of many occurrences of a unit pattern. A texture is then defined by the unit pattern and the law that determines the arrangement. In natural textures it is often difficult to determine the unit pattern and the placement rule, even though the regularity of the arrangement is evident (for example sand, pressed cork, pebbles); sometimes, on the other hand, the unit pattern and the placement rule are evident (for example wire nets, herring bone tweed). The law which determines the arrangement may also imply a deformation or variation of the unit pattern. F o r instance, two different occurrences of the same pattern may be not identical in the shape and/or in the size. The only requirement is that the unit pattern together with the placement rule determines in a unique way the structure of the texture. In other words it has to be possible to find a particular occurrence of the unit pattern P, and a function f depending on some parameter p, such that all the other occurrences P' of the unit pattern are obtained as images of P b y f f o r different values of p. For instance p could be the distance between P and P'. A texture may be made up of the arrangement of more than one unit pattern. Figure l(a) is an example of such a situation. In Fig. l(a) the texture has two unit patterns, namely a square and a triangle. In the schematization given here this case is reduced to the case of a unique pattern by considering a compound unit pattern, see Fig. l(b). In fact, the texture appearing in Fig. l(a) may be considered as generated from a quadrangular unit pattern, see Fig. l(c), which is structured. Its structure can be described as a square tangent to an empty square, tangent to a triangle, aligned. However, this microstructure needs not to be considered.
A formal system for texture languages
55
E]LOL
I-] ½ {-I ½ (o)
E]F (b)
(c)
FIG. 1. Texture made of a structured unit pattern. In order to clarify how the p a p e r has been divided, a m o r e detailed description of how textures have been schematized is here given. Textures considered here are m a d e of unit patterns that are polygonals. A first distinction is m a d e on the basis of the polygonals: segments, open polygonals, polygons. Further restrictions imposed on unit patterns---for example open polygonals and polygons must be not self-intersecting--are described in sections 2, 3 and 4. It has to be noted that, for the sake of simplicity, the various occurrences of the unit pattern composing a texture can be tangent but not intersecting. Intersection is only allowed in the case of segments. Some examples of textured scenes are shown as a further explanation (see Figs. 2-7).
/
/ FIG. 2. Texture made of translated segments.
FIG. 3. Texture made of rotated segments.
56
LUIGIA CARLUCCI
FIG. 4. Texture with collinear segments.
FIG. 5. Texture made of translated polygonals.
FIG. 6. Texture made of translated segment-likepolygonals.
FIG. 7. Texture made of translated tangent squares.
1.2. Pattern languages; texture languages Terminology and techniques used for string languages can, in many cases, be applied also to pattern languages. The linguistic approach to pattern recognition takes advantage from the recent improvements in the field of natural and artificial languages (see MILLER (4) and ROSEN~LO(7)). The syntax of string languages is often defined as follows : first of all a finite non empty set of symbols is given, i.e. an alphabet A such that any string in the language can be uniquely decomposed into its component symbols which belong to A. For any such A there exists an infinite set U, the set of all strings of symbols belonging to A. U is called the linguistic universe generated by A. Any specification that selects a subset L of U is called a formal syntax of the language L.
A formal system for texture languages
57
In an analogous way, in the two-dimensional case a set A' of primitives and a set of connecting laws that substitute the one dimension juxtaposition are defined. The linguistic universe U' is obtained from the primitives by means of these laws. Any subset L' of U' is a pattern language. A mean of specifying L' is called a formal syntax for the pattern language L'. There are many ways of defining such classes of patterns or pattern languages L': by specifying how patterns of the language can be generated, or by defining a procedure for deciding whether or not an arbitrary pattern belongs or not to the language. The first method relates to pattern generation and synthesis, the second one relates to pattern recognition and analysis. The formal system defined here belongs to the first kind of language definition. A method is defined for the generation of textures, namely a metagrammar which, whenever some parameters are given (unit pattern and placement rule) makes it possible to define all the textures belonging to the class determined by these parameters. Obviously a language can be defined also by union and intersection of a finite number of these classes. There are many ways to define such a metagrammar. The method used here allows to define a parsing procedure for textures, based on the syntax. This topic may be the subject for successive work : to devise a procedure which, given a texture, determines the parameters of the texture, i.e. the unit pattern and the placement rule, on the basis of the syntactic rules given here. This kind of analysis may be useful for various purposes : to "mend" a texture, to find the edges between two texturized regions, to determine surface slant. These results can then be useful, for instance, in the detection of solid objects in a three-dimensional scene. The method used here has also been chosen because the tree structured representation of objects may be interpreted as the data structure for the representation of textures in a computer memory. The formal system has been defined in the following way : two sets of symbols (vocabulary) are defined, namely a denumerable set of names (elementary selectors, i.e. names of tree branches) and the set of real numbers (elementary objects, i.e. measures of length, of arcs, cartesian coordinates in the plane, etc.). Starting from these symbols, tree structured objects are defined. In the following, tree structured objects will be named "trees" or "objects". The notation used here for the representation of trees has already been used for the formal definition of programming languages. A detailed exposition may be found in LUCAS.~3~A few concepts will be introduced here. An extended exposition may be found in the appendix. Compound trees are finite constructs consisting in a finite set of branches. Each of them points to a node which is either an elementary object or a compound tree having the same structure and which is called "immediate component" of the tree. A tree is uniquely defined by the list of the names of its branches and by the objects selected by these branches. The order in which branches appear in the formation of a tree is not relevant. Therefore two trees differing only for the order in which two branches appear are considered identical. Tree structured objects represent points, segments, open polygonals, polygons and textures. A point is represented by a tree which has two branches. Each of them selects an elementary object: One represents the x-coordinate, the other one represents the ycoordinate. A segment is represented by a tree with two immediate components, which are the two extrema of the segment. A polygonal is represented by a tree with n + 1 immediate components; n of them are points, namely the vertices of the polygonal, while the other immediate component is a number which specifies how many vertices the polygonal has.
58
LUIGIA CARLUCCI
The linguistic universe is defined as the set of all the tree structured objects having n + 1 immediate components, namely n segments, open polygonals or polygons and an immediate component which is an elementary object, i.e. the number n. In other words the linguistic universe contains all compound trees whose immediate components are a number n and n trees representing occurrences of some unit pattern in a texture. Classes of textures will be defined by imposing some restriction on the segments, open polygonals, or polygons composing the object. Namely some predicate is introduced which has to be satisfied by the immediate components for the object being considered a texture belonging to a certain class. These predicates are the formalization of the placement rule that, together with the unit pattern, determines the texture. Before going into a detailed description of the formal system only few remarks are pertinent. While in the other two-dimensional grammars, generally the symbols of the vocabulary are arcs, segments, arrows, here the symbols are abstract entities; therefore the phrases of the languages are not scenes but structural descriptions of scenes. Trees handled here are commutative. Therefore, in describing textures, the order in which the occurrences of the unit pattern are described is not relevant. In one-dimensional languages the vocabulary is finite and the linguistic universe, which is the set of all finite sequences of symbols from the vocabulary, is denumerable. Here the vocabulary, which is the union of two sets : a denumerable set of names ES and the set of all real numbers R, is not denumerable. The linguistic universe is a set of finite constructs, namely trees having a finite number of branches (a finite subset of ES) and a finite set of terminal nodes (a finite subset of R). The linguistic universe is not denumerable. Trees which are equal except for the values of the real numbers attached to their terminal nodes are said to belong to the same class then the linguistic universe may be considered as the union of a denumerable set of classes, each class being constituted of a nondenumerable set of trees. 2. T E X T U R E L A N G U A G E S . T E X T U R E S M A D E O F S E G M E N T S 2.1. The formal system The formal system here defined is based on a vocabulary which is the union of two sets : ES and EO. ES, the set of elementary selectors, is a denumerable set of names :
ES = {s-x, s-y, s-h, s-t, s-p, s-l} ~
{s-o,, s-colt, = I, 2, 3. . . . }.
In the above formula s-x and s-y are the selectors of the coordinates of some particular point. Selectors s-h and s-t indicate the head and tail extremum in some segment, while s-p selects the number of occurrences of a unit pattern in a texture, and s-l selects the number of vertices in a polygonal. Finally s-v, is the selector of the i-th vertex in a polygonal and s-co~ is the selector of the i-th occurrence of the unit pattern in a texture. EO, the set of elementary objects, is equal to R, the set of real numbers. Both EO and R will be used in the following for denoting the set of elementary objects. On R the operations of the ordinary arithmetic and the natural ordering are defined.
A formal system for texture languages
59
Starting from this vocabulary, namely ES ~ EO, finite constructs having tree structure are defined according to the rules explained in Appendix A2. 2.1.1. Points. First of all the definition of tree structured objects representing points in the plane is given. Starting from ES and EO this set is defined as follows : P = {/Zo((S-X: r l ) ; (s-y : r 2 ) ) l r I E EO A r 2 ~. EO}. The elements of P are called "objects representing points", or, shortly "points". They will be denoted as p, and their graphic representation is shown in Fig. 8.
/
$-x
f~
$-y
\ f2
FIo. 8. Graphic representation of a point.
They are defined by the composition operator #o which uses two selectors : s-x pointing to the object rl and s-y pointing to r 2. On the set P a two-place function D is defined corresponding to the euclidean distance in the plane. Let p and p' be elements of P D(p, p') = [(s-x(p) - s-x(p')) 2 + (s-y(p)-- s-y(p'))2] 1/2. Two predicates are defined on the elements of P. The first one, is-sl(pl, P2, P3), has three arguments and is satisfied if the three corresponding elements of P represent points on the same straight line. It is defined by a conditional expression (see Appendix A1). L e t p i e P , i = 1,2,3 is-sl(Pl, P2, P3) = (Pl = P2) ~ T;
(:12) [2 ~ R A S-X(p3) = 2" S-X(p 1) q- (1 -- 2)" S'x(p2) A s-y(p3)= 2. s-y(p~)+(1--2), s - y ( p 2 ) ] - - * T; T~F. Note that if the first and second points coincide the predicate is true, otherwise a linear relation must be satisfied. The second predicate, analogous to the previous one is true if and only if the third argument is a point belonging to the segment determined by the first two arguments. Let p i e P , i = 1,2,3 is'sg(Pl,P2,Pa) = -"~(P~ = P2) A (:12 such that 2 e R A 0 < 2 < 1
^ s-x(p3) = 2. s-x(pO + (1 - 2)- s-x(P2) A s-y(p3) = 2. s-y(p~)+(1 -~.). s-y(p2) ] ~ T; T~F.
LUIGIA CARLUCCI
60
2.1.2. Segments. The set of tree structured objects representing oriented segments in the plane is defined as follows : S O = {#o((s-h :p) ; (s-t :P'))IP ~ P A p' ~ P:.. Elements of S O will be denoted as seg. If points p and p' are equal the segment degenerates in a point. The graphic representation of a segment is shown in Fig. 9(a) or, more explicitly, in Fig. 9(b) where rl, r2, ra and r4 are real numbers.
/
s - x
s-h
s-f
s-h
s-f
s -y
s - x
/\1\
fl
\
f~
s
f3
-¥
f4
(b) FIG.9. Graphic representations of a segment.
In the following segments are not considered oriented. F o r this reason an equivalence relation is defined on S O which identifies segments having the same extrema. First of all the operator ITH(seg) is defined on the elements of S °. Given a segment seg ~ S °. ITH(seg) is the segment seg' obtained interchanging the head and the tail of seg. Let seg E S O ITH(seg) = #o((s-h :s-t(seg)) ; (s-t :s-h(seg))). If seg' = ITH(seg), it is obvious that seg = ITH(seg'). The equivalence relation is defined as follows : given seg and seg' belonging to S °, seg "-, seg' if and only if either seg = seg' or seg = ITH(seg'). Being reflexive, symmetric and transitive this relation is an equivalence relation. Let S = S ° / " ; S will be the set of all unoriented segments. Nevertheless, its elements will be denoted as the elements of S O. On the elements of S two real valued functions are defined representing the length and the slope of segments. L : S --* R ÷ L(seg) = D(s-h(seg), s-t(seg)).
A formal system for texture languages
61
The second one is defined by a conditional expression:
"0:S--,
-
~, 7~
O(seg) = s-x- s-h(seg) = s-x . s-t(seg) ~ ~" . r s - y . s - h ( s e g ) - s - y .s-t(seg)~ T ---, arctg ~ - x - ~ . s-t(seg)_]" Note that the symbol "" • " denotes composition of selectors, namely paths on the trees. A three-argument real valued function is now defined on P. This function, given three elements of P, namely p, p' and p" yields the value of the angle determined by the segments whose extrema are p and p', p' and p" respectively. The angle is defined as follows A(p,p',p") = (p = p' V p' = p") ~ O; T ~ [0(#o((s-h :p'): (s-t :p)))-O(#o((s-h:p'): (s-t :p")))/mod n]. The range of A is then [0, rt[. It is obvious that A(p, p', p") = A(p", p', p). The translation operator Tis now defined. Given a segment seg ~ S and an ordered pair of real number (a, b) the translation operator yields the segment obtained translating seg by the vector (a, b). T~.b(seg ) = /~o( ( S-X . s-h :s-x . s-h(seg) + a ) ; ( s-x . s-t: s-x . s-t(seg) + a ) ; ( s - y . s-h :s-y. s-h(seg)+b) ; ( s - y . s-t: s-y. s-t(seg)+b) ). If seg' = Ta,b(seg) then seg = T_a _h(seg'). A two-place predicate, namely is-gt(seg, seg') is now defined on S. This predicate, corresponding to a length-modifying translation, is satisfied if and only if, given an ordered pair of real numbers (a, b) and two segments seg and seg', seg' lies on the straight line determined translating seg by the vector (a, b). By means of a conditional expression : is-gt~.b(seg, seg') = [O(seg) = O(seg') A is-sl(s-h( T~.b(seg) ), s-t( T,.b(seg) ), s-h(seg') )] --* T; T~F. Namely, the two segments must have the same slope and the head of seg' must be collinear with the extrema of Ta,b(seg). Given seg and (a, b) there are infinitely many seg' satisfying the predicate is.gt(seg, seg'). Obviously is-gta,b(seg, seg') is true if and only if is-gt_a,_~(seg', seg) is true. The two-place predicate is-coil is now defined on S; it is true if and only if the two arguments are segments on the same straight line. is-coll(seg, seg') = O(seg) = O(seg') A is-sl( s-h(seg), s-t(seg), s-h(seg' )) --* T; T~F. is-coll(seg, seg') is true whenever is-coll(seg', seg) is true.
62
LUIGIA CARLUCCI
A two-place partial operator J is defined on the elements of S in the following way : Given two segments seg and seg', if they are collinear, d(seg, seg') is defined as the segment obtained by connecting seg and seg', otherwise it is undefined. Let seg and seg' be in S; let X = {s-h(seg), s-t(seg), s-h(seg'), s-t(seg')} and let p and p' be elements of X such that D(p, p') = maxpt,p2~ x D(pl, P2). Then by means of a conditional expression : J(seg, seg') = is-c611(seg, seg') ~ # o ( ( s - h : p ) ; (s-t:p')): T --+ undefined
Note that the segment which is the result of the J operator is uniquely determined, since segments are unoriented. Note that J(seg, seg') = J(seg', seg) in the sense that, whenever they are defined, they have the same value. 2.2. Textures made of segments Starting from the set S of all segments, the linguistic universe generated by them is now defined. Let Us ° = {bto((S-p :n) ; {(s-co i :segi)li = 1 , . . . , n})[n e N A (Vi)(seg i ~ S)~ where N is the set of natural numbers without 0, N = { 1, 2 . . . . } ~ EO. The elements of Us ° will be denoted by ts. Remember that s-coi (selector of the i-th component of the texture) points to a segment which is the i-th occurrence of the unit pattern in the texture, while s-p (selector of the number of parts) points to a terminal node which specifies the number of immediate components pointed by s-col occurring in the object. It has to be noted that the order in which the segments are numbered is not relevant in the specification of a texture. F o r this reason an equivalence relation is defined on Us °. Two elements of Us ° are equivalent, ts "~ ts', ifand only ifthey differ at most for the order in which the segments appear. Let Us denote Us°/~-. The elements of Us will be denoted as ts, that is in the same way as the elements of Us °. A texture language, that is a subset of Us, is defined by imposing some restrictions on the segments occurring in the generic ts belonging to it. Many cases have been distinguished : (1) ts in which collinear segments appear; (2) ts in which no segments are collinear. Furthermore : (a) ts made of parallel segments; (b) ts made of rotated segments. The case (2)~a) is now discussed. Let Tsa be the subset of Us defined in the following way : Tsa = {ts = lZo((s-p :n) ; {(s-co s:segl)[i = 1. . . . . n}) e Usl[Vi = I . . . . . n O{segi) = a] A [Vi, j = 1. . . . . n if i ~ j then ---n is-coll(segi, segi)]}. In words, the elements of Ts° are objects whose immediate components are parallel, not collinear segments, a is a real number such that 7Z
---~
7~
A formal system for texture languages
63
F o r each a, subsets of Tsa may be defined by means of the previously described functions and predicates. For example, the following formula defines, by means of the L function, the language of all the textures whose unit pattern is a segment and whose placement rule is the following : all the segments are parallel, no segments are collinear, all the segments have the same length.
Tsla = { t s e Tsa[ 31e R such that V i = I . . . . . s-p(ts)
L(s-codts)) = l}.
Other languages can be defined by imposing that all the segments appearing in an object have an extremum (or both) on a line (or one two lines). Classes of textures may be also defined by imposing that all the occurrences of a segment in a texture are equidistant or that the distance between the occurrences varies according to an assigned law (see Fig. 2). Let ~, be the set of all one-to-one functions ~o:{0 . . . . ,n} ~ R such that ¢p(O) = 0
Tst, = { t s e Ts,[ ]q~, ~ e e#s.vt,s ~ such that Vi = 1. . . . . s-pits) s-co~ts) = To,_ l~.*,- ,~( s'c°l(ts) )} ~0 and ~ determine the variation of distance between two segments in the texture. All the segments are equidistant in the case that ¢#(i) = i(p(1) and ~k(i) = i~k(1) for i = 1 , . . . , s - p i t s ) - 1. Using the predicate is-gt., b it is possible to define a broader class by removing all restrictions on the length of the segments occurring in a texture. The case of textures made up of segments having a varying orientation is now sketched (see Fig. 3). In this case segments belonging to straight lines which pass through the same point are considered. Let
Tp = {ts = # o ( ( s - p : n ) ; {(s-coi:segi)[i = 1. . . . . n})[ ]peP such that Vi=
1. . . . . n
is-sl(s-h(segi), s-t(segi), p) /~ Vi, j = 1 , . . . , n ifiq:j then is-coll(segl, seg 3)} . Languages may be defined as subsets of Tp by imposing restrictions on (1) the length of segments (2) the position of head and/or tail of segments (3) the angles between segments. For example, let • , be the set of all functions f:{1 . . . . . n}
- 2'21~
-2'2
64
LUIGIA CARLUCCI
that are one-to-one with respect to the first argument, i.e. for i :~ j .
f ( i , a) ~ f ( j , a) Tr = { ts ~ Tpl 3 f ~ ~ _ p , s )
such that
V i = 2 . . . . . s-p(ts) O(s-coi(ts)) = f ( i , O ( s - c o t ( t s ) ) ) } .
The set T r is the set of all the textures in which the angle between two consecutive segments varies according to a certain lawf. F o r what concerns textures in which collinear segments appear they may be studied from two points of view. Only the case of parallel segments is now discussed, for the case of rotated segments is analogous. The hypothesis is made that two collinear segments are not overlapping. The "'hole'" between two collinear segments m a y be considered either irrelevant (random) or relevant (determining the structure of the texture). In the first case a language L for textures with holes is specified by associating it to a language L' for textures without holes. L' is obtained from L by connecting all the collinear segments of the textures in L by means of the J operator. In the second case texture languages are defined in a more complex way since the presence of the holes determines the structure. For example: let Tsha be the following set Tsha = {ts = # o ( ( s - P :n) ; { ( s - c o i :segi)[i = 1 . . . . .
n~)l(Vi = 1 . . . . .
n O(segi) = a)
A (:lkl,k2~N
such that kl, k2 > 1 A k t • k 2 = n A V r = 1. . . . .
kl-1
A Vj = 0 .....
k2-1
is-coll(seg,+ ~.g~ , seg,+ 1 + j.g~) A --7 is-over(seg,+ j.k~ , segr+ 1 + j.g,)~
where is-over is a two-place predicate on S defined as follows is-over(seg, seg') = is-sg( s-h(seg), s- t(seg), s-h(seg') ) V is-sg(s-h(seg), s.t(seg), s-t(seg')) ~ T; T~F.
Two languages may be the following: H = {ts ~ TshalVj = 1 . . . . , k 2 - i A V m = 1 , . . . ,
kl
L ( s - c o m + j,k,(ts)) = f ( j ) L ( s - c o m ( t s ) ) }
w h e r e f i s a function from {1 . . . . . n]. into R . and: Hs = (ts~TshalVi=
2 . . . . . k 2 - 1 A Vl = 1. . . . . k l
is-sl(s-h . s-co~(ts), s-h . s-cog, +l(ts), s-h . s-cot + i.g,(ts)) A is-sl(s-t, s-cot(ts), s-t.s-COk, + t(ts), S - t ' s - c o I + i.g,(ts)) 1.
A formal system for texture languages
65
H N Hs is also an interesting language ; an example of texture belonging to it is the one shown in Fig. 4 : the length of corresponding segments varies according to a certain law and the extrema of corresponding segments belong to two straight lines. 3. T E X T U R E S M A D E O F O P E N P O L Y G O N A L S In this section textures in which the unit pattern is an open polygonal are introduced. O p e n polygonals considered here are nonselfintersecting and their occurrences in a texture are also nonselfintersecting. They are divided into two kinds: (1) generic open polygonals and (2) segment-like open polygonals. As an example for case (1) see Fig. 5, for case (2) see Fig. 6. O p e n polygonals of the second kind are those which may be inscribed in a rectangle whose sides are in a small ratio. This case is treated by analogy with the case of textures made of segments since a texture whose unit pattern is an open polygonal of this kind m a y be considered as the " r a n d o m d e f o r m a t i o n " of a texture m a d e of segments. O p e n polygonals are defined as an ordered sequence of n vertices, n > 2. The case of two vertices is excluded, since in this case the polygonal is reduced to a segment. Let be, for every integer n > 3 :
OP ° = {op = #o((s-l:n) ; {(s-v,:p,~li = 1. . . . . n})[is-opn(op)~. OP ° is the set of all polygonals having n vertices and it is defined by the predicate is-opn. which is the following : is-op,(op) = op = #o((s-l:n) ; {(s-vi:pi)[i = 1. . . . . n}) A ['v'i = 1. . . . . n p i e P ] A [Vi, j = 1. . . . . n - 1 ; V p e P if
i ~ j A p~ {p,,p,+l,pj, pj+ll then -'-7 [is-sg(1)i, Pi+ 1, P) A isosg(p j, p j+ 1, P)]]
A [Vi, j = 1. . . . . n i f i ~ j t h e n p , # p j ] A IV, = 1. . . . . n-2---nis-sg(pi, Pi+l,Pi+2) ] --* T; T~F. R e m e m b e r that s-l (selector of length) is a selector that points to a terminal node: the integer n u m b e r attached to it specifies how m a n y vertices the polygonal has. On the contrary, s-v, (selector of vertex) is the selector pointing to the i-th vertex of the polygonal. The predicate is-op~ says that for an object to be an open polygonal all the vertices must be points. T w o segments belonging to the polygonal cannot have c o m m o n points, except at m o s t one vertex. Vertices are different from each other (in particular the polygonal is open) and three consecutive vertices are not aligned. Let now define the union
o P° = L ) o e ° . n>_3
An open polygonal may be described in two ways according to the order of the sequence of
66
LUIGIA CARLUCCI
vertices. An equivalence relation = is defined on O P ° for identifying the two representations of the same open polygonal. As usual, the quotient set : OP = OP°/~
will denote the set of all the unoriented open polygonals. On the elements of OP operators and predicates may be defined: for instance, an operator that defines the translation of a polygonal, one that defines the rotation of a polygonal around a fixed point, a predicate that is true for two polygonals if and only if one of them is a translated part of the other one. Let Uop ° = { top = # o ( ( s ' P : n ) ; { (s-coi: opi)li = 1. . . . . n }In ~ S V i = 1. . . . . n opi ~ O P A is-nsi(tpo)}
be the set of all objects whose immediate components are an integer n and n polygonals. Uop ° is the linguistic universe generated by the polygonals. The predicate is-nsi(top) Specifies that polygonals occurring in an object are not self intersecting. Namely : is-nsi(top) = Vi, j = 1. . . . . n V p ~ P
Vl= 1,...,nj_ t
Vk
=
1 ....
, hi-
1
ifi~j
then
[is'sg(Pik, Pik+ 1, P) A is-sg(pil, Pit+ l, P)] "-* T; T-,
F.
where n, nl have an obvious meaning. As in the case of segments on the linguistic universe an equivalence relation ~- is defined : two elements are equivalent if they differ at most for the order of their immediate components selected by s-coi. T h e quotient set will be then defined : Uop = U o p ° / ~ - .
Example of languages of textures whose unit patterns are polygonals may be obtained by imposing that all the occurrences of the polygonals are translated or rotated from an assigned one, that all the polygonals have an assigned number of vertices, that corresponding vertices lie on straight lines, etc. The definition of such languages is quite long due to the complexity of predicate and function specifications. The reader interested in further details and examples about this section and the following one may s e e CARLUCCIJ10) As mentioned before elements of U o p made of segment-like polygonals are reduced to elements of Us in the following way. A segment-like polygonal may be inscribed, in some reasonable way, in a rectangle whose height h and basis b are in a small ratio. A segment seg whose length is b is then associated to the polygonal (if changes to it may be a segment which intersects the polygonal and is parallel to one of the basis of the inscribing rectangle, at a distance hi2 from it). Given top ~ Uop, if d is the maximal height among all the polygonals occurring in it, then top is said to be d-represented by ts = lao((S-p :s-p(top)) ; { ( s - c o l ; segi)li = 1. . . . . s-p(top)})
A formal system for texture languages
67
where ts ~ Us. F o r the elements of U o p made of segment-like polygonals a language L is defined by imposing that an element top belongs to L if and only if its d-repres, ntant in U s belongs to a certain language in Us. By varying these restrictions, further interesting languages may be defined : for example restrictions on the variation of h~ may be imposed. 4. T E X T U R E S M A D E O F P O L Y G O N S Textures whose unit patterns are polygons are now considered. First ofall the definition of the set of polygons is given : a polygon is an ordered sequence of n vertices (n > 4), the first vertex being equal to the last one. Three consecutive vertices are not aligned, and two sides of the polygon are nonselfinters, cting. More precisely let pol = #o((S-I: n ) ; { (s-vi:pi)[i = 1. . . . . n}).
The set of all polygons is defined as follows Pol ° = ~.) {pol [ is-pol.(pol)}. n~4
F o r all integers n _> 4 the predicate is-pol,~pol) is defined in an analogous way to the predicate is-op.(op). is-pol.(pol) = pol = #o((s-l :n) ; { (s-vi:pi)[i = 1. . . . . n}) A [Vi = 1 . . . . . n p i ~ P
A Pl = P~]
A [Vi, j = 1 , . . . , n - 1
if i ~ j
A [Vi = 1. . . . . n - 2 A[Vi, j=
i ..... n-1
then Pi ~ Pj]
---lis-sl(pi,pi+l,Pi÷2) ] Vp~P
if(/~jAp¢{p,,p,+l,pj,p~+l})
then---~[is-sg(pi, pi+ l , p ) A is-sg(pj, pj+ l , p ) ] ] ~ T; T~F.
Two polygons are considered equal if they differ only for the order of vertices. The sense in which they are numbered must be the same (clockwise or counterclockwise) but the first vertex may change. In order to define the equivalence relation on Pol o a mapping from (1 . . . . , n} onto ( 1. . . . , n} is defined as follows. Let X = { 1 , . . . , n} and let k be an element of X. By means of a conditional expression, the mapping 7rk is defined as follows : ~k:X ~ X •k(i) = 1 <__ i <__ k-'* n - k + i ; k<_i<_ n~i-k+l.
The equivalence relation is then defined : let pol, pol' ~ Pol o pol ~- pol' if and only if s-l(pol) = s-l(pol') -- n and there exists k e {1 . . . . . n} such that V i = 1 . . . . . n s-v,~pol) = s-v,k(o(pol'). As usual, Pol = P o l ° / ' , .
LUIGIA CARLUCCI
68
On the elements of Pol operators and predicates m a y be defined : for instance, a predicate which is true if and only if two polygons have a c o m m o n part (i.e. if they are tangent), or a predicate which is true when they have the same shape. Operators that determine the translation, the rotation, the scale change of a polygon, the inversion of the sense in which vertices are numbered, etc. The linguistic universe is defined as:
Upo ° = I tpo = ~o(( S-p :n) {(s-co i :poli~li = 1. . . . . n})l(Vi = 1. . . . . n poli ~ Pol) A is-tpo(tpo)}. Here is-tpo(tpo) is a predicate which specifies that two polygons occurring in a texture must have either (1) no points in c o m m o n or (2) one point in c o m m o n or 13) if they have two points in c o m m o n , the entire part between them in common. Therefore they cannot intersect. F o r this predicate definition (which is quite long) see CARLUCCI.I lo~ On Upo °, which is the linguistic universe generated by polygons, the equivalence relation ~ is defined that identifies two objects differing at most for the order of the s-co i components. As usual, let Upo = U p o ° / ~ - . As an example, the language is here presented of all the textures whose unit pattern is a square and whose placement rule allows translated and tangent occurrences. Thus the final result is a square lattice. An instance of texture belonging to this language is shown in Fig. 7. A predicate and an operator on pol ~ Pol are introduced first.
is-square(pol) = s-l(pol) = 5 A [:ld > 0 such that Vi = 1. . . . . 4 D(s-v,{pol), s-vi+ l(pol)) = d] A A(s-vl(pol), s-v2(pol), s-v3(pol) ) = - ~r~T ; 2 T~F. Ta.b(pol) = pol' ~ Pol where
pol' = #o( ( S-I : s-l(pol) ) ; { ( s-v~ : #o( ( S-X "s-x . s-v~4pol) + a ~ ; (s-y :s-y. s-v(pol)+ b)))li = 1. . . . . s-l(pol)})
is-square allows polygons with 4 equal sides and a right angle, while Ta, b is the translation operator. Then the language is' Tst = {tpo = #o((S'p :n) ; {(s-co i :poll)li -- 1. . . . . n}) e Upot ~ i = 1. . . . . n is-square(poli)]
A []k,k';k,k'
> 1 A kk' = n such that
Vj=0 ..... k-l,
Vl = 0 . . . . . k ' - I
pol~.g+~+ 1 = Tj.dj.a(PO/1)]} where
d = D(S-Vl(pOl O, s-v2(polt)).
A formal system for texture languages
69
C O N C L U D I N G REMARKS A formal system for describing languages based on the tree representation of objects has been presented. As examples of the application of this method, languages for textures whose unit patterns are segments, open polygonals or polygons have been discussed. It has to be noted that the description of languages given here is independent of the particular system of cartesian axes chosen, for all predicates on textures are based on absolute measurements. Note that, as an obvious extension, this method can be used for defining languages for textures made of a more complex unit pattern (for instance a blob) or textures made of more than one unit pattern. This formal system, based on the tree representation of objects is quite general. It has been used first for defining the abstract syntax of programming languages and an abstract machine which interprets languages. This definition technique looks appropriate not only for textures but it is very promising for the entire linguistic approach to picture processing. In fact this formal system may also be used for the definition of other kinds of picture languages by means of a similar vocabulary and an appropriate definition of the tree structured compound objects for representing pictures. The author's feeling is that picture language definitions will not be in general substantially more complicated than the definitions given here for texture languages. Acknowledgements--My gratitude to Professor A. GRASSELLIand to my colleagues for the conversations on the subject of this work. In particular I would like to thank U. MONTANAmfor his useful suggestions in writing the paper.
REFERENCES 1. J. J. GlaSON, The Senses Considered as Perceptual System. Houghton Mifflin, Boston (1966). 2. R. M. PICKETT,Perceiving visual texture: a literature survey with remarks on the theory and application of man as a pattern receiver, Ph.D. Thesis, Harvard University, School of Public Health (1966). 3. P. LUCAS,P. LAUERand H. S~GL~r~ER, Method and notations for the formal definition of Programming Languages, IBM Laboratory, Vienna, Tech. Rep. TR 25087 (1968). 4. W. F. MILLERand A. C. SHAW, Linguistic methods in picture processing: a survey, AFIPS, FJCC33, part l, 279, 290 (1968). 5. M. PUZIN, Simulation of cellular patterns by computer graphics, M.Sc. Thesis, University of Toronto, Department of Computer Science, Tech. Rep. 10 (1969). 6. P. CONROY,Simulation of texture by computer graphics, M.SC. Thesis, University of Toronto, Department of Computer Science, Tech. Rep. 14 (1969). 7. A. ROSinED, Picture Processing by Computer, Chap. 10. Academic Press, New York (1969). 8. A. ROSENFI~.Dand E. B. TROY, Visual texture analysis, University of Maryland, Computer Science Center, Tech. Rep. 70-116 (1970). 9. B. S. LIPKIN and A. ROSENFELD(Eds.), Picture Processing and Psychopictorics. Academic Press, New York (1970). 10. L. CARLUCCI,Texture languages, l.E.l., Tech. Rep. B71-4, Pisa (1971).
APPENDIX In paragraph A 1 a summary of the notational conventions used in this work is given. In A2, an informal illustration of the method of definition of tree structured objects is exposed.
70
LUIGIA CARLUCCI
A1. Notational conventions The notation used in this work has been taken mostly from predicate calculus and set theory. Some notations are taken from LISP, for example conditional expressions. (In the system, many predicates and functions are defined by means of conditional expressions.) A conditional expression has the following form : Pl ~ el ;
Pn --* en
where p ~ , . . . , Pn are truth values and e t , . . . , e, are values. The conditional expression above evaluates ei where i is the smallest integer for which pi is true and Pi is false for allj such that 1 < j < i. If such an i does not exist the expression has no value. A2. Definition of tree structured objects What follows has been written with the aim to clarify the method which has been used for the definition of the system. In LUCAS(3) it is possible to find the complete and detailed exposition of what here is only sketched. A2.1. Objects. An object will in general be composed of components which are themselves objects. The components of an object are uniquely named so that, using the names, one can refer to the components. The names used to name components of objects are called selectors and it is assumed that there is a finite or countably infinite set S of symbols defined for that purpose. In the following the symbols s, st, s2 . . . . stand for selectors, the symbols A, A1, A 2 , . . . stand for arbitrary objects. For selecting components of given objects an operation is introduced. Given an object A the result of the application of a selector s to A is the component of A whose name in the formation of A is s, if any. By analogy with functional application the operation is represented by :
s(A) and reads "s applied to A". The application of s to A is said to yield the s-component of A. Since objects will be identified with linguistic expressions, i.e. finite constructs, the only interesting class of objects are those having a finite number of components and a finite depth of nesting. The latter means that the successive application of selectors to a given object will, after a finite number ofsteps, result in an object which has no further component, i.e. no selector can be meaningfully applied to it. Objects of this kinds are called elementary objects. It is assumed that there is a set of symbols EO to represent elementary objects. The symbols, eo, eot, eo2 • • • will in the sequel stand for elementary objects ; objects that are not elementary are called compound objects. A named object is a pair (s :A) where s is a selector and A is an elementary or compound object.
A formal system for texture languages
71
A compound object can now be uniquely described by a finite non empty set of named objects, either compound or elementary, where the names must be mutually different, i.e. a compound object is described by the set:
{(s~ :Ax>,...
#o((Sl :A1);... ;
(2) A compound object described by/Zo((S 1 :Ax) . . . . . (sn:An)) is represented by the tree shown in Fig. 10
&
$1
// \
®
s2
®
s.
®
FIG. 10. Graphic representation of a compound object.
where for (~), ( ~ , . . . ,
( ~ the respective tree representations have to be inserted.
A2.3. Composite selectors. By analogy with the functional composition, an operation for selectors is introduced in the following way : sl
. s~
. . . . . s.(A)
=
s~(sd
. . .
(s.(A))...)).
s l . s 2 - . . . . s, is called a composite selector. In the following X, gt, Z2 . . . . stand for composite selectors. It is possible to introduce the identity relation on objects by means of composite selectors in the following way : A1 = A2 if and only if either A1 = A2 = eo
72
LUIGIA CARLUCCI
or Z(A 1) = z(A2) for all selectors ~. Note that the direction bottom to top in the tree corresponds to the direction left to right in the composition of selectors. Example : see Fig. 11.
/
s3
$4
\ 003
O0 2
FIG. 11, Graphic representation of a compound object. Composition of selectors.
A2.4. Extension of the meaning o f # o. Some functions that operate on objects may be defined, namely #, of which/~o is a particular case, and 6. In the definition of texture languages the/a and 6 functions have never been used, so it has been preferred to omit their definition and consequently, the definition of Po as a particular case of/~ (see LucAs~31). In the following an extension of the meaning of Po, as defined in A2.1, is given./~o has been defined for finite lists of pairs, namely for finite lists of named objects: (~l :A1) . . . . . ( ~ , : A . ) . Here the extension is given for the case that the arguments are not a list, but an unordered finite set specified by some predicate. Let P(X, A) be a predicate defined on pairs : (selector:object) which is true for a finite non empty set of pairs. Then
~o({<~ :A>Ip~x, A)}) = go(
i.to({
{