- Email: [email protected]
Combinatorics and Image Processing
GRAPHICAL MODELS AND IMAGE PROCESSING
Vol. 59, No. 5, September, pp. 265–277, 1997 ARTICLE NO. IP970437
Combinatorics and Image Processing A. Bretto, J. Azema, H. Cherifi, and B. Laget Laboratoire de traitement du signal et instrumentation (UMR 5516), Equipe d’inge´nierie de la vision, Site GIAT Industries, 3, rue Javelin Pagnon BP 505, 42007 Saint-Etienne Cedex 1, France E-mail: [email protected] Received January 31, 1995; revised May 30, 1997; accepted June 18, 1997
problems in many areas of mathematics and computer science including computational geometry [28, 32], geographic connectivity analysis [14], pattern analysis [23], artificial intelligence [13], and computer vision [21]. As for structural relations, in numerous situations, graphs have turned out to provide the most appropriate tool for setting up the mathematical model. This is certainly one reason why graph theory has expanded so rapidly during the past decades [16]. The main drawback of proximity graphs is their use of binary neighborhood relations. Although binary relations are relevant for many basic situations, they cannot apprehend the structuration process of objects of an arbitrary nature. An image is an organization of objects in a space, but it cannot be disconnected from the human visual system and interpretation processes, and the appropriate relational algebra is not necessarily a binary one. The idea of looking at a family of sets as a generalized graph took shape around 1960. By regarding each set as a generalized edge one obtains a structure called a hypergraph [4]. Hypergraph theory has proved to be extremely useful for solving numerous applications in several fields of human activity [12, 16, 27]. It is thus natural to use this combinatoric tool in image processing. Such a general frame provides a new basis for models in different areas such as segmentation, restoration, and coding. In this paper, we introduce a new image neighborhood model built on hypergraph theory, which allows combinatorial and geometrical approaches to image processing. In Section 2 we begin with some preliminaries on graph and hypergraph theory. Section 3 presents the mathematical model required for this framework. Section 4 introduces an application to image segmentation. A region growing algorithm based on a neighborhood hypergraph is given. The Helly property [5] is used to improve the performances of this algorithm. This basic property of hypergraph theory tied to a geometrical visibility notion lets one develop a preprocessing algorithm. Section 5 gives some examples of experiments to validate the model and illustrates the segmentation algorithm performances.
In this paper, we introduce an image combinatorial model based on hypergraph theory. Hypergraph theory is an efficient formal frame for developing image processing applications such as segmentation. Under the assumption that a hypergraph satisfies the Helly property, we develop a segmentation algorithm that partitions the image by inspecting packets of pixels. This process is controlled by a homogeneity criterion. We also present a preprocessing algorithm that ensures that the hypergraph associated with any image satisfies the Helly property. We show that the algorithm is convergent. A performance analysis of the model and of the segmentation algorithm is included. 1997 Academic Press
1. INTRODUCTION
Images are obtained as the result of the impression left by the light sent or reflected by objects of the world onto a photosensitive surface. One of the main challenges of disciplines such as visual psychophysics, computer vision, and robotics is to understand how, from the local properties of an image and in a way that is largely independent of image acquisition device, stable and reliable information about the world can be obtained. One of the most important approaches in research fields in which mathematics is applied is to construct a model of a concrete situation in order to understand it better and, possibly, to influence it. One way to face the complex problems arising in computer vision is to analyze them at a more basic level. Each step of the interpretation process has its appropriate model. In recent years significant progress in image analysis has been witnessed. This has been the result of the development of new powerful models and the growth of computer capabilities. In many fields of research, objects and interrelations are represented by graphs. Proximity graphs are those in which points are fixed in n-dimensional space and adjacency is determined by the closeness of a pair of points relative to other points in the set. These graphs are useful in solving 265
1077-3169/97 $25.00 Copyright 1997 by Academic Press All rights of reproduction in any form reserved.
266
BRETTO ET AL.
2. COMBINATORIC BACKGROUND
Graphs are mathematical objects that can be used to model networks, data structures, process scheduling, computation, and a variety of other systems in which the relations between the objects in the system play a dominant role. Hypergraphs generalize the concept of a graph in order to cope with additional combinatorial problems. Our objective in this section is to introduce the terminology of graph and hypergraph theory and to define some familiar classes of graphs.
2.1. Elementary Graph Theoretic Definitions In the past quarter century we have seen a remarkable rise of graph theory as an important area of mathematics [3]. This theory can be seen as a unified framework for many problems in different fields such as traffic networks, electrical circuits, and biology. It would be difficult to give a complete account of the area of graph problems. There have been several good accounts in the literature [4]. Our purpose is to give an overview of graph theory. A graph is a couple G 5 (V; E ), where V is a set of elements called vertices, and E is a set of unordered pairs of members of V called edges. Given a graph G, we denote by G(x) the neighborhood of a vertex x, i.e., the set formed by all vertices adjacent to x: G(x ) 5 hy [ V, hx, yj [ E j. The number of neighborhood vertices x if the degree of x (we write dx). A graph is loopless, if it does not contain an edge of type (x, x). If dx is constant, the graph is called regular. We refer to Fig. 1 for a geometric presentation of a graph G. The vertices of the graphs are shown as points,
FIG. 2. Representation of the hypergraph. In this example we have the set of vertices S 5 hx1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 j, the set of hyperedges is E1 5 hx1 , x2 , x5 , x9 j, E2 5 hx2 , x3 , x4 j, E3 5 hx2 , x7 j, E4 5 hx5 , x6 , x7 , x8 , x9 j, the star centered on x2 is H(x2) 5 hE1 , E2 , E3 j, with degree dx2 5 3, and the star centered on x4 is H(x4) 5 hE2j with degree dx4 5 1.
while the edges are shown as lines connecting pairs of points.
2.2. Elementary Hypergraph Theoretic Definitions Recently, the need to go beyond the strict frame of graph theory to solve combinatorial problems of optimization has arisen. The initial idea that gave rise to the hypergraph theory was to extend certain classical results of graph theory. Theoretical aspects of hypergraphs are presented in [4]. As our main interest in this paper is to use combinatorial models, we will introduce basic tools that are needed. 2.2.1. Basic Concepts A hypergraph H on a set S is a family (Ei )i[I of nonempty subsets of S called hyperedges with
i[I
;i [ I I 5 h1, 2, . . . , nj n [ N. Let us note H 5 (S; (Ei )i[I ). For x [ S, a star of H (with center x ) is the set of hyperedges which contains x, and is called H(x ). The degree of x is the cardinality of the star H(x ) denoted by dx 5 card(H (x )).
FIG. 1. Example of graph G. In this graph the set of vertices is V 5 hx1 , x2 , x3 , x4j, the set of edges is E 5 hx1 , x2 j E2 5 hx2 , x3 j, E3 5 hx3 , x4 j, E4 5 hx2 , x4 jj, the neighborhood of x2 is G(x2) 5 hx1 ; x3 ; x4 j, and the degree of x2 is dx2 5 3.
A hypergraph is often represented on the plane by points standing for the vertices; a hyperedge with cardinality 1 will be represented by a loop; a hyperedge with cardinality 2 will be represented by a line connecting its two elements, and a hyperedge with cardinality .2 will be represented by a closed line surrounding its elements. Figure 2 is an example of the representation of a hypergraph.
COMBINATORICS AND IMAGE PROCESSING
267
Remark 1. A loopless graph with no isolated vertex is therefore a particular hypergraph with every hyperedge having cardinality 2. The hypergraph theory uses a very simple axiom that often leads to simplification in classical problems in combinatoric optimization. 2.2.2. Neighborhood Hypergraph Given a graph G, the hypergraph having the vertices of G as vertices and the neighborhood of these vertices as hyperedges (including these vertices) is called the neighborhood hypergraph of G. To each graph we can associate a neighborhood hypergraph: FIG. 3. Neighborhood systems.
HG 5 (S, (Ex 5 x < G(x ))). The neighborhood hypergraph associated with the example in Fig. 1 is S 5 hx1 , x2 , x3 , x4j Ex1 5 hx1 , x2j, Ex2 5 hx1 , x2 , x3 , x4 j, Ex3 5 hx2 , x3 , x4 j, Ex4 5 hx2 , x3 , x4 j.
where Zn identifies the feature intensity level and X identifies a set of points called pixels in the image. A digital image is defined on a grid and one of the most commonly needed pieces of information in the image plane is the neighborhood relationship among the image pixels. Now these notions will be briefly stated again. 3.1.2. Grids
Neighborhood hypergraphs have been widely used in operational research and mathematical computation [2, 11]. Nevertheless, little work has been done on the properties of these hypergraphs. 3. IMAGE PROCESSING AND NEIGHBORHOOD RELATIONS
Digital images are special data structures in which relationships among parts are crucial in investigating neighborhood relations. A number of mathematically defined models have already been used in image analysis [15, 29, 30, 36]. In this section some notions of graph theory applied to digital images are given. We introduce an alternative representation of the neighborhood relations which lets one define a new image neighborhood model.
3.1. Basic Concepts 3.1.1. Image There are several types of images: light intensity images, thermal images, etc. In general, we describe a continuous image by a two-dimensional function f (x, y), where (x, y ) denotes the spatial coordinates and f (x, y) the feature value of (x, y). A digital image is a two-dimensional discrete function which has been digitalized both in spatial coordinates and in magnitude feature value. Throughout this paper a digital image will be represented by the application I: X # Z2 R Zn with n $ 1,
The point sets investigated in image processing are finite sets, and the neighborhood of a point contains only a finite number of other points. In image processing we need a combinatorial topology since notions such as neighborhood and connectivity can be defined exactly by elements of a discrete space. In discrete geometry [36], mathematical morphology [30], and digital topology [20], a grid is a loopless regular graph, associated with a regular lattice L of Rn, which is triangular, square, or hexagonal. In image processing, the most commonly used grids are the 4-connected grids and the 8-connected grids defined on a square lattice of R2. Throughout this paper we will be concerned only with square lattices. Figure 3 shows the neighbors of a lattice point in such a grid. The 4-neighborhood system can be formalized using the distance d1(x, y ) 5 ui 2 ku 1 u j 2 l u, where (i, j ) and (k, l ) denote, respectively, the coordinates of x and y on the lattice. The 4-neighbors of a point x on a lattice are given by the following relation: ;x [ L: G(x ) 5 hy [ L, d1(x, y ) 5 1j (4-connected grid). To define the 8-neighborhood system we use the following distance:
268
BRETTO ET AL.
In image processing the notion of a neighborhood is a key concept. It is characterized by two essential properties: • A pixel is not a neighbor of itself. • If the pixel x is a neighbor of the pixel y, then y is a neighbor of x. In other words, the neighborhood relation is an irreflexive and symmetric homogeneous relation that can be represented by a loopless graph.
3.3. Neighborhood Relations and Hypergraph Model
FIG. 4. Neighborhood system at order 2.
dy(x, y ) 5 maxhui 2 ku, u j 2 l uj. The 8-neighbors of a point x on a lattice are defined by the following relation: ;x [ L: G(x ) 5 hy [ L, dy(x, y ) 5 1j (8-connected grid). Let r be the minimum number of edges between any two vertices of a grid; we can call a neighborhood system of order n associated with the vertex x the set of vertices defined by Gn (x ) 5 h y [ R, 1 # r(x, y ) # nj. Figure 4 shows the neighbors of order 2 of a lattice point. Remark 2. In the terminology of mathematical morphology, neighborhood systems of order n can be obtained from neighborhood systems of order 1 (called structuring element) by n successive dilations [18].
3.2. Neighborhood Relations and Graphs In the process of organization of different classes of objects, the introduction of relations is a crucial step. However, although the concept of relation is very simple, its formalization is not without influence on the objective and subjective representations of these objects. Relations between elements of the same set are called homogenous. More precisely, a homogeneous relation R on S is a subset of the cartesian product S 3 S. Elements (x, y ) [ S are said to be in relation R if (x, y ) [ R. Basically a graph is just a homogeneous relation where the edges represent the pairs of elements for which the relation holds.
In a graph theory, the neighborhood notion is always built through a homogeneous relation. In the human visual system the perception of an object is dependent on its surroundings. The relations used in this process are built between the object and its neighborhood. Such a relation belongs to the class of heterogeneous relations. This concept relates elements of two different sets and generalizes homogeneous relations. Formally, given two different sets S and E, a subset R of S 3 E is called a heterogeneous relation. Such a relation can be described by a hypergraph. Let H 5 (S, E 5 ((Ei)i[I ) be a hypergraph; the pairs (x, Ei ), where x belongs to the hyperedge Ei , define the heterogeneous relation. In image analysis a hypergraph associates a pixel x to its neighborhood, defined by a given neighborhood relation. On a grid Gn , to each pixel x we can associate a neighborhood Gn,a (x ), according to a predicate a. The predicate a may be completely arbitrary provided it is useful for a task domain. It may be defined on a set of points, it may use gray levels, color, depth, motion, or some symbolic representation of a set of gray levels, or it may be a combination of several predicates, and so on. Formally a heterogeneous neighborhood relation Gn,a on an image I is a mapping on X with values in the power set P(X ) of X defined by Gn,a (x ) 5 I21[I(x ) 2 a; I (x ) 1 a] > Gn (x ),
(1)
where x is a pixel, I(x ) is the feature intensity level of x, and I21(B) is the reciprocal image of B by I such that I21(B) 5 hx [ X, I(x) [ Bj with B [ P(X ). Remark 3. We use the reciprocal image because I is not necessarily a bijection. In practice the image cardinality is different from the cardinality of the feature intensity set, yet a necessary condition for bijectivity is that the cardinality of the two sets are equal. To this neighborhood we can associate a hypergraph defined by Hn,a 5 (X, (x < Gn,a (x ))).
COMBINATORICS AND IMAGE PROCESSING
This hypergraph will be called the image neighborhood hypergraph (INH). The INH offers new facilities for handling the topology and geometry of the image. We suggest that such a general frame can provide a new basis for models in different applications such as segmentation, restoration, and coding. Indeed, it is more appropriate to describe the neighborhood relations that can be used in such applications.
4. AN APPLICATION TO IMAGE SEGMENTATION
Segmentation of an image is a process to obtain meaningful simpler constituents. Four types of methodologies are used widely in attempts to solve this problem. Threshold techniques are based on partition of the feature values. Edge detection searches for parts in which a transition occurs from one uniform region to another. Region-based methods use a uniformity notion to build the partition of the image. Hybrid techniques combine edge and region criteria. We present here an algorithm for image segmentation which is based on the conventional region growing postulate of uniformity within a region. Region growing starts from small regions that are uniform and expands them s far as possible without violating a predicate. The general procedure is to compare a pixel to one of its neighbors. If the predicate is satisfied, the pixel is said to belong to the same region as this neighbor. This process can be apprehended as a graph construction. If a neighborhood hypergraph model is used, then the picture can be represented by a union of stars. So rather than grouping pixels one can aggregate stars according to a homogeneity criterion. This lets one build the partition by packets of pixels rather than inspect local properties by pairs. As the goal of this paper is to present some preliminary results dealing with picture segmentation as a combinatoric problem, for simplicity we assume that the predicate a is a constant value. We propose an elementary algorithm for gray level images. A preprocessing algorithm incorporating more global information is described next. Indeed, due to noise, the star’s cardinalities can be fairly small even in a homogeneous region. The Helly filter presented in Section 4.2 lets one optimize this parameter.
4.1. Covering and Selection Algorithm The algorithm proceeds in two parts. In the first part, a covering of the image by a minimal set of stars is computed. In the second part, selected stars are aggregated to obtain the regions. Let Hn,a be a neighborhood hypergraph associated with the (p 3 q)-image.
269
Step 1. Start with a covering of the image by a minimal set of stars. E 5 hH (x1), H (x2), . . . H(xn)j. The minimal cover that is used has the following property. Any pixel of the image belongs to at most one hyperedge of at least one star of the set E. By using a minimal cover, one limits the number of stars to be examined in later steps. To apprehend this step one can refer to Fig. 6. In this example, if we consider the pixel v6 , then it belongs to one hyperedge, E6 . Because a 5 20, it belongs to H(v18), H(v20), H(v21). In this case the minimal cover must include at least one of these stars. Step 2. Build star aggregate areas. For any area, proceed like this: —Let g(xi) be the gray level of the star center H(xi ). Examine any star H(xj ) intersecting with the star H(xi ). —If g(xj ) is within the interval [ g(xi ) 2 a, g(xi ) 1 a], the star H(xj ) is aggregated with H(xi ). —Repeat this process with the stars that intersect with at least one star of the area. Step 3. Reduce the number of areas. —For any intersecting area compute the following parameters: *center of gravity, *minimum gray level, *maximum gray level, *medium gray level. (These last three parameters are computed on star centers.) —If one center of gravity belongs to the intersection when two areas have parameters which are similar (6a), aggregate them. (Every star can belong to several aggregation areas (2a 1 1 different areas at most)). Step 4. Assign each star to an aggregation area to obtain a partition. Begin with the area containing the greatest number of stars and assign a common index to each of them. The index corresponds to this area. —Remove these stars from other areas to which they may belong. —Repeat the process until all the stars have been assigned. At the end of this step each star center has been assigned to an area. Step 5. Assign the pixels generating edges. —If a pixel belongs to several stars, it is assigned to the area of the star center whose gray level is the closest to its own. Finally the pixels which are neither star centers nor edge generating are assigned to the area of the neighboring pixel which has already been assigned and whose gray level is the closest.
270
BRETTO ET AL.
• Hn has the Helly property if and only if Gn is defined by dy . This theorem states that a necessary condition to confer the Helly property on the neighborhood hypergraph associated with a digital image, is to use the 8-neighborhood system of order n. Remark 4. In the following sections, we will use only the 8-neighborhood system of order n. 4.2.2. Helly Filter Algorithm
FIG. 5. Suppose that the hypergraphs have only three hyperedges, then the hypergraph (a) has the Helly property.
4.2. Helly Filter 4.2.1. Helly Property An important property of hypergraph theory is the socalled Helly property. This property generalizes the geometric notion of visibility. It is derived from convex geometry and has been generalized to hypergraphs by Berge [4]. A hypergraph has the Helly property if each family of hyperedges intersecting 2 3 2 (intersecting family) has a nonempty intersection (belongs to a star). Figure 5 shows two examples of intersecting hyperedges. • Fig. 5(a): A family of three hyperedges intersecting 2 by 2 with a nonempty intersection. • Fig. 5(b): A family of three hyperedges intersecting 2 by 2 with an empty intersection. This basic property is of great interest in hypergraph study. It helps to characterize specific hypergraph classes such as unimodular hypergraphs and balanced hypergraphs. As a digital image has geometrical and combinatorial aspects the Helly property is particularly suited to image processing. Let an image be represented by its neighborhood hypergraph. If this hypergraph has the Helly property, the centers of the stars characterize the common neighborhood relations of the stars. Therefore a star center is representative of the whole neighborhood. These centers may be a sufficient representation to extract global information. In discrete geometry it is important to define the grids that satisfy the Helly property. In [7] we prove the following theorem. THEOREM 1. Let Gn be a grid defined by a square network and let Hn be the neighborhood hypergraph associated with it.
Although the Helly property is not linked with the physical formation of an image, the smoothness of local properties can often help one construct a hypergraph which satisfies this property without drastically modifying the original image. We now present an algorithm for transforming an image so that it satisfies the Helly property. For any x element in the domain of I, proceed from Step 1 to Step 3. Let x < Gn,a (x ) be the edge generated by x in Hn,a . Step 1. Compute the set of intersecting families. Compute Ax the set of edges of Hn,a associated with the vertices of the neighborhood of x: Ax 5
< (y < G
n,a ( y)).
y[Gn(x)
Search in P(Ax) for the set of intersecting families. • Build the graph (GIx ) of the intersections of the edges Ax defined by: —the set of vertices V 5 Gn(x ), —the set of edges E defined by ( y1 , y2) [ E if and only if ( y1 < Gn,a ( y1)) > ( y2 < Gn,a ( y2)) ? B. Step 2. Search for the set of maximal intersecting families Ax . • Search for the complete maximal subgraphs of the graph GIx . (This problem is N 2 P complete but in practice, n is small, and thus Gn(x ) has a finite number of neighboring vertices.) Step 3. Compute the x gray level. Let Fx1, Fx2, . . . , Fxl be the set of the maximal intersecting families. (To avoid processing the same family several times, and to process every vertex, we will deal only with the families whose center of gravity is x. x is the center of gravity of the set of families Fx1 , Fx2 , . . . , Fxl , if y [ G(x ) for some y < Gn,a (y ) [ Fxi.) Fxi , is centered on x if the center of gravity of Fxi is x. (x is adjacent to any Fxi).
COMBINATORICS AND IMAGE PROCESSING
271
FIG. 6. A figure representing the hyperedges of H1,20 associated with the pixels adjacent to x. They make up a set called Ax . To make the drawing more easily understandable, the hyperedges centered on the pixels with even numbers are represented with continuous lines. Those centered on odd pixels are represented with dotted lines. Pixel indices are ordered according to the Freeman code.
• If l 5 1 (there is only one family with x as a center) and if x [ Fx1 then Fx1 is a star (the partial subhypergraph has the Helly property). —Else if l . 1 of (if l 5 1 and x Ó Fx1) build the star as follows. —Let Sx 5
• First, let Ax denote the set of hyperedges of the image centered on the pixels v1 , v2 , . . . , v8 neighbors of x: Ax 5 hE1 , E2 , E3 , E4 , E5 , E6 , E7 , E8 j E1 5 hv1 , v10 j E2 5 hv2 , v3 , v12 j E3 5 hv3 , v2 , v4 , v5 , v12 , v13 , v14 j E4 5 hv4 , v3 , v5 , v13 , v14 , v16 , v17 j E5 5 hv5 , v3 , v4 , v16 , v18 j E6 5 hv6 , v18 , v19 , v20 , v21 j E7 5 hv7 , v8 , v20 , v21 j E8 5 hv8 , v7 , v9 , v21 j.
FIG. 7. A figure representing the graph GIx of the intersections of the hyperedges of Ax .
272
BRETTO ET AL.
• Then we build the graph of the intersections of the hyperedges of Ax , denoted as GIx as shown in Fig. 7. It is known that the maximum intersecting families correspond to the maximum complete subgraphs of GIx . Thus, we have the set Fx of the maximum intersecting families in accordance with Fx 5 hFx1 , Fx2 , Fx3 j with Fx1 5 hEv2 , Ev3 , Ev4 , Ev5 j, Fx2 5 hEv6 , Ev7 , Ev8 j, Fx3 5 hEv5 , Ev6 j,
As x Ó Sx , its gray level value is given by I(x ) 5 round
S
D
Imax(Sx ) 1 Imin(Sx ) 5 round(97.5). 2
Remark 5. Suppose that the image is defined on a neighborhood system other than the 8-neighborhood system of order n. According to Theorem 1 this one does not have the Helly property. So we cannot confer this property on the neighborhood hypergraph associated with the image (this one being defined using relation 1 which depends on the grid).
4.3. Properties of the Algorithms 4.3.1. Properties of Covering and Selection Algorithm
As in standard region growing, the choice of the seed is critical. Nevertheless, since this algorithm works on aggregates of pixels it may be hoped that it is less dependent Fx4 5 hEv1 j. than segmentation algorithms dealing with pixels. As the image size is finite (p 3 q), the convergence is Thus, the set of pixels Sx is the following: obvious. The minimal Helly families covering up the image can be found in constant time. The cover takes at most Sx 5 hv1 , v2 , v3 , v4 , v5 , v6 , v7 , v8 , v9 , v10 , v11 , v12 , v13 , v14 , the Helly families number and the partition is computed v16 , v18 , v19 , v20 , v21 j. in a time equivalent to the maximum intersecting Helly families number. • We calculate the maximum distance between two gray levels of Sx: 4.3.2. Properties of the Helly Filter
and
DImax 5 Imax 2 Imin 5 I(v7) 2 I(v13) 5 115 2 60
THEOREM 2. The Helly filter is convergent, and if p is the number of neighbors of a vertex on the grid, p2 is an upper bound of the iteration number of the filter.
Proof. At the ith process of the algorithm, a vertex whose gray level has been changed previously has its gray 5 55 .2a 5 40. level changed again if x is again possibly the center of We then remove the pixel whose gray level is the farthest gravity of H(x ), Fxp , . . . . . . . , Fxp1j . (H(x ) is the star centered on x and Fxp1j is a maximal intersecting family that from the one of x, namely v13 . It follows that can be a star.) The gray level of the neighbor y of x is Sx 5 hv1 , v2 , v3 , v4 , v5 , v6 , v7 , v8 , v9 , v10 , v11 , v12 , v13 , v14 , changed if y is a candidate as gravity center of H(x ), H(y), Fxp , . . . . . . . , Fxp1j , Fyp , . . . , Fyp1k . v16 , v18 , v19 , v20 , v21 j —The cardinality of a star is increasing. Let H(x) be a star built at the ith process (the neighborhood relation and being reflexive, it is sufficient to reason on x). Let H(x), H( y), Fxp , ......., Fxp1j be the intersecting families DImax 5 115 2 65 5 50 .2a 5 40. at the i 1 1 ith process. Any generator vertex of the edges We go through this process again until DImax is less than of these families are the neighbors of x; thus their gray levels are at (1a, 2a) from I(x) and consequently will not 2a. Eventually, we have be eliminated by building the new star. The new middle will be defined at more or less the gray level of these Sx 5 hv2 , v6 , v7 , v8 , v9 , v12 , v16 , v18 , v19 , v20 , v21 j vertices. H(x) can only take edges in the families Fxp1j . —If the number of intersecting families accepting x as and a center of gravity is finite and the cardinality of H(x) is DImax 5 115 2 80 5 35 ,2a 5 40. limited, then the convergence is proved.
COMBINATORICS AND IMAGE PROCESSING
273
—A vertex x, on the grid having p neighbors having themselves p neighbors, will have its gray level changed p2 times at most. n THEOREM 3. If the set of pixels for which the gray level values must be modified are changed after the whole image has been inspected, the output of the Helly filter is unique. Proof. Let x be a pixel whose value has to be modified by the Helly filter. Suppose that it is processed at the ith rank in one case, and at the ith rank in another case (i ? j ). As the gray level values are changed at the end of an iteration, its neighbors have the same gray level values in both cases. Consequently the result of the processing which is only neighbor dependent is rank independent. n 5. EXPERIMENTAL RESULTS
We now turn to several experiments made with the algorithms introduced above. The first set of tests is made on a synthetic image. It is used to compare a standard region growing algorithm [17] using the same predicate as the proposed one in terms of subjective and objective quality of segmentation, speed, and stability. The standard region growing algorithm is abbreviated SRA, while the covering and selection algorithm is abbreviated CSA. The second set of tests is used to judge whether the model agrees with the physical nature of digital images. In other words, if the Helly property is a natural property of digital images then the model will be appropriate. A third set of tests is used to evaluate the performance of the segmentation algorithm proposed in Section 5.1, with different homogeneity levels for various types of natural images. The results of CSA segmentation are compared to classical region growing based segmentation. For all these tests, we use a 8-neighborhood system of order 1.
5.1. Experimental Properties of the Algorithms on Synthetic Images For this analysis, we generated a sequence of 128 3 128 synthetic images made of two regions, with a difference in gray levels of 40 and separated by a vertical transition at x 5 64. We added Gaussian noise with zero mean and standard deviation of s 5 4, 8, 10, 15, 20, 30. 5.1.1. Quality of the Segmentation As an objective measure of quality, we use the number of regions generated by the algorithms. For all images studied, the covering and selection algorithm produces fever regions than the standard region growing algorithm. If the Helly filter is used as a preprocessing step the number
FIG. 8. Evolution of the number of regions as a function of a. For a synthetic image with noise s 5 4, (a) standard region growing algorithm, (b) covering and selection algorithm, (c) Helly filter and standard region algorithm, (d) Helly filer and covering and selection algorithm.
of regions decreases. Figure 8 is a typical example of these behaviors. In terms of subjective quality we can establish a classification of the algorithms. Whatever the case, CSA gives better results than SRA. Using the Helly filter always improves the quality. However, it is more efficient with CSA. The differences are all the more obvious as the noise level increases. Performance of these two algorithms is very poor when s . 15. Figure 9 gives a visual illustration of these observations. In each case the value of a used is the one which corresponds to the best visual quality. Table 1 summarized the parameter values for this example. 5.1.2. Computational Time The computational time required by CSA is approximately twice that required by SRA. However, no particular care has been taken to optimize the code. Noise does not affect the value of the computation time. Helly filter computational cost is greatly dependent on the noise level. The efficiency comparison is summaried in Table 2. 5.1.3. Stability To examine the effect of seed position on stability, we choose this value at random. Fifteen seeds were generated for the set of images. If the value of a is appropriate the results of the segmentation are similar when the noise level is low (s , 10). Each time the results differ only very small regions are concerned. Visually these differences are imperceptible. If the regions are noisy the segmentation may be incorrect. The Helly filter associated with CSA proves to be adequate to prevent this up to a certain point.
274
BRETTO ET AL.
TABLE 2 Processing Time Image
SRA
CSA
Helly filter
s54 s 5 10
1.8s 2.0s
2.5s 3.7s
3.9s 5.1s
Note. The algorithms are processed on a SUN Sparc 5 workstation. User time is reported.
5.2. Experimental Properties of the Algorithms on Natural Images In this section we present experimental results obtained with three 512 3 512 8-bit natural images shown in Figs. 10a, 11a, and 12a. We evaluate the adequacy of the model
FIG. 9. (a) Synthetic image with noise s 5 10, (b) standard region growing segmentation, (c) covering and selection segmentation, (d) Helly filter and standard region segmentation, (e) Helly filter and covering and selection segmentation.
TABLE 1 Parameter Values for the Images in Fig. 9 Algorithms
a
Number of regions
SRA CSA Helly 1 CSA Helly 1 SRA
6 19 19 6
1811 13 8 1611
FIG. 10. (a) Original image map, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 63, b 5 76, (e) Helly filter and SRA segmentation a 5 63, b 5 76.
275
COMBINATORICS AND IMAGE PROCESSING
TABLE 3 Intersecting Families Which Are Not Stars a
5
10
15
20
30
Percentage for smoke Percentage for Tiffany Percentage for map
3.1 18.3 16.3
1.5 11.25 9.8
0.67 7.9 5.7
0.35 4.4 2.7
0.15 2.1 1.9
and then we compare the segmentation results of the proposed algorithm with a standard region growing algorithm result. 5.2.1. Model Performance For these images we compare the number of intersecting families and the number of stars for various values of the homogeneity criterion. Table 3 shows the percentage of nonstar families for five different values of alpha. As expected, we observe asymptotic behavior with increasing values of alpha for all the images under study. Indeed, if the homogeneity criterion covers the entire range of feature values every point of the grid is the center of a star. Intersecting families which are not stars are representative of areas in the image where feature values have higher variations than the homogeneity criterion. They are correlated with the discontinuities of the image. For a smooth image such as Smoke the model is valid even for low values of the homogeneity criterion. For images with high contrast zones, such as Tiffany, the knee of the curves can be interpreted as the minimal value of alpha necessary to apprehend the complexity of the image.
We consider the number of regions generated by the algorithms. Generally if a region is considered to be homogeneous, the least number of regions is desired. In the table, we show the results for both algorithms. For CSA 1 Helly filter the results are given for two values of the a parameter. The results for SRA are given when used directly on the original image and also when applied to the same image previously processed by Helly filtering. The reported values of the control parameters a and b are the ones that give the best visual quality to a human interpreter. These values have been obtained through a trial and error based adjustment. As shown in Table 4, the Helly 1 CSA segmentation results in a small number of regions as compared to both SRA implementations. The results of the segmentation are also given as follows. In each example, the original image and the segment images are reported. In the segmented images, each region is represented by the mean value of its intensities. These results demonstrate that the CSA algorithm should produce significantly better segmentations than SRA. One can see that all the results show a very satisfying regional homogeneity. This result can be explained by the Helly filtering that harmonizes the pixel values in a neighborhood. Nevertheless this effect does not interfere with the emergence of small homogeneous regions. As the homogeneity criterion value decreases, it can be observed that different new regions appear and the segmentation provides more accurate region limits. In the ‘‘map’’ image shown in Fig. 10, where thin elongated regions appear, one
TABLE 4 Parameter Values for Figs. 10–12
5.2.2. Segmentation Performance In this section we will illustrate the effectiveness of the overall algorithm (CSA 1 Helly filter). To compare with our method, we consider the standard region growing algorithm. As natural images are far more complex than the synthetic images used in the first set of tests, we add to the previous predicate used in SRA another parameter to control the aggregation process. A pixel x is aggregated to a region Ri if its gray level I(x ) is within a and if it is within the mean value of the region for which it is candidate: x [ Ri ⇔ for y [ Ri and x [ G( y), uI(x ) 2 I( y)u # a and uI(x ) 2 Meanz[Ri (I(z ))u # b. b is a measure of how different the pixel is from the region it adjoins. As in segmentation of a given scene, there is rarely one unambigous partition and the human interpreter is often the arbiter of the segmentation quality. Thus the algorithms are evaluated on two fronts.
(a) Parameter values for the images of Fig. 10 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 63 63
b
Number of regions of Map
76 76
2357 406 3516 3122
(b) Parameter values for the images of Fig. 11 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 38 38
b
Number of regions of Smoke
76 76
80 20 349 302
(c) Parameter values for the images of Fig. 12 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 46 46
b
Number of regions of Tiffany
76 76
1514 492 3874 2985
276
BRETTO ET AL.
various approaches of image analysis. This model, based on hypergraph theory, allows us to use all the mathematical background of combinatorics. We have presented propositions for solving basic image processing problems such as segmentation deduced from this model. At this point our goal is not to optimize the performance of these algorithms, but only to suggest that this new approach can be an effective approach to image processing. Through this application we showed that the hypergraph associated to an image lets one process a picture with very simple assumptions. Naturally the algorithms presented can be improved in several ways (sufficient conditions for a hypergraph satisfy the Helly property, the uniformity criterion, adaptativity, parameters and decision rules, etc.). Now that the model has been validated we are trying to extend it to other fields of image processing, such as restoration and coding.
FIG. 11. (a) Original image Smoke, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 38, b 5 51 Helly filter and SRA segmentation a 5 38, b 5 51.
can see that the segmentation process is very powerful. Indeed, most of these regions are segmented integrally: the spatial resolution of the approach is very fine. In the image Smoke shown in Fig. 11, CSA has preserved the small white region in the lower right-hand corner of the image while SRA has produced more than a unique region. One can also see that SRA cannot recover the begining of the stream. It can be seen in Fig. 12 that the major features on the face are found in the image of Tiffany. For both values of a, the eyes and the nails are particularly preserved by CSA while SRA fails to recover these features, although a slight improvement is gained by the Helly filtering preprocessing. 6. CONCLUSION
Throughout this paper our emphasis has been on introducing a new model of digital imaging which can unify the
FIG. 12. (a) Original Image Tiffany, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 51, b 5 76, (e) Helly filter and SRA segmentation a 5 51, b 5 76.
COMBINATORICS AND IMAGE PROCESSING
ACKNOWLEDGMENT The authors are grateful to the referees for their helpful comments leading to improvements of the manuscript.
REFERENCES 1. H. Adeli and S. L. Hung, Machine Learning—Neuronal Networks, Genetic Algorithms and Fuzzy Systems, Wiley, New York, 1994. 2. H. J. Bandelt and E. Pesch, Dismantling absolute retracts of reflexive graphs, Eur. J. Combin. 10, 1989, 211–220. 3. C. Berge, Graphs, North-Holland, Amsterdam, 1985. 4. C. Berge, Hypergraphs, North-Holland, Amsterdam, 1987. 5. C. Berge, The Helly property, Southeast Asian Math. Soc. Bull. 1, 1979, 16–19. 6. C. A. Bouman and M. Shapiro, A systematic way for region-based on Markov field model. PR 27, 1994, 162–177. 7. A. Bretto, Convexite´ abstraite, Hypergraphe de voisinage et proprie´te´ de Helly, une application a` la the´orie de l’image. Ph.D. thesis, Universite´ de Saint Etienne, 1994. 8. A. Bretto and B. Laget, Neighborhood hypergraph and image analysis, in Signal Process. II, 5–9 September, 1994, Fontainebleau. 9. F. Cheevasuvit, H. Maitre, and D. Vidal-Madjar, A robust method for picture segmentation based on split and merge procedure, Comput. Vision Graphics Image Process. 34, 1986, 268–281. 10. S. Y. Chen, W. C. Lin, and C. T. Chen, Split and merge image segmentation based on localized feature analysis and statistical tests, CVGIP: Graphics Models Image Process. 53, 1993, 457–475. 11. F. F. Dragan, Domination in quadrangle-free helly graphs, Cybernet. Syst. Anal. 29 (6), 1993, 822–829. 12. R. Fagin, Degrees of acyclicity for hypergraphs and relational database schemes, J. Assoc. Comput. Mach. 30, 1983, 514–550. 13. FUZZ–IEEE, Third IEEE International Conference on Fuzzy Systems, Orlando, FL, June 26–July 2, 1994. 14. K. Geman and D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intel. PAMI-6, 1984, 721–741. 15. S. R. Gabriel and R. R. Sokal, A new statistical approach to geographic variations analysis, Syst. Zool. 18, 1969, 259–278. 16. M. Gondran and M. Minoux, Graphs and Algorithms, Wiley, New York, 1984.
277
17. R. C. Gonzalez and P. Wintz, Digital Image Processing, Addison– Wesley, Reading, MA, 1977. 18. H. J. A. M. Heijmans, Morphological Image Operators, Academic Press, San Diego, 1994. 19. I. Y. Kim and H. S. Yang, A multiscale random field model for Bayesian image segmentation, T-IP 3, 1994, 841–852. 20. V. A. Kovalevsky, Finite topology as applied to image processing, Comput. Vision Graphics Image Process. 46, 1989, 141–161. 21. M. D. Levine, Vision in Man and Machines, McGraw–Hill, New York, 1985. 22. J. S. Lim, Two dimensional signal and image processing, Signal Processing Series, Prentice Hall, New York, 1990. 23. H. Niemann, Pattern Analysis and Understanding, Springer-Verlag, Berlin/New York, 1992. 24. J. O’Rourke, Computational Geometry, Cambridge Univ. Press, Cambridge, UK, 1994. 25. J. Pach, New Trends in Discrete and Computational Geometry, Springer-Verlag, Berlin, 1993. 26. N. R. Pal and S. K. Pal, A review on image segmentation techniques, Pattern Recog. 26 (9), 1993, 1277–1294. 27. J. L. Peterson, Petri nets, Comput. Surv. 9 (3), 1997, 223–252. 28. F. P. Preparata and M. I. Shamos, Computational Geometry, SpringerVerlag, New York, 1985. 29. A. Rosenfeld, Digital Image Processing, Academic Press, San Diego, 1982. 30. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, San Diego, 1982. 31. J. Serra, Mathematical Morphology, Vol. 2, Academic Press, San Diego, 1988. 32. K. J. Supowit, The relative neighborhood graph with an application to minimum spaning trees, J. Assoc. Comput. Mach. 30, 1983, 428–447. 33. J. T. Tou and R. C. Gonzalez, Pattern Recognition Principles, Addison–Wesley, Reading, MA, 1974. 34. M. Tuceryant and T. Chorzempa, Relative sensitivity of a family of closest point graphs in computer vision applications, Pattern Recognit. 24, (5), 1991, 361–373. 35. International Workshop on Graph Grammars, Williamsburg VA, November 14–16, 1994. 36. K. Voss, Discrete Images, Objects and Functions in Zn, Algorithms and Combinatorics, Springer-Verlag, Berlin, New York, 1990. 37. S. W. Zucker, Region growing: childhood and adolescence, Comput. Vision Graphics Image Process. 5, 1976, 382–399.
Vol. 59, No. 5, September, pp. 265–277, 1997 ARTICLE NO. IP970437
Combinatorics and Image Processing A. Bretto, J. Azema, H. Cherifi, and B. Laget Laboratoire de traitement du signal et instrumentation (UMR 5516), Equipe d’inge´nierie de la vision, Site GIAT Industries, 3, rue Javelin Pagnon BP 505, 42007 Saint-Etienne Cedex 1, France E-mail: [email protected] Received January 31, 1995; revised May 30, 1997; accepted June 18, 1997
problems in many areas of mathematics and computer science including computational geometry [28, 32], geographic connectivity analysis [14], pattern analysis [23], artificial intelligence [13], and computer vision [21]. As for structural relations, in numerous situations, graphs have turned out to provide the most appropriate tool for setting up the mathematical model. This is certainly one reason why graph theory has expanded so rapidly during the past decades [16]. The main drawback of proximity graphs is their use of binary neighborhood relations. Although binary relations are relevant for many basic situations, they cannot apprehend the structuration process of objects of an arbitrary nature. An image is an organization of objects in a space, but it cannot be disconnected from the human visual system and interpretation processes, and the appropriate relational algebra is not necessarily a binary one. The idea of looking at a family of sets as a generalized graph took shape around 1960. By regarding each set as a generalized edge one obtains a structure called a hypergraph [4]. Hypergraph theory has proved to be extremely useful for solving numerous applications in several fields of human activity [12, 16, 27]. It is thus natural to use this combinatoric tool in image processing. Such a general frame provides a new basis for models in different areas such as segmentation, restoration, and coding. In this paper, we introduce a new image neighborhood model built on hypergraph theory, which allows combinatorial and geometrical approaches to image processing. In Section 2 we begin with some preliminaries on graph and hypergraph theory. Section 3 presents the mathematical model required for this framework. Section 4 introduces an application to image segmentation. A region growing algorithm based on a neighborhood hypergraph is given. The Helly property [5] is used to improve the performances of this algorithm. This basic property of hypergraph theory tied to a geometrical visibility notion lets one develop a preprocessing algorithm. Section 5 gives some examples of experiments to validate the model and illustrates the segmentation algorithm performances.
In this paper, we introduce an image combinatorial model based on hypergraph theory. Hypergraph theory is an efficient formal frame for developing image processing applications such as segmentation. Under the assumption that a hypergraph satisfies the Helly property, we develop a segmentation algorithm that partitions the image by inspecting packets of pixels. This process is controlled by a homogeneity criterion. We also present a preprocessing algorithm that ensures that the hypergraph associated with any image satisfies the Helly property. We show that the algorithm is convergent. A performance analysis of the model and of the segmentation algorithm is included. 1997 Academic Press
1. INTRODUCTION
Images are obtained as the result of the impression left by the light sent or reflected by objects of the world onto a photosensitive surface. One of the main challenges of disciplines such as visual psychophysics, computer vision, and robotics is to understand how, from the local properties of an image and in a way that is largely independent of image acquisition device, stable and reliable information about the world can be obtained. One of the most important approaches in research fields in which mathematics is applied is to construct a model of a concrete situation in order to understand it better and, possibly, to influence it. One way to face the complex problems arising in computer vision is to analyze them at a more basic level. Each step of the interpretation process has its appropriate model. In recent years significant progress in image analysis has been witnessed. This has been the result of the development of new powerful models and the growth of computer capabilities. In many fields of research, objects and interrelations are represented by graphs. Proximity graphs are those in which points are fixed in n-dimensional space and adjacency is determined by the closeness of a pair of points relative to other points in the set. These graphs are useful in solving 265
1077-3169/97 $25.00 Copyright 1997 by Academic Press All rights of reproduction in any form reserved.
266
BRETTO ET AL.
2. COMBINATORIC BACKGROUND
Graphs are mathematical objects that can be used to model networks, data structures, process scheduling, computation, and a variety of other systems in which the relations between the objects in the system play a dominant role. Hypergraphs generalize the concept of a graph in order to cope with additional combinatorial problems. Our objective in this section is to introduce the terminology of graph and hypergraph theory and to define some familiar classes of graphs.
2.1. Elementary Graph Theoretic Definitions In the past quarter century we have seen a remarkable rise of graph theory as an important area of mathematics [3]. This theory can be seen as a unified framework for many problems in different fields such as traffic networks, electrical circuits, and biology. It would be difficult to give a complete account of the area of graph problems. There have been several good accounts in the literature [4]. Our purpose is to give an overview of graph theory. A graph is a couple G 5 (V; E ), where V is a set of elements called vertices, and E is a set of unordered pairs of members of V called edges. Given a graph G, we denote by G(x) the neighborhood of a vertex x, i.e., the set formed by all vertices adjacent to x: G(x ) 5 hy [ V, hx, yj [ E j. The number of neighborhood vertices x if the degree of x (we write dx). A graph is loopless, if it does not contain an edge of type (x, x). If dx is constant, the graph is called regular. We refer to Fig. 1 for a geometric presentation of a graph G. The vertices of the graphs are shown as points,
FIG. 2. Representation of the hypergraph. In this example we have the set of vertices S 5 hx1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 j, the set of hyperedges is E1 5 hx1 , x2 , x5 , x9 j, E2 5 hx2 , x3 , x4 j, E3 5 hx2 , x7 j, E4 5 hx5 , x6 , x7 , x8 , x9 j, the star centered on x2 is H(x2) 5 hE1 , E2 , E3 j, with degree dx2 5 3, and the star centered on x4 is H(x4) 5 hE2j with degree dx4 5 1.
while the edges are shown as lines connecting pairs of points.
2.2. Elementary Hypergraph Theoretic Definitions Recently, the need to go beyond the strict frame of graph theory to solve combinatorial problems of optimization has arisen. The initial idea that gave rise to the hypergraph theory was to extend certain classical results of graph theory. Theoretical aspects of hypergraphs are presented in [4]. As our main interest in this paper is to use combinatorial models, we will introduce basic tools that are needed. 2.2.1. Basic Concepts A hypergraph H on a set S is a family (Ei )i[I of nonempty subsets of S called hyperedges with
i[I
;i [ I I 5 h1, 2, . . . , nj n [ N. Let us note H 5 (S; (Ei )i[I ). For x [ S, a star of H (with center x ) is the set of hyperedges which contains x, and is called H(x ). The degree of x is the cardinality of the star H(x ) denoted by dx 5 card(H (x )).
FIG. 1. Example of graph G. In this graph the set of vertices is V 5 hx1 , x2 , x3 , x4j, the set of edges is E 5 hx1 , x2 j E2 5 hx2 , x3 j, E3 5 hx3 , x4 j, E4 5 hx2 , x4 jj, the neighborhood of x2 is G(x2) 5 hx1 ; x3 ; x4 j, and the degree of x2 is dx2 5 3.
A hypergraph is often represented on the plane by points standing for the vertices; a hyperedge with cardinality 1 will be represented by a loop; a hyperedge with cardinality 2 will be represented by a line connecting its two elements, and a hyperedge with cardinality .2 will be represented by a closed line surrounding its elements. Figure 2 is an example of the representation of a hypergraph.
COMBINATORICS AND IMAGE PROCESSING
267
Remark 1. A loopless graph with no isolated vertex is therefore a particular hypergraph with every hyperedge having cardinality 2. The hypergraph theory uses a very simple axiom that often leads to simplification in classical problems in combinatoric optimization. 2.2.2. Neighborhood Hypergraph Given a graph G, the hypergraph having the vertices of G as vertices and the neighborhood of these vertices as hyperedges (including these vertices) is called the neighborhood hypergraph of G. To each graph we can associate a neighborhood hypergraph: FIG. 3. Neighborhood systems.
HG 5 (S, (Ex 5 x < G(x ))). The neighborhood hypergraph associated with the example in Fig. 1 is S 5 hx1 , x2 , x3 , x4j Ex1 5 hx1 , x2j, Ex2 5 hx1 , x2 , x3 , x4 j, Ex3 5 hx2 , x3 , x4 j, Ex4 5 hx2 , x3 , x4 j.
where Zn identifies the feature intensity level and X identifies a set of points called pixels in the image. A digital image is defined on a grid and one of the most commonly needed pieces of information in the image plane is the neighborhood relationship among the image pixels. Now these notions will be briefly stated again. 3.1.2. Grids
Neighborhood hypergraphs have been widely used in operational research and mathematical computation [2, 11]. Nevertheless, little work has been done on the properties of these hypergraphs. 3. IMAGE PROCESSING AND NEIGHBORHOOD RELATIONS
Digital images are special data structures in which relationships among parts are crucial in investigating neighborhood relations. A number of mathematically defined models have already been used in image analysis [15, 29, 30, 36]. In this section some notions of graph theory applied to digital images are given. We introduce an alternative representation of the neighborhood relations which lets one define a new image neighborhood model.
3.1. Basic Concepts 3.1.1. Image There are several types of images: light intensity images, thermal images, etc. In general, we describe a continuous image by a two-dimensional function f (x, y), where (x, y ) denotes the spatial coordinates and f (x, y) the feature value of (x, y). A digital image is a two-dimensional discrete function which has been digitalized both in spatial coordinates and in magnitude feature value. Throughout this paper a digital image will be represented by the application I: X # Z2 R Zn with n $ 1,
The point sets investigated in image processing are finite sets, and the neighborhood of a point contains only a finite number of other points. In image processing we need a combinatorial topology since notions such as neighborhood and connectivity can be defined exactly by elements of a discrete space. In discrete geometry [36], mathematical morphology [30], and digital topology [20], a grid is a loopless regular graph, associated with a regular lattice L of Rn, which is triangular, square, or hexagonal. In image processing, the most commonly used grids are the 4-connected grids and the 8-connected grids defined on a square lattice of R2. Throughout this paper we will be concerned only with square lattices. Figure 3 shows the neighbors of a lattice point in such a grid. The 4-neighborhood system can be formalized using the distance d1(x, y ) 5 ui 2 ku 1 u j 2 l u, where (i, j ) and (k, l ) denote, respectively, the coordinates of x and y on the lattice. The 4-neighbors of a point x on a lattice are given by the following relation: ;x [ L: G(x ) 5 hy [ L, d1(x, y ) 5 1j (4-connected grid). To define the 8-neighborhood system we use the following distance:
268
BRETTO ET AL.
In image processing the notion of a neighborhood is a key concept. It is characterized by two essential properties: • A pixel is not a neighbor of itself. • If the pixel x is a neighbor of the pixel y, then y is a neighbor of x. In other words, the neighborhood relation is an irreflexive and symmetric homogeneous relation that can be represented by a loopless graph.
3.3. Neighborhood Relations and Hypergraph Model
FIG. 4. Neighborhood system at order 2.
dy(x, y ) 5 maxhui 2 ku, u j 2 l uj. The 8-neighbors of a point x on a lattice are defined by the following relation: ;x [ L: G(x ) 5 hy [ L, dy(x, y ) 5 1j (8-connected grid). Let r be the minimum number of edges between any two vertices of a grid; we can call a neighborhood system of order n associated with the vertex x the set of vertices defined by Gn (x ) 5 h y [ R, 1 # r(x, y ) # nj. Figure 4 shows the neighbors of order 2 of a lattice point. Remark 2. In the terminology of mathematical morphology, neighborhood systems of order n can be obtained from neighborhood systems of order 1 (called structuring element) by n successive dilations [18].
3.2. Neighborhood Relations and Graphs In the process of organization of different classes of objects, the introduction of relations is a crucial step. However, although the concept of relation is very simple, its formalization is not without influence on the objective and subjective representations of these objects. Relations between elements of the same set are called homogenous. More precisely, a homogeneous relation R on S is a subset of the cartesian product S 3 S. Elements (x, y ) [ S are said to be in relation R if (x, y ) [ R. Basically a graph is just a homogeneous relation where the edges represent the pairs of elements for which the relation holds.
In a graph theory, the neighborhood notion is always built through a homogeneous relation. In the human visual system the perception of an object is dependent on its surroundings. The relations used in this process are built between the object and its neighborhood. Such a relation belongs to the class of heterogeneous relations. This concept relates elements of two different sets and generalizes homogeneous relations. Formally, given two different sets S and E, a subset R of S 3 E is called a heterogeneous relation. Such a relation can be described by a hypergraph. Let H 5 (S, E 5 ((Ei)i[I ) be a hypergraph; the pairs (x, Ei ), where x belongs to the hyperedge Ei , define the heterogeneous relation. In image analysis a hypergraph associates a pixel x to its neighborhood, defined by a given neighborhood relation. On a grid Gn , to each pixel x we can associate a neighborhood Gn,a (x ), according to a predicate a. The predicate a may be completely arbitrary provided it is useful for a task domain. It may be defined on a set of points, it may use gray levels, color, depth, motion, or some symbolic representation of a set of gray levels, or it may be a combination of several predicates, and so on. Formally a heterogeneous neighborhood relation Gn,a on an image I is a mapping on X with values in the power set P(X ) of X defined by Gn,a (x ) 5 I21[I(x ) 2 a; I (x ) 1 a] > Gn (x ),
(1)
where x is a pixel, I(x ) is the feature intensity level of x, and I21(B) is the reciprocal image of B by I such that I21(B) 5 hx [ X, I(x) [ Bj with B [ P(X ). Remark 3. We use the reciprocal image because I is not necessarily a bijection. In practice the image cardinality is different from the cardinality of the feature intensity set, yet a necessary condition for bijectivity is that the cardinality of the two sets are equal. To this neighborhood we can associate a hypergraph defined by Hn,a 5 (X, (x < Gn,a (x ))).
COMBINATORICS AND IMAGE PROCESSING
This hypergraph will be called the image neighborhood hypergraph (INH). The INH offers new facilities for handling the topology and geometry of the image. We suggest that such a general frame can provide a new basis for models in different applications such as segmentation, restoration, and coding. Indeed, it is more appropriate to describe the neighborhood relations that can be used in such applications.
4. AN APPLICATION TO IMAGE SEGMENTATION
Segmentation of an image is a process to obtain meaningful simpler constituents. Four types of methodologies are used widely in attempts to solve this problem. Threshold techniques are based on partition of the feature values. Edge detection searches for parts in which a transition occurs from one uniform region to another. Region-based methods use a uniformity notion to build the partition of the image. Hybrid techniques combine edge and region criteria. We present here an algorithm for image segmentation which is based on the conventional region growing postulate of uniformity within a region. Region growing starts from small regions that are uniform and expands them s far as possible without violating a predicate. The general procedure is to compare a pixel to one of its neighbors. If the predicate is satisfied, the pixel is said to belong to the same region as this neighbor. This process can be apprehended as a graph construction. If a neighborhood hypergraph model is used, then the picture can be represented by a union of stars. So rather than grouping pixels one can aggregate stars according to a homogeneity criterion. This lets one build the partition by packets of pixels rather than inspect local properties by pairs. As the goal of this paper is to present some preliminary results dealing with picture segmentation as a combinatoric problem, for simplicity we assume that the predicate a is a constant value. We propose an elementary algorithm for gray level images. A preprocessing algorithm incorporating more global information is described next. Indeed, due to noise, the star’s cardinalities can be fairly small even in a homogeneous region. The Helly filter presented in Section 4.2 lets one optimize this parameter.
4.1. Covering and Selection Algorithm The algorithm proceeds in two parts. In the first part, a covering of the image by a minimal set of stars is computed. In the second part, selected stars are aggregated to obtain the regions. Let Hn,a be a neighborhood hypergraph associated with the (p 3 q)-image.
269
Step 1. Start with a covering of the image by a minimal set of stars. E 5 hH (x1), H (x2), . . . H(xn)j. The minimal cover that is used has the following property. Any pixel of the image belongs to at most one hyperedge of at least one star of the set E. By using a minimal cover, one limits the number of stars to be examined in later steps. To apprehend this step one can refer to Fig. 6. In this example, if we consider the pixel v6 , then it belongs to one hyperedge, E6 . Because a 5 20, it belongs to H(v18), H(v20), H(v21). In this case the minimal cover must include at least one of these stars. Step 2. Build star aggregate areas. For any area, proceed like this: —Let g(xi) be the gray level of the star center H(xi ). Examine any star H(xj ) intersecting with the star H(xi ). —If g(xj ) is within the interval [ g(xi ) 2 a, g(xi ) 1 a], the star H(xj ) is aggregated with H(xi ). —Repeat this process with the stars that intersect with at least one star of the area. Step 3. Reduce the number of areas. —For any intersecting area compute the following parameters: *center of gravity, *minimum gray level, *maximum gray level, *medium gray level. (These last three parameters are computed on star centers.) —If one center of gravity belongs to the intersection when two areas have parameters which are similar (6a), aggregate them. (Every star can belong to several aggregation areas (2a 1 1 different areas at most)). Step 4. Assign each star to an aggregation area to obtain a partition. Begin with the area containing the greatest number of stars and assign a common index to each of them. The index corresponds to this area. —Remove these stars from other areas to which they may belong. —Repeat the process until all the stars have been assigned. At the end of this step each star center has been assigned to an area. Step 5. Assign the pixels generating edges. —If a pixel belongs to several stars, it is assigned to the area of the star center whose gray level is the closest to its own. Finally the pixels which are neither star centers nor edge generating are assigned to the area of the neighboring pixel which has already been assigned and whose gray level is the closest.
270
BRETTO ET AL.
• Hn has the Helly property if and only if Gn is defined by dy . This theorem states that a necessary condition to confer the Helly property on the neighborhood hypergraph associated with a digital image, is to use the 8-neighborhood system of order n. Remark 4. In the following sections, we will use only the 8-neighborhood system of order n. 4.2.2. Helly Filter Algorithm
FIG. 5. Suppose that the hypergraphs have only three hyperedges, then the hypergraph (a) has the Helly property.
4.2. Helly Filter 4.2.1. Helly Property An important property of hypergraph theory is the socalled Helly property. This property generalizes the geometric notion of visibility. It is derived from convex geometry and has been generalized to hypergraphs by Berge [4]. A hypergraph has the Helly property if each family of hyperedges intersecting 2 3 2 (intersecting family) has a nonempty intersection (belongs to a star). Figure 5 shows two examples of intersecting hyperedges. • Fig. 5(a): A family of three hyperedges intersecting 2 by 2 with a nonempty intersection. • Fig. 5(b): A family of three hyperedges intersecting 2 by 2 with an empty intersection. This basic property is of great interest in hypergraph study. It helps to characterize specific hypergraph classes such as unimodular hypergraphs and balanced hypergraphs. As a digital image has geometrical and combinatorial aspects the Helly property is particularly suited to image processing. Let an image be represented by its neighborhood hypergraph. If this hypergraph has the Helly property, the centers of the stars characterize the common neighborhood relations of the stars. Therefore a star center is representative of the whole neighborhood. These centers may be a sufficient representation to extract global information. In discrete geometry it is important to define the grids that satisfy the Helly property. In [7] we prove the following theorem. THEOREM 1. Let Gn be a grid defined by a square network and let Hn be the neighborhood hypergraph associated with it.
Although the Helly property is not linked with the physical formation of an image, the smoothness of local properties can often help one construct a hypergraph which satisfies this property without drastically modifying the original image. We now present an algorithm for transforming an image so that it satisfies the Helly property. For any x element in the domain of I, proceed from Step 1 to Step 3. Let x < Gn,a (x ) be the edge generated by x in Hn,a . Step 1. Compute the set of intersecting families. Compute Ax the set of edges of Hn,a associated with the vertices of the neighborhood of x: Ax 5
< (y < G
n,a ( y)).
y[Gn(x)
Search in P(Ax) for the set of intersecting families. • Build the graph (GIx ) of the intersections of the edges Ax defined by: —the set of vertices V 5 Gn(x ), —the set of edges E defined by ( y1 , y2) [ E if and only if ( y1 < Gn,a ( y1)) > ( y2 < Gn,a ( y2)) ? B. Step 2. Search for the set of maximal intersecting families Ax . • Search for the complete maximal subgraphs of the graph GIx . (This problem is N 2 P complete but in practice, n is small, and thus Gn(x ) has a finite number of neighboring vertices.) Step 3. Compute the x gray level. Let Fx1, Fx2, . . . , Fxl be the set of the maximal intersecting families. (To avoid processing the same family several times, and to process every vertex, we will deal only with the families whose center of gravity is x. x is the center of gravity of the set of families Fx1 , Fx2 , . . . , Fxl , if y [ G(x ) for some y < Gn,a (y ) [ Fxi.) Fxi , is centered on x if the center of gravity of Fxi is x. (x is adjacent to any Fxi).
COMBINATORICS AND IMAGE PROCESSING
271
FIG. 6. A figure representing the hyperedges of H1,20 associated with the pixels adjacent to x. They make up a set called Ax . To make the drawing more easily understandable, the hyperedges centered on the pixels with even numbers are represented with continuous lines. Those centered on odd pixels are represented with dotted lines. Pixel indices are ordered according to the Freeman code.
• If l 5 1 (there is only one family with x as a center) and if x [ Fx1 then Fx1 is a star (the partial subhypergraph has the Helly property). —Else if l . 1 of (if l 5 1 and x Ó Fx1) build the star as follows. —Let Sx 5
• First, let Ax denote the set of hyperedges of the image centered on the pixels v1 , v2 , . . . , v8 neighbors of x: Ax 5 hE1 , E2 , E3 , E4 , E5 , E6 , E7 , E8 j E1 5 hv1 , v10 j E2 5 hv2 , v3 , v12 j E3 5 hv3 , v2 , v4 , v5 , v12 , v13 , v14 j E4 5 hv4 , v3 , v5 , v13 , v14 , v16 , v17 j E5 5 hv5 , v3 , v4 , v16 , v18 j E6 5 hv6 , v18 , v19 , v20 , v21 j E7 5 hv7 , v8 , v20 , v21 j E8 5 hv8 , v7 , v9 , v21 j.
FIG. 7. A figure representing the graph GIx of the intersections of the hyperedges of Ax .
272
BRETTO ET AL.
• Then we build the graph of the intersections of the hyperedges of Ax , denoted as GIx as shown in Fig. 7. It is known that the maximum intersecting families correspond to the maximum complete subgraphs of GIx . Thus, we have the set Fx of the maximum intersecting families in accordance with Fx 5 hFx1 , Fx2 , Fx3 j with Fx1 5 hEv2 , Ev3 , Ev4 , Ev5 j, Fx2 5 hEv6 , Ev7 , Ev8 j, Fx3 5 hEv5 , Ev6 j,
As x Ó Sx , its gray level value is given by I(x ) 5 round
S
D
Imax(Sx ) 1 Imin(Sx ) 5 round(97.5). 2
Remark 5. Suppose that the image is defined on a neighborhood system other than the 8-neighborhood system of order n. According to Theorem 1 this one does not have the Helly property. So we cannot confer this property on the neighborhood hypergraph associated with the image (this one being defined using relation 1 which depends on the grid).
4.3. Properties of the Algorithms 4.3.1. Properties of Covering and Selection Algorithm
As in standard region growing, the choice of the seed is critical. Nevertheless, since this algorithm works on aggregates of pixels it may be hoped that it is less dependent Fx4 5 hEv1 j. than segmentation algorithms dealing with pixels. As the image size is finite (p 3 q), the convergence is Thus, the set of pixels Sx is the following: obvious. The minimal Helly families covering up the image can be found in constant time. The cover takes at most Sx 5 hv1 , v2 , v3 , v4 , v5 , v6 , v7 , v8 , v9 , v10 , v11 , v12 , v13 , v14 , the Helly families number and the partition is computed v16 , v18 , v19 , v20 , v21 j. in a time equivalent to the maximum intersecting Helly families number. • We calculate the maximum distance between two gray levels of Sx: 4.3.2. Properties of the Helly Filter
and
DImax 5 Imax 2 Imin 5 I(v7) 2 I(v13) 5 115 2 60
THEOREM 2. The Helly filter is convergent, and if p is the number of neighbors of a vertex on the grid, p2 is an upper bound of the iteration number of the filter.
Proof. At the ith process of the algorithm, a vertex whose gray level has been changed previously has its gray 5 55 .2a 5 40. level changed again if x is again possibly the center of We then remove the pixel whose gray level is the farthest gravity of H(x ), Fxp , . . . . . . . , Fxp1j . (H(x ) is the star centered on x and Fxp1j is a maximal intersecting family that from the one of x, namely v13 . It follows that can be a star.) The gray level of the neighbor y of x is Sx 5 hv1 , v2 , v3 , v4 , v5 , v6 , v7 , v8 , v9 , v10 , v11 , v12 , v13 , v14 , changed if y is a candidate as gravity center of H(x ), H(y), Fxp , . . . . . . . , Fxp1j , Fyp , . . . , Fyp1k . v16 , v18 , v19 , v20 , v21 j —The cardinality of a star is increasing. Let H(x) be a star built at the ith process (the neighborhood relation and being reflexive, it is sufficient to reason on x). Let H(x), H( y), Fxp , ......., Fxp1j be the intersecting families DImax 5 115 2 65 5 50 .2a 5 40. at the i 1 1 ith process. Any generator vertex of the edges We go through this process again until DImax is less than of these families are the neighbors of x; thus their gray levels are at (1a, 2a) from I(x) and consequently will not 2a. Eventually, we have be eliminated by building the new star. The new middle will be defined at more or less the gray level of these Sx 5 hv2 , v6 , v7 , v8 , v9 , v12 , v16 , v18 , v19 , v20 , v21 j vertices. H(x) can only take edges in the families Fxp1j . —If the number of intersecting families accepting x as and a center of gravity is finite and the cardinality of H(x) is DImax 5 115 2 80 5 35 ,2a 5 40. limited, then the convergence is proved.
COMBINATORICS AND IMAGE PROCESSING
273
—A vertex x, on the grid having p neighbors having themselves p neighbors, will have its gray level changed p2 times at most. n THEOREM 3. If the set of pixels for which the gray level values must be modified are changed after the whole image has been inspected, the output of the Helly filter is unique. Proof. Let x be a pixel whose value has to be modified by the Helly filter. Suppose that it is processed at the ith rank in one case, and at the ith rank in another case (i ? j ). As the gray level values are changed at the end of an iteration, its neighbors have the same gray level values in both cases. Consequently the result of the processing which is only neighbor dependent is rank independent. n 5. EXPERIMENTAL RESULTS
We now turn to several experiments made with the algorithms introduced above. The first set of tests is made on a synthetic image. It is used to compare a standard region growing algorithm [17] using the same predicate as the proposed one in terms of subjective and objective quality of segmentation, speed, and stability. The standard region growing algorithm is abbreviated SRA, while the covering and selection algorithm is abbreviated CSA. The second set of tests is used to judge whether the model agrees with the physical nature of digital images. In other words, if the Helly property is a natural property of digital images then the model will be appropriate. A third set of tests is used to evaluate the performance of the segmentation algorithm proposed in Section 5.1, with different homogeneity levels for various types of natural images. The results of CSA segmentation are compared to classical region growing based segmentation. For all these tests, we use a 8-neighborhood system of order 1.
5.1. Experimental Properties of the Algorithms on Synthetic Images For this analysis, we generated a sequence of 128 3 128 synthetic images made of two regions, with a difference in gray levels of 40 and separated by a vertical transition at x 5 64. We added Gaussian noise with zero mean and standard deviation of s 5 4, 8, 10, 15, 20, 30. 5.1.1. Quality of the Segmentation As an objective measure of quality, we use the number of regions generated by the algorithms. For all images studied, the covering and selection algorithm produces fever regions than the standard region growing algorithm. If the Helly filter is used as a preprocessing step the number
FIG. 8. Evolution of the number of regions as a function of a. For a synthetic image with noise s 5 4, (a) standard region growing algorithm, (b) covering and selection algorithm, (c) Helly filter and standard region algorithm, (d) Helly filer and covering and selection algorithm.
of regions decreases. Figure 8 is a typical example of these behaviors. In terms of subjective quality we can establish a classification of the algorithms. Whatever the case, CSA gives better results than SRA. Using the Helly filter always improves the quality. However, it is more efficient with CSA. The differences are all the more obvious as the noise level increases. Performance of these two algorithms is very poor when s . 15. Figure 9 gives a visual illustration of these observations. In each case the value of a used is the one which corresponds to the best visual quality. Table 1 summarized the parameter values for this example. 5.1.2. Computational Time The computational time required by CSA is approximately twice that required by SRA. However, no particular care has been taken to optimize the code. Noise does not affect the value of the computation time. Helly filter computational cost is greatly dependent on the noise level. The efficiency comparison is summaried in Table 2. 5.1.3. Stability To examine the effect of seed position on stability, we choose this value at random. Fifteen seeds were generated for the set of images. If the value of a is appropriate the results of the segmentation are similar when the noise level is low (s , 10). Each time the results differ only very small regions are concerned. Visually these differences are imperceptible. If the regions are noisy the segmentation may be incorrect. The Helly filter associated with CSA proves to be adequate to prevent this up to a certain point.
274
BRETTO ET AL.
TABLE 2 Processing Time Image
SRA
CSA
Helly filter
s54 s 5 10
1.8s 2.0s
2.5s 3.7s
3.9s 5.1s
Note. The algorithms are processed on a SUN Sparc 5 workstation. User time is reported.
5.2. Experimental Properties of the Algorithms on Natural Images In this section we present experimental results obtained with three 512 3 512 8-bit natural images shown in Figs. 10a, 11a, and 12a. We evaluate the adequacy of the model
FIG. 9. (a) Synthetic image with noise s 5 10, (b) standard region growing segmentation, (c) covering and selection segmentation, (d) Helly filter and standard region segmentation, (e) Helly filter and covering and selection segmentation.
TABLE 1 Parameter Values for the Images in Fig. 9 Algorithms
a
Number of regions
SRA CSA Helly 1 CSA Helly 1 SRA
6 19 19 6
1811 13 8 1611
FIG. 10. (a) Original image map, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 63, b 5 76, (e) Helly filter and SRA segmentation a 5 63, b 5 76.
275
COMBINATORICS AND IMAGE PROCESSING
TABLE 3 Intersecting Families Which Are Not Stars a
5
10
15
20
30
Percentage for smoke Percentage for Tiffany Percentage for map
3.1 18.3 16.3
1.5 11.25 9.8
0.67 7.9 5.7
0.35 4.4 2.7
0.15 2.1 1.9
and then we compare the segmentation results of the proposed algorithm with a standard region growing algorithm result. 5.2.1. Model Performance For these images we compare the number of intersecting families and the number of stars for various values of the homogeneity criterion. Table 3 shows the percentage of nonstar families for five different values of alpha. As expected, we observe asymptotic behavior with increasing values of alpha for all the images under study. Indeed, if the homogeneity criterion covers the entire range of feature values every point of the grid is the center of a star. Intersecting families which are not stars are representative of areas in the image where feature values have higher variations than the homogeneity criterion. They are correlated with the discontinuities of the image. For a smooth image such as Smoke the model is valid even for low values of the homogeneity criterion. For images with high contrast zones, such as Tiffany, the knee of the curves can be interpreted as the minimal value of alpha necessary to apprehend the complexity of the image.
We consider the number of regions generated by the algorithms. Generally if a region is considered to be homogeneous, the least number of regions is desired. In the table, we show the results for both algorithms. For CSA 1 Helly filter the results are given for two values of the a parameter. The results for SRA are given when used directly on the original image and also when applied to the same image previously processed by Helly filtering. The reported values of the control parameters a and b are the ones that give the best visual quality to a human interpreter. These values have been obtained through a trial and error based adjustment. As shown in Table 4, the Helly 1 CSA segmentation results in a small number of regions as compared to both SRA implementations. The results of the segmentation are also given as follows. In each example, the original image and the segment images are reported. In the segmented images, each region is represented by the mean value of its intensities. These results demonstrate that the CSA algorithm should produce significantly better segmentations than SRA. One can see that all the results show a very satisfying regional homogeneity. This result can be explained by the Helly filtering that harmonizes the pixel values in a neighborhood. Nevertheless this effect does not interfere with the emergence of small homogeneous regions. As the homogeneity criterion value decreases, it can be observed that different new regions appear and the segmentation provides more accurate region limits. In the ‘‘map’’ image shown in Fig. 10, where thin elongated regions appear, one
TABLE 4 Parameter Values for Figs. 10–12
5.2.2. Segmentation Performance In this section we will illustrate the effectiveness of the overall algorithm (CSA 1 Helly filter). To compare with our method, we consider the standard region growing algorithm. As natural images are far more complex than the synthetic images used in the first set of tests, we add to the previous predicate used in SRA another parameter to control the aggregation process. A pixel x is aggregated to a region Ri if its gray level I(x ) is within a and if it is within the mean value of the region for which it is candidate: x [ Ri ⇔ for y [ Ri and x [ G( y), uI(x ) 2 I( y)u # a and uI(x ) 2 Meanz[Ri (I(z ))u # b. b is a measure of how different the pixel is from the region it adjoins. As in segmentation of a given scene, there is rarely one unambigous partition and the human interpreter is often the arbiter of the segmentation quality. Thus the algorithms are evaluated on two fronts.
(a) Parameter values for the images of Fig. 10 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 63 63
b
Number of regions of Map
76 76
2357 406 3516 3122
(b) Parameter values for the images of Fig. 11 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 38 38
b
Number of regions of Smoke
76 76
80 20 349 302
(c) Parameter values for the images of Fig. 12 Algorithms
a
Helly 1 CSA Helly 1 CSA SRA Helly 1 SRA
15 25 46 46
b
Number of regions of Tiffany
76 76
1514 492 3874 2985
276
BRETTO ET AL.
various approaches of image analysis. This model, based on hypergraph theory, allows us to use all the mathematical background of combinatorics. We have presented propositions for solving basic image processing problems such as segmentation deduced from this model. At this point our goal is not to optimize the performance of these algorithms, but only to suggest that this new approach can be an effective approach to image processing. Through this application we showed that the hypergraph associated to an image lets one process a picture with very simple assumptions. Naturally the algorithms presented can be improved in several ways (sufficient conditions for a hypergraph satisfy the Helly property, the uniformity criterion, adaptativity, parameters and decision rules, etc.). Now that the model has been validated we are trying to extend it to other fields of image processing, such as restoration and coding.
FIG. 11. (a) Original image Smoke, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 38, b 5 51 Helly filter and SRA segmentation a 5 38, b 5 51.
can see that the segmentation process is very powerful. Indeed, most of these regions are segmented integrally: the spatial resolution of the approach is very fine. In the image Smoke shown in Fig. 11, CSA has preserved the small white region in the lower right-hand corner of the image while SRA has produced more than a unique region. One can also see that SRA cannot recover the begining of the stream. It can be seen in Fig. 12 that the major features on the face are found in the image of Tiffany. For both values of a, the eyes and the nails are particularly preserved by CSA while SRA fails to recover these features, although a slight improvement is gained by the Helly filtering preprocessing. 6. CONCLUSION
Throughout this paper our emphasis has been on introducing a new model of digital imaging which can unify the
FIG. 12. (a) Original Image Tiffany, (b) CSA segmentation with a 5 15, (c) CSA segmentation with a 5 25, (d) SRA segmentation a 5 51, b 5 76, (e) Helly filter and SRA segmentation a 5 51, b 5 76.
COMBINATORICS AND IMAGE PROCESSING
ACKNOWLEDGMENT The authors are grateful to the referees for their helpful comments leading to improvements of the manuscript.
REFERENCES 1. H. Adeli and S. L. Hung, Machine Learning—Neuronal Networks, Genetic Algorithms and Fuzzy Systems, Wiley, New York, 1994. 2. H. J. Bandelt and E. Pesch, Dismantling absolute retracts of reflexive graphs, Eur. J. Combin. 10, 1989, 211–220. 3. C. Berge, Graphs, North-Holland, Amsterdam, 1985. 4. C. Berge, Hypergraphs, North-Holland, Amsterdam, 1987. 5. C. Berge, The Helly property, Southeast Asian Math. Soc. Bull. 1, 1979, 16–19. 6. C. A. Bouman and M. Shapiro, A systematic way for region-based on Markov field model. PR 27, 1994, 162–177. 7. A. Bretto, Convexite´ abstraite, Hypergraphe de voisinage et proprie´te´ de Helly, une application a` la the´orie de l’image. Ph.D. thesis, Universite´ de Saint Etienne, 1994. 8. A. Bretto and B. Laget, Neighborhood hypergraph and image analysis, in Signal Process. II, 5–9 September, 1994, Fontainebleau. 9. F. Cheevasuvit, H. Maitre, and D. Vidal-Madjar, A robust method for picture segmentation based on split and merge procedure, Comput. Vision Graphics Image Process. 34, 1986, 268–281. 10. S. Y. Chen, W. C. Lin, and C. T. Chen, Split and merge image segmentation based on localized feature analysis and statistical tests, CVGIP: Graphics Models Image Process. 53, 1993, 457–475. 11. F. F. Dragan, Domination in quadrangle-free helly graphs, Cybernet. Syst. Anal. 29 (6), 1993, 822–829. 12. R. Fagin, Degrees of acyclicity for hypergraphs and relational database schemes, J. Assoc. Comput. Mach. 30, 1983, 514–550. 13. FUZZ–IEEE, Third IEEE International Conference on Fuzzy Systems, Orlando, FL, June 26–July 2, 1994. 14. K. Geman and D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intel. PAMI-6, 1984, 721–741. 15. S. R. Gabriel and R. R. Sokal, A new statistical approach to geographic variations analysis, Syst. Zool. 18, 1969, 259–278. 16. M. Gondran and M. Minoux, Graphs and Algorithms, Wiley, New York, 1984.
277
17. R. C. Gonzalez and P. Wintz, Digital Image Processing, Addison– Wesley, Reading, MA, 1977. 18. H. J. A. M. Heijmans, Morphological Image Operators, Academic Press, San Diego, 1994. 19. I. Y. Kim and H. S. Yang, A multiscale random field model for Bayesian image segmentation, T-IP 3, 1994, 841–852. 20. V. A. Kovalevsky, Finite topology as applied to image processing, Comput. Vision Graphics Image Process. 46, 1989, 141–161. 21. M. D. Levine, Vision in Man and Machines, McGraw–Hill, New York, 1985. 22. J. S. Lim, Two dimensional signal and image processing, Signal Processing Series, Prentice Hall, New York, 1990. 23. H. Niemann, Pattern Analysis and Understanding, Springer-Verlag, Berlin/New York, 1992. 24. J. O’Rourke, Computational Geometry, Cambridge Univ. Press, Cambridge, UK, 1994. 25. J. Pach, New Trends in Discrete and Computational Geometry, Springer-Verlag, Berlin, 1993. 26. N. R. Pal and S. K. Pal, A review on image segmentation techniques, Pattern Recog. 26 (9), 1993, 1277–1294. 27. J. L. Peterson, Petri nets, Comput. Surv. 9 (3), 1997, 223–252. 28. F. P. Preparata and M. I. Shamos, Computational Geometry, SpringerVerlag, New York, 1985. 29. A. Rosenfeld, Digital Image Processing, Academic Press, San Diego, 1982. 30. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, San Diego, 1982. 31. J. Serra, Mathematical Morphology, Vol. 2, Academic Press, San Diego, 1988. 32. K. J. Supowit, The relative neighborhood graph with an application to minimum spaning trees, J. Assoc. Comput. Mach. 30, 1983, 428–447. 33. J. T. Tou and R. C. Gonzalez, Pattern Recognition Principles, Addison–Wesley, Reading, MA, 1974. 34. M. Tuceryant and T. Chorzempa, Relative sensitivity of a family of closest point graphs in computer vision applications, Pattern Recognit. 24, (5), 1991, 361–373. 35. International Workshop on Graph Grammars, Williamsburg VA, November 14–16, 1994. 36. K. Voss, Discrete Images, Objects and Functions in Zn, Algorithms and Combinatorics, Springer-Verlag, Berlin, New York, 1990. 37. S. W. Zucker, Region growing: childhood and adolescence, Comput. Vision Graphics Image Process. 5, 1976, 382–399.