Fuzzy Set Theoretic Tools for Image Analysis

ADVANCES IN ELECTRONICS AND ELECTRON PHYSICS, VOL. 88

Fuzzy Set Theoretic Tools for Image Analysis SANKAR K. PAL Machine Intelligence Unit, Indian Statistical Institute, Calcutta, India

I. Introduction . . . . . . . . . . . . . . . . . . . . 11. Uncertainties in a Recognition System and Relevance of Fuzzy Set Theory 111. Image Ambiguity and Uncertainty Measures . . . . . . . . . . A. Grayness Ambiguity Measures . . . . . . . . . . . . . B. Spatial Ambiguity Measures Based on Fuzzy Geometry of Image . . IV. Flexibility in Membership Functions . . . . . . . . . . . . A. Bound Functions . . . . . . . . . . . . . . . . . B. Spectral Fuzzy Sets . . . . . . . . . . . . . . . . . V. Some Examples of Fuzzy Image-Processing Operations . . . . . . A. Threshold Selection (Fuzzy Segmentation) . . . . . . . . . B. Contour Detection . . . . . . . . . . . . . . . . . C. Optimum Enhancement Operator Selection . . . . . . . . . D. Fuzzy Skeleton Extraction and FMAT . . . . . . . . . . . V1. Feature/Knowledge Acquisition, Matching, and Recognition . . . . VII. Fusion of Fuzzy Sets and Neural Networks: Neuro-Fuzzy Approach . . VIII. Use of Genetic Algorithms . . . . . . . . . . . . . . . IX. Discussion . . . . . . . . . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . . . . . .

. . . . . . . . . . . .

. . . . . .

. . . .

. . . .

241 249 25 1 252 256 260 26 1 262 264 264 261 269 269 212 218 288 290 29 1 292

I. INTRODUCTION

Pattern recognition and machine learning form a major area of research and development activity that encompasses the processing of pictorial and other nonnumerical information obtained from interaction among science, technology, and society. The second motivation for this spurt of activity in this field is the need for the people to communicate with the computing machines in their natural mode of communication. The third and most important motivation is that the scientists are also concerned with the idea of designing and making automata that can carry out certain tasks as we human beings do. The most salient outcome of these is the concept of fifth-generation computing systems. Machine recognition of patterns (Tou and Gonzalez, 1974; Duda and Hart, 1973) can be viewed as a two-fold task, consisting of learning the invariant and common properties of a set of samples characterizing a class and of deciding that a new sample is a possible member of the class by 241

Copyright 0 1994 by Academic Press, Inc. All rights of reproduction in any form reserved. ISBN 0-12-014730-0

248

SANKAR K. PAL

noting that it has properties common to those of the set of samples. Therefore, the task of pattern recognition by a computer can be described as a transformation from the measurement space M to the feature space F and finally to the decision space D. When the input pattern is a gray-tone image, the measurement space involves some processing tasks such as enhancement, filtering, noise reduction, segmentation, contour extraction, and skeleton extraction, in order to extract salient features from the image pattern. This is what is basically known as image processing. The ultimate aim is to make its understanding, recognition, and interpretation from the processed information available from the image pattern. Such a complete image recognition/ interpretation system is called a vision system, which may be viewed as consisting of low, mid, and high levels. In a pattern-recognition or vision system, the uncertainty can arise at any phase of the aforesaid tasks resulting from the incomplete or imprecise input information, the ambiguity/vagueness in input image, the ill-defined and/or overlapping boundaries among the classes or regions, and the indefiniteness in defining/extracting features and relations among them. Any decision taken at a particular level will have an impact on all higherlevel activities. It is therefore required for a recognition system to have sufficient provision for representing these uncertainties involved at every stage, so that the ultimate output (results) of the system can be associated with the least uncertainty (and not be affected or biased very much by the earlier or lower-level decisions). This chapter describes various fuzzy set theoretic tools and explores their effectiveness in representing/describing various uncertainties that might arise in an image-recognition system and the ways these can be managed in making a decision. Some examples of uncertainties that arise often in the process of recognizing a pattern are given in Section 11. Section 111 provides a definition of image and describes various fuzzy set theoretic tools for measuring information on grayness ambiguity and spatial ambiguity in an image. Concepts of bound functions and spectral fuzzy sets for handling uncertainties in membership functions are also discussed in Section IV. Their applications to low-level vision operations (e.g., segmentation, skeleton extraction, and edge detection), whose outputs are crucial and responsible for the overall performance of a vision system, are then presented in Section V for demonstrating the effectiveness of these tools in managing uncertainties by providing both soft and hard decisions. Their usefulness in providing quantitative indices for autonomous operations is also explained. Section VI describes the issues of feature/primitive extraction, knowledge acquisition and syntactic classification, and the features of DempsterShafer theory and rough set theory in this context. An application of the

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

249

multivalued recognition system for detecting curved structures from remotely sensed image is also described. Some of the recent attempts on fusion of the theories of fuzzy sets and neural networks for efficient handling of uncertainty (in the sense of parallel processing, robustness, and overall performance) are described in Section VII. The concept of genetic algorithms and its possible use are explained in Section VIII. 11. UNCERTAINTIES IN A RECOGNITION SYSTEM AND

RELEVANCE OF FUZZYSETTHEORY

Some of the uncertainties that one encounters often while designing a pattern-recognition or vision (Gonzalez and Wintz, 1987; Rosenfeld and Kak, 1982) system will be explained in this section. Let us consider, first of all, the problem of processing and analyzing a gray-tone image pattern. A gray-tone image possesses some ambiguity within the pixels due to the possible multivalued levels of brightness. This pattern indeterminacy is due to inherent vagueness rather than randomness. The conventional approach to image analysis and recognition consists of segmenting (hard partitioning) the image space into meaningful regions, extracting its different features (e.g., edges, skeletons, centroid of an object), computing the various properties of and relationships among the regions, and interpreting and/or classifying the image. Since the regions in an image are not always crisply defined, uncertainty can arise at every phase of the aforesaid tasks. Any decision taken at a particular level will have an impact on all higher-level activities. Therefore, a recognition system (or vision system) should have sufficient provision for representing the uncertainties involved at every stage, i.e., in defining image regions, its features, and relations among them, and in their matching, so that it retains as much as possible the information content of the original input image for making a decision at the highest level. The ultimate output (result) of the system will then be associated with least uncertainty (and unlike conventional systems it will not be biased or affected very much by the lower level decisions), For example, consider the problem of object extraction from a scene. Now, the question is, how can someone define exactly the target or object region in a scene when its boundary is ill-defined? Any hard thresholding made for its extraction will propagate the associated uncertainty to the following stages, and this might affect its feature analysis and recognition. Similar is the case with the tasks of contour extraction and skeleton extraction of a region. From the aforesaid discussion, it becomes therefore convenient, natural, and appropriate to avoid committing ourselves to a specific (hard) decision

250

SANKAR K. PAL

(e.g., segmentation/thresholding, edge detection, and skeletonization) by allowing the segments or skeletons or contours to be fuzzy subsets of the image; the subsets being characterized by the possibility (degree) of a pixel belonging to them. Similarly, for describing and interpreting ill-defined structural information in a pattern, it is natural to define primitives (such as line, corner, curve) and relations among them using labels of fuzzy sets. For example, primitives that d o not lend themselves to precise definition may be defined in terms of arcs with varying grades of membership from 0 to 1 representing its belonging to more than one class. The production rules of a grammar may similarly be fuzzified to account for the fuzziness in physical relation among the primitives, thereby increasing the generative power of a grammar for syntactic recognition (Fu, 1982) of a pattern. The incertitude in an image pattern may be explained in terms of grayness ambiguity or spatial (geometrical) ambiguity or both. Grayness ambiguity means indefiniteness in deciding a pixel as white or black. Spatial ambiguity refers to indefiniteness in shape and geometry (e.g., in defining centroid, sharp edge, perfect focussing, and so on) of a region. There is another kind of uncertainty that may arise from the subjective judgment of an operator in defining the grades of membership of the object regions. This has been explained in Section IV in terms of uncertainty in membership function. Let us now consider the case of a decision theoretic approach to pattern classification. With the conventional probabilistic and deterministic classifiers (Duda and Hart, 1973; Tou and Gonzalez, 1974), the features characterizing the input patterns are considered to be quantitative (numerals) in nature. The patterns having imprecise or incomplete information are usually ignored or discarded from their designing and testing processes. The impreciseness (or ambiguity) may arise from various reasons. For example, instrumental error or noise corruption in the experiment may lead to partial or partially reliable information available on a feature measurement F,such as, F is about 500, say, or F is between 400 and 500, say. Again, in some cases the expense incurred in extracting the exact value of a feature may be high, or it may be difficult to decide on the actual salient features to be extracted. On the other hand, it may become convenient to use the linguistic variables and hedges, e.g., small, medium, high, very, more or less, and the like, in order to describe the feature information (e.g., F is very small). In such cases, it is not appropriate to give exact representation to uncertain feature data. Rather, it is reasonable to represent uncertain feature information by fuzzy subsets. Again, the uncertainty in classification or clustering of patterns may arise from the overlapping nature of the various classes. This overlapping may result from fuzziness or randomness. In the conventional classification

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

25 1

technique, it is usually assumed that a pattern may belong to only one class, which is not necessarily true. A pattern may have degrees of membership in more than one class. It is therefore necessary to convey this information while classifying a pattern or clustering a data set. Similarly, consider the problem of determining the boundary or shape of a class from its sampled points or prototypes. There are various approaches (Murthy, 1988; Edelsbrunner er al., 1983; Tousant, 1980) described in the literature that attempt to provide an exact shape of the pattern class by determining the boundary such that it contains (passes through) some of the sample points. This need not be true. It is necessary to extend the boundaries to some extent to represent the possible uncovered portions by the sampled points. The extended portion should have lower possibility to be in the class than the portions explicitly highlighted by the sample points. The size of the extended regions should also decrease with the increase of the number of sample points. This leads one to define a multivalued or fuzzy (with continuum grade of belonging) boundary of a pattern class (Mandal er al., 1992b). In the following section we will be explaining various fuzzy-set theoretical tools for image analysis (which were developed based on the realization that many of the basic concepts in pattern analysis, e.g., the concept of an edge or a corner, do not lend themselves to precise definition) and the way of using them for handling uncertainties in the process of recognizing an image pattern. 111. IMAGE AMBIGUITY AND UNCERTAINTY MEASURES

An L level image X(M x N ) can be considered as an array of fuzzy singletons, each having a value of membership denoting its degree of possessing some property (e.g., brightness, darkness, edginess, blurredness, texture). In the notation of fuzzy sets one may therefore write that

x = (px(x,,):rn

= 1 , 2 ,..., M ; n = 1 , 2,..., N )

(1)

where px(x,,,) denotes the grade of possessing such a property p by the ( m , n)th pixel. This property p of an image may be defined using global information, local information, positional information, or a combination of them depending on the problem. Again, the aforesaid information can be used in a number of ways (in their various functional forms), depending on individual’s opinion and/or the problem to hand, to define a requisite membership function for an image property. Basic principles and operations of image processing and pattern recognition in the light of fuzzy set theory are available in (Pal and Dutta Majumder, 1986).

252

SANKAR K . PAL

We will be explaining in this section the various image information measures (arising from both fuzziness and randomness) and tools, and their relevance for the management of uncertainty in different operations for processing and analysis. These are classified mainly in two groups, namely grayness ambiguity/uncertainty and spatial arnbiguityhncertainty, A , Grayness Ambiguity Measures

The definitions of some of the measures that were formulated to represent grayness ambiguity in an image X with dimension M x N and levels L (based on individual pixel as well as a collection of pixels) are listed below. 1. rth Order Fuzzy Entropy

H'(X)

=

(-1jk)

c Icl(sf)loglP(sf)l + (1 i

- P(Sf)I log(1 - fl(sf)II

i

= 1,2,

...,k

(2)

where sf denotes the ith combination (sequence) of r pixels in X ;k is the number of such sequences; and p(sf) denotes the degree to which the combination si, as a whole, possesses some image property p . 2. Hybrid Entropy

with

Hhy(x) = -Pwlog E, - Pb log Eb

Ew = (1/MM C m

Eb

= (l/MN)

(3)

Cn Pmnex~(1- p m n )

Cm Cn (1 - prnn)eXP(pmn)

(4)

m = 1 , 2,..., M ; n = 1 , 2,..., N Here pmn denotes the degree of whiteness of the (m,n)th pixel. Pw and Pb denote probability of occurrences of white (p,, = 1) and black (pmn= 0) pixels respectively; and E, and Ebdenote the average likeliness (possibility) of interpreting a pixel as white and black respectively. 3. Correlation

= 1

ifX,+X2=0

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

253

m = 1,2 ,..., M ; n = 1,2 ,..., N Here pYmn and pUlrnn denote the degree of possessing the properties pl and p2 respectively by the (m,n)th pixel and C(pl,p2) denotes the correlation between two such properties pl and p2 (defined over the same domain). These expressions (Eqs. 2 through 6) are the versions extended to the two-dimensional image plane from those defined (Murthy et al., 1985; Pal and Pal, 1992a) for a fuzzy set. H‘(X) gives a measure of the average amount of difficulty in taking a decision whether any subset of pixels of size r possesses an image property or not. Note that, no probabilistic concept is needed to define it. If r = 1, H‘(X) reduces to (nonnormalized) entropy as defined by De Luca and Termini (1972). Hhy(X),on the other hand, represents an amount of difficulty in deciding whether a pixel possesses a certain property pmn or not by making a prevision on its probability of occurrence. (It is assumed here that the fuzziness occurs because of the transformation of the complete white (0) and black pixels (1) through a degradation process; thereby modifying their values to lie in the intervals [0,0.5] and [0.5,1] respectively). Therefore, if pmn denotes the fuzzy set “object region” then the amount of ambiguity in deciding x,,,, a member of object region is conveyed by the term hybrid entropy depending on its probability of occurrence. In the absence of fuzziness (i.e., with exact defuzzification of the gray pixels to their respective black or white version), Hhy reduces to the two-state classical entropy of Shannon (1948), the states being black and white. Since a fuzzy set is a generalized version of an ordinary set, the entropy of a fuzzy set deserves to be a generalized version of classical entropy by taking into account not only the fuzziness of the set but also the underlying probability structure. In that respect, Hhy can be regarded as a generalized entropy such that classical entropy becomes its special case when fuzziness is properly removed. Note that the Eqs. (2) and (3) are defined using the concept of logarithmic gain function. Similar expressions using exponential gain function, i.e., defining the entropy of an n-state system have been given by Pal and Pal (1989a, 1991a,b; 1992a,b).

H=

i

pie’-pi,

i = 1,2,

..., n

(7)

all these terms, which given an idea of indefiniteness or fuzziness of an image, may be regarded as the measures of average intrinsic information

254

SANKAR K. PAL

that is received when one has to make a decision (as in pattern analysis) in order to classify the ensembles of patterns described by a fuzzy set. H'(X) has the following properties: Pr 1: H' attains a maximum if pi = 0.5 for all i. Pr 2: H' attains a minimum if pi = 0 or 1 for all i. Pr 3: H' 2 H*', where H*' is the rth-order entropy of a sharpened version of the fuzzy set (or an image). Pr 4: H' is, in general, not equal to H',where H' is the rth-order entropy of the complement set, Pr 5: H' I Hr+' when all pi E [0.5, 11. H' 2 H'+' when all pi E [0,0.5]. The sharpened or intensified version of X is such that ~ . r * ( x m n )2

p.r(xrnn)

if~x(xrnn12 0.5

and

(8) ~ x 4 x m n )5 px(xrnn)

if M X r n n )

5

0.5

When r = 1, the property 4 is valid only with the equal sign. Property 5 (which does not arise for r = 1) implies that H' is a monotonically nonincreasing function of r for pi E [0,0.5] and a monotonically nondecreasing function of r for pi E [ O S , 11 (when the "min" operator has been used to get the group membership value). When all pi values are the same, H ' ( X ) = H 2 ( X ) = = H'(X). This is because the difficulty in taking a decision regarding possession of a property on an individual is the same as that of a group selected therefrom. The value of H' would, of course, be dependent on the pi values. Again, the higher the similarity among singletons (supports) the quicker is the convergence to the limiting value of H'. Based on this observation, an index of similarity of supports of a fuzzy set may be defined as S = H 1 / H 2 (when H 2 = 0, H' is also zero and S is taken as 1). Obviously, when p i E I0.5, I] and the min operator is used to assign the degree of possession of the property by a collection of supports, S will lie in [0, I] as H' s H'". Similarly, when pi E [0,0.5], S may be defined as H 2 / H ' so that S lies in [O, 11. The higher the value of S, the more alike (similar) are the supports of the fuzzy set with respect to the fuzzy property p . This index of similarity can therefore be regarded as a measure of the degree to which the members of a fuzzy set are alike. Therefore, the value of first-order fuzzy entropy (H') can only indicate whether the fuzziness in a set is low or high. In addition to this, the value of H', r > 1 also enables one to infer whether the fuzzy set contains similar

-

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

255

supports (or elements) or not. The similarity index thus defined can be successfully used for measuring interclass and intraclass ambiguity (i.e., class homogeneity and contrast) in pattern recognition and image processing problems. H ' ( X ) is regarded as a measure of the average amount of information (about the gray levels of pixels) that has been lost by transforming the classical pattern (two-tone) into a fuzzy (gray) pattern X.Further details on this measure with respect t o image processing problems are available in Pal and King (1981a, b), Pal (1982), and Pal and Dutta Majumder (1986). It is to be noted that H ' ( X ) reduces to zero whenever ,urn,,is made 0 or 1 for all (m,n), no matter whether the resulting defuzzification (or transforming process) is correct or not. In the following discussion it will be clear how Hhy takes care of this situation. Let us now discuss some of the properties of Hhy(X).In the absence of fuzziness when MNPb pixels become completely black (pmn= 0) and MNP, pixels become completely white ( P , ~ , ,= l), then E , = P,,,,Eb = Pb and Hhy boils down to the two state classical entropy Hc

=

- P w log P, - Pb log Pb,

(9)

the states being black and white. Thus Hhyreduces to H, only when a proper defuzzification process is applied to detect (restore) the pixels. IH,, - H,I can therefore be treated as an objective function for enhancement and noise reduction. The lower the difference, the lesser is the fuzziness associated with the individual symbol and the higher will be the accuracy in classifying them as their original value (white or black). (This property is lacking with the H ' ( X ) measure and the measure of Xie and Bedrosian (1984), which always reduces to zero or some constant value irrespective of the defuzzification process). In other words, IHhy - H,l represents an amount of information that was lost by transforming a two-tone image to a gray tone. For a given P, and Pb, (P, + Pb = 1 , 0 IP,, Pb Il), of all possible defuzzifications, the proper defuzzification of the image is the one for which Hh,, is minimum. If p,,,,, = 0.5 for all (m,n) then E, = Eb and

Hhy = -log(0.5 exp 0.5)

(10)

i.e., H,,,, takes a constant value and becomes independent of P, and pb. This is logical in the sense that the machine is unable to make a decision on the pixels since all p,, values are 0.5. Let us now consider the measure correlation C ( p I ,p2) of Eq. (5). This has the following properties.

256

SANKAR K. PAL

(a) If for higher values of pl(x), p2(x) takes higher values and the converse is also true, then C(pl ,p2)must be very high. (b) If with increase of x, both p1and p2 increase, then C ( p l ,p2) > 0. (c) If with increase of x, p1 increases and p2 decreases or vice versa then C ( P ~ , CcC0. ~)

(4 C(PIYP1) =

1.

(el C(P1, PI) 2 C(Pl P2). (f) C(P1 1 - PI) = -1. (g) C(p1 P2) = C(P2 Pl). (h) - 1 5 C(PI,P2) 1. (i) C(pl ,p2) = -C(1 - p1,~ 2 ) . ci) C(PIYP2) = C(1 - P l , 1 - P2). Correlation of an image indicates the characteristics of relative variation between its two properties p l and p 2 , Based on these characteristics, bound functions are defined as shown in Section 1V.A.If one of these properties is considered to be the nearest crisp (two-tone) property of the other (say, p1 = 1 if p2 > 0.5, and p1 = 0 if p2 5 0.5), then C(pl,p2)lies in [0, 11. In other words, if p2 denotes a bright-image plane of an image X having crossover point at s, say, and is dependent only on gray level, then p1 represents its closest two-tone version threshold at s. Therefore, by varying s of the p2 plane, an optimum version of p2 (i.e., optimum fuzzy segmented version of the image) can be obtained for which correlation is maximum. Various segmentation algorithms based on transitional correlation and within class correlation have been derived (Pal and Ghosh, 1992a) using the co-occurrence matrix, Recently fuzzy divergence has been introduced by Bhandari et al. (1992, 1993) for measuring grayness ambiguity. Before leaving this section, it should be mentioned that there have been several attempts recently made on image information and uncertainty measures (Pal, 1992a) based on classical entropy and gray-level statistics. These include conditional entropy, hybrid entropy, higher-order entropy, and positional entropy (Pal and Pal, 1989b, 1991a, 1992a,b). Y

Y

Y

= Y

B. Spatial Ambiguity Measures Based on Fuzzy Geometry of Image

Many of the basic geometric properties of and relationships among regions have been generalized to fuzzy subsets. Such an extension, called fuzzy geometry (Rosenfeld, 1984; Pal and Rosenfeld, 1988, 1991; Pal and Ghosh, 1990, 1992b), includes the topological concept of connectedness, adjacency and surroundedness, convexity, area, perimeter, compactness, height, width, length, breadth, index of area coverage, major axis, minor axis, diameter, extent, elongatedness, adjacency, and degree of adjacency. Some

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

257

of these geometrical properties of a fuzzy digital image subset (characterized by piecewise constant membership function p x (xmJ or simply p ) are listed below with illustrations. These may be viewed as providing measures of ambiguity in the geometry (spatial domain) of an image. 1. Compactness (Rosenfeld, 1984)

where = &&

and P(P) =

Ci,j,k

I&) - W)llA(i,j, k)l.

(12)

Here, a ( p ) denotes area of p , and p ( p ) , the perimeter of p, is just the weighted sum of the lengths of the arcs A ( i , j , k)along which the region p ( i ) and p ( j ) meet, weighted by the absolute difference of these values. Physically, compactness means the fraction of maximum area (that can be encircled by the perimeter) actually occupied by the object. In the nonfuzzy case, the value of compactness is maximum for a circle and is equal to 1/4n. In the case of the fuzzy disk, where the membership value is only dependent on its distance from the center, this compactness value is 2 1/4z. Of all possible fuzzy disks, compactness is therefore minimum for its crisp version.

Example 1. Let p be of the form 0.2 0.4 0.3 0.2 0.7 0.6 0.6 0.5 0.6

1

Then area a(p) = 4.1, perimeter p ( p ) = 2.3, and comp(p) = 0.775. 2 . Height and Width (Rosenfeld, 1984) and

So, height/width of a digital picture is the sum of the maximum membership values of each row/column. For the fuzzy subset p of Example 1, height is h ( p ) = 0.4 + 0.7 + 0.6 = 1.7, and width is w(p) = 0.6 + 0.7 + 0.6 = 1.9.

258

SANKAR K. PAL

3 . Length and Breadth (Pal and Ghosh, 1990, 1992b)

and

The length/breadth of an image fuzzy subset gives its longest expansion in the column/row direction. If p is crisp, pmn = 0 or 1; then lengthjbreadth is the maximum number of pixels in a column/row. Comparing Eqs, ( 1 5 ) and (16) with (13) and (14), we notice that the lengthlbreadth takes the summation of the entries in a column/row first and then maximizes over different columnshows, whereas the height/width first maximizes the entries in a column/row and then sums over different columnshows. For the fuzzy subset p in Example 1, / ( p ) = 0.4 + 0.7 + 0.5 = 1.6, and breadth is b ( p ) = 0.6 + 0.5 + 0.6 = 1.7. 4. Index ofArea Coverage (Pal and Ghosh, 1990, 1992b)

In the nonfuzzy case, the index of area coverage (IOAC) has value of one for a rectangle (placed along the axes of measurement). For a circle this value is nr2/(2r* 2r) = d 4 . ZOA C of a fuzzy image represents the fraction (which may be improper also) of the maximum area (that can be covered by the length and breadth of the image) actually covered by the image. For the fuzzy subset p of example 1, the maximum area that can be covered by its length and breadth is 1.6 x 1.7 = 2.72, whereas the actual area is 4.1, so the IOAC = 4.1/2.72 = 1.51. Note the difference between IOAC(p) and comp(p). Again, note the following relationships and

I(X)/h(X)I1 b ( X ) / w ( X )I1.

When equality holds for Eq. (18) the object is either vertically or horizontally oriented.

5 . Major Axis (Pal and Ghosh, 1992b) Find the length of the object. Now rotate the axes through an angle 8, 8 varying between 0" and 90". The angle for which length is maximum

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

259

is said to be the angle of inclination of the object (with the vertical). The corresponding axis along which the length is maximum is said to be the major axis. The length along the major axis denotes the expansion of the object. 6 . Minor Axis (Pal and Ghosh, 1992b)

The axis perpendicular to major axis, for which breadth is maximum, is defined as the minor axis of the object. 7. Center of Gravity (Pal and Ghosh, 1992b) The center of gravity (CG) of an object can be defined in various ways. Two such definitions are given here. (a) CG of an object can be defined as the point of intersection of the major and the minor axes. (b) Take any pixel as the center. Take a neighborhood of radius r. Find the energy (area) of the circle. Now shift the center of the circle over all the pixels of the object. The center for which the energy is maximum is defined as the CG. If there is any tie, then increase the radius and obtain the CG.

For the fuzzy subset p of Example 1, length is [ ( p ) = 1.6, and breadth is

b ( p ) = 1.7. Now if we rotate the object by 45" then its length is 41) = 0.6 + 0.7 + 0.6 = 1.9. Hence the object is inclined at an angle of 45" with vertical axis. So by major axis of this image we mean the axis inclined at an angle of 45" with the vertical. Similarly the minor axis of this object is inclined at an angle of 45" with horizontal. Trivially the CG of this object is through the pixel having membership 0.7.

8. Density (Pal and Ghosh, 1992b)

where N denotes the number of supports of p (i.e., summation is taken over pixels for which p is nonzero). The maximum value of density is one, and this value occurs only for a nonfuzzy case. Density can be used for finding the CG of an image. If we break the image into different regions, then the region having the maximum density may be regarded as containing the CG.

260

SANKAR K. PAL

9. Degree ofAdjacency (Pal and Ghosh, 1992b)

The degree to which two regions S and T of an image are adjacent is defined as

Here d(p) is the shortest distance between p and q, 4 is a border pixel (BP) of T, and p is a border pixel of S. The other symbols have the same meaning as in the previous discussion. The degree of adjacency of two regions is maximum (= 1) only when they are physically adjacent, i.e., d ( p ) = 0, and their membership values are also equal, i.e., p ( p ) = dq). If two regions are physically adjacent then their degree of adjacency is determined only by the difference of their membership values. Similarly, if the membership values of two regions are equal their degree of adjacency is determined by their physical distance only. The readers may note the difference between Eq. (20) and the adjacency definition given in Rosenfeld (1984).

IV. FLEXIBILITY IN MEMBERSHIP FUNCTIONS Since the theory of fuzzy sets is a generalization of the classical set theory, it has greater flexibility to capture faithfully the various aspects of incompleteness or imperfection (i.e., deficiencies) in information of a situation. The flexibility of fuzzy-set theory is associated with the elasticity property of the concept of its membership function. The grade of membership is a measure of the compatibility of an object with the concept represented by a fuzzy set. The higher the value of membership, the lesser will be the amount (or extent) to which the concept represented by a set needs to be stretched to fit an object. Since the grade of membership is both subjective and dependent on context, some difficulty of adjudging the membership value still remains. In other words, the problem is how to assess the membership of an element to a set. This is an issue where opinions vary, giving rise to uncertainties. Two operators, namely bound functions (Murthy and Pal, 1992) and spectralfuzzy sets (Pal and Das Gupta, 1992) have recently been defined to analyze the flexibility and uncertainty in membership function evaluation. These are explained below along with their significance in image analysis and pattern-recognition problems.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

26 1

A . Bound Functions

Consider, for example, a fuzzy set “tall.” This is represented by an S-type function that is a nondecreasing function of height. Now, the question is, can any such nondecreasing function be taken to represent the above fuzzy set? Intuitively, the answer is “no.” Bounds for such an S-type membership function p have recently been reported (Murthy and Pal, 1992) based on the properties of fuzzy correlation (Murthy et al., 1985). The correlation measure between two membership functions pl and p, relates the variation in their functional values. The main properties on which the correlation was formulated are as follows:

P I : If for higher values of p l , pz takes higher values, and for lower values of p , , p2 also takes lower values, then C ( p l ,p2) > 0. P2: If p , increases and p2 increases then C ( p , ,p2) > 0. P3: If p1 increases and p2 decreases then C(pl,p2) < 0. P2 and P3 should not be considered in isolation of P , . Had this been the case, one can cite several examples when both pl and p2 increase, but C(pl,pz) < 0; and p , increases and p, decreases but C(pl, p2) > 0. Subsequently, the types of membership functions that should preferably be avoided in representing fuzzy sets are categorized with the help of correlation. Bound functions h , and h, are accordingly derived in order to restrict the variation in the p function. They are

= 1,

where

E

l - & I X l l

. ,

= 0.25. The bounds for membership function p are such that

h,(x) 5 p(x) Ih,(x)

for x E [O, 11

For x belonging to any arbitrary interval, the bound functions will be changed proportionately. For h , I p 5 h 2 , C(h,, h,) 1 0 , C(h,,p ) 2 0 and C(h,,p ) 2 0. The function p lying in between k, and h2 does not have most of its variation concentrated (1) in a very small interval, (2) toward one of the end points of the interval under consideration, and (3) toward both the end points of the interval under consideration. In other words, the membership function p of a fuzzy set should not have, in any interval of the domain, an abrupt change from, nonmembership to membership or viceversa, because this can make the representation of a fuzzy set crisp.

262

SANKAR K. PAL

It is to be noted that Zadeh’s standard S function (Pal and Dutta Majumder, 1986; Zadeh et al., 1975) satisfies these bounds. The significance of the bound functions in selecting an S-type function p for an image-segmentation problem has been reported in detail in (Murthy and Pal, 1990). It has been shown that for detecting a minimum in the valley region of a histogram, the window length w of the function p : [0, w] -, [0, I ] should be less than the distance between two peaks around that valley region. The ability to make the fuzzy set theoretic approach flexible and robust will be demonstrated further in Section V.

B. Spectral Fuzw Sets The concept of spectral fuzzy sets is used where, instead of a single unique membership function, a set of functions reflecting various opinions on membership elements is available, so that each membership grade is attached to one of these functions. By giving due respect to all the opinions available for further processing, it reduces the difficulty (ambiguity) in selecting a single function. A spectral fuzzy subset F having n supports is characterized by a set or a band (spectrum) of r membership functions (reflecting r opinions) and may be represented as F =

u j

[

I

Upb(xj)/xj i

,

xj E Y, i = 1,2, ..., r ; j = 1 , 2 , ..., n

(23)

where r, the number of membership functions, may be called the cardinality of the opinion set. &(xj) denotes the degree of belonging of xi to the set F according to the ith membership function. The various properties and operations related to it have been defined by Pal and Das Gupta (1992). The incertitude or ambiguity associated with this set is two-fold, namely ambiguity in assessing a membership value to an element (d,) and ambiguity in deciding whether an element can be considered to be a member of the set or not (d2).Obviously, d2 is related to a fuzzy set, and its functional nature is the same as H’ (Eq. 2 with r = 1). On the other hand, d , reflects the amount of disparity (disagreement) within opinions because of the spectral nature. Regarding d , , it has been observed that human beings do not find it very difficult to assign memberships t o elements that have either very low or very high possibility of belonging to that set. In other words, the difference in opinions is low for those elements (supports) whose degree of inclusion or possibilities of belonging to a set is subjectively very low or very high. The variation in opinion, on the other hand, is high for those supports whose degree of belonging is fairly medium. For example, consider a spectral fuzzy set labeled “tall men” over the range 5 ft to 7 ft.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

263

The difference in opinions (or difficulty in assessing a membership) as expressed would be higher around 5 ft 10 in. than around 5 ft or 7 ft. Similarly, if someone is asked to bring a full glass of water several times with the same glass, the variation in the amount of water will be less compared to the case when half a glass of water is asked for. Similar observations may be made when the task is performed by several people. It therefore appears that the variation in membership function assignment is high for the elements having fairly medium belonging. Based on this concept, spectral index 8 is defined as (Pal and Das Gupta, 1992) 1 O(F) = d,(F) = n

where S= -

eli,

1

r/2(r - I ) ’ 1 (r + 1)/2r’

j = 1,2, ..., n

(24)

if r is even

if r is odd

8 provides, in a global sense, a quantitative measure of the average differences (or disagreement) or opinion, in assigning a membership value to a supporting element. The (dis)similarity between the concept of spectral fuzzy sets and those of the other tools such as probabilistic fuzzy set, interval-valued fuzzy set, fuzzy set of type 2 or ultrafuzzy set (Klir and Folger, 1988; Hirota, 1981; Turksen, 1986; Mizumoto and Tanaka, 1976; Zadeh, 1984) (which have also considered the difficulty in settling a definite degree of fuzziness or ambiguity) has been explained in Pal and Das Gupta (1992). The concept has been found to be significantly useful (Pal and Das Gupta, 1992) in segmentation of ill-defined regions where the selection of a particular threshold becomes questionable as far as its certainty is concerned. In other words, questions may arise such as “where is the boundary” or “what is the certainty that a level 1, say, is a boundary between object and background?” The opinions on these queries may vary from individual to individual because of the differences in opinion in assigning membership values to the various levels. In handling this uncertainty, the algorithm gives due respect to various opinions on membership of gray levels for object region, minimizes the image ambiguity d( = d, + d,) over the resulting band of membership functions and then makes a soft decision by providing a set of thresholds (instead of a single

264

SANKAR K. PAL

one) along with their certainty values. A hard (crisp) decision obviously corresponds to one with maximum d value, i.e., the level at which opinions differ most. The problems of edge detection and skeleton extraction (where incertitude arises from ill-defined regions and various opinions on membership values), and any expert system type application (where differences in experts' opinions leads to an uncertainty) may also be similarly handled within this framework. V. SOMEEXAMPLES OF FUZZYIMAGE PROCESSING OPERATIONS

Let us now describe some algorithms to show how the aforesaid information measures and geometrical properties (which reflect grayness ambiguity and spatial ambiguity in an image) can be incorporated in handling uncertainties in various operations, e.g., gray-level thresholding, enhancement, contour detection, and skeletonization by avoiding hard decisions and providing output in both fuzzy and nonfuzzy (as a special case) versions. It is t o be noted that these low-level operations (particularly image segmentation and object extraction) play a major role in an image-recognition system. As mentioned in Section 11, any error made in this process might propagate to feature extraction and classification. A . Threshold Selection (Fuzzy Segmentation) Given an L-level image X of dimension M x N with minimum and ,,,/ respectively, the algorithm for its fuzzy maximum gray values lminand segmentation into object and background may be described as follows:

Step I : Construct the membership plane using the standard S function of Zadeh (Zadeh, 1965; Pal and Dutta Majumder, 1986) as Pmn

= ~ ( 1= ) S(I; a, 6, C)

(28)

(called bright-image plane if the object regions possess higher gray values) or pmn= p ( l ) = 1 - S(I; a, b, C) (29) (called dark-image plane if the object regions possess lower gray values) with crossover point b and a band width A b = b - a = c - b. Step 2: Compute the parameter I ( X ) , where I(X) represents either grayness ambiguity or spatial ambiguity (as designated by H', correlation, compactness, IOAC, and adjacency, say) or both (i.e., product of grayness and spatial ambiguities).

265

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

Step 3: Vary b between lminand I,,,,, and select those b for which I ( X ) has local minima or maxima depending on I ( X ) .(Maxima correspond for the correlation measure only.) Among the local minima/maxima, let the global one have crossover point at s. The level s, therefore, denotes the crossover point of the fuzzy image plane

pmn, which has minimum grayness and/or geometrical ambiguity. The

pmnplane then can be viewed as a fuzzy segmented version of the image X.

For the purpose of nonfuzzy segmentation, we can take s as the threshold (or boundary) for classifying or segmenting an image into object and background. Faster methods of computation of the fuzzy parameters are explained in Pal and Ghosh (1992b). Note that w = 2 A b is the length of the window (such that [0, w ] [0, 11) that was shifted over the entire dynamic range. As w decreases, the p(xmn) plane tends to have more intensified contrast around the crossover point, thus resulting in a decrease of ambiguity in X.As a result, the possibility of detecting some undesirable thresholds (spurious minima) increases because of the smaller value of A b . On the other hand, an increase in w results in a higher value of fuzziness and thus leads toward the possibility of losing some of the weak minima. The criteria regarding the selection of membership functions and the length of window (i.e., w) have been reported recently in Murthy and Pal (1990, 1992), assuming continuous functions for both histogram and membership function. For a fuzzy set “bright image plane,” the membership function p : [0, w] + [0, 11 should be such that +

1. p is continuous, p(0) = 0, p ( w ) = 1 2. p is monotonically nondecreasing, and 3. p(x) = 1 - p(w - x ) for all x E [0, w ] . Furthermore, p, should satisfy the bound criteria derived based on the correlation (Section 1V.A). If, instead of a single membership function, we have a set of monotonically nondecreasing functions to represent a collection of various opinions on the bright membership plane px(x,,,,) and we wish to give due respect to all of these opinions, then the concept of spectral fuzzy sets (Section 1V.B) can be used to minimize the parameter spectral index (Eq. 24) in addition to one of those represented by Z(X) to manage this uncertainty. Consequently, it will make a soft decision by providing a set of thresholds associated with their respective certainty values. Details on this issue are available in Pal and Das Gupta (1992). Let us now describe another way of extracting an object by minimizing higher-order entropy (Eq. 2) of both object and background regions using

266

SANKAR K . PAL 1

0.5+------,

1 F

r

0

_ . * -

4

c

e

*

.

I> ‘ \

.. \

t

FIGURE1. Inverse z function (solid line) for computing object and background entropy.

an inverse n function as shown by the solid line in Fig. 1. Unlike the previous algorithm, the membership function does not need any parameter selection to control the output. Suppose s is the assumed threshold so that the gray level ranges [l, s] and [s + 1, L ] denote, respectively, the object and background of the image X. The inverse n-type function to obtain p,,, values of X is generated by taking unionofS[x;(s-(L-s)),s,L]and 1 - S [ x ; l , s , ( s + s - l)],whereS denotes the standard S function. The resulting function, as shown by the solid line, makes p lie in [ O S , 11. Since the ambiguity (difficulty) in deciding a level as a member of the object or the background is maximum for the boundary level s, it has been assigned a membership value of 0.5 (i.e., crossover point). Ambiguity decreases (i.e., degree of belonging to either object or background increases) as the gray value moves away from s on either side. The p,, thus obtained denotes the degree of belonging of a pixel x,,, to either object or background. Since s is not necessarily the midpoint of the entire gray scale, the membership function (solid line of Fig. 1) may not be a symmetric one. It is further to be noted that one may use any linear or nonlinear equation (instead of the standard S function) to represent the membership function in Fig. 1. Therefore, the task of object extraction is to: Step 1: Compute the rth order fuzzy entropy of the object Hh and the background H;1 considering only the spatially adjacent sequences of pixels present within the object and background respectively. Use the “min” operator to get the membership value of a sequence of pixels. Step 2: Compute the total rth order fuzzy entropy of the partitioned image as H,’ = Hh + HL. Step 3: Minimize H,’ with respect to s to get the threshold for object background classification.

267

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

Referring back to Section III.A, it is seen that H 2 reflects the homogeneity among the supports in a set in a better way than H ' does. The higher the value of r, the stronger is the validity of this fact. Thus, considering the problem of object-background classification, H' seems to be more sensitive (as r increases) to the selection of the appropriate threshold; i.e., the improper selection of the threshold is more strongly reflected by H' than Hr- 1 . For example, the thresholds obtained by the H Zmeasure have more validity than those by H' (which only takes into account the histogram information). Similar arguments hold good for even higher order (r > 2) entropy. The methods of object extraction (or segmentation) described above are all based on gray-level thresholding. Another way of doing this task is by pixel classification. The details on this technique using fuzzy c-means, fuzzy isodata, fuzzy dynamic clustering and fuzzy relaxation are available in Pal and Dutta Majumder (1986), Bezdek (1981), Kandel(1982), Pedrycz (1990), Lim and Lee (1990), Pal and Mitra (1990), Rosenfeld and Kak (1982), Dave and Bhaswan (1991), and Gonzalez and Wintz (1977). B. Contour Detection

Edge detection is also an image-segmentation technique where the contours/ boundaries of various regions are extracted based on the detection of discontinuity in grayness. The key factors of this approach are: 1. Most of the information of an image lies on the boundaries between different regions where there is a more or less abrupt change in gray levels, and 2. The human visual systems seem to make use of edge detection, but not of thresholding .

To formulate a fuzzy edge-detection algorithm, let us describe an edginess measure based on H' (Eq. 2 ) that denotes an amount of difficulty in deciding whether a pixel can be called an edge or not (Pal and Pal, 1990). Let N& be a 3 x 3 neighborhood of a pixel at ( x , y ) such that N& = ((X,Y), (x - 1 , Y h (x + 1,Yh (X,Y - 11, (X,Y

+ I),

( x - 1,Y - l ) , ( x - l , y + l ) , ( X + 1,Y - l ) , ( X + 1,Y

+ 1)1

(30)

The edge-entropy HZ of the pixel ( x , y ) , giving a measure of edginess at ( x , y ) , may be computed as follows. For every pixel ( x , y ) , compute the average, maximum, and minimum values of gray levels over N;. Let us denote the average, maximum, and minimum values by A v g , M a x , Min

268

SANKAR K. PAL

A

FIGURE2.

R

B

C

function for computing edge entropy.

respectively. Now define the following parameters. D = mux(Mux - Aug, Aug - Min]

(31)

B = Aug

(32)

A = B - D

(33)

C=B+D

(34)

A n-type membership function (Fig. 2) is then used to compute pxy for all ( x , y) E N A , such that p(A) = p(C) = 0.5 and p ( B ) = 1. It is to be noted that pv 2 0.5. Such a p v , therefore, gives the degree to which a gray level is close to the average value computed over Nx,y. In other words, it represents a fuzzy set pixel intensity close to its average value, averaged over N l , y . When all pixel values over Nx,y are either equal or close to each other (i.e., they are within the same region), such a transformation will make all pxy = 1 or close to 1. In other words, if there is no edge, pixel values will be close to each other, and the p values will be close to one, thus resulting in a low value of H I . On the other hand, if there is an edge (dissimilarity in then thep values will be more away from unity, thus gray values over N,,,), resulting in a high value of HI. Therefore, the entropy H' over Nx,,ycan be viewed as a measure of edginess (H&) at the point ( x , y ) . The higher the value of H:y, the stronger is the edge intensity and the easier is its detection. Such an entropy plane will represent the fuzzy edge-detected version of the image. As mentioned before, there are several ways in which one can define a n-type function as shown in Fig. 2. The proposed entropic measure is less sensitive to noise because of the use of a dynamic membership function based on a local neighborhood. The method is also not sensitive to the direction of edges. Other edginess measures and algorithms based on fuzzy set theory are available in Pal and Dutta Majumder (1986), PalandKing(l98la), PalandKing(1983), andPal(1986).

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

269

C. Optimum Enhancement Operator Selection

When an image is processed for visual interpretation, it is ultimately up to the viewers to judge its quality for a specific application and how well a particular method works. The process of evaluation of image quality therefore becomes subjective, which makes the definition of a well-processed image an elusive standard for comparison of algorithm performance. Again, it is customary to have an iterative process with human interaction in order to select an appropriate operator for obtaining the desired processed output. For example, consider the case of contrast enhancement using a nonlinear functional mapping. Not every kind of nonlinear function will produce a desired (meaningful) enhanced version. The questions that automatically arise are “Given an arbitrary image, which type of nonlinear functional form will be best suited without prior knowledge on image statistics (e.g., in remote applications like space autonomous operations where frequent human interaction is not possible) for highlighting its object?” and “Knowing the enhancement function, how can one quantify the enhancement quality for obtaining the optimal one?” Regarding the first question, even if the image statistics are given, it is possible only to estimate approximately the function required for enhancement, and the selection of the exact fucctional form still needs human interaction in an iterative process. The second question, on the other hand, needs individual judgment, which makes the optimum decision subjective. The method of optimization of the fuzzy geometrical properties and entropy has been found recently (Kundu and Pal, 1990) to be successful in providing quantitative indices in order to avoid such human iterative interaction in selecting an appropriate nonlinear function and to make the task of subjective evaluation objective. The use of fuzzy enhancement in hybrid coding of an image is described in Nasrabadi et al. (1983). Further discussion on this issue is made in Section VIII. D. Fuzzy Skeleton Extraction and FMA T

Let us now explain two methods for extracting the fuzzy skeleton of an object from a gray-tone image without getting involved in its (questionable) hard thresholding. The first one is based on minimization of the parameter ZOAC (Eq. 17) or compactness (Eq. 1 1 ) with respect to a-cuts (a-cut of a fuzzy set A comprises all elements of X whose membership value is greater than or equal to a, 0 < a 5 1) over a fuzzy core line (or skeleton) plane. The membership value of a pixel to the core line plane depends on its

270

SANKAR K. PAL

property of possessing maximum intensity, and property of occupying vertically and horizontally middle positions from the &-edge(pixels beyond which the membership value in the fuzzy segmented image becomes less than or equal to E , E > 0) of the object (Pal, 1989). If a nonfuzzy (or crisp) single pixel width skeleton is deserved, it can be obtained by a contour tracing algorithm (Pal et al., 1983) which takes into account the direction of contour. Note that the original image can not be reconstructed, like the other conventional techniques of gray skeleton extraction (Rosenfeld, and Kak, 1982; Levi and Montanari, 1970; Peleg and Rosenfeld, 1981; Salari and Siy, 1984) from the fuzzy skeleton obtained here. The second method is based on fuzzy medial axis transformation (FMAT) (Pal and Rosenfeld, 1991) using the concept of fuzzy disks. A fuzzy disk with center P is a fuzzy set in which membership depends only on the distance from P. For any fuzzy set f , there is a maximal fuzzy disk gPf 5 f centered at every point P , and f is the sup of the gPf ’s. (Moreover, iff is fuzzy convex, so is every gPf, but not conversely.) Let us call a set Sf of points f-sufficient if every gPf 5 gQf for some set of Q in S,; evidently f is then the sup of the g@’s. In particular, in a digital image, the set of Q’s at which gf is a (nonstrict) local maximum is f-sufficient. This set is called the fuzzy medial axis off, and the set of g@’s is called the fuzzy medial axis transformation (FMAT) off. These definitions reduce to the standard one i f f is a crisp set. For a gray-tone image X (denoting the nonnormalized fuzzy bright image plane), the FMAT algorithm computes, first of all, various fuzzy disks centered at the pixels and then retains a few (as small as possible) of them, as designated by gQs, so that their union can represent the entire image X. That is, the pixel value at any point t can be obtained from a union operation, as t has membership value equal to its own gray value (i.e., equal to its nonnormalized membership value to the bright image plane) in one of those retained disks. For example, consider a 5 x 5 image X as shown in Fig. 3. The lower-left pixel of intensity 4 has coordinate (1, 1). Fuzzy disks (upright square of odd side length) for all the border pixels have values 15,Oj except the one at position (1, 1) for which gP = (4,O).The pixels having intensity 6 have disk values of ( 6 , 5 , 0 )except the one at (2,2), for which it is ( 6 , 4 , 0 ) .The center pixel has gP = (7,6,41. In these sets of disk values, the first entry denotes the nonnormalized membership value of the pixel itself to that disk (i.e., membership value of the disk at r = 0). The consecutive entries denote similarly the memberships at r = 1,2, ... . The pixels constituting the fuzzy medial axis are marked bold. (Note that if we had the pixel intensity 4 of X replaced by 5 , the FMAT would have been reduced to only one disk with g(3,3) = (736,511.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

I

27 1

1

( 5

5

5

5

51

5

6

6

6

5

5

6

2

6

5

5

6

6

6

5

4

5

5

5

5

FIGURE3. A 5 x 5 digital image. Pixels belonging to fuzzy medial axis (FMA) are marked bold. Pixels belonging to reduced fuzzy medial axis (RFMA) are underlined.

In order to restore the deleted pixels, simply put all the disk values of FMA pixels at those locations back. In case a location has more than one such value, select the largest one. It is to be noted that the representation in Fig. 3 is redundant, i.e., some more disks can further be deleted without affecting the reconstruction. The redundancy in pixels (fuzzy disks) from the fuzzy medial axis output can be reduced by considering the criterion gPf(t) 5 sup g&(t), i = 1,2, ... instead of gPf(t) Ig@(t). In other words, eliminate many other gPf's for which there exists a set of gQ"s whose sup is greater than or equal to gP'. For example, the point at location (3,4) in Fig. 3 can be removed because it is contained in the union of the fuzzy disks around (3,3) and (2,4) for (4,411, i.e., g(3,4) 5: supIg(3,3), g(2,4)1 (or 5: S U P W ,31, g(4,4)1) for all pixels in X.Similar is the case with the pixel at location (4,3), which can also be removed. The pixels representing the final reduced MA are underlined in Fig. 3. Let RFMAT denote the FMAT after reducing its redundancy. To demonstrate its applicability on a real image let us consider Fig. 4(a) as input. Figure 4(b) denotes its RFMAT output. Therefore, the fuzzy medial axis provides a good skeleton of the darker (higher-intensity) pixels in an image apart from its exact representation. FMAT of an image can be considered as its core (prototype) version for the purpose of image matching. It is to be mentioned here that such a representation may not be economical in a practical situation. The details on this feature and the possible approximation in order to make it practically feasible are available in Pal and Wang (1991; 1992).

272

SANKAR K. PAL

FIGURE4. (a) 36 x 60 “S” image. (b) RFMAT output of “S” image.

Note that the membership values of the disks contain the information of image statistics. For example, if the image is smooth, the disk will not have abrupt change in its values. On the other hand, it will have abrupt change in case the image has salt-and-pepper noise or edginess. The concept of fuzzy MAT can therefore be used as spatial filtering (both high-pass and low-pass) of an image by manipulating the disk values to the extent desired and then putting them back while reconstructing the processed image.

VI. FEATURE/KNOWLEDGE ACQUISITION, MATCHING, AND RECOGNITION

In the previous sections, we have discussed, in detail, various measures (both fuzzy set theoretic and classical) for ambiguity in an image and their applications in representing and handling the various uncertainties that might arise in some of the important operations in image processing and analysis. The processed output can be obtained in both fuzzy and crisp

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

213

(as a special case) forms. As mentioned before, these operations (particularly image segmentation and object extraction) play major roles in an imagerecognition system. Any error made in this process might propagate to the higher level tasks, Le., in feature extraction, description, and classification/ analysis. Let us now explain, in brief, some of the approaches to show how the uncertainty in the tasks of feature extraction, boundary/shape detection of classes, learning and matching in a pattern recognition system can, in general, be represented and managed with the notion of fuzzy set theory. In picture-recognition and scene-analysis problems, the structural information is very abundant and important, and the recognition process includes not only the capability to assign the input pattern to a pattern class, but also the capacity to describe the characteristics of the pattern that make it ineligible for assignment to another class. In such cases complex patterns are described as hierarchical or treelike structures of simpler subpatterns, each simpler subpattern is again described in terms of even simpler subpatterns, and so on. Evidently, for this approach to be advantageous, the simplest subpatterns, called pattern primitives, are to be selected. Another activity that needs attention in this connection is the subject of shape analysis that has become an important subject in its own right. Shape analysis is of primal importance in feature/primitive selection and extraction problems. Description of shape can be done in two ways, e.g., in terms of scalar measurements and through structural descriptions. In this connection, it needs to be mentioned that shape description algorithms should be information-preserving in the sense that it is possible to reconstruct the shapes with some reasonable approximation from the descriptors. As described in Section 111.B, the fuzzy geometrical parameters also provide scalar measurements of shape of a gray image. Having extracted these fuzzy geometrical properties of an image, one can go by the decision theoretic approaches for its recognition. The fuzzy measures have recently been used by Leigh and Pal (1992) for motion-frame analysis. The way uncertainty arising from impreciseness and incompleteness in input pattern information can be handled heuristically has been reported recently in Pal and Mandal, (1992) by developing a linguistic recognition system based on approximate reasoning. The system can take input features either in linguistic form (F is very small, say) or in quantitative form (i.e., F is 500) or mixed form (Fis about 500) or set form (e.g., F is between 400 and 500). An input pattern has been viewed as consisting of various combinations of the three primary properties, e.g., small, medium, and high, possessed by its different features to some degree. The system provides a natural output decision in linguistic form, which is associated with a confidence factor denoting the degree of certainty of the decision. There have also been some attempts (Nath and Lee, 1983; Yager, 1981) to provide the design concept

274

SANKAR K. PAL

of a classifier that needs a priori knowledge from the experts in linguistic form, It is to be noted that the patterns having imprecise or incomplete information are usually ignored or discarded from the designing and testing processes of the conventional decision theoretic or syntactic classifiers (Duda and Hart, 1973; Tou and Gonzalez, 1974). For feature selection and extraction problems using fuzzy set theory, the readers may refer to the papers by Pal and Chakraborty (1986), Pal (1992b), Dave and Bhaswan (1991), Bezdek and Anderson, 1985), Bezdek and Castelaz (1977) and Di Gesu and Maccarone (1986). A multivalued recognition system (Mandal et al., 1992a) based on the concept of fuzzy sets has been formulated recently. This system is capable of handling various imprecise inputs and in providing multiple class choices corresponding to any input. Depending on the geometric complexity (Mandal et al., 1992b) and the relative positions of the pattern classes in the feature space, whole feature space is decomposed into some overlapping regions. The system uses Zadeh’s compositionai rule of inference (Zadeh, 1977) in order to recognize the samples. Application of this system to IRS (Indian remote sensing) imagery for detecting curved structures has been reported by Mandal (1 992). In a remotely sensed image, the regions (objects) are usually ill-defined because of both grayness and spatial ambiguities. Moreover, the gray value assigned to a particular pixel of a remotely sensed image is the average reflectance of different types of ground covers present in the corresponding pixel area (36.25rn x 36.2% for the Indian remote sensing [IRS] imagery). Therefore, a pixel may represent more than one class with a varying degree of belonging. For detecting the curved structures, the recognition system (Mandal et al., 1992a) is initially applied on an IRS image to classify (based on the spectral knowledge of the image) its pixels into six classes corresponding to six land cover types namely, pond water, turbid water, concrete structure, habitation, vegetation, and open space. The green and infrared band information, being sensitive than other band images to discriminate various land cover types, are used for the classification. The clustered images are then processed for detecting the narrow concrete structure curves. These curves include, basically, the roads and railway tracks. The width of such attributes has an upper bound, which was considered there to be three pixels for practical reasons. So all the pixels lying on the concrete structure curves with width not more than three pixels were initially considered as the candidate set for the narrow curves. Because of the low-pixel resolutions (36.2% x 36.25m for IRS imagery) of the remotely sensed images, all existing portions of such real curve segments may not be reflected as concrete structures, and, as a result, the candidate

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

275

pixel set may constitute some broken curve segments. In order to identify the curves in a better extent, a traversal through the candidate pixels was used. Before traversing process, one also needs to thin the candidate curve patterns so that a unique traversal can be made through the existing curve segments with candidate pixels. Thus, the total procedure to find the narrow concrete structure curves consists of three parts: (1) selecting the candidate pixels for such curves, (2) thinning the candidate curve patterns, and (3) traversing the thinned patterns to make some obvious connections between different isolated curve segments. The multiple choices provided by the classifier in making a decision are utilized to a great extent in the traversal algorithm. Some of the movements are governed by only the second and combined choices. After the traversal, the noisy curve segments (i.e., with insignificant lengths) are discarded from the curve patterns. The residual curve segments represent the skeleton version of the curve patterns. To complete the curve pattern, the concrete structure pixels lying in the eight neighboring positions corresponding to the pixels on the above-obtained narrow curve patterns are now put back. This resultant image represents the narrow concrete structure curves corresponding to an image frame. The effectiveness of the methods has been demonstrated on an IRS image frame representing a part of the city Calcutta. Figures 5a and 5b show the input of the image in Green and infrared bands respectively. Figure 5c shows the clustered version into six regions, and Fig. 5d demonstrates the detected narrow concrete structure curves. The results are found to agree well with the ground truths. The classification accuracy of the recognition system (Mandal et a / . , 1992a) is not only found to be good, but its stability of providing multiple choices in making decisions is also found t o be very effective in detecting the roadlike structures from IRS images. Let us now consider the syntactic approach of description and recognition of an image based on the primitives extracted from the structural information of its shape. The syntactic approach t o pattern recognition involves the representation of a pattern by a string of concatenated subpatterns called primitives. These primitives are considered to be the terminal alphabets of a formal grammar whose language is the set of patterns belonging to the same class. The task of recognition therefore involves a parsing of the string. Because of the ill-defined character of the structural information, the uncertainty may arise both in defining primitives and in relations among them. In order to handle them, the syntactic approach has incorporated the concept of fuzzy sets at two levels. First, the pattern primitives are themselves considered to be labels of fuzzy sets, i.e., such subpatterns as “almost circular arcs,” “gentle,” “fair,” and “sharp” curves are considered. Secondly, the structural relations among the subpatterns may be fuzzy, so

276

SANKAR K . PAL

FIOURE 5. IRS Calcutta images in (a) green band, (b) infrared band, (c) clustered image in six classes, and (d) detected roads along with a bridge (enclosed by dark lines).

that the formal grammar is fuzzified by the weighted production rules, and the grade of membership of a string is obtained by min-max composition of the grades of the production used in the derivations. For example, the primitives like line and curve may be viewed in terms of arcs with varying grades of membership from 0 to 1; 0 representing a straight line and 1 representing a sharp arc. Based on this concept, an algorithm was developed (Pal and Dutta Majumder, 1986; Pal et al., 1983) for automatic

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

277

extraction of primitives from gray-tone edge-detected images by defining membership functions for vertical line, horizontal line, and oblique line from the angle of inclination, and the degree of arcness of a line segment from the coordinates of its end points. Its effectiveness in recognizing x-ray images, hand and wrist bones, and nuclear patterns of brain neurosecretory cells is demonstrated in (Pathak and Pal, 1986b; Kwabwe et a / . , 1985; Azimi et at., 1982; Pal and Bhattacharyya, 1990). A similar interpretation of the shape parameters of triangle, rectangle and quadrangle in terms of membership for “approximate isoceles triangles,” “approximate equilateral triangles,” “approximate right triangle,’’ and so on has also been made (Huntsberger et at., 1986) for their classification in a color image. In order to represent the uncertainty in physical relations among the primitives, the production rules of a formal grammar are fuzzified to account for the fuzziness in relation among the primitives, thereby increasing the generative power of a grammar. Such a grammar is called fuzzy grammar (Lee and Zadeh, 1969; Thomason, 1973; DePalma and Yau, 1975). A concept of fractionally fuzzy grammars (Pathak et al., 1984) has also been introduced with a view to improving the effectiveness of a syntactic recognition system. It has been observed (Pathak and Pal, 1986b; Pathak et at., 1984) that the incorporation of the element of fuzziness in defining sharp, fair, and gentle curves in the grammars enables one to work with a much smaller number of primitives. By introducing fuzziness in the physical relations among the primitives, it was also possible to use the same set of production rules and nonterminals at each stage. This is expected to reduce, to some extent, the time required for parsing in the sense that parsing needs to be done only once at each stage, unlike the case of the nonfuzzy approach, where each string has to be parsed more than once, in general, at each stage. However, this merit has to be balanced against the fact that the fuzzy grammars are not as simple as the corresponding nonfuzzy grammars. Recently, rule-based systems have gained popularity in pattern recognition and high-level vision activities. By modeling the rules and facts in terms of fuzzy sets, it is possible to make inferences using the concept of approximate reasoning. Such a system has been designed recently (Nafarieh and Keller, 1991) for automatic target recognition using about 40 rules. A knowledge-based approach using Dempster-Shafer theory of evidence (Shafer, 1976) has also been formulated (Korvin et al., 1990) for managing uncertainty in object-recognition problems when features fail to be homogeneous. Meaningful pay-offs are defined in this context. The problem is tackled by considering masses with fuzzy focal elements. An evidential approach to problem solving was also developed when a large number of knowledge systems (which might give contradictory or inconsistent

278

SANKAR K. PAL

information) is available (Korvin et al., 1990). It is to be mentioned in this connection that the definitions of credibility and plausibility of DempsterShafer theory of evidence when the evidence and propositions are both fuzzy in nature are available in Yen (1990) and Pal and Das Gupta (1990). Another way of handling uncertainty in knowledge acquistition based on the theory of rough sets (Pawlak, 1982) is reported in Grzymala-Busse (1988). The approach considered uncertainties arising from the inconsistencies in different actions of different experts for the same object, or from the different actions of the same expert for different objects described by the same values of conditions. The method involves learning from examples. For a set of conditions of the information systems, and a given action of an expert, lower and upper approximations of a classification, generated by the action, have been computed with the help of rough set theory. Based on these approximations, the rules produced from the information stored in a data base are categorized as certain and possible. The certain rules may be propagated separately during the inference process, producing new certain rules. Similarly, the possible rules may be propagated in a parallel way. Fuzzy set theory and rough set theory are independent and offer alternative approaches to deal with uncertainty. However, there is a connection between rough set theory and Dempster-Shafer theory, though they have been developed separately. Dempster-Shafer theory uses the belief function as a main tool, whereas the rough set theory makes use of the family of all sets with common lower and upper approximations (Pawlak, 1985; Grzymala-Busse, 1988). VII. FUSIONOF FUZZYSETSAND NEURAL NETWORKS: NEURO-FUZZY APPROACH Artificial neural networks are signal-processing systems that emulate the human brain, i.e., the behavior of biological nervous systems, by providing a mathematical model of combination of numerous neurons connected in a network. Human intelligenceand discriminating power are mainly attributed to the massively connected network of biological neurons in the human brain. The collective computational abilities of the densely interconnected nodes or processors may provide a material technique, at least to a great extent, for solving complex real-life problems in a manner a human being does (Pao, 1989; Kosko, 1992; Bezdek and Pal, 1992; Lippman, 1989; Ghosh et al., 1991, 1992a,b, 1993; Pal and Mitra, 1992; Burr, 1988; Proc. 1st Int. Conf. on Fuzzy Logic and Neural Networks, 1990b; Proc. 1st IEEE Int. Conf. on Fuzzy Systems, 1992a; Proc. 2nd Int. Conf. on Fuzzy Logic and Neural Networks, 1992b).

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

279

We see that the fuzzy set theoretic models try to mimic human reasoning and the capability of handling uncertainty, whereas the neural network models attempt to emulate the architecture and information representation schemes of human brain. The fusion of these two new technologies therefore promise enormous intellectual and material gains in the field of computer and system science by incorporating the similarity in their logical operations and learning processes, and combining their individual merits. The fusion or integration is mainly tried out in the following ways or in any combination of them. 1 . Incorporating fuzziness into neural network frameworks. This includes assigning fuzzy labels to training samples, fuzzifying the input data, and obtaining output in terms of fuzzy sets (Fig. 6).

Neural

labels

network

FIGURE 6 . Neural network implementing fuzzy classifier.

2. Making the individual neuron fuzzy (input to such a neuron is a fuzzy set and the output also is a fuzzy set). Activity of the networks involving fuzzy neurons is a fuzzy process (Fig. 7).

FIGURE7 . Block diagram of fuzzy neuron.

3. Designing neural networks guided by fuzzy logic formalism (i.e., designing neural networks to implement fuzzy logic) and realization of membership functions representing fuzzy sets by neural networks (Fig. 8).

AnteccdeiiL

clauses

Neural network

Error

Consequelit clauses

FIGURE 8. Neural network implementing fuzzy logic.

280

SANKAR K. PAL

4. Changing the basic characteristics of the neurons so that they perform

the operations used in fuzzy set theory (like fuzzy union, intersection, aggregation) instead of doing the standard multiplication and addition operations (Fig. 9).

FIGURE9. Neural network implementing fuzzy connectives.

5 . Modeling the error or instability or energy function of a neural

network based systems using measures of fuzzineduncertainty of a set (Fig. 10).

Puzzy

Neural network

Nonfuzzy output

FIGURE10. Layered network implementing self-organization.

The first way of integration is to incorporate the concept of fuzziness into a neural network framework, i.e., to build fuzzy neural networks. For example, the target output of the neurons in the output layer during training can be fuzzy label vectors. In this case the network itself is functioning as a fuzzy classifier. Keller and Hunt (1985) first suggested the incorporation of the concept of fuzzy pattern recognition into perceptron (single layer). They described a method for fuzzifying the labeled target data used for training the perceptron. Instead of giving hard labels to the training samples, membership functions denoting their degrees of belonging to the classes were used as labels. Instead of using the weight updation as

w+ w + ex, (c is a constant and

(35)

X, is the input data) they used

w w + JUlk - U*klmCX, 4-

(36)

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

28 1

where m is a constant and uik denotes the degree of belonging of xk to the ith class. Incorporation of membership function in the label vectors also acted as a good stopping criterion for linearly nonseparable classes (where the classical perceptron oscillates). The concept of fuzzy sets has also been introduced by Pal and Mitra (1992) in designing classifiers (both supervised and unsupervised) for uncertainty analysis and recognition of patterns using Kohonen’s model and the multilayer perceptron. A self-organizing artificial neural network capable of fuzzy partitioning of patterns has been developed that takes membership values to linguistic properties (e.g., low, medium, and high) along with some contextual class information to constitute the input vector. An index of disorder based on mean square distance between input and weight vectors has been defined in order to provide a quantitative measure for the ordering of the output space. The method based on the multilayer perceptron, on the other hand, involves assignment of appropriate weights to the backpropagated errors depending on the membership values at the corresponding outputs. During training, the learning rate is gradually decreased until the network converges to a minimum error solution. The performance is compared with that of the conventional model and Bayes’s classifier. It has been shown that these modified versions provide better performance for certain nonconvex decision regions (Pal and Mitra, 1993) as compared to the conventional ones. Though the effectiveness of the classifiers is demonstrated on some artificial data and speech data, the problem of image recognition under uncertainty can easily be dealt with within this framework. For example, the fuzzy geometrical properties (Section 1II.B) of a pattern can be used as features for learning the network parameters. The fuzzy segmented version, fuzzy edge-detected version, or fuzzy skeleton of an image may also be used along with their degrees (values) of ambiguities for the purpose of network training and its recognition. Sanchez (1990) has pointed out the noticeable similarities (like training by example, dynamic adjustment of changes in the environment, ability to generalize, tolerance to noise, graceful degradation at the border of the domain of expertise, and ability to discover new relations between variables) between neural networks and expert systems. He developed a fuzzy version of the connectionist expert system of Gallant (1988). In such an expert classification system knowledge base is generated from a training set of examples and is stored as connection strengths. The weight ( w i j ) between the input and the hidden layers are linguistic labels of fuzzy sets (identified by membership function) characterizing the variation of the input neurons and are assumed to be known. The weights between the hidden layer and the output layers (bij)are determined by training. The output of the neurons are computed by combining the weights for

282

inferencing as

SANKAR K. PAL

q ( t ) = min(w@,b, J

(37)

A possible utility of such an expert system in biomedical domain is also

stated. The second way to incorporate fuzziness into the standard neural network is by making the individual neurons fuzzy (Lee and Lee, 1975; Yamakawa and Tomada, 1989). The idea is originally introduced by Lee and Lee (1975). The classical model of a neuron (McCulloch and Pitts, 1943) assumes its activity as an all-or-none process. It fails to model a type of imprecision that is associated with the lack of sharp transition from the occurrence to nonoccurrence of an event. Some of the concepts of fuzzy set theory are employed by Lee and Lee to define a fuzzy neuron, which is a generalization of the classical neuron. The activity of a fuzzy neuron is a fuzzy process (Lee and Lee, 1975). The input to such a neuron is a fuzzy set, and the outputs are equal to some positive numbers pj’s (0 < pj Il), if it is firing, and zero if it is quiet, pj denoting the degree to which thejth output is fired, the output is, therefore, also a fuzzy set. Unlike conventional neurons, such a neuron has multiple outputs. A fuzzy neural network is defined as a collection of interconnected fuzzy neurons. The utility of fuzzy neural networks to synthesize fuzzy automata is also investigated by them. The third fusion methodology is to use neural networks for a variety of computational tasks within the framework of a preexisting fuzzy model (i.e., implementation of fuzzy-logic formalism using neural networks). The use of multilayer feed forward neural network for implementing fuzzy logic rules (if-then rules) in introduced by Keller and coworkers (Keller and Tahani, 1991; Keller and Tahani, 1992; Keller et a/., 1992; Keller and Krishnapuram, 1992). It has been shown that the networks designed for implementing fuzzy rules can learn and extrapolate complex relationships between antecedents and consequent clauses for rules containing single, conjunctive, and disjunctive antecedent clauses. For rules having conjunctive clauses, the architecture has a fixed number of neurons in the input layer for each antecedent clause, a set of neurons in the hidden layer connected only to the neurons in the input layer associated with each antecedent clause. The neurons in the output layer are connected to all the neurons in the hidden layer. For implementing rules with disjunctive antecedent clauses, one more hidden layer was necessary. In Keller et a/. (1992), attempts are made to embed apriori knowledge of each rule directly into the weights of the network. In other networks the standard back-propagation learning algorithm is applied for learning weights. An attempt is also made by Takagi et al. (1992) to design structured neural networks to perform if-then fuzzy inference rules.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

283

Attempts are also made to have membership function representation by neural networks (Ishibuchi and Tanaka, 1990; Takagi and Hayashi, 1991). A method has been suggested by Ishibuchi and Tanaka (1990) to identify real-valued and interval-valued membership functions from a set of given input-output data using a feed-forward layered neural network and backpropagation of error. Suggestions are also given to design membership functions of fuzzy neurons in Yamakawa and Furukawa (1992). A great deal of effort has also been given to design neural networkdriven optimal decision rules for fuzzy controllers (Gupta et al., 1989; Yager, 1992; Hayashi et al., 1992). A system for implementing fuzzy logic controllers using a neural network is designed in Yager (1992). The linguistic values associated with the fuzzy control rules are realized by separate neural-network blocks. The network is crafted depending on the inference structure provided by fuzzy logic involving intersection operations of fuzzy sets. The importance of different rules in the system is learned by operating the whole system and employing a rule that is of the form of the generalized delta rule. Suggestions are also given (Berenji, 1992; Berenji and Khedkar, 1992) for learning and tuning of fuzzy logic controllers based on reinforcement learning. Emphasis is mainly to adjust membership functions of the linguistic labels used in control rules. A fuzzy version of Kohonen’s self-organizing feature map algorithm is developed by Huntsberger and Ajmarangsee (1990) in order to generate continuous valued outputs (representing the degree of belonging) by adding one layer to the original Kohonen network. Fuzziness is also incorporated in the learning rate by replacing the learning rate usually found in Kohonentype update rules for the weight vectors with fuzzy membership of the nodes in each class. The proposed update rule is where &k is the fuzzy membership of input standard updating rule

?(t

xk

in class i instead of the

+ 1) = Y(t)+ a[& - Y(tN

(39)

with a as a constant. They also have shown that the results produced by this fuzzy version of Kohonen’s algorithm are similar to those obtained by fuzzy c-means algorithms (Bezdek, 1981). Parallel implementations of this technique are also suggested. Further modification on the rate of learning is done by Bezdek et a/. (1992), and a relationship between the fuzzy version of Kohonen’s algorithm and the fuzzy c-means algorithm is established. Another way of fusion is to change the integration/transformation operation performed at each node so that they perform some sort of fuzzy aggregation (i.e., fuzzy union, intersection, aggregation). In Krishnapuram

284

SANKAR K. PAL

and Lee (1992a,b) fuzzy set connectives are used in multilayer network structures suitable for pattern recognition and other decision-making systems. Various union, intersection, generalized mean, and multiplicative hybrid operators (which are used by fuzzy sets to aggregate imprecise information in order to arrive at a decision in uncertain environments) are implemented by layered networks. A generalized mean operator was introduced by Yager (1978). Its form is

where wis are the relative importance given to corresponding inputs xis and C;= wi = 1. The hybrid (compensatory) model used was the y-model of Zimmerman and Zysno (1983) and is expressed as y = ni(xiG;)'-Y(l - n,(l - Xi)6i)Y

(41)

where 6;= rn and 0 5 y i 1; xi E [0, 11 are the inputs, bi is the weight associated with xi and y controls the degree of compensation. The hybrid operator can behave as union, intersection, or mean operator with different sets of parameters, which can be learned through training procedure. An iterative algorithm to determine the type of aggregation functions and its parameters at each node in the network is also provided, thereby making the network more flexible. The learning procedure involved is the same as that of the MLP. The training procedure of the multiplicative y-model is slow. To achieve faster convergence the additive y-model is studied, under the above framework, by Keller and Chen (1992) as an alternative connective in such networks. Gupta (1992) suggested the use of generalized AND (which can be expressed using the notation of triangular norms) and OR (represented by triangular conorm) operations for fuzzy signals (signals bounded by the graded membership function over the unit interval [0, 11) instead of multiplication and summation operations as used in standard neural networks. Thus for fuzzy inputs, x ( t ) E [0, 11" and synaptic strengths w ( t ) E [0, 11" the weighted synaptic signal z ( t ) E [0, 11" is defined as zi(t) = w i ( t ) A N D x i ( t ) ,

i = 1,2,

..., n,

(42)

and the aggregated input to a neuron is ui(t) = O R z j ( t ) . i

(43)

The nonlinear mapping with threshold wo E [0, 11 is then defined as

~ ; ( t=) [ui(t) OR wo(t)]"

(44)

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

285

where a, is a positive quantity. For 0 < a I1 the operation corresponds to dilation operation of a fuzzy set, and for CY > 1 it corresponds to concentration operation. Pedrycz (1991) tried to introduce fuzziness in neural networks in a different way. He pointed out the analogies between structures involving composite operators and a certain class of neural networks. Links are established between neural network architectures and relational systems in terms of fuzzy relational equations. The proposed architecture is based exclusively on set theoretic operations. The individual neurons perform logical operations (like max, min) which are mainly used in set theory instead of arithmetic operations. The problem of learning of connection strengths or weights was also studied, and relevant learning rules were proposed. A performance index, called equality index, is also introduced keeping track of these logical operations. Pedrycz has also suggested (1992) a design of neural networks to implement logic operations used in fuzzy set theory. The fifth way to integrate the concepts of fuzzy sets and neural networks is to use the fuzziness measures/uncertainty measures of a fuzzy set to model the error in neural networks. An attempt is made in this context by Ghosh el a/. (1993) to incorporate various fuzziness measures in a multilayer network for performing (unsupervised) self-organizing tasks in image processing, in general, and object extraction in particular. The network architecture is basically a feed forward one with back propagation of error (Fig. l l ) , but unlike conventional MLP it does not require any supervised learning. Each layer has M x N neurons for an M x N image (each neuron corresponding to an image pixel). Each neuron is connected to the corresponding neuron in the previous layer and its neighbors. Another structural difference from the standard MLP is that there exists a feedback

FIGURE1 1. Schematic representation of self-organizing rnultilayer neural network.

286

SANKAR

K. PAL

path from the output to the input layer. The status of neurons in the output layer is described as a fuzzy set. A fuzziness measure (e.g., index of fuzziness and entropy as mentioned in Section 1II.A) of this set is used as a measure of error in the system (instability of the network) and is back-propagated to correct weights. The input value (Ui)to the ith neuron in any layer (except the input layer) is calculated using the formula

where Wo is the connection strength between the ith neuron of one layer andjth neuron of the previous layer, and oj is the output status of the j t h neuron of the previous layer. j can either belong to the neighborhood of i, or j = i of the previous layer. The output is then obtained as 1

Starting from the input layer, this way the input pattern is passed on to the output layer and the corresponding output states are calculated. The output value of each neuron lies in [0, 11. After the weights have been adjusted by back-propagating the fuzziness measure of the output status of the neurons (which is treated as a fuzzy set) properly, the output of the neurons in the output layer is fed back to the corresponding neurons in the input layer. The second pass is then continued with this as input. The iteration (updating of weights) is continued as in the previous case until the network stabilizes, i.e., the error value (measure of fuzziness) becomes negligible. For example, the expression for weight updating for quadratic index of fuzziness (Kauffmann, 1980) is AY; =

tt( - oj)f'(lj)oi q(l - oj)f'(lj)oi

if 0 5 oj I0.5 if 0.5 < oj 5 1.0

(47)

for the output layer and

for hidden layers; where 6 k = -aE/aZk, and q is a proportionality constant. In the converged state the ON neurons constitute one class, and the OFF neurons another. Figure 12 demonstrates the variation of the learning rate for different fuzziness measures. In Kios and Liu (1992) an approach is provided to design optimal network architecture by optimization of fuzziness of a set.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

287

I I

I

I

I I I

I I

I

I I I

I

I I I

I

I

1

1-

Logarithmic entropy

II

I

~

/

‘\iF

Linear index of fuzziness

\

/

Quadratic index

\

0 0

0.2

0.4 0.6 Status of a neuron

0.8

1.0

FIGURE 12. Rate of learning with variation of output status for different error measures,

288

SANKAR K. PAL

Note that these attempts of integration are mainly in the field of pattern recognition and to some extent in fuzzy logic control. Literature on neurofuzzy image-processing is not adequate at this moment. For further references on this issue one can refer to Kosko (1992); Takagi (1990); Bezdek and Pal (1992); Werbos (1992); Proc. Int. Joint Conf. on Neural Networks (1989); Archer and Wang (1991); Mitra and Pal (1992); Carpenter et al. (1991); Proc. 2nd Joint Tech. Workshop on Neural Networks and Fuzzy Logic (1990a); Proc. 1st IEEE Int. Conf. on Fuzzy Systems, Znt. J. of Approximate Reasoning, vol. 6, no. 2 (1992); IEEE Tr. on Neural Networks, vol. 3, no. 5 (1992); Int. J . of Pattern Recognition and AI, vol. 6, no. 1 (1992). VIII.

U S E OF

GENETIC ALGORITHMS

Genetic algorithms (GAS) (Goldberg, 1989; Davis, 1991) are highly parallel, mathematical, adaptive search procedures (i.e., problem-solving methods) based loosely on the processes or mechanics of natural genetics and Darwinian survival of the fittest. They model operations found in nature to form an efficient search that is effective across a broad spectrum of problems. These algorithms apply genetically inspired operators to populations of potential solutions in an iterative fashion, creating new populations while searching for an optimal (or near-optimal) solution to the problem at hand. Population is a key word here: the fact that many points in the space are searched in parallel sets genetic algorithms apart from other search operators. Another important characteristic of genetic algorithms is that they are very effective when searching (e.g., optimizing) function spaces that are not smooth or continuous functions that are very difficult (or impossible) to search using calculus based methods. Genetic algorithms are also blind; that is, they know nothing of the problem being solved other than payoff or penalty information, GAS differ from many conventional search algorithms in the following ways. They consider many points in the search space simultaneously, not a single point, and therefore have less chance of converging to local optima. They deal directly with strings of characters representing the parameter sets, not the parameters themselves. They use probabilistic rules to guide their searching process instead of deterministic rules. GAS find out the global near-optimal solution employing three basic operations-reproduction/selection, crossover, and mutation-over a limited number of strings (chromosomes) called population. A string is a coded version of the parameter set. For example, a binary string of length p q can be considered as a chromosomal (string) representation of the

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

289

parameter set @arjI i = 1,2, ...,p ] , where each substring of length q is assumed to be the representative of each parameter. Reproduction is a process in which individual strings are copied according to their objective function values, f,called the fitness function. These strings are then entered into a mating pool, a tentative new population, for further genetic operator action. The crossover generates offspring for the new generation using the highly fitted strings (parents) selected randomly from the mating pool. Each pair of strings undergoes crossing over as follows: An integer position k is selected uniformly at random between 1 and I - 1, where I is the string length greater than 1. Two new strings are created by swapping all characters from position k + 1 to 1. Mutation is the occasional (with small probability) random alteration of the value of a string position. A random bit position of a random string is selected and is replaced by another alphabet. In dealing with pattern analysis problems, GAS may be helpful in determining the appropriate membership functions, rules, and parameter space, and in providing a reasonably suitable solution. For this purpose, a suitable fuzzy fitness function needs to be defined depending on the problem. Fuzziness may also be incorporated in the encoding process by introducing a membership function representing the degree of similarity/ closeness between the chromosome parameters (strings). For example, consider a scene analysis problem where the relations among various segments (or objects) may be defined in terms of fuzzy labels such as close, around, partially behind, or occluded. Given a labelling of each of the segments, the degrees to which each relationship fits each pair of segments can be measured. These measures can be combined to define an overall fuzzy fitness function. Given this fitness function, the relations among objects, and the relations among classes to which the objects belong, a genetic algorithm searches the space to find the best solution in determining a class to be associated most appropriately to each object. An approach based on genetic algorithm for scene labeling is reported in Ankenbrandt et al. (1990). Let us now consider the problem of contrast enhancement of an image by gray-level modification. Given an image it is difficult t o select a functional form that will be best suited without prior knowledge of image statistics. Even if we are given the image statistics it is possible only to estimate approximately the function required for enhancement, and the selection of the exact functional form still needs human interaction in an iterative process. Bhandari el al. (1993) attempted to demonstrate the suitability of GAS in automatically selecting an optimum set of 12 parameter values of a generalized enhancement function that maximizes some fitness function.

290

SANKAR K. PAL

The algorithm used both spatial and grayness ambiguity measures (as mentioned in Section 1II.A) as the fitness value. The algorithm was implemented on images having compact and elongated (noncompact) objects and found to produce satisfactory results. The algorithm does not need iterative visual interaction and prior knowledge of image statistics in order to select the appropriate enhancement function. Convergence of the algorithm is experimentally verified. Since the domains of the parameters here are continuous, one needs to increase the length of the strings to obtain a more accurate solution. Some attempts in applying the GAS for classification, segmentation, primitive extraction and vision problems are reported in Belew and Brooker (1991). The basic idea is to use the GA to search efficiently the hyper-space of parameters in order to maximize some desirable criteria. In Section V.D, we have seen that the task of extracting fuzzy medial axis transformation (FMAT) of an image involves enormous computation, and it is not guaranteed even if the resulting output provides a compact minimal set for image representation. Searching based on GAS may be helpful in this case. Primitive extraction and aggregation is another area where GAS may be useful. Some recent applications in determining optimal set of weights for neural networks are available in Whitley et al. (1990), Bornholdt and Graugenz (1992), and Machado and Rocha (1992). It has been found that the backpropagation technique of multilayer perceptron may be avoided, thereby improving its computational time and the possibility of getting stuck to local minima. It is to be mentioned here that the GAS are computationally expensive. Moreover, one should be careful in selecting the initial population and the recombination operators. IX. DISCUSSION The problem of pattern analysis and image recognition under fuzziness and uncertainty has been considered. The role of fuzzy logic in representing and managing the uncertainties (which might arise in a recognition system) was explained. Various fuzzy set theoretic tools for measuring information on grayness ambiguity and spatial ambiguity in an image were listed along with their characteristics. Some examples of image-processing operations (e.g., segmentation, skeleton extraction, and edge detection), whose outputs are responsible for the overall performance of a recognition (vision) system, were considered in order to demonstrate the effectiveness of these tools in managing uncertainties by providing both soft and hard decisions. Uncertainty in determining a membership function in this regard and the tools for its management were also explained. Apart from representing and managing

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

29 1

uncertainties, the tools based on fuzzy set theory can also be used for providing quantitative measures in order to avoid the subjectivejudgment on the quality of processed output and to avoid human intervention in autonomous operations. Most of the algorithms and tools described here were developed recently by the author with his colleagues. Some of the illustrations were taken from the existing literature and put here together in a unified framework. Processing of color images has not been considered here. Some recent results on color image information and processing in the notion of fuzzy logic are available in Lim and Lee (1990), Xie (1990), and Pal (1991). Uncertainties involved in other parts of a recognition system, such as primitive extraction/analysis and syntactic classification and knowledge acquisition, were discussed. An application of multivalued approach to IRS image analysis has been demonstrated. Recent attempts of researchers on fusion of fuzzy set theory and neural networks for better handling of uncertainty (in the sense of robustness, performance and parallel processing) in pattern-analysis problems have been mentioned. It may be mentioned here that neuro-fuzzy processing should continue to be a thrust research area at least for the next decade. Finally, the key features of genetic algorithms along with the possibility of successful use in this context were explained. Research is in progress at NASA’s Johnson Space center in making application of the aforesaid tolls and the recognition algorithm in space autonomous operations (e.g., camera tracking system and collision avoidance in Mars rover control [Lea et ai., 1989, 1990a,b,c, 1991, 1992) for supporting an unmanned mission. Various expert system shells based on fuzzy logic are now commercially available. Fuzzy logic chips developed by Togai and Watanabe at Bell Laboratories can be used in fuzzy-rule-based expert systems that do not require a high degree of precision. The fuzzy computer developed by Y amakawa of Kumamoto University has shown great promise in processing linguistic data at high speed and with remarkable robustness (Rogers and Hosiai, 1990; Proc. 2nd Congress ofthe Int. Fuzzy Systems Assoc., 1987). This may be an important step toward the development of a sixth-generation computer capable of processing common-sense knowledge. This capability is a prerequisite for solving many A1 problems, e.g., recognition of handwritten text and speech, machine translation, summarization, and image understanding that do not lend themselves to cost-effective solution within the bounds (limitations) of conventional technology. ACKNOWLEDGMENTS

The author acknowledges Mr. A. Ghosh, D. Bhandari, and D. P. Mandal for their assistance in preparing the manuscript.

292

SANKAR K. PAL

REFERENCES Ankenbrandt, C., Buckles, B., and Petry, F. (1990). Putt. Recog. Lett. 11, 285-293. Archer, N. P., and Wang, S. (1991). IEEE Transactions on Systems, Man, and Cybernetics 21, 735-742. Azimi Sadjadi, M. R., King, R. A., and Pal, S. K. (1982). IEE Proceedings, Pt.-F 129, 373. Belew, R. K., and Booker, L. B. (eds.) (1991). Proc. Fourth Int. Conf. on Genetic Algorithms. University of California Press, San Diego. Berenji, H. R. (1992). In?. J. of Approximate Reasoning 6, 267-292. Berenji, H. R., and Khedkar, P. (1992). IEEE Transactions on Neural Networks 3, 724-740. Bezdek, J . C. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York. Bezdek, J. C., and Anderson, 1. M. (1985). IEEE Trans. Syst., Man and Cyberns. 15,637-641. Bezdek, J. C., and Castelaz, P. F. (1977). IEEE Trans. Syst., Man and Cyberns. SMC-7, 87-92. Bezdek, J . C., and Pal, S. K. (eds.) (1992). Fuzzy Models for Pattern Recognition: Methods that Search for Structures in Data. IEEE Press, New York. Bezdek, J . C., Tsao, E. C. K., and Pal, N. R. (1992). Proc. First IEEE Int. Conf. on Fuzzy Systems. San Diego, pp. 1035-1046. Bhandari, D., and Pal, N. R. (1993). Information Sciences 67, 209-228. Bhandari, D., Pal, N. R., and Dutta Majumder, D. (1992). Pattern Recognition Letters 13, 851-867. Bhandari, D., Pal, S. K., and Kundu, M. K. (1993). Proc. IEEE Int. Con5 on Fuzzy Systems. California, 1408- I41 3. Bornholdt, S., and Graugenz, D. (1992). Neural Networks 5, 327-334. Burr, D. J. (1988). IEEE Transactions on Acoustic, Speech, and Signal Processing 36, 1162-1 168. Carpenter, C. A., Crossberg, S., and Rosen, D. B. (1991). Neural Networks 4. Davis, L. (1991). Hand Book of Genetic Algorithms. Van Nostrand and Reinhold, New York. Dave, R., and Bhaswan, K. (1991). Proc. NAFIPS’ 91, University of Missouri-Columbia, pp. 195-199. De Luca, A., and Termini, S. (1972). Inform and Control 20, 301-312. DePalma, G. F., and Yau, S. S. (1975). “Fractionally Fuzzy Grammars with Applications t o Pattern Recognition.” In Fuzzy Sets and Their Applications to Cognitive and Decision Processes (Zadeh, L. A. el al., eds.). Academic Press, London. Di Gesu, V., and Maccarone, M. C. (1986). Pattern Recognition 19, 63-72. Duda, R., and Hart, P. (1973). Pattern Classification and Scene Analysis. John Wiley & Sons, New York. Edelsbrunner, H., Kirkpatrick, D. G., and Seidel, R. (1983). IEEE Trans. Inform. Theory 29. 551. Fu, K . S . (1982). Syntactic Pattern Recognition and Applications. Academic Press, London. Gallant, S . 1. (1988). Communication of the Association for Computing Machinery 31, 152-169. Ghosh, A,, and Pal, S. K. (1992a). Putt. Recog. Lett. 11, 387-397. Ghosh, A., Pal, N. R., and Pal, S. K. (1991). Biological Cybernetics 66, 151-158. Ghosh, A., Pal, N. R., and Pal, S. K. (1992b). Int. J . Putt. Recog. and Artificial Intell. 6, 989-1008. Ghosh, A., Pal, N. R., and Pal, S. K . (1993). IEEE Transactions on Fuzzy Systems 1, 54-68.

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

293

Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, Massachusetts. Gonzalez, R. C., and Wintz, P. (1987). Digital Image Processing, 2nd ed. Addison-Wesley, Reading, Massuchusetts. Grzyrnala-Busse, J. W. (1988). J. Intelligent and Robotic Systs. 1, 3-16. Gupta, M. M. (1992). Proc. Second In/. Conj. on Fuzzy Logic and Neural Networks. Iizuka, Japan, 157-160. Gupta, M. M., Pedrycz, W., and Kiszka, J . (1989). Proc. Third IFSA Congress. Seattle, 258-261. Hayashi, Y., Czogala, E., and Buckley, J . J . (1992). Proc. First IEEE Int. Conj. on Fuzzy Systems. San Diego, USA, 197-202. Hirota, K. (1981). Fuzzy Sets and Systs, 5, 31-46. Huntsberger, T. L., and Ajjimerangsee, P. (1990). Int. J . General Sysfems 16, 357-372. Huntsberger, T. L., Rangarajan, C., and Jayaramamurthy, S. N. (1986). IEEE Trans. Comp. C-35, 145-156. Ishibuchi, H . , and Tanaka, H. (1990). Proc. Firs( In/. Conf. on Fuzzy Logic and Neural Networks. Iizuka, Japan, 179-182. Kandel, A, (1982). Fuzzy Techniques in Pattern Recognition. Wiley Interscience, New York. Kaufrnann, A. (1980). Introduction to the Theory oJFuzzy Subsets-Fundamental Theoretical Elements, Vol. 1 . Academic Press, New York. Keller, J . M., and Chen, Z. (1992). Proc. Second Int. Conf. on Fuzzy Logic and Neural Networks. Iizuka, Japan, 85-87. Keller, J . M . , and Hunt, D. J. (1985). IEEE Transactions an Paltern Analysis and Machine Intelligence 1,693-699. Keller, J. M., and Krishnapuram, R. (1992). IEEE Transactions on Neural Networks 3 , 761-769. Keller, 3 . M., and Tahani. H. (1991). Int. J . Approximate Reasoning 6 , 221-240. Keller, J. M., and Tahani, H. (1992). Information Sciences 62, 205-221. Keller, J . M., Yager, R. R., and Tahani, H. (1992). Fuzzy Sets and Systems 45, 1-12. Kios, K. J., and Liu, N. (1992). IEEE Transactions on Neural Nefworks 3, 280-291. Klir, G. J., and Folger, T. A. (1988). Fuzzy Sets, Uncertainty, and Information. Prentice Hall, Englewood Cliffs, New Jersey. Korvin, A. de, Kleyle, R., and Lea, R. (1990). Ini. J. Intell. Systs. 5, 293-306. Kosko, B. (1992). Neural Networks and Fuzzy Systems. Prentice-Hall, Englewood Cliffs, New Jersey. Krishnapurarn, R., and Lee, J . (1992a). Fuzzy Sets and Systems 46, 11-27. Krishnapuram, R., and Lee, J. (1992b). Neural Networks 5, 335-350. Kundu, M. K., and Pal, S. K. (1990). Part. Recog. Lett. 11, 811-829. Kwabwe, S. A,, Pal, S. K., and King, R. A . (1985). In/. J . Syst. Sci. 16, 403-413. Lea, R. N., Giarratano, J., Fritz, R. H., and Jani, Y. K. (1990a). Proc. 8th International Congress of Cybernetics and Systems. New York. Lea, R. N., and Jani, Y. (1992). In/. J. Approximate Reasoning 6 , 151-184. Lea, R. N., Jani, Y. K., and Berenji, H. (1990b). Proc. 5th IEEEInternationalSymposiumon Intelligent Control 2, 903. Lea, R. N., Murphy, M. G., and Walters, L. (1991). Proc. NAFIPS 91, University o f Missouri-Columbia, 154. Lea, R. N., Togai, M.. Teichrow, J., and Jani, Y. (1989). Proc. IFSA 89 Congress., 23-29. Lea, R. N., Walters, L., and Jani, Y. K. (1990~).Proc. 1st International Symposium on Measurement and Control in Robotics, D.3.1 , I . Lee, E . T., and Zadeh, L. A. (1969). Inform. Sci. 1, 421-434.

294

SANKAR K. PAL

Lee, S. C., and Lee, E. T. (1975). Mathematical Biosciences 23, 151-177. Leigh, A. B., and Pal. S. K. (1992). Proc. NAFIPS-92, 69-80. Levi, G., and Montanari, U. (1970). Inform. and Control 17, 62-91. Lim, Y. W., and Lee, S. U. (1990). Pattern Recognition 23, 935-952. Lippmann, R. P. (1989). IEEE Communications Magazine, 47-64. Machado, R. J., and Rocha, A. F. D. (1992). Proc. IEEE 1st Int. Conf. on Fuzzy Systems, FUZZ-IEEE’92. San Diego, 493-500. Mandal, D. P. (1992). “A Multivalued Approach for Uncertainty Management in Pattern Recognition Problems Using Fuzzy Sets.” Ph.D. Thesis, Indian Statistical Institute, Calcutta. Mandal, D.P., Murthy, C. A., and Pal, S. K. (1992a). IEEE Trans. System, Man & Cybern. SMC-22, 307-320. Mandal, D. P., Murthy, C. A,, and Pal, S. K. (1992b). Int. J. General Systems 20, 307-339. McCulloch, W. S., and Pitts, W. (1943). Bulletin ojMarhematica1 Biophysics 5. 115-133. Mitra, S., and Pal, S. K. (1994). IEE Trans. Syst.. Man and Cyberns. 24(2), in press. Mizumoto, M., and Tanaka, K. (1976). Inform. and Control 31, 312-340. Murthy, C. A. (1988). “On Consistent Estimation of Classes in R2 in the Context of Cluster Analysis.” Ph.D. Thesis, lndian Statistical Institute, Calcutta. Murthy, C. A., and Pal, S. K. (1990). Pal?. Recog. Lett. 11, 197-206. Murthy, C. A., and Pal, S. K. (1992). Inform. Sci. 60. 107-135. Murthy, C. A., Pal, S. K., and Dutta Majumder, D. (1985). Fuzzy Sets and Systs. 7, 23-38. Nafarieh, A., and Keller, J. (1991). I n f . J. Intell. Systs. 6, 295-312. Nasrabadi, N. M., Pal, S. K., and King, R. A. (1983). EIectronics Lett. 19, 63. Nath, A. K., and Lee, T. T. (1983). Fuzzy Sets and Systs. 11, 265. Pal, N. R., and Pal, S. K. (1989a). fEE Proc. 136, Pt. E , 284-295. Pal, N. R., and Pal, S. K. (1989b). Signal Processing 16, 97-108. Pal, N. R., and Pal, S. K. (1991a). IEEE Trans. Syst.. Man and Cyberns. SMC-21, 1260-1270. Pal, N. R., and Pal, S. K. (1991b). Int. J. Pall. Recog. and Artvicial Intell. 5, 459-483. Pal, N. R., and Pal, S. K. (1992a). Inform. Sci. 61, 211-231. Pal, N. R., and Pal, S. K. (1992b). fnjorm. Sci. 66, 119-137. Pal, S. K. (1982). IEEE Trans. Patt. Anal. and Machine Intell. PAMI-4, 204-208. Pal, S. K. (1986). Patt. Recog. Lett. 4, 51-56. Pal, S. K. (1989). Part. Recog. Lett. 10, 17-23. Pal, S. K. (1991). Int. J. Syst. Sci. 22, 511-549. Pal, S. K. (1992a). J. ojScient$ic and Industrial Research 51, 71-98. Pal, S. K. (1992b). Inform. Sci. 64, 165-179, Pal, S. K., and Bhattacharyya, A. (1990). Patt. Recog. Lett. 11, 443-452. Pal, S. K., and Chakraborty, B. (1986). IEEE Trans. Syst., Man and Cyberns. 16, 754-760. Pal, S. K., and Das Gupta, A. (1990). Proc. I n t . Conj. Fuzzy Logic & Neural Nefworks (IIZUKA’90). Iizuka, Japan, vol. 1. 299-302. Pal, S. K., and Das Gupta, A. (1992). Inform. Sci. 65. 65-97. Pal, S. K., and Dutta Majumder, D. (1986). Fuzzy Mathemafical Approach to Pattern Recognition. John Wiley and Sons, (Halsted Press), New York. Pal, S. K., and Ghosh, A. (1990). Patt. Recog. Lett. 11, 831-841. Pal, S. K., and Ghosh, A. (1992a). Inform. Sci. 62, 223-250. Pal, S. K., and Ghosh, A. (1992b). Fuzzy Sets and Sysls. 48, 23-40. Pal, S. K., and King, R. A. (1981a). Electronics Letters. 17, 302-304. Pal, S. K.,and King, R. A. (1981b). IEEE Trans. Syst.. Man and Cyberns. SMC-11,494-501. Pal, S. K., and King, R. A. (1983). IEEE Trans. Pattern Anal. Machine Intell. PAMI-5, 69-77,

FUZZY SET THEORETIC TOOLS FOR IMAGE ANALYSIS

295

Pal, S. K., King, R. A., and Hashim, A. A. (1983). IEEE Trans. Syst., Man and Cyberns. SMC-13, 94-100. Pal, S. K., and Mandal, D. P. (1992). Inform. Sci. 61, 135-161. Pal, S. K., and Mitra, S. (1990). Putt. Recog. Lett. 11, 525-535. Pal, S. K., and Mitra, S. (1992). IEEE Transactions on Neural Networks 3, 683-697. Pal, S. K., and Mitra, S. (1993). Information Sciences (accepted). Pal, S. K., and Pal, N. R. (1990). Proc. INDO-US Workshop on Spectrum Analysis in One and Two Dimensions. New Delhi, NBH Oxford Publishing Co., New Delhi, 285-300. Pal, S. K., and Rosenfeld, A. (1988). Patr. Recog. Lett. 7 , 77-86. Pal, S. K., and Rosenfeld, A. (1991). Putt. Recog. Lett. 10, 585-590. Pal, S. K., and Wang, L. (1991). Proc. IFSA’91 Congress. Brussels, Belgium, 167-170. Pal, S. K., and Wang, L. (1992). Fuzzy Sets and Systems 50, 15-34. Pao, Y. H. (1989). Adaptive Pattern Recognition aird Neural Networks. Addison-Wesley, Reading, Massachussets. Parsi, B. K., and Parsi, B. K. (1990). Biological Cybernetics 62, 415-423. Pathak, A,, and Pal, S. K. (1986a). Putt. Recog. Lett. 4, 63-69. Pathak, A,, and Pal, S. K. (1986b). IEEE Trans. Syst., Man and Cyberns. SMC-16, 657-667. Pathak, A,, Pal, S. K., and King, R. A. (1984). Putt. Recog. Lett. 2, 193. Pawlak, Z. (1982). Int. J. Inform. Comp. Sci. 11, 341-356. Pawlak, Z. (1985). Fuzzy Sets and Sysrs. 17, 99-102. Pedrycz, W. (1990). Partern Recognition 23, 121-146. Pedrycz, W. (1991). IEEE Transactions on Pattern Analysis and Machine Intelligence 13, 289-297. Pedrycz, W . (1992). IEEE Transactions on Neural Networks 3, 770-775. Peleg, S . , and Rosenfeld, A. (1981). IEEE Trans. Putt. Anal. Mach. Intell. P A M I - 3 , 208-210. Proc. of the Second Congress of the lnternational Fuzzy Systems Association (1987). Tokyo, Japan. Proc. Int. Joint Conf. on Neural Networks (1989). Washington DC, USA. Proc. Second Joint Technology Workshop on Neural Networks and Fuzzy Logic (1990a). NASA Conference Publication 10061, Johnson Space Center, Houston, Texas, USA, April 10-13. Proc. Int. Conf. Fuzzy Logic and Neural Networks (1990b). IIZUKA’90, Kyusu Institute of Technology, Iizuka, Fukuoka, Japan, July 22-24. Proc. First IEEE International Conference on Fuzzy Systems (1992a). San Diego, USA. Proc. Second International Conference on Fuzzy Logic and Neural Networks (1992b). lizuka, Japan. Rogers, M., and Hoshiai, Y. (1990). Newsweek, May 28, 46. Rosenfeld, A. (1984). Putt. Recog. Lett. 2, 31 1-317. Rosenfeld, A., and Kak, A. C. (1982). Digital Picture Processing, vol. 2. Academic Press, New York. Salari, E., and Siy, P. (1984). IEEE Trans. Syst.. Man and Cyberns. SMC-14, 524-528. Sanchez, E. (1990). Proceedings First Int. Conf. on Fuzzy Logicand NeuralNetworks. lizuka, Japan, 31-35. Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press, Princeton, New Jersey. Shannon, C.E. (1948). Bell. Syst. 7ech. Jour. 27, 379. Special issue on fuzzy logic and neural networks (1992). IEEE Transactions on Neural Networks 3. Special issue on neural networks (1992). Int. J . of Pattern Recognition and Artificial Intelligence 6 .

296

SANKAR K. PAL

Special issue on fuzzy logic and neural networks (1992). Int. J. Approximate Reasoning 6. Takagi, H. (1990). In Proc. Int. Conf. Fuzzy Logic and Neural Networks, IIZUKA’90, Kyusu Institute of Technology, Iizuka, Fukuoka, Japan, July 22-24, 13-26. Takagi, H., Suzuki, N., Koda, T., and Kojima, Y. (1992). IEEE Transactions on Neural Networks 3, 752-760. Takagi, H., and Hayashi, I. (1991). Int. J. OfApproximate Reasoning 5, 191-212. Thomason, M. G. (1973). Patfern Recognition 5, 383-390. Tou, J. T., and Gonzalez, R. C. (1974). Pattern Recognition Principles. Addison-Wesley, London. Tousant, G. T. (1980). Proc. 5th Int. Conf, Pattern Recognition. Miami Beach, Florida, 1324- 1347.

Turksen, I. B. (1986). Fuzzy Sets and Systs. 20, 191-210. Werbos, P. J. (1992). Int. J. of Approximate Reasoning 6, 185-219. Whitley, D., Starkweather, T., and Bogart, C. (1990). Parallel Computing 14, 347-361. Xie, W. X. (1990). Fuzzy Sets and Systs. 36, 157-165. Xie, W . X., and Bedrosian, S. D. (1984). IEEE Trans. Syst., Man and Cyberns. 14, 151. Yager, R. R. (1978). Fuzzy Sets and Systems 1, 87-95. Yager, R. R. (1981). Int. J. Comp. Inf. Sci. 10, 141. Yager, R. R. (1992). Fuzzy Sers and Systems 48, 53-64. Yamakawa, T., and Furukawa, M. (1992). Proc. First IEEE Int. Conf. on Fuzzy Systems. San Diego, USA, 75-82. Yamakawa, T., and Tomada, S. (1989). Proceedings Third ZFSA Congress, Seattle, 30-38. Yen, J. (1990). IEEE Trans. Syst., Man and Cyberns. SMC-20,559-570. Zadeh, L. A. (1965). Inform. Control 8, 338-353. Zadeh, L. A. (1977). Synthese 30, 407-428. Zadeh, L. A. (1984). IEEE Spectrum August, 26-32. Zadeh, L. A., Fu, K. S., Tanaka, K., and Shimura, M. (1975). Fuzzy Sets and Their Applications to Cognitive and Decision Processes. Academic Press, London. Zimmerman, H. J., and Zysno, P. (1983). Fuzzy Sets and Systems 10, 243-260.