- Email: [email protected]
Textural filters based on the texture spectrum
Pattern Recognition. Vol. 24. N o 12, pp. 1187 1195. 1991 Printed in Great Brttain
t11131-32113.'91 $3 110 + .(XI Pergamon Press pie 1991 Pattern Recognition Society
T E X T U R A L FILTERS BASED ON THE TEXTURE SPECTRUM DONG-CHEN HE and LI WANG Centre d'Applications et de Recherches en T616d6tection (CARTEL). Universit6 de Sherbrooke, Sherbrooke (Qu6bec), Canada J1K 2R1
(Received 6 February 1991; in revised form 26 April 1991; received fi~r publication 22 May 1991) Abstract~Conventional digital filtering techniques, based on classical Fourier analysis (that is, lowpass. high-pass and band-pass), are widely used in digital image processing. Unsatisfaction may be encountered when applying these filters to texture analysis of images, where one needs some specific spatial filters which are able to transform an image in the sense of texture rather than the spectral properties. Such textural filters can be designed in the texture spectrum domain and they are of interest for texture analysis. An example is given in this paper, and has been applied to four of Brodatz's natural images. The result shows a promising potential of the texture spectrum for designing textural filters. Texture analysis Texture filtering
Texture spectrum Pattern recognition
Texture enhancement
I. INTRODUCTION Digital filtering techniques constitute an important part of the set of image transformations and are widely used in image processing and pattern recognition. Their principal applications include edge detection, noise suppression, smoothing, recognition and enhancement of images. Over the past years. many digital filtering methods and techniques have been developed. ~1-~)They are often divided into two categories:linear and non-linear filters. Based on traditional Fourier analysis, the lowpass, high-pass and band-pass are the three basic classical linear filters. They can be combined to form a wide variety of more complicated filters. The origin of these filtering techniques can be found in classical analog signal processing, where filtering a signal means to modify its Fourier spectrum in such a way as to eliminate, or attenuate, some undesirable frequency components, and to transmit others unaltered. Unsatisfactory results may be encountered when applying these filters to texture analysis, because the classical Fourier analysis has its limitation in the characterization of texture. In texture analysis, one is interested in the relative intensity relations between the pixels in a small neighborhood, not in their absolute intensity values, nor in the spatial relationships of pixels in a large scale. It would be difficult to represent or describe efficiently spatial relationships of texture only using a few spatial frequency components of the Fourier spectrum. The complex spatial relationships could not be processed satisfactorily by only using the traditional low-pass, high-pass or band-pass filters. What one needs would be some kind of specific spatial filtering technique, termed here texturalfiltering. Such filters can comp-
Image processing
lement the traditional Fourier filtering techniques. and will be useful specially for texture analysis. Over the last few years, morphological filtering has been introduced and developed for image processing. Is'6) This filtering method, based on some basic operations of mathematical morphology (erosion/dilation, opening/closing and so on), 17~ is an example of non-linear image transformations, where the geometric features of an input image, such as its peaks and valleys, are locally modified using some structuring elements. Several applications may be found in the field of image processing and pattern recognitionJ 8'91 However, the method has mainly been used for binary images and is more suitable for shape representation and analysis. Recently, He and Wang have introduced a texture spectrum method for texture analysisJ ~°-131 In this new statistical approach, an image is characterized by its texture spectrum, making it possible to consider a digital filter from the point of view of texture analysis. The purpose of this paper is to demonstrate this possibility and to give an example of the design of a digital textural filter in the texture spectrum domain. 2. TEXTURE UNIT AND TEXTURE SPECTRUM
This section gives a brief review of the texture unit and texture spectrum, which have been introduced and described in detail in previous work/10-1-~
2.1. Texture unit Considering a digital image stored in a square raster form, each pixel is surrounded by eight neighboring pixels. The local texture information for a pixel can be extracted from a neighborhood of 3 × 3 1187
1188
D.-C. HE and L. WANG
a
b
h g
value of the central pixel and Vi is the intensity value of the neighboring pixel i, we define the corresponding texture unit as a set containing eight elements: TU = {El, E2 . . . . . Es}, where E/is determined by the formula:
c d
f
e
f O ifVi<-(Vo-A) Ei=
Fig. 1. Eight clockwise successive ordering ways of the eight elements of texture units: the first element E~ may take eight possible positions from a to h.
neighbourhood
1
if(Vo-A)
2
ifV i>(Vo+A) for i = 1,2 . . . . . 8,
Texture Unit
63
28
45
2
88
40
35
2
67
40
21
2
0
TU number 2 0
1
0
• TU = {2,0,2,0,0,1,2,2}
V = {40,63,28,45,35,21,40,67,88}
"
N~
=
6095
Fig. 2. Example of transforming a neighborhood to a texture unit with the texture unit number under the ordering way a and A = 0.
where the A represents a small positive value and the element Ei occupies the same position as the pixel i. As each element of TU has one of three possible values, the combination of all the eight elements results in 3 s = 6561 possible texture units in total.
2.2. Labeling texture units There is no unique way to label and order the 6561 different texture units. In our study, the texture unit n u m b e r (Nrv) is found by using the following formula: 8
',
IL,t-/
:'2!,i.
t.q?(i
55bl
Fig. 3. Example of a texture spectrum calculated from the image of Fig. 5(D).
pixeis, which represents the smallest complete unit (in the sense of having all eight directions surrounding the pixel). Given a neighborhood of 3 x 3 pixels, which will be denoted by a set containing nine elements: V = {V0, Vt . . . . . V8}, where V0 represents the intensity
Nrv = ~ Ei × 3 i-1, i=1
Nru E {0, 1, 2 . . . . . 6560},
where E; is the ith element of the texture unit set TU = {El, E2 . . . . . E8}. In addition, the eight elements may be ordered differently. If the eight elements are clockwise ordered, the first element may take eight possible positions from top-left (a) to middle-left (h). As shown in Fig. 1, the 6561 texture units can be labeled by the above formula under eight different ordering ways (from a to h). Figure 2 gives an example of
Tcxture filters
transforming an image neighborhood to a texture unit under the ordering method a and with A = 0. 2.3. Texture spectrum The previously defined set of 6561 texture units describes the local texture aspect of a given pixel, that is, the relative grey level relationships between the central pixel and its eight neighbors. Thus, the statistics on the occurrence frequency function of all the texture units over a whole image should reveal texture information of the image to be analyzed. We will term the occurrence frequency function of all the texture units the texture spectrum, with the abscissa indicating texture unit number Nrc and the ordinate representing its occurrence frequency. In practice, a real image is usually composed of two parts: texture elements and random noise or background. The greater the proportion of texture components to background, the better that the texture may be perceived by human vision. In the texture spectrum, an increase in pcrcentage of texture components in an image will result in a tendency to form a particular distribution of peaks. In addition, different textures are composed of particular texture units with different distribution in their texture spectra. In this way, the global texture of an image can be characterized by its texture spectrum. It should be noted that the labeling method chosen may affect the relative positions of texture units in the texture spectrum, but does not change their frequency values in the texture spectrum. Figure 3 shows an example of a texture spectrum obtained from the texture image of Fig. 5(D). 2.4. Texture characterization The texture spectrum approach has been evaluated in our previous studies 1~-13~ for texture characterization and texture classification using some of Brodatz's natural images. It was also used with success to distinguish different lithological units from a Synthetic Aperture Radar (SAR) image/t4) Texture spectrum can also be used to locate prominent texturc edges in digital images. (15~ These preliminary results allow us to make the assumption that different texturc images have correspondingly different spectra. In other words, the texture aspect of an image can be characterized by its texture spectrum. As discussed above, the local texture aspect of a given pixcl and its neighbors, that is the microtexture of an image, is described by the set of texture units, while the global texture characteristics of the whole image arc revealed by the corresponding texture spectrum obtained from a moving window. The choice of the size of the window depends on the texture structure of the original image to be analyzed. A small window can characterize fine textures while macrotextures should be qualified using a large window.lit, i~ These types of problems are often addressed in the texture classification of images.
1189
3. TEXTURALFILTERING Textural filtering means here to transform an image from the point of view of texture units and texture spectrum. In this section, we present an example, termed here a texture noise suppression filter. The filter to be considered is designed to remove spectral noise as well as the regional intensity background of an image. The latter may be called textural noise. Such a filter is relevant to texture analysis because a real image usually contains both spectral and textural components, one influencingthe other. The low-pass filter is usually applied to eliminate spectral noise but is unable to remove the regional background intensity variation. The texture noise suppression filter will remove both spectral and textural noise. The basic idea of such a filter is that the spectral or/and textural noise can be eliminated by some averaging operations over a large region, and that the texture aspect of the processed image may be preserved by keeping its texture spectrum unchanged. In other words, an image will be filtered by averaging operations in such a way that the output image has the same texture spectrum as that of the input image. In practice, this can be achieved by the following procedure: (1) calculate the texture spectrum for the image to be filtered. This operation gives the occurrence statistics of all the texture units of the whole image in the form of a spectrum F = f(Nrc), whcrc f(Nlr) represents the occurrence function of the texture unit numbered Nrt., N-co = O, 1, 2 . . . . . 6560: (2) carry out an averaging operation for all the neighborhoods of 3 × 3 pixels, which belong to the same texture unit. Thus we obtain an averaged standard sample of 3 × 3 pixels for each texture unit. which can be denoted by the set:
S(Nrt;) = {~'l.(Nrc), (Zl(Nrt:) . . . . . l?x(Nlc)}, where Nrc = 0, 1, 2 . . . . . 6560, is the texture unit number;
S(NrL,) represents the standard sample set for the texture unit numbered Ntc: (/,(Nrc), i = 0, 1, 2 . . . . . 8), denotes the ith element of the set S(N,rc), which is the mean intensity value of all the V,s within the image, which belong to the same texture unit. This averaging operation is performed over the whole image with respect to the texture unit number Nr~,, so that the spectral and textural noises are removed from S(Nrt,,); (3) transform the original image in such a manner that each pixel will be replaced by the averaged standard values with respect to Nrt,,. A given pixel is contained in nine neighborhoods or nine texture unit sets around it (Fig. 4). So, its final intensity value will be the average of the standard values of these nine elements, that is: 1 ~
V/= ~ ~, 1/,(Nrc,(i)). '~ 1 = 0
1190
D.-C. HE and L. WANG
I
9
I0
II
24
1
2
23
8
0
IO
II
12
I
2
3
8
0
4
II
12
13
2
3
14
0
4
15
24
1
23
8
~=JVTu(1) s(.)
b=Nrv(2) S(b) P~
s(~) P~ d =NTU(4) Vo
S(d)"
22:7
9
10
11
12
13
24
1
2
3
14
o
23
8
0
4
15
22
?
6
5
18
21
20
19
18
17
D.
I
2
8
0
7
6
2
3
14
0
4
15
6
5
16
0
4
15
6
5
16
19
18
17
8
0
4
7
6
5
20
19
18
=Nru(O)
,.
P,
it
s(,)
1,1 -- i x E , ' . o v,
/=,vTu(s) s(t)
V,
e=NTu(e) S(s) °
I;',
h=,VTU(r)
S(h)" V|
= NTu(S) S(i) o r i g i n a l grid
•
nine n e i g h b o r h o o d s
•
nine stsndard samvles n i n e e l e m e n t s used" for a v e r a g e calculation
Fig. 4. Nine neighborhoods to be considered and the nine standard values used for the average calculation. where I , ' / d e n o t e s the final intensity value for the pixel considered; Nrv(i) represents the texture unit number of the element i, i = 0, 1 . . . . . 8; ~"i(Nru(i)) is the ith element of the standard set S(Nrv(i)). This further averaging operation is always carried out with respect to the texture unit number, so that
the texture spectrum of the whole image remains unchanged on the whole. Figure 4 shows how to perform the 3rd step of the filtering procedure. The first column represents a grid of 5 × 5 pixels used to calculate the final value of the central pixel considered. These pixeis are labeled from 0 to 24. The second column of Fig. 4 illustrates the nine neighborhoods of 3 x 3 pixels to be considered. They are labeled from 0 to 8. For
T e x t u r e filters
1191
A'
A
B
>~, ~".~ ; ~ . ' 7 2 ~'~-. ": "F "7.~':~-- ."7 ~,'~<,~" ~'~."li~.'~ ~,,,~....,~. ~ , :~ . . . . . . . . ~__~..-~,,,'~_~4~. .
C
r~ ~ , a .• ~ -
~ ~ .. ~. ~
#
- --
rf~,
,...,,~, re
~
,r
I /
t4~'~-~r[i"
, .,
'
,'.
'
r,
¢
k , j . ' , ' ~"yz ,
;
''~ t
r,cr
, rJ
I
,
•
~,. ~,
. . . . . ,.~...-~ 7 . %• . ~ ". ~o * ? ". ~~. ~ . ~ -. ~ : ,•
'
0
l,l
I "~
;'~,
i
C
.',..,.'.~- .-==-.q-';,...~,~4_.E"..,~,-.'.,?,." -.,~.~, , %,.. ~"~..~;..,_z. • . ~:,.-,.,,~_.-:..:.~
#
- ¢,.
,
.¢; ,/~ t.,~
v.
-
, . s, -.. . . ,t '. . . ..,, - . . . - . ~ .
•
,?
... ~,?~ ~ ..." ,...
.-:. ,-.¢,
,.:
.
.
'.'~ ' ' ; " ~ i ' :'. v /
;:"'I~'.~"' t,
#~' "
I.'
,,'~,,:.~ I . ' . , . , . : , , . ,
'!t ~ " ' : ' '.
•
~A:~.~-
,,,',,- •
If ] ' ;
I,
: ¢, , , .'~"z ,
~
~'-..
"
/.
p cv.,
"~" ';.-.';.-
D
~ . , ~ - ~ - . ,
--'.'-:r- e'~.~'~;",:,,.":-.,-~:'L'..,:,, " -:.:'-.'.7".," ~£~7,~ "; "'; ~ r ~.','.'.",, ",-, .7 ,t 2 . . . ~ a. ~,;~,.~* ~ " ~" ~.~ .~ ~"~: "~" ~.':;->':,"5.','-~'~,J": ~;'~,'~ ":.';r.C.,:'..¢ ," ~,.
-
I
,'.
.
'
,
" ' "
I~
"
''
.
.
"
~
.~
~*I
~/-
t t"/r t (,',#~ l ..,4.:'~',. ~ :.C',',.• (t'. ,I "' ~,.:,- ':":,. ~ ".
Fig. 5. F o u r o f B r o d a t z ' s t e x t u r e i m a g e s
{A,
B,
D ) a n d the result of t e x t u r a l filtering ( A ' , B ' , ( " . D').
C,
~
1192
D.-C. HE and L. WANG
A-
B!
~'
"'1"'1"'1
C
D-~
cq
D~
'"1"'1
Fig. 6. Intensity histograms of the four texture images before filtering (the upper part) and after filtering (the lower part).
each neighborhood, the corresponding texture unit number Nrt~ can be easily obtained by the formula presented in Section 2.2. Given the texture unit number NTU and from the statistical result of the 2nd step of the procedure, the corresponding standard sample set S(NTu) = {V~I(NTv), Vl(Nr~,) . . . . . l?8(Nru)} can be identified for each neighborhood. The third column of Fig. 4 shows the nine standard sample sets, without intensity values. For each standard sample set, the element which overlaps the original center pixel (that is the pixel numbered 0), will be selected to contribute to the averaging operation of the nine elements. These nine elements are indicated in the third column of grids in Fig. 4. 4. APPLICATIONSAND DISCUSSION The previously described textural filtering technique of noise suppression has been applied to four natural texture images extracted from Brodatz's album, t16) These natural images have been digitized with the M.A.M.A. machine. ¢17) Each image is represented by 256 x 256 pixels. They are, respectively, the images of (A) pressed calf leather, (B) water, (C) pressed cork and (D) fur hide of unborn calf. Figure 5 shows the four original texture images
(the left part) and the filtered results (the right part). Figures 6 and 7 present, respectively, the corresponding histogram and texture spectrum for each texture image before and after the filtering operation. With the comparison of these figures, we note that: (1) texture spectra of the filtered images have the same appearance as those of their original images, providing that the proposed filtering operation preserves textural information in the original images; (2) by removing the spectral and textural noises from the original images, the filtering operation enhances subjectively the textural perception within the images, especially for images (B) and (D). These two images have a relatively large variation in background intensity. For image (B), its intensity increases from the top to the bottom; and for image (D), it changes from left to right. After filtering this variation is attenuated considerably. This improvement of textural perception in an image would be useful for the visual interpretation and analysis of digital images, as, for example, in the processing, classification and interpretation of remotely sensed landscape scenes. The distinction and separation of the textural aspect from the spectral intensity background variation are important and of benefit in texture image analysis, because a real scene to be analyzed is usu-
Texture filters
1193
| .j 0
|
,
v
,--IT
.
.
.
i
.
.
.
.
~
. . . .
w
w
w
Q
O
U=
-
|
l'i
D
|
-i
-I1
-I1
1194
D.-C. HE and L. WANO
ally composed of these two parts and one influences the other when processing a digital image. The filter presented here, as an example of textural filters, is intended specially for this attempt. The result obtained by applying the filter to four of Brodatz's natural images shows the success of such a filter. In this example, the regional intensity background variation and spectral noise are eliminated, and visual perception of texture is obviously enhanced, while the texture spectrum is the same for the input and output images. Such noise suppression and textural enhancement may be also useful for traditional methods of texture analysis, that is structural and statistical approaches/~s) The texture features derived from a co-occurrence matrix are often used in practice for characterization of the texture aspect. 119-221 It is known that these features are influenced by regional intensity background variation. The same situation can be encountered in other methods of texture feature extraction. In these cases, the textural filtering presented here may be applied before the procedure of conventional texture feature extraction. Thc quality of these features would be improved. The proposed textural filtering technique has been used for this purpose in a geological study using remotely sensed data. q'~ In that application, the discriminating performance of some of Haralick's features derived from the co-occurrence matrix has been improved for distinguishing several lithological units. 5. CONCLUDINGSUMMARY The texture spectrum method has been recently introduced for texture analysis. The main intention of this paper is to propose a digital filter particularly suited for texture analysis purposes. The key concept of this approach is to design the filters in the texture spectrum domain, as the latter characterizes the textural aspect of an image in the form of a spectrum. These textural filters modify the local properties of an image therefore being non-linear filtering techniques. They can complement the traditional digital filters for image processing, and are specially of interest for texture analysis. The preliminary results obtained from the applications of such filters to some of Brodatz's natural images and remotely sensed image data show a promising potential of the texture spectrum for designing textural filters. Certainly, the present work is only the beginning of this approach. Many further studies must be undertaken for designing various textural filters based on the texture spectrum, to meet the different requirements of texture analysis.
Acknowledgements--This research was supported by an NSERC (Natural Sciences and Engineering Research Council of Canada) operation grant (No. OGP0089833) and a FCAR (Fonds pour la Formation de Chercheurs et l'Aide h la Recherche du Ou6bcc, Canada) grant (No. 91NC0602) to D.-C. He.
REFERENCES
1. R. C. Gonzalez and P. Wintz, Digital Image Processing, 2nd Edn. Addison Wesley, Reading, Massachusetts (1987). 2. D. T. Kuan, A. A. Sawchuk, T. C. Strand and P. Chavel, Adaptive noise smoothing filter for images with signal dependent noise, IEEE Trans. Pattern Anal. Mach. lntell. PAMI-7, 165-177 (1985). 3. W. K. Pratt, Digital Image Processing. Wiley, New York (1978). 4. A. Rosenfeld and A. C. Kak, Digital Picture Processing, 2nd Edn, Vols 1 and II. Academic Press, New York (1982). 5. P. Maragos, Tutorial on advances in morphological image processing and analysis, Opt. Engng 26,623--632 (1987). 6. P. Maragos and R. W. Schafer, A unification of linear, median, order-statistics, and morphological filters under mathematical morphology, Proc. IEEE 1985 Int. Conf. Acoust. Speech Signal Process., Tampa, FL, pp. 34.8.1-34.8.4 (1985). 7. J. Serra, Image Analysis and Mathematical Morphology. Academic Press, New York (1982). 8. M. M. Skolnick, Application of morphological transformations to the analysis of two-dimensional electrophoretic gels of biological materials, Comput., Vision, Graphics Image Process. 35, 306 (1986). 9. P. Maragos and R. W. Schafer, Application of morphological filtering to image analysis and processing, Proc. IEEE 1986 Int. Conf. Acoust. Speech Signal Process., Tokyo, Japan (1986). 10. D. C. He and L. Wang, Texture features based on texture spectrum, Pattern Recognition 24, 391-399 (1991). 11. L. Wang and D. C. He, A new statistical approach for texture analysis, Photogrammetric Engng Remote Sensing 56, 61-66 (1990). 12. D. C. He and L. Wang, Texture unit, texture spectrum and texture analysis, IEEE Trans. Geosci. Remote Sensing 28, 509-512 (1990). 13. L. Wang and D. C. He, Texture classification using texture spectrum, Pattern Recognition 23, 905-910 (1990). 14. D. C. He and L. Wang, Recognition oflithological units in airborne SAR images using new texture features, Int. J. Remote Sensing 11, 2337-2344 (1990). 15. D. C. He and L. Wang, D~tection des contours de texture d'images num~riques, Int. J. Remote Sensing 12,651-657 (1991). 16. P. Brodatz, Texture--a Photographic Album for Artists and Designers. Reinhold, New York (1967). 17. J. Guibert, P. Charvin and P. Stoclet, M.A.M.A. Project, a new measuring machine in Paris, Proc. 78th Coll. Int. Astron. Union, Asiago, Italy, pp. 165--167 (1983). 18. R. M. Haralick, Statistical and structural approaches to texture, Proc. IEEE 67,786-804 (1979). 19. R. M. Haralick, K. Shanmugan and I. Dinstein, Textural features for images classification, IEEE Trans. Syst. Man Cybern. SMC-8, 610-621 (1973). 20. R. W. Conners, Toward a set of statistical features which measure visually perceivable qualities of texture, Proc. Pattern Recognition Image Process. Conf., pp. 382-390 (1979). 21. D. C. He, L. Wang and J. Guibert, Texture features extraction, Pattern Recognition Lett. 6,269-273 (1987). 22. D. C. He, L. Wang and J. Guibert, Texture discrimination based on an optimal utilization of texture features, Pattern Recognition 21, 141-146 (1988). 23. L. Wang, D. C. He and A. Fabbri, Texture filtering for SAR image processing, IEEE Trans. Geosci. Remote Sensing 28, 735-737 (1990).
Texture filters
About the AUfllor--DONG-CHEN HE graduated from the Petroleum Institute of East-China in 1982. Hc received a D.E.A. (Dipi6me d'l~tudes Approfondies) and his Ph.D. in automatics and signal processing from the Department of Astrophysics, Universit6 de Nice, France, in 1984 and 1988, respectively. From 1988 to 1989, he was with Horler Information Inc. in Ottawa, Canada. Since 1989. hc has been working at the "Centre d'Applications et de Recherches en T616d6tection (CARTEL) de l'Universit6 de Sherbrooke", Sherbrooke, Canada. He has been a Professor in the Department of Geography and Remote Sensing at the Universit6 de Sherbrooke since June 1991. His research activities ccnter around image processing, texture analysis, pattern recognition and remc~te sensing.
About the Author--Lt WANG graduated from the Polytechnic University of itefci, China in 1982. She received a D.E.A. (Dipl6me d'l~tudes Approfondies) and a Ph.D. from the Univcrsit6 de Nice, Fr~mcc. in 1983 and 1987, respectively. She received the qualification of "'El~vc Dipl6m6 de l'Ecolc Pratique des Hautes I~tudes", in Paris, France, in 1988. From 1988 to 1989, she was with the Canada Ccntrc for Remote Sensing, in Ottawa, Canada, as a post-doctoral visiting fellow of the Natural Sciences and Engineering Research Council (NSERC) of Canada. Since 1990, she has been working at the "Ccntrc d'Applications et de Recherches en T616d6tection (CARTEL) de l'Universit6 dc Sherbrooke". Sherbrooke, Canada. Her current research interests include remote sensing, image processing, texture analysis and integration of data for resource evaluation.
I195
t11131-32113.'91 $3 110 + .(XI Pergamon Press pie 1991 Pattern Recognition Society
T E X T U R A L FILTERS BASED ON THE TEXTURE SPECTRUM DONG-CHEN HE and LI WANG Centre d'Applications et de Recherches en T616d6tection (CARTEL). Universit6 de Sherbrooke, Sherbrooke (Qu6bec), Canada J1K 2R1
(Received 6 February 1991; in revised form 26 April 1991; received fi~r publication 22 May 1991) Abstract~Conventional digital filtering techniques, based on classical Fourier analysis (that is, lowpass. high-pass and band-pass), are widely used in digital image processing. Unsatisfaction may be encountered when applying these filters to texture analysis of images, where one needs some specific spatial filters which are able to transform an image in the sense of texture rather than the spectral properties. Such textural filters can be designed in the texture spectrum domain and they are of interest for texture analysis. An example is given in this paper, and has been applied to four of Brodatz's natural images. The result shows a promising potential of the texture spectrum for designing textural filters. Texture analysis Texture filtering
Texture spectrum Pattern recognition
Texture enhancement
I. INTRODUCTION Digital filtering techniques constitute an important part of the set of image transformations and are widely used in image processing and pattern recognition. Their principal applications include edge detection, noise suppression, smoothing, recognition and enhancement of images. Over the past years. many digital filtering methods and techniques have been developed. ~1-~)They are often divided into two categories:linear and non-linear filters. Based on traditional Fourier analysis, the lowpass, high-pass and band-pass are the three basic classical linear filters. They can be combined to form a wide variety of more complicated filters. The origin of these filtering techniques can be found in classical analog signal processing, where filtering a signal means to modify its Fourier spectrum in such a way as to eliminate, or attenuate, some undesirable frequency components, and to transmit others unaltered. Unsatisfactory results may be encountered when applying these filters to texture analysis, because the classical Fourier analysis has its limitation in the characterization of texture. In texture analysis, one is interested in the relative intensity relations between the pixels in a small neighborhood, not in their absolute intensity values, nor in the spatial relationships of pixels in a large scale. It would be difficult to represent or describe efficiently spatial relationships of texture only using a few spatial frequency components of the Fourier spectrum. The complex spatial relationships could not be processed satisfactorily by only using the traditional low-pass, high-pass or band-pass filters. What one needs would be some kind of specific spatial filtering technique, termed here texturalfiltering. Such filters can comp-
Image processing
lement the traditional Fourier filtering techniques. and will be useful specially for texture analysis. Over the last few years, morphological filtering has been introduced and developed for image processing. Is'6) This filtering method, based on some basic operations of mathematical morphology (erosion/dilation, opening/closing and so on), 17~ is an example of non-linear image transformations, where the geometric features of an input image, such as its peaks and valleys, are locally modified using some structuring elements. Several applications may be found in the field of image processing and pattern recognitionJ 8'91 However, the method has mainly been used for binary images and is more suitable for shape representation and analysis. Recently, He and Wang have introduced a texture spectrum method for texture analysisJ ~°-131 In this new statistical approach, an image is characterized by its texture spectrum, making it possible to consider a digital filter from the point of view of texture analysis. The purpose of this paper is to demonstrate this possibility and to give an example of the design of a digital textural filter in the texture spectrum domain. 2. TEXTURE UNIT AND TEXTURE SPECTRUM
This section gives a brief review of the texture unit and texture spectrum, which have been introduced and described in detail in previous work/10-1-~
2.1. Texture unit Considering a digital image stored in a square raster form, each pixel is surrounded by eight neighboring pixels. The local texture information for a pixel can be extracted from a neighborhood of 3 × 3 1187
1188
D.-C. HE and L. WANG
a
b
h g
value of the central pixel and Vi is the intensity value of the neighboring pixel i, we define the corresponding texture unit as a set containing eight elements: TU = {El, E2 . . . . . Es}, where E/is determined by the formula:
c d
f
e
f O ifVi<-(Vo-A) Ei=
Fig. 1. Eight clockwise successive ordering ways of the eight elements of texture units: the first element E~ may take eight possible positions from a to h.
neighbourhood
1
if(Vo-A)
2
ifV i>(Vo+A) for i = 1,2 . . . . . 8,
Texture Unit
63
28
45
2
88
40
35
2
67
40
21
2
0
TU number 2 0
1
0
• TU = {2,0,2,0,0,1,2,2}
V = {40,63,28,45,35,21,40,67,88}
"
N~
=
6095
Fig. 2. Example of transforming a neighborhood to a texture unit with the texture unit number under the ordering way a and A = 0.
where the A represents a small positive value and the element Ei occupies the same position as the pixel i. As each element of TU has one of three possible values, the combination of all the eight elements results in 3 s = 6561 possible texture units in total.
2.2. Labeling texture units There is no unique way to label and order the 6561 different texture units. In our study, the texture unit n u m b e r (Nrv) is found by using the following formula: 8
',
IL,t-/
:'2!,i.
t.q?(i
55bl
Fig. 3. Example of a texture spectrum calculated from the image of Fig. 5(D).
pixeis, which represents the smallest complete unit (in the sense of having all eight directions surrounding the pixel). Given a neighborhood of 3 x 3 pixels, which will be denoted by a set containing nine elements: V = {V0, Vt . . . . . V8}, where V0 represents the intensity
Nrv = ~ Ei × 3 i-1, i=1
Nru E {0, 1, 2 . . . . . 6560},
where E; is the ith element of the texture unit set TU = {El, E2 . . . . . E8}. In addition, the eight elements may be ordered differently. If the eight elements are clockwise ordered, the first element may take eight possible positions from top-left (a) to middle-left (h). As shown in Fig. 1, the 6561 texture units can be labeled by the above formula under eight different ordering ways (from a to h). Figure 2 gives an example of
Tcxture filters
transforming an image neighborhood to a texture unit under the ordering method a and with A = 0. 2.3. Texture spectrum The previously defined set of 6561 texture units describes the local texture aspect of a given pixel, that is, the relative grey level relationships between the central pixel and its eight neighbors. Thus, the statistics on the occurrence frequency function of all the texture units over a whole image should reveal texture information of the image to be analyzed. We will term the occurrence frequency function of all the texture units the texture spectrum, with the abscissa indicating texture unit number Nrc and the ordinate representing its occurrence frequency. In practice, a real image is usually composed of two parts: texture elements and random noise or background. The greater the proportion of texture components to background, the better that the texture may be perceived by human vision. In the texture spectrum, an increase in pcrcentage of texture components in an image will result in a tendency to form a particular distribution of peaks. In addition, different textures are composed of particular texture units with different distribution in their texture spectra. In this way, the global texture of an image can be characterized by its texture spectrum. It should be noted that the labeling method chosen may affect the relative positions of texture units in the texture spectrum, but does not change their frequency values in the texture spectrum. Figure 3 shows an example of a texture spectrum obtained from the texture image of Fig. 5(D). 2.4. Texture characterization The texture spectrum approach has been evaluated in our previous studies 1~-13~ for texture characterization and texture classification using some of Brodatz's natural images. It was also used with success to distinguish different lithological units from a Synthetic Aperture Radar (SAR) image/t4) Texture spectrum can also be used to locate prominent texturc edges in digital images. (15~ These preliminary results allow us to make the assumption that different texturc images have correspondingly different spectra. In other words, the texture aspect of an image can be characterized by its texture spectrum. As discussed above, the local texture aspect of a given pixcl and its neighbors, that is the microtexture of an image, is described by the set of texture units, while the global texture characteristics of the whole image arc revealed by the corresponding texture spectrum obtained from a moving window. The choice of the size of the window depends on the texture structure of the original image to be analyzed. A small window can characterize fine textures while macrotextures should be qualified using a large window.lit, i~ These types of problems are often addressed in the texture classification of images.
1189
3. TEXTURALFILTERING Textural filtering means here to transform an image from the point of view of texture units and texture spectrum. In this section, we present an example, termed here a texture noise suppression filter. The filter to be considered is designed to remove spectral noise as well as the regional intensity background of an image. The latter may be called textural noise. Such a filter is relevant to texture analysis because a real image usually contains both spectral and textural components, one influencingthe other. The low-pass filter is usually applied to eliminate spectral noise but is unable to remove the regional background intensity variation. The texture noise suppression filter will remove both spectral and textural noise. The basic idea of such a filter is that the spectral or/and textural noise can be eliminated by some averaging operations over a large region, and that the texture aspect of the processed image may be preserved by keeping its texture spectrum unchanged. In other words, an image will be filtered by averaging operations in such a way that the output image has the same texture spectrum as that of the input image. In practice, this can be achieved by the following procedure: (1) calculate the texture spectrum for the image to be filtered. This operation gives the occurrence statistics of all the texture units of the whole image in the form of a spectrum F = f(Nrc), whcrc f(Nlr) represents the occurrence function of the texture unit numbered Nrt., N-co = O, 1, 2 . . . . . 6560: (2) carry out an averaging operation for all the neighborhoods of 3 × 3 pixels, which belong to the same texture unit. Thus we obtain an averaged standard sample of 3 × 3 pixels for each texture unit. which can be denoted by the set:
S(Nrt;) = {~'l.(Nrc), (Zl(Nrt:) . . . . . l?x(Nlc)}, where Nrc = 0, 1, 2 . . . . . 6560, is the texture unit number;
S(NrL,) represents the standard sample set for the texture unit numbered Ntc: (/,(Nrc), i = 0, 1, 2 . . . . . 8), denotes the ith element of the set S(N,rc), which is the mean intensity value of all the V,s within the image, which belong to the same texture unit. This averaging operation is performed over the whole image with respect to the texture unit number Nr~,, so that the spectral and textural noises are removed from S(Nrt,,); (3) transform the original image in such a manner that each pixel will be replaced by the averaged standard values with respect to Nrt,,. A given pixel is contained in nine neighborhoods or nine texture unit sets around it (Fig. 4). So, its final intensity value will be the average of the standard values of these nine elements, that is: 1 ~
V/= ~ ~, 1/,(Nrc,(i)). '~ 1 = 0
1190
D.-C. HE and L. WANG
I
9
I0
II
24
1
2
23
8
0
IO
II
12
I
2
3
8
0
4
II
12
13
2
3
14
0
4
15
24
1
23
8
~=JVTu(1) s(.)
b=Nrv(2) S(b) P~
s(~) P~ d =NTU(4) Vo
S(d)"
22:7
9
10
11
12
13
24
1
2
3
14
o
23
8
0
4
15
22
?
6
5
18
21
20
19
18
17
D.
I
2
8
0
7
6
2
3
14
0
4
15
6
5
16
0
4
15
6
5
16
19
18
17
8
0
4
7
6
5
20
19
18
=Nru(O)
,.
P,
it
s(,)
1,1 -- i x E , ' . o v,
/=,vTu(s) s(t)
V,
e=NTu(e) S(s) °
I;',
h=,VTU(r)
S(h)" V|
= NTu(S) S(i) o r i g i n a l grid
•
nine n e i g h b o r h o o d s
•
nine stsndard samvles n i n e e l e m e n t s used" for a v e r a g e calculation
Fig. 4. Nine neighborhoods to be considered and the nine standard values used for the average calculation. where I , ' / d e n o t e s the final intensity value for the pixel considered; Nrv(i) represents the texture unit number of the element i, i = 0, 1 . . . . . 8; ~"i(Nru(i)) is the ith element of the standard set S(Nrv(i)). This further averaging operation is always carried out with respect to the texture unit number, so that
the texture spectrum of the whole image remains unchanged on the whole. Figure 4 shows how to perform the 3rd step of the filtering procedure. The first column represents a grid of 5 × 5 pixels used to calculate the final value of the central pixel considered. These pixeis are labeled from 0 to 24. The second column of Fig. 4 illustrates the nine neighborhoods of 3 x 3 pixels to be considered. They are labeled from 0 to 8. For
T e x t u r e filters
1191
A'
A
B
>~, ~".~ ; ~ . ' 7 2 ~'~-. ": "F "7.~':~-- ."7 ~,'~<,~" ~'~."li~.'~ ~,,,~....,~. ~ , :~ . . . . . . . . ~__~..-~,,,'~_~4~. .
C
r~ ~ , a .• ~ -
~ ~ .. ~. ~
#
- --
rf~,
,...,,~, re
~
,r
I /
t4~'~-~r[i"
, .,
'
,'.
'
r,
¢
k , j . ' , ' ~"yz ,
;
''~ t
r,cr
, rJ
I
,
•
~,. ~,
. . . . . ,.~...-~ 7 . %• . ~ ". ~o * ? ". ~~. ~ . ~ -. ~ : ,•
'
0
l,l
I "~
;'~,
i
C
.',..,.'.~- .-==-.q-';,...~,~4_.E"..,~,-.'.,?,." -.,~.~, , %,.. ~"~..~;..,_z. • . ~:,.-,.,,~_.-:..:.~
#
- ¢,.
,
.¢; ,/~ t.,~
v.
-
, . s, -.. . . ,t '. . . ..,, - . . . - . ~ .
•
,?
... ~,?~ ~ ..." ,...
.-:. ,-.¢,
,.:
.
.
'.'~ ' ' ; " ~ i ' :'. v /
;:"'I~'.~"' t,
#~' "
I.'
,,'~,,:.~ I . ' . , . , . : , , . ,
'!t ~ " ' : ' '.
•
~A:~.~-
,,,',,- •
If ] ' ;
I,
: ¢, , , .'~"z ,
~
~'-..
"
/.
p cv.,
"~" ';.-.';.-
D
~ . , ~ - ~ - . ,
--'.'-:r- e'~.~'~;",:,,.":-.,-~:'L'..,:,, " -:.:'-.'.7".," ~£~7,~ "; "'; ~ r ~.','.'.",, ",-, .7 ,t 2 . . . ~ a. ~,;~,.~* ~ " ~" ~.~ .~ ~"~: "~" ~.':;->':,"5.','-~'~,J": ~;'~,'~ ":.';r.C.,:'..¢ ," ~,.
-
I
,'.
.
'
,
" ' "
I~
"
''
.
.
"
~
.~
~*I
~/-
t t"/r t (,',#~ l ..,4.:'~',. ~ :.C',',.• (t'. ,I "' ~,.:,- ':":,. ~ ".
Fig. 5. F o u r o f B r o d a t z ' s t e x t u r e i m a g e s
{A,
B,
D ) a n d the result of t e x t u r a l filtering ( A ' , B ' , ( " . D').
C,
~
1192
D.-C. HE and L. WANG
A-
B!
~'
"'1"'1"'1
C
D-~
cq
D~
'"1"'1
Fig. 6. Intensity histograms of the four texture images before filtering (the upper part) and after filtering (the lower part).
each neighborhood, the corresponding texture unit number Nrt~ can be easily obtained by the formula presented in Section 2.2. Given the texture unit number NTU and from the statistical result of the 2nd step of the procedure, the corresponding standard sample set S(NTu) = {V~I(NTv), Vl(Nr~,) . . . . . l?8(Nru)} can be identified for each neighborhood. The third column of Fig. 4 shows the nine standard sample sets, without intensity values. For each standard sample set, the element which overlaps the original center pixel (that is the pixel numbered 0), will be selected to contribute to the averaging operation of the nine elements. These nine elements are indicated in the third column of grids in Fig. 4. 4. APPLICATIONSAND DISCUSSION The previously described textural filtering technique of noise suppression has been applied to four natural texture images extracted from Brodatz's album, t16) These natural images have been digitized with the M.A.M.A. machine. ¢17) Each image is represented by 256 x 256 pixels. They are, respectively, the images of (A) pressed calf leather, (B) water, (C) pressed cork and (D) fur hide of unborn calf. Figure 5 shows the four original texture images
(the left part) and the filtered results (the right part). Figures 6 and 7 present, respectively, the corresponding histogram and texture spectrum for each texture image before and after the filtering operation. With the comparison of these figures, we note that: (1) texture spectra of the filtered images have the same appearance as those of their original images, providing that the proposed filtering operation preserves textural information in the original images; (2) by removing the spectral and textural noises from the original images, the filtering operation enhances subjectively the textural perception within the images, especially for images (B) and (D). These two images have a relatively large variation in background intensity. For image (B), its intensity increases from the top to the bottom; and for image (D), it changes from left to right. After filtering this variation is attenuated considerably. This improvement of textural perception in an image would be useful for the visual interpretation and analysis of digital images, as, for example, in the processing, classification and interpretation of remotely sensed landscape scenes. The distinction and separation of the textural aspect from the spectral intensity background variation are important and of benefit in texture image analysis, because a real scene to be analyzed is usu-
Texture filters
1193
| .j 0
|
,
v
,--IT
.
.
.
i
.
.
.
.
~
. . . .
w
w
w
Q
O
U=
-
|
l'i
D
|
-i
-I1
-I1
1194
D.-C. HE and L. WANO
ally composed of these two parts and one influences the other when processing a digital image. The filter presented here, as an example of textural filters, is intended specially for this attempt. The result obtained by applying the filter to four of Brodatz's natural images shows the success of such a filter. In this example, the regional intensity background variation and spectral noise are eliminated, and visual perception of texture is obviously enhanced, while the texture spectrum is the same for the input and output images. Such noise suppression and textural enhancement may be also useful for traditional methods of texture analysis, that is structural and statistical approaches/~s) The texture features derived from a co-occurrence matrix are often used in practice for characterization of the texture aspect. 119-221 It is known that these features are influenced by regional intensity background variation. The same situation can be encountered in other methods of texture feature extraction. In these cases, the textural filtering presented here may be applied before the procedure of conventional texture feature extraction. Thc quality of these features would be improved. The proposed textural filtering technique has been used for this purpose in a geological study using remotely sensed data. q'~ In that application, the discriminating performance of some of Haralick's features derived from the co-occurrence matrix has been improved for distinguishing several lithological units. 5. CONCLUDINGSUMMARY The texture spectrum method has been recently introduced for texture analysis. The main intention of this paper is to propose a digital filter particularly suited for texture analysis purposes. The key concept of this approach is to design the filters in the texture spectrum domain, as the latter characterizes the textural aspect of an image in the form of a spectrum. These textural filters modify the local properties of an image therefore being non-linear filtering techniques. They can complement the traditional digital filters for image processing, and are specially of interest for texture analysis. The preliminary results obtained from the applications of such filters to some of Brodatz's natural images and remotely sensed image data show a promising potential of the texture spectrum for designing textural filters. Certainly, the present work is only the beginning of this approach. Many further studies must be undertaken for designing various textural filters based on the texture spectrum, to meet the different requirements of texture analysis.
Acknowledgements--This research was supported by an NSERC (Natural Sciences and Engineering Research Council of Canada) operation grant (No. OGP0089833) and a FCAR (Fonds pour la Formation de Chercheurs et l'Aide h la Recherche du Ou6bcc, Canada) grant (No. 91NC0602) to D.-C. He.
REFERENCES
1. R. C. Gonzalez and P. Wintz, Digital Image Processing, 2nd Edn. Addison Wesley, Reading, Massachusetts (1987). 2. D. T. Kuan, A. A. Sawchuk, T. C. Strand and P. Chavel, Adaptive noise smoothing filter for images with signal dependent noise, IEEE Trans. Pattern Anal. Mach. lntell. PAMI-7, 165-177 (1985). 3. W. K. Pratt, Digital Image Processing. Wiley, New York (1978). 4. A. Rosenfeld and A. C. Kak, Digital Picture Processing, 2nd Edn, Vols 1 and II. Academic Press, New York (1982). 5. P. Maragos, Tutorial on advances in morphological image processing and analysis, Opt. Engng 26,623--632 (1987). 6. P. Maragos and R. W. Schafer, A unification of linear, median, order-statistics, and morphological filters under mathematical morphology, Proc. IEEE 1985 Int. Conf. Acoust. Speech Signal Process., Tampa, FL, pp. 34.8.1-34.8.4 (1985). 7. J. Serra, Image Analysis and Mathematical Morphology. Academic Press, New York (1982). 8. M. M. Skolnick, Application of morphological transformations to the analysis of two-dimensional electrophoretic gels of biological materials, Comput., Vision, Graphics Image Process. 35, 306 (1986). 9. P. Maragos and R. W. Schafer, Application of morphological filtering to image analysis and processing, Proc. IEEE 1986 Int. Conf. Acoust. Speech Signal Process., Tokyo, Japan (1986). 10. D. C. He and L. Wang, Texture features based on texture spectrum, Pattern Recognition 24, 391-399 (1991). 11. L. Wang and D. C. He, A new statistical approach for texture analysis, Photogrammetric Engng Remote Sensing 56, 61-66 (1990). 12. D. C. He and L. Wang, Texture unit, texture spectrum and texture analysis, IEEE Trans. Geosci. Remote Sensing 28, 509-512 (1990). 13. L. Wang and D. C. He, Texture classification using texture spectrum, Pattern Recognition 23, 905-910 (1990). 14. D. C. He and L. Wang, Recognition oflithological units in airborne SAR images using new texture features, Int. J. Remote Sensing 11, 2337-2344 (1990). 15. D. C. He and L. Wang, D~tection des contours de texture d'images num~riques, Int. J. Remote Sensing 12,651-657 (1991). 16. P. Brodatz, Texture--a Photographic Album for Artists and Designers. Reinhold, New York (1967). 17. J. Guibert, P. Charvin and P. Stoclet, M.A.M.A. Project, a new measuring machine in Paris, Proc. 78th Coll. Int. Astron. Union, Asiago, Italy, pp. 165--167 (1983). 18. R. M. Haralick, Statistical and structural approaches to texture, Proc. IEEE 67,786-804 (1979). 19. R. M. Haralick, K. Shanmugan and I. Dinstein, Textural features for images classification, IEEE Trans. Syst. Man Cybern. SMC-8, 610-621 (1973). 20. R. W. Conners, Toward a set of statistical features which measure visually perceivable qualities of texture, Proc. Pattern Recognition Image Process. Conf., pp. 382-390 (1979). 21. D. C. He, L. Wang and J. Guibert, Texture features extraction, Pattern Recognition Lett. 6,269-273 (1987). 22. D. C. He, L. Wang and J. Guibert, Texture discrimination based on an optimal utilization of texture features, Pattern Recognition 21, 141-146 (1988). 23. L. Wang, D. C. He and A. Fabbri, Texture filtering for SAR image processing, IEEE Trans. Geosci. Remote Sensing 28, 735-737 (1990).
Texture filters
About the AUfllor--DONG-CHEN HE graduated from the Petroleum Institute of East-China in 1982. Hc received a D.E.A. (Dipi6me d'l~tudes Approfondies) and his Ph.D. in automatics and signal processing from the Department of Astrophysics, Universit6 de Nice, France, in 1984 and 1988, respectively. From 1988 to 1989, he was with Horler Information Inc. in Ottawa, Canada. Since 1989. hc has been working at the "Centre d'Applications et de Recherches en T616d6tection (CARTEL) de l'Universit6 de Sherbrooke", Sherbrooke, Canada. He has been a Professor in the Department of Geography and Remote Sensing at the Universit6 de Sherbrooke since June 1991. His research activities ccnter around image processing, texture analysis, pattern recognition and remc~te sensing.
About the Author--Lt WANG graduated from the Polytechnic University of itefci, China in 1982. She received a D.E.A. (Dipl6me d'l~tudes Approfondies) and a Ph.D. from the Univcrsit6 de Nice, Fr~mcc. in 1983 and 1987, respectively. She received the qualification of "'El~vc Dipl6m6 de l'Ecolc Pratique des Hautes I~tudes", in Paris, France, in 1988. From 1988 to 1989, she was with the Canada Ccntrc for Remote Sensing, in Ottawa, Canada, as a post-doctoral visiting fellow of the Natural Sciences and Engineering Research Council (NSERC) of Canada. Since 1990, she has been working at the "Ccntrc d'Applications et de Recherches en T616d6tection (CARTEL) de l'Universit6 dc Sherbrooke". Sherbrooke, Canada. Her current research interests include remote sensing, image processing, texture analysis and integration of data for resource evaluation.
I195