- Email: [email protected]
Wavelet transform and adaptive neuro-fuzzy inference system for color texture classification
Available online at www.sciencedirect.com
Expert Systems with Applications Expert Systems with Applications 34 (2008) 2120–2128 www.elsevier.com/locate/eswa
Wavelet transform and adaptive neuro-fuzzy inference system for color texture classification Abdulkadir Sengur Firat University, Department of Electronics and Computer Science, 23119 Elazig, Turkey
Abstract The wavelet domain features have been intensively used for texture classification and texture segmentation with encouraging results. More of the proposed multi resolution texture analysis methods are quite successful, but all the applications of the texture analysis so far are limited to gray scale images. This paper investigates the usage of Wavelet transform (WT) and Adaptive neuro-fuzzy inference system (ANFIS) for color texture classification problem. The proposed scheme composed of a wavelet domain feature extractor and an ANFIS classifier. Both entropy and energy features are used on wavelet domain. Different color spaces are considered in the experimental studies. The performed experimental studies show the effectiveness of the wavelet transform and ANFIS structure for color texture classification problem. The overall success rate is over 96%. 2007 Elsevier Ltd. All rights reserved. Keywords: Wavelet decomposition; ANFIS; Texture classification; Feature extraction; Entropy; Energy correlation
1. Introduction Texture analysis plays an important role in many image processing tasks, ranging from remote sensing to medical image processing, computer vision applications and natural scenes. A number of texture analysis methods have been proposed in the past decades (e.g., Sklansky, 1978) but most of them are used for gray scale images which represent the amount of visible light at the pixel’s position while ignoring the color information. The performance of such methods can be improved by adding the color information because besides texture, color is the most important properties especially when dealing with real world images (Van de Wouwer, Scheunders, Livens, & Van Dyck, 1999). Texture can be defined as a local statistical pattern of texture primitives in observer’s domain of interest. Texture classification aims to assign texture labels to unknown textures, according to training samples and classification rules. Two major issues are critical for texture classification: the texture classification algorithms and texture feature extraction. The main aim of texture classification is to find a best E-mail address: ksengur@firat.edu.tr 0957-4174/$ - see front matter 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.02.032
matched category for a given texture among existing textures. Texture has been analyzed extensively and many texture recognition schemes have been proposed (Haralick, Shanmugan, & Dinstein, 1973; Reed & du Buf, 1993; Van de Wouwer et al., 1999). The important property all have in common is that they constitute an appropriate model for relationship between the adjacent pixels of a neighborhood. A number of researchers have proposed algorithms for texture analysis. But all the applications of the texture analysis so far are limited to gray scale images (Sengur, Turkoglu, & Ince, 2007). But recently both color and texture information is used by several researchers (Panjwani & Healey, 1995; Van de Wouwer et al., 1999). A method has been proposed by Caelli and Reye (1993). They extract features from three spectral channels by using three multiscale isotropic filters. Van de Wouwer et al. (1999), proposed wavelet energy-correlation signatures and derive the transformation of these signatures upon linear color space transformation. The wavelet domain based co-occurrence matrix method and the second order statistical features were used for color texture classification in the study of Arivazhagan, Ganesan, and Angayarkanni
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
(XXXX). A set of features are derived and color texture classification is done for different combination of the features and for different color models. Karkanis, Iakovidis, Maroulis, Karras, and Tzivras (2003), proposed a new approach for detection of tumors in colonoscopic video. The proposed algorithm is based on a color feature extraction scheme to represent the different regions in the frame sequence. The scheme is built on the wavelet transform. The features called as color wavelet covariance are based on the covariance of second order textural measurement. A linear discriminant analysis is used for classification of the image regions. Panjwani and Healey proposed an unsupervised algorithm which uses Markov random field models for color textures classification (Panjwani & Healey, 1995). These models characterized a texture in terms of spatial interaction within each color plane and interaction between different color planes. Crouse et al. has proposed a framework for statistical signal modeling based on the wavelet domain hidden Markov tree (Crouse, Nowak, & Baraniuk, 1998). The algorithm provided an efficient approach to modeling of wavelet coefficients that are often found in real world images. Xu et al., have shown that the wavelet coefficients have certain inter-dependences between color planes (Qing, Jie, & Siyi, 2005). They used wavelet domain hidden Markov model for color texture analysis. The proposed approach is used for modeling the dependences between color planes as well as the interactions across scales. The wavelet coefficients at the same location scale and sub-band, but different color planes are grouped into one vector and a multivariate Gaussian mixture model is employed for approximating the marginal distribution of the wavelet coefficient vectors in one scale. Despite these examples, there is not enough study for color texture analysis as was noted by Van de Wouwer et al. (1999). Multi-resolution representations give rise to an interesting class of texture analysis. Especially wavelets provide a convenient way to obtain a multi-resolution aspect to the texture images (Chen & Chen, 1999). In this paper, a color texture image classification scheme is proposed which uses Wavelet transform and ANFIS classifier. Wavelet entropies and wavelet energies of each color plane at different scales are used for forming the feature vector of each color texture. The experimental studies show the effectiveness of the proposed classification system. The experimental results were also compared with the results of wavelet energy correlation signatures. The influence of the choice of color space representation on classification performance is also investigated. The organization of this paper is as follows, in Section 2; it is given the theory for wavelet transform, feature extraction for color texture images, ANFIS and the extraction of the energy correlation signatures. Several brief definitions can be seen at this section. In Section 3; the methodology and the implementation of the proposed process is given. In Section 4; experimental study is introduced and the classification results are shown. In Section 5; we finally conclude the study.
2121
2. Preliminaries In this section, the theoretical foundations for the expert system used in the presented study are given in the following subsections. 2.1. Wavelet transform Wavelet transforms are finding inverse use in fields as diverse as telecommunications and biology. Because of their suitability for analyzing non-stationary signals, they have become a powerful alternative to Fourier methods in many medical applications, where such signals abound (Daubechies, 1988). The main advantages of wavelets is that they have a varying window size, being wide for slow frequencies and narrow for the fast ones, thus leading to an optimal time-frequency resolution in all the frequency ranges. Furthermore, owing to the fact that windows are adapted to the transients of each scale, wavelets lack the requirement of stationary. The (continuous) wavelet transform of a 1D signal f(x) is defined as: Z þ1 ðW w f Þða; bÞ ¼ hf ; wða; bÞi ¼ f ðxÞwða;bÞ ðxÞ dx 1 ð1Þ xb 1=2 wða;bÞ ¼ a w a where a is the scaling factor, b is the translation parameter related to the location of the window, and w*(x) is the transforming function. The latter is also called the mother wavelet, which is the prototype for generating the other window functions. The extension to the 2D case is usually performed by using a product of 1D filters. In practice, the transform is computed by applying a separable filter bank to the image: Ln ðbÞ ¼ ½H x ½H y Ln1 #2;1 #1;2 ðbÞ
ð2Þ
Dn1 ðbÞ ¼ ½H x ½Gy Ln1 #2;1 #1;2 ðbÞ
ð3Þ
Dn2 ðbÞ ¼ ½Gx ½H y Ln1 #2;1 #1;2 ðbÞ
ð4Þ
Dn3 ðbÞ ¼ ½Gx ½Gy Ln1 #2;1 #1;2 ðbÞ
ð5Þ
where b 2 R2, * denotes the convolution operator, #2,1 (#1,2) sub-sampling along the rows (columns) and L0 = I(x) is the original image. H and G are low and band pass filters, respectively. Ln is obtained by low pass filtering so it is called low resolution image at scale n. The Dni are obtained by band pass filtering in a specific direction. So these parameters contain directional detail information at scale n. The original image is thus represented by a set of sub images at several scales. 2.2. Architecture of adaptive-network-based fuzzy inference system An adaptive network, as its name implies, is a network structure consisting of nodes and directional links through
2122
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
which the nodes are connected. Moreover, parts or all of the nodes are adaptive, which means each output of these nodes depends on the parameters pertaining to this node and the learning rule specifies how these parameters should be changed to minimize a prescribe error measure. The ANFIS is a fuzzy Sugeno model put in the framework of adaptive systems to facilitate learning and adaptation (Guler & Ubeyli, 2005; Jang, 1992). Such framework makes the ANFIS modelling more systematic and less reliant on expert knowledge. To present the ANFIS architecture, two fuzzy if-then rules based on a first order Sugeno model are considered:
where ai, bi and ci are the parameters of the membership function, governing the bell shaped functions accordingly. Layer 2: The nodes in this layer are fixed. These are labelled M to indicate that play the role of a simple multiplier. The outputs of these nodes are given by: O2i ¼ wi ¼ lAi ðxÞlBi ðyÞ i ¼ 1; 2
which are the so-called firing strengths of the rules. Layer 3: In the third layer, the nodes are also fixed nodes. They are labelled with N, indicating that they play a normalization role to the firing strengths from the previous layer. The outputs of this layer can be represented as: wi i ¼ 1; 2 ð10Þ O3i ¼ wi ¼ w1 þ w2
Rule 1: If (x is A1) and (y is B1) then (f1 = p1x + q1y + r1) Rule 2: If (x is A2) and (y is B2) then (f2 = p2x + q2y + r2) where x and y are the inputs, Ai and Bi are the fuzzy sets, fi are the outputs within the fuzzy region specified by the fuzzy rule, pi, qi and ri are the design parameters that are determined during the training process. The ANFIS architecture to implement these two rules is shown in Fig. 1, in which a circle indicates a fixed node, whereas a square indicates an adaptive node.
which are the so-called normalized firing strengths. Layer 4: In the fourth layer, the nodes are adaptive nodes. The output of each node in this layer is simply the product of the normalized firing strength and a first order polynomial (for a first order Sugeno model). Thus, the outputs of this layer are given by:
Layer 1: In the first layer, all the nodes are adaptive nodes. i is the degree of the membership of the input to the fuzzy membership function represented by the node: O1i ¼ lAi ðxÞ; O1i
i ¼ 1; 2
¼ lBi2 ðyÞ;
O4i ¼ wi fi ¼ wi ðpi x þ qi y þ ri Þ i ¼ 1; 2
ð7Þ
where lAi ðxÞ; lBi 2 ðyÞ can adopt any fuzzy membership function. For example, if the bell shaped membership function is employed; lAi ðxÞ is given by: lAi ðxÞ ¼ 1þ
1 bi 2
It can be observed that there are two adaptive layers in this ANFIS architecture, namely the first layer and the fourth layer. In the first layer, there are three modifiable parameters {ai, bi, ci}, which are related to the input membership functions. These parameters are the so-called premise
ð8Þ
xci ai
Layer1
Layer 2
Layer 3
Layer 4 x y
Layer 5
A1 x
M
w1
N
w1 w1 f1
A2 S B1 y
M B2
ð11Þ
Layer 5: In the fifth layer, there is only one single fixed node labelled with S. This node performs the summation of all incoming signals. Hence, the overall output of the model is given by: P2 2 X wi fi O5i ¼ wi fi ¼ i¼1 ð12Þ w1 þ w2 i¼1
ð6Þ
i ¼ 3; 4
ð9Þ
w2
w2 f 2
N
w2 x
Fig. 1. ANFIS architecture.
y
f
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
parameters. In the fourth layer, there are also three modifiable parameters {pi, qi, ri}, pertaining to the first order polynomial. These parameters are so-called consequent parameters (Guler & Ubeyli, 2005; Jang, 1992). 2.3. Learning algorithm of ANFIS The task of the training algorithm for this architecture is tuning all the modifiable parameters to make the ANFIS output match the training data. Note here that parameters ai, bi and ci of the membership function are fixed, and describe the sigma, slope and centre of the bell membership functions, respectively. Thus, the output of the ANFIS model can be written as: f ¼
w1 w2 f1 þ f2 w1 þ w2 w1 þ w2
ð13Þ
Substituting Eq. (10) into Eq. (13) yields: f ¼ w1 f1 þ w2 f2
ð14Þ
Substituting the fuzzy if-then rules into Eq. (14), it becomes: f ¼ w1 ðp1 x þ q1 y þ r1 Þ þ w2 ðp2 x þ q2 y þ r2 Þ
ð15Þ
After rearrangement, the output can be expressed as: f ¼ ðw1 xÞp1 þ ðw1 yÞq1 þ ðw1 Þr1 þ ðw2 xÞp2 þ ðw2 yÞq2 þ ðw2 Þr2
ð16Þ
which is a linear combination of the modifiable consequent parameters p1, q1, r1, p2, q2 and r2. The least squares method can be used to identify the optimal values of these parameters easily. When the premise parameters are not fixed, the search space becomes larger and the convergence of the training becomes slower. A hybrid algorithm combining the least squares method and the gradient descent method is adopted to solve this problem. The hybrid algorithm is composed of a forward pass and a backward pass. The least squares method (forward pass) is used to optimize the consequent parameters with the premise parameters fixed. Once the optimal consequent parameters are found, the backward pass starts immediately. The gradient descent method (backward pass) is used to adjust optimally the premise parameters corresponding to the fuzzy sets in the input domain. The output of the ANFIS is calculated by employing the consequent parameters found in the forward pass. The output error is used to adapt the premise parameters by means of a standard back propagation algorithm. It has been proven that this hybrid algorithm is highly efficient in training the ANFIS (Guler & Ubeyli, 2005; Jang, 1992). 2.4. Color texture feature extraction The color of a pixel is typically represented with the RGB tristimulus values, each corresponding to the red,
2123
green and blue frequency bands of the visible light spectrum. Color is an important feature in the three dimensional RGB color space which contains information regarding the spectral distribution of light complementary to the gray scale information (Van de Wouwer et al., 1999). Moreover, texture is the discriminating information that differentiates the regions in an image. Thus, using only color information or using only gray scale texture is not a complementary work for real world texture images. Both properties increase the efficiency of the algorithms for texture analysis. Since texture is essentially a multi scale phenomenon, multi resolution approaches such as wavelets, perform well for texture analysis. A characterization of texture is usually based on the local information that appears within a neighborhood distribution of the gray levels. The proposed methodology focuses on a single scale in order to extract the relevant information. Recent studies have come to the conclusion that a spatial/frequency representation, which preserves both global and local information, is adequate for the characterization of texture. The wavelet transform offers a tool for spatial/frequency representation by decomposing the original images to the corresponding scales. When decomposition level decreases in the spatial domain, it increases in the frequency domain providing zooming capabilities and local characterization of the image. Since the low-frequency image produced by the transformation does not contain major texture information and the most significant information of a texture often appears in the middle-frequency channels. Entropy is a quantity that is widely used in information theory and is based on probability theory (Phan & MicheliTzanakou, 2000). Entropy is a common concept in many fields, mainly in mechanic, image processing and signal processing. The general form of the entropy function is shown as follows; H ðX Þ ¼
n X
pi log2 pi
ð17Þ
i¼1
where X is a random variable which can be one of the values x1 ; x2 ; . . . ; xn with probability p1 ; p2 ; . . . ; pn . Note that if pi = 0, then 0log2 0 is defined to be 0. Thus, H(X) can be interpreted as representing the amount of uncertainty that exists in the value of X. In information theory, entropy value is considered to be an average amount of information received when the value of X is observed. In this paper, we used norm entropy. The Norm entropy HN(s) is defined as follows; H N ðxÞ ¼
n X
jxi jp
for ð1 6 p < 2Þ
ð18Þ
i¼0
where x is signal and xi is ith component of the given x variable (Coifman & Wickerhauser, 1992). Energy is one of the most commonly used features for texture analysis. In this study we used the averaged l2-norm which is defined as follows:
2124
El2 ¼
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128 n 1 X 2 ðxi Þ N i¼1
ð19Þ
where N is the number of the components in the x variable. 2.5. Energy correlation signatures Van de Wouwer et al. (1999) has defined the wavelet energy correlation signatures as follows; C XB;jm ;X n ¼
Nj X Nj 1 X wX m wX n N j xN j i¼0 l¼0 B;j;l;i B;j;l;i
ð20Þ
m is the wavelet coefficient at (l,i) location at j Where wXB;j;l;i scale in B (B 2 {LH,HL,HH}) sub-band and Xm is the color space (m = 1, 2, 3). The set fC XB;jm ;X n jm; n ¼ 1; 2; 3; m 6 n; B 2 fLH; HL; HHg; j ¼ 1; 2; 3; . . . ; J g is called the wavelet energy signatures. They capture the energy distribution of the wavelet coefficients over the scale, sub band and color space for m = n and the others (m 5 n) represent the covariance between different color spaces.
3. Methodology The proposed color texture classification algorithm is illustrated at Fig. 2. The steps involved in color texture classification are as follows: Step-1: The input to the WT and ANFIS based color texture classification system is the color textures of size 128 · 128. These color texture images of size 128 · 128 was constructed randomly by subdividing the each color texture image of size 512 · 512. Thus, obtained input color texture images may overlap. For gray scale texture classification, the color information is discharged by gray scale transformation. For color textures the red, green and blue components are decomposed from the original color texture. Step-2: This step involves both feature extraction and classification. Fig. 2 shows the proposed structure for classification of color textures. Feature extraction is the key for pattern recognition so that it is arguably the most important component of designing the intelligent system based on pattern recognition since even the best classifier will perform poorly if the features are not chosen well. (i) Wavelet layer: this layer is responsible for feature extraction from textures. The feature extraction process has three stages: (1) Wavelet decomposition: for wavelet decomposition of the color textures, the pyramid structure was used at depth m = 1. As we mentioned before, the gray scale textures are generated by computing the luminance of the color texture images, hereby discarding the color information.
On the other hand, for color texture images, each color component (RGB) is wavelet transformed separately. Wavelet decomposition was applied to the texture images using four different wavelet filters. These filters are Bior4.4, coif4, sym4 and db7, respectively. (2) Wavelet entropy: an entropy-based criterion describes information-related properties for an accurate representation of a given signal. Entropy is a common concept in many fields, mainly in image processing and signal processing. We calculated the norm entropy as defined in Eq. (18) of the textures at the terminal node obtained from wavelet decomposition. In norm entropy, P is the power and must be such that 1 6 P < 2. (3) Wavelet energy: beside wavelet entropy features, we use the energy values of each subband of the wavelet decomposed textures to the WT and ANFIS structure. Energy of each channel can be computed by using Eq. (19). (ii) ANFIS layer: this layer realizes the intelligent classification using features from wavelet layer. The training parameters and the structure of the ANFIS is shown in Table 1. The hybrid learning algorithm has been used in ANFIS structure because it is highly efficient in training (Guler & Ubeyli, 2005). A Gauss type membership functions and two membership functions is used in the ANFIS model.
4. Experiments and discussions Sixteen real world RGB color texture images of size 512 · 512 from different natural scenes are used in the experimental studies. Fig. 3 shows the color texture images (Internet: University of Oulu texture database, 2005). A data base of 1920 color image regions of 16 texture classes of size 128 · 128 was constructed randomly by subdividing the each color texture image. The 320 of the color texture sub images were used for training the ANFIS. This means that 20 sample sub images for each texture classes is used for training. Hundred sub images for each texture class were constructed for testing the ANFIS. This also means that totally 1600 sub images were used for test. The following feature vectors were constructed: (a) Intensity (gray scale) images were obtained from the RGB form, thereby discarding the color information. We obtain one-level wavelet decomposition, and use the HH, LH and HL where H and L stand for the high pass and low pass band in each of the horizontal
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
2125
Sub image (128x128)
Color texture image (512x512) Sub image (128x128)
Feature Extraction Block
Randomly selected sub image (128x128)
R
G
B
LL
LH
LL
LH
LL
LH
HL
HH
HL
HH
HL
HH
Feature Extraction Wavelet Entropy Wavelet Energy
error
+
∑
Classification Block
Adaptive Neuro-Fuzzy Inference System
- desired output
Fig. 2. The proposed color texture classification scheme.
and vertical orientations. The features were extracted which have been explained at Section 2.4. This process constructed a feature vector of size 6. (b) Each R, G and B component was wavelet transformed using one-level decomposition and the proposed feature extraction scheme is employed. This process con-
structed a feature vector of size 18 (3 color channels · 3 wavelet detail images · 2 feature values). (c) For comparison of the proposed method with the energy correlation signatures which was explained in Section 2.5. Each R, G and B color component was wavelet transformed using one-level decomposi-
2126
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
Fig. 3. Color texture images; from left to right and top to bottom: Grass, Flowers1, Flowers2, Bark1, Clouds, Fabric7, Leaves, Metal, Misc, Tile, Bark2, Fabric2, Fabric3, Food1, Water, and Food2.
tion and wavelet energy correlation signatures were calculated. This process constructed 18 features. (d) We conduct the same experiment as in experiment b, except that the RGB space is replaced by the K1K2K3 color space. The related formulations about the K1K2K3 color space is given in Appendix A. Each K1, K2 and K3 component was wavelet transformed using one-level decomposition and the proposed feature extraction scheme is employed. This process also constructed a feature vector of size 18 (3 color channels · 3 wavelet detail images · 2 feature values). The experiments are conducted based on the methodology which is illustrated in Fig. 2 and the experimental results are presented in Tables 2–6, respectively. The tables are designed as if the rows indicate the four different wavelet filter types and the columns indicate the color texture images. The correct classification rates are indicated for all color texture types and the related wavelet filters. The p parameter which is used for norm entropy function is fixed
to the 1,5. Table 2 indicates the experimental results for gray scale texture images. As we mentioned earlier, gray scale texture images were obtained after the color information is discharged by gray scale transformation. Flowers, Clouds and Misc are correctly classified for all wavelet filter types when the experimental studies carried out with all feature sets (feature vector 1, feature vector 2, etc.). The correct classification rate is 100% for these texture images. This high correct classification rate is obtained for these color texture types because of their homogeneity structure. On the other hand, the similar high correct classification rates are not obtained for the rest of the gray scale texture types. Another important property for Table 2 is that Bior4.4 wavelet filter is gained much more accuracy than the other wavelet filter types. The overall correct classification rates for gray scale texture images can be seen at Table 6. One observes from Table 6 that features and the subsequent classification performance is significantly improved when color information is added to the texture property. For example, for Bior4.4 wavelet filter type, the correct classification rate is almost 90% for feature vector 1 (gray
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128 Table 1 ANFIS architecture and training parameters
Table 6 Total correct classification rates for feature vectors (%)
Architecture The number of layers Type of input membership functions Training parameters Learning rule
Momentum constant Sum-squared error Epochs number to sum squared error
2127
Feature set 1
Feature set 2
Feature set 3
Feature set 4
90.4375 85.7500 88.0625 81.5000
96.2500 95.5000 93.1250 93.3750
93.5000 93.2500 92.1250 92.0000
97.6250 97.0625 95.0000 94.8750
5 Output: 1 Gauss
Bior 4.4 Coif 4 Sym 4 db7
Hybrid learning algorithm (back-propagation for nonlinear parameters (ai, ci) and Least square errors for Linear parameters (pi, qi, ri, si, ssi, ppi, ui)) 0.98 0.00001 1
images. The overall correct classification rates are nearly 95% for all wavelet filter types. Hundred percent correct classification rate is obtained for Flowers, Clouds, Misc, Fabric2 and Water texture images. Moreover, for Bior4.4, 6 color texture images are recognized with a success rate of 100%. On the other hand, the similar high correct classification rates are obtained with the other wavelet filters type (Table 3). Our goal is investigating the color texture classification improvements. Thus we compare our proposal with the wavelet energy correlation signatures. We also use the ANFIS structure for classification of the wavelet energy correlation features. The structure of the ANFIS and the parameter variables are given in Table 1. Table 4 shows the experimental results of the wavelet energy correlation
level texture features), this correct classification rate is increased to almost 95% when the color information is also added (Feature vector 2). Table 3 indicates the classification results of the proposed method for color texture Table 2 Correct classification rates for feature vector 1 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
84 80 81 81
100 100 100 100
78 73 74 70
81 70 80 53
100 100 100 100
89 86 85 81
89 85 86 71
92 90 91 89
100 100 100 100
94 91 90 88
85 76 84 83
90 71 88 87
85 84 82 81
96 91 81 75
99 95 98 84
85 80 89 61
Table 3 Correct classification rates for feature vector 2 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
94 91 88 91
100 100 100 100
92 88 91 90
98 97 88 86
100 100 100 100
94 94 89 90
96 97 89 95
92 97 91 90
100 100 100 100
94 89 90 88
93 94 90 87
100 100 100 100
96 90 85 85
100 100 99 99
100 100 100 100
91 91 90 93
Table 4 Correct classification rates for feature vector 3 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
92 88 86 88
100 100 100 100
90 85 88 90
89 93 89 82
100 100 100 100
90 86 88 86
91 94 88 91
90 92 91 92
100 100 100 100
90 88 90 90
90 89 90 88
100 100 90 100
86 89 85 84
100 97 99 91
100 99 100 98
88 92 90 92
Table 5 Correct classification rates for feature vector 4 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
95 93 90 92
100 100 100 100
95 90 91 93
99 100 91 89
100 100 100 100
96 95 93 91
100 98 95 96
93 94 95 94
100 100 100 100
96 95 93 90
96 96 90 91
100 100 100 100
100 95 91 90
99 100 99 100
100 100 100 100
93 97 92 92
2128
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
signatures. As it is seen from the Table 6, for the feature vector 3, the overall correct classification rate is higher then the feature vector 1. Flowers, Clouds and Misc are also correctly classified (100%) for all wavelet filter types. The others are classified different success rates and correct classification rates are also satisfactory. But our proposal generates much more accurate results. The maximum correct classification rates are obtained for the correlation signatures when Bior4.4 and coif 4 wavelet types were used. And finally, Table 5 indicates the experimental results when we have changed our color space. The best results are obtained with the K–L color space (I1I2I3 color space). Grass, Flowers, Clouds, Misc, Fabric 2 and Water texture images are also classified 100% success rate. The overall correct classification rates are almost 96% for all wavelet filter types for this feature set. Note that the first component of the K–L transform is the intensity, which indeed proved to be an important feature. The other two axes then represent the image information which is statistically uncorrelated with intensity. These features can be interpreted as the extra information which is not present in texture or color separately, and which can be denoted as color texture information. 5. Conclusions In this paper, we have discussed the effect of the wavelet domain features and ANFIS classifier on the texture classification problem. The main aim of the study is combining the color and texture information to improve the classification of the texture images. We proposed a system which uses the wavelet domain entropy and energy values of the red, green and blue component of the RGB texture images. Among the three wavelet based methods that we examined, our proposed system for color texture provides best classification result. Experimental studies and subsequent results using a set of real world colored texture images, are showed the usefulness of the wavelet entropy and energy for color texture analysis. The results show that color is an important component for improving the classification results for texture analysis problem. In this study several important parameters such as wavelet decomposition level, wavelet filter type and norm entropy parameter value are constant. Anyway to select the best decomposition level is an important issue, furthermore the selecting best wavelet filter type and the best p parameter value for norm entropy will be studied in the future. On the other hand, proposed color texture classification scheme can be used for color texture segmentation. As we know that color texture segmentation is more important for computer vision applications and content based image retrieval. For instance, for segmentation of the color texture images, the wavelet entropy and wavelet energy values are computed for a small window centered on each pixel of the image, resulting in one dimensional feature vector per
pixel. Each pixel is then assigned to a particular image region by neural network (supervised) or clustering techniques (unsupervised). Appendix A. Color Transformation The K–L space (Karhunen–Loeve transform): 1 0 10 1 K1 0:333 0:333 0:333 R B C B CB C 0 0:500 A@ G A @ K2 A ¼ @ 0:500 0
K3
0:500
1:000
0:500
B
References Arivazhagan, S., Ganesan, L., & Angayarkanni, V. (XXXX). Color texture classification using wavelet transform. In Proceedings of the sixth international conference on computational intelligence and multimedia applications (ICCIMA’05). Caelli, T., & Reye, D. (1993). On the classification of image regions by color texture and shape. Pattern Recognition, 26(4), 461–470. Chen, C. C., & Chen, C. C. (1999). Filtering methods for texture discrimination. Pattern Recognition Letters, 20, 783–790. Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713–718. Crouse, M. S., Nowak, R. D., & Baraniuk, R. G. (1998). Wavelet based statistical signal processing using hidden Markov model. IEEE Transactions on Signal Processing, 46(4), 886–902. Daubechies, I. (1988). Orthogonal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41, 909–996. Guler, I., & Ubeyli, D. E. (2005). Adaptive neuro-fuzzy inference system for classification of EEG signals using wavelet coefficients. Journal of Neuroscience Methods, 148, 113–121. Haralick, R. M., Shanmugan, K., & Dinstein, I. (1973). Texture for image classification. IEEE Transactions on Systems Man and Cybernetics, 3(3), 610–621. Jang, J.-S. R. (1992). Self-learning fuzzy controllers based on temporal back propagation. IEEE Transactions on Neural Network, 3(5), 714–723. Karkanis, S. A., Iakovidis, D. K., Maroulis, D. E., Karras, D. A., & Tzivras, M. (2003). Computer-aided tumor detection in endoscopic video using color wavelet features, Information Tech. in Biomedicine. IEEE Transactions, 7(3), 141–152. Panjwani, D. K., & Healey, G. (1995). Markov random field models for unsupervised segmentation of textured color images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 939–954. Phan, F., & Micheli-Tzanakou, E. (2000). Supervised and unsupervised pattern recognition: feature extraction and computational intelligence. CRC Press. Qing, Xu, Jie, Yang, & Siyi, Ding (2005). Color texture analysis using the wavelet-based hidden Markov model. Pattern Recognition Letters, 26(11), 1710–1719. Reed, T. R., & du Buf, J. M. H. (1993). A review of recent texture segmentation and feature extraction techniques. CVGIP: Image Understanding, 57(3), 359–372. Sengur, A., Turkoglu, I., & Ince, M. C. (2007). Wavelet packet neural networks for texture classification. Expert Systems with Applications, 32(2), 527–533. Sklansky, J. (1978). Image segmentation and feature extraction. IEEE Transactions on Systems Man and Cybernetics, 8, 237–247. Internet: University of Oulu texture database (2005). http://www.outex. oulu.fi/outex.php. Van de Wouwer, G., Scheunders, P., Livens, S., & Van Dyck, D. (1999). Wavelet correlation signatures for color texture characterization. Pattern Recognition, 32(3), 443–451.
Expert Systems with Applications Expert Systems with Applications 34 (2008) 2120–2128 www.elsevier.com/locate/eswa
Wavelet transform and adaptive neuro-fuzzy inference system for color texture classification Abdulkadir Sengur Firat University, Department of Electronics and Computer Science, 23119 Elazig, Turkey
Abstract The wavelet domain features have been intensively used for texture classification and texture segmentation with encouraging results. More of the proposed multi resolution texture analysis methods are quite successful, but all the applications of the texture analysis so far are limited to gray scale images. This paper investigates the usage of Wavelet transform (WT) and Adaptive neuro-fuzzy inference system (ANFIS) for color texture classification problem. The proposed scheme composed of a wavelet domain feature extractor and an ANFIS classifier. Both entropy and energy features are used on wavelet domain. Different color spaces are considered in the experimental studies. The performed experimental studies show the effectiveness of the wavelet transform and ANFIS structure for color texture classification problem. The overall success rate is over 96%. 2007 Elsevier Ltd. All rights reserved. Keywords: Wavelet decomposition; ANFIS; Texture classification; Feature extraction; Entropy; Energy correlation
1. Introduction Texture analysis plays an important role in many image processing tasks, ranging from remote sensing to medical image processing, computer vision applications and natural scenes. A number of texture analysis methods have been proposed in the past decades (e.g., Sklansky, 1978) but most of them are used for gray scale images which represent the amount of visible light at the pixel’s position while ignoring the color information. The performance of such methods can be improved by adding the color information because besides texture, color is the most important properties especially when dealing with real world images (Van de Wouwer, Scheunders, Livens, & Van Dyck, 1999). Texture can be defined as a local statistical pattern of texture primitives in observer’s domain of interest. Texture classification aims to assign texture labels to unknown textures, according to training samples and classification rules. Two major issues are critical for texture classification: the texture classification algorithms and texture feature extraction. The main aim of texture classification is to find a best E-mail address: ksengur@firat.edu.tr 0957-4174/$ - see front matter 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.02.032
matched category for a given texture among existing textures. Texture has been analyzed extensively and many texture recognition schemes have been proposed (Haralick, Shanmugan, & Dinstein, 1973; Reed & du Buf, 1993; Van de Wouwer et al., 1999). The important property all have in common is that they constitute an appropriate model for relationship between the adjacent pixels of a neighborhood. A number of researchers have proposed algorithms for texture analysis. But all the applications of the texture analysis so far are limited to gray scale images (Sengur, Turkoglu, & Ince, 2007). But recently both color and texture information is used by several researchers (Panjwani & Healey, 1995; Van de Wouwer et al., 1999). A method has been proposed by Caelli and Reye (1993). They extract features from three spectral channels by using three multiscale isotropic filters. Van de Wouwer et al. (1999), proposed wavelet energy-correlation signatures and derive the transformation of these signatures upon linear color space transformation. The wavelet domain based co-occurrence matrix method and the second order statistical features were used for color texture classification in the study of Arivazhagan, Ganesan, and Angayarkanni
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
(XXXX). A set of features are derived and color texture classification is done for different combination of the features and for different color models. Karkanis, Iakovidis, Maroulis, Karras, and Tzivras (2003), proposed a new approach for detection of tumors in colonoscopic video. The proposed algorithm is based on a color feature extraction scheme to represent the different regions in the frame sequence. The scheme is built on the wavelet transform. The features called as color wavelet covariance are based on the covariance of second order textural measurement. A linear discriminant analysis is used for classification of the image regions. Panjwani and Healey proposed an unsupervised algorithm which uses Markov random field models for color textures classification (Panjwani & Healey, 1995). These models characterized a texture in terms of spatial interaction within each color plane and interaction between different color planes. Crouse et al. has proposed a framework for statistical signal modeling based on the wavelet domain hidden Markov tree (Crouse, Nowak, & Baraniuk, 1998). The algorithm provided an efficient approach to modeling of wavelet coefficients that are often found in real world images. Xu et al., have shown that the wavelet coefficients have certain inter-dependences between color planes (Qing, Jie, & Siyi, 2005). They used wavelet domain hidden Markov model for color texture analysis. The proposed approach is used for modeling the dependences between color planes as well as the interactions across scales. The wavelet coefficients at the same location scale and sub-band, but different color planes are grouped into one vector and a multivariate Gaussian mixture model is employed for approximating the marginal distribution of the wavelet coefficient vectors in one scale. Despite these examples, there is not enough study for color texture analysis as was noted by Van de Wouwer et al. (1999). Multi-resolution representations give rise to an interesting class of texture analysis. Especially wavelets provide a convenient way to obtain a multi-resolution aspect to the texture images (Chen & Chen, 1999). In this paper, a color texture image classification scheme is proposed which uses Wavelet transform and ANFIS classifier. Wavelet entropies and wavelet energies of each color plane at different scales are used for forming the feature vector of each color texture. The experimental studies show the effectiveness of the proposed classification system. The experimental results were also compared with the results of wavelet energy correlation signatures. The influence of the choice of color space representation on classification performance is also investigated. The organization of this paper is as follows, in Section 2; it is given the theory for wavelet transform, feature extraction for color texture images, ANFIS and the extraction of the energy correlation signatures. Several brief definitions can be seen at this section. In Section 3; the methodology and the implementation of the proposed process is given. In Section 4; experimental study is introduced and the classification results are shown. In Section 5; we finally conclude the study.
2121
2. Preliminaries In this section, the theoretical foundations for the expert system used in the presented study are given in the following subsections. 2.1. Wavelet transform Wavelet transforms are finding inverse use in fields as diverse as telecommunications and biology. Because of their suitability for analyzing non-stationary signals, they have become a powerful alternative to Fourier methods in many medical applications, where such signals abound (Daubechies, 1988). The main advantages of wavelets is that they have a varying window size, being wide for slow frequencies and narrow for the fast ones, thus leading to an optimal time-frequency resolution in all the frequency ranges. Furthermore, owing to the fact that windows are adapted to the transients of each scale, wavelets lack the requirement of stationary. The (continuous) wavelet transform of a 1D signal f(x) is defined as: Z þ1 ðW w f Þða; bÞ ¼ hf ; wða; bÞi ¼ f ðxÞwða;bÞ ðxÞ dx 1 ð1Þ xb 1=2 wða;bÞ ¼ a w a where a is the scaling factor, b is the translation parameter related to the location of the window, and w*(x) is the transforming function. The latter is also called the mother wavelet, which is the prototype for generating the other window functions. The extension to the 2D case is usually performed by using a product of 1D filters. In practice, the transform is computed by applying a separable filter bank to the image: Ln ðbÞ ¼ ½H x ½H y Ln1 #2;1 #1;2 ðbÞ
ð2Þ
Dn1 ðbÞ ¼ ½H x ½Gy Ln1 #2;1 #1;2 ðbÞ
ð3Þ
Dn2 ðbÞ ¼ ½Gx ½H y Ln1 #2;1 #1;2 ðbÞ
ð4Þ
Dn3 ðbÞ ¼ ½Gx ½Gy Ln1 #2;1 #1;2 ðbÞ
ð5Þ
where b 2 R2, * denotes the convolution operator, #2,1 (#1,2) sub-sampling along the rows (columns) and L0 = I(x) is the original image. H and G are low and band pass filters, respectively. Ln is obtained by low pass filtering so it is called low resolution image at scale n. The Dni are obtained by band pass filtering in a specific direction. So these parameters contain directional detail information at scale n. The original image is thus represented by a set of sub images at several scales. 2.2. Architecture of adaptive-network-based fuzzy inference system An adaptive network, as its name implies, is a network structure consisting of nodes and directional links through
2122
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
which the nodes are connected. Moreover, parts or all of the nodes are adaptive, which means each output of these nodes depends on the parameters pertaining to this node and the learning rule specifies how these parameters should be changed to minimize a prescribe error measure. The ANFIS is a fuzzy Sugeno model put in the framework of adaptive systems to facilitate learning and adaptation (Guler & Ubeyli, 2005; Jang, 1992). Such framework makes the ANFIS modelling more systematic and less reliant on expert knowledge. To present the ANFIS architecture, two fuzzy if-then rules based on a first order Sugeno model are considered:
where ai, bi and ci are the parameters of the membership function, governing the bell shaped functions accordingly. Layer 2: The nodes in this layer are fixed. These are labelled M to indicate that play the role of a simple multiplier. The outputs of these nodes are given by: O2i ¼ wi ¼ lAi ðxÞlBi ðyÞ i ¼ 1; 2
which are the so-called firing strengths of the rules. Layer 3: In the third layer, the nodes are also fixed nodes. They are labelled with N, indicating that they play a normalization role to the firing strengths from the previous layer. The outputs of this layer can be represented as: wi i ¼ 1; 2 ð10Þ O3i ¼ wi ¼ w1 þ w2
Rule 1: If (x is A1) and (y is B1) then (f1 = p1x + q1y + r1) Rule 2: If (x is A2) and (y is B2) then (f2 = p2x + q2y + r2) where x and y are the inputs, Ai and Bi are the fuzzy sets, fi are the outputs within the fuzzy region specified by the fuzzy rule, pi, qi and ri are the design parameters that are determined during the training process. The ANFIS architecture to implement these two rules is shown in Fig. 1, in which a circle indicates a fixed node, whereas a square indicates an adaptive node.
which are the so-called normalized firing strengths. Layer 4: In the fourth layer, the nodes are adaptive nodes. The output of each node in this layer is simply the product of the normalized firing strength and a first order polynomial (for a first order Sugeno model). Thus, the outputs of this layer are given by:
Layer 1: In the first layer, all the nodes are adaptive nodes. i is the degree of the membership of the input to the fuzzy membership function represented by the node: O1i ¼ lAi ðxÞ; O1i
i ¼ 1; 2
¼ lBi2 ðyÞ;
O4i ¼ wi fi ¼ wi ðpi x þ qi y þ ri Þ i ¼ 1; 2
ð7Þ
where lAi ðxÞ; lBi 2 ðyÞ can adopt any fuzzy membership function. For example, if the bell shaped membership function is employed; lAi ðxÞ is given by: lAi ðxÞ ¼ 1þ
1 bi 2
It can be observed that there are two adaptive layers in this ANFIS architecture, namely the first layer and the fourth layer. In the first layer, there are three modifiable parameters {ai, bi, ci}, which are related to the input membership functions. These parameters are the so-called premise
ð8Þ
xci ai
Layer1
Layer 2
Layer 3
Layer 4 x y
Layer 5
A1 x
M
w1
N
w1 w1 f1
A2 S B1 y
M B2
ð11Þ
Layer 5: In the fifth layer, there is only one single fixed node labelled with S. This node performs the summation of all incoming signals. Hence, the overall output of the model is given by: P2 2 X wi fi O5i ¼ wi fi ¼ i¼1 ð12Þ w1 þ w2 i¼1
ð6Þ
i ¼ 3; 4
ð9Þ
w2
w2 f 2
N
w2 x
Fig. 1. ANFIS architecture.
y
f
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
parameters. In the fourth layer, there are also three modifiable parameters {pi, qi, ri}, pertaining to the first order polynomial. These parameters are so-called consequent parameters (Guler & Ubeyli, 2005; Jang, 1992). 2.3. Learning algorithm of ANFIS The task of the training algorithm for this architecture is tuning all the modifiable parameters to make the ANFIS output match the training data. Note here that parameters ai, bi and ci of the membership function are fixed, and describe the sigma, slope and centre of the bell membership functions, respectively. Thus, the output of the ANFIS model can be written as: f ¼
w1 w2 f1 þ f2 w1 þ w2 w1 þ w2
ð13Þ
Substituting Eq. (10) into Eq. (13) yields: f ¼ w1 f1 þ w2 f2
ð14Þ
Substituting the fuzzy if-then rules into Eq. (14), it becomes: f ¼ w1 ðp1 x þ q1 y þ r1 Þ þ w2 ðp2 x þ q2 y þ r2 Þ
ð15Þ
After rearrangement, the output can be expressed as: f ¼ ðw1 xÞp1 þ ðw1 yÞq1 þ ðw1 Þr1 þ ðw2 xÞp2 þ ðw2 yÞq2 þ ðw2 Þr2
ð16Þ
which is a linear combination of the modifiable consequent parameters p1, q1, r1, p2, q2 and r2. The least squares method can be used to identify the optimal values of these parameters easily. When the premise parameters are not fixed, the search space becomes larger and the convergence of the training becomes slower. A hybrid algorithm combining the least squares method and the gradient descent method is adopted to solve this problem. The hybrid algorithm is composed of a forward pass and a backward pass. The least squares method (forward pass) is used to optimize the consequent parameters with the premise parameters fixed. Once the optimal consequent parameters are found, the backward pass starts immediately. The gradient descent method (backward pass) is used to adjust optimally the premise parameters corresponding to the fuzzy sets in the input domain. The output of the ANFIS is calculated by employing the consequent parameters found in the forward pass. The output error is used to adapt the premise parameters by means of a standard back propagation algorithm. It has been proven that this hybrid algorithm is highly efficient in training the ANFIS (Guler & Ubeyli, 2005; Jang, 1992). 2.4. Color texture feature extraction The color of a pixel is typically represented with the RGB tristimulus values, each corresponding to the red,
2123
green and blue frequency bands of the visible light spectrum. Color is an important feature in the three dimensional RGB color space which contains information regarding the spectral distribution of light complementary to the gray scale information (Van de Wouwer et al., 1999). Moreover, texture is the discriminating information that differentiates the regions in an image. Thus, using only color information or using only gray scale texture is not a complementary work for real world texture images. Both properties increase the efficiency of the algorithms for texture analysis. Since texture is essentially a multi scale phenomenon, multi resolution approaches such as wavelets, perform well for texture analysis. A characterization of texture is usually based on the local information that appears within a neighborhood distribution of the gray levels. The proposed methodology focuses on a single scale in order to extract the relevant information. Recent studies have come to the conclusion that a spatial/frequency representation, which preserves both global and local information, is adequate for the characterization of texture. The wavelet transform offers a tool for spatial/frequency representation by decomposing the original images to the corresponding scales. When decomposition level decreases in the spatial domain, it increases in the frequency domain providing zooming capabilities and local characterization of the image. Since the low-frequency image produced by the transformation does not contain major texture information and the most significant information of a texture often appears in the middle-frequency channels. Entropy is a quantity that is widely used in information theory and is based on probability theory (Phan & MicheliTzanakou, 2000). Entropy is a common concept in many fields, mainly in mechanic, image processing and signal processing. The general form of the entropy function is shown as follows; H ðX Þ ¼
n X
pi log2 pi
ð17Þ
i¼1
where X is a random variable which can be one of the values x1 ; x2 ; . . . ; xn with probability p1 ; p2 ; . . . ; pn . Note that if pi = 0, then 0log2 0 is defined to be 0. Thus, H(X) can be interpreted as representing the amount of uncertainty that exists in the value of X. In information theory, entropy value is considered to be an average amount of information received when the value of X is observed. In this paper, we used norm entropy. The Norm entropy HN(s) is defined as follows; H N ðxÞ ¼
n X
jxi jp
for ð1 6 p < 2Þ
ð18Þ
i¼0
where x is signal and xi is ith component of the given x variable (Coifman & Wickerhauser, 1992). Energy is one of the most commonly used features for texture analysis. In this study we used the averaged l2-norm which is defined as follows:
2124
El2 ¼
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128 n 1 X 2 ðxi Þ N i¼1
ð19Þ
where N is the number of the components in the x variable. 2.5. Energy correlation signatures Van de Wouwer et al. (1999) has defined the wavelet energy correlation signatures as follows; C XB;jm ;X n ¼
Nj X Nj 1 X wX m wX n N j xN j i¼0 l¼0 B;j;l;i B;j;l;i
ð20Þ
m is the wavelet coefficient at (l,i) location at j Where wXB;j;l;i scale in B (B 2 {LH,HL,HH}) sub-band and Xm is the color space (m = 1, 2, 3). The set fC XB;jm ;X n jm; n ¼ 1; 2; 3; m 6 n; B 2 fLH; HL; HHg; j ¼ 1; 2; 3; . . . ; J g is called the wavelet energy signatures. They capture the energy distribution of the wavelet coefficients over the scale, sub band and color space for m = n and the others (m 5 n) represent the covariance between different color spaces.
3. Methodology The proposed color texture classification algorithm is illustrated at Fig. 2. The steps involved in color texture classification are as follows: Step-1: The input to the WT and ANFIS based color texture classification system is the color textures of size 128 · 128. These color texture images of size 128 · 128 was constructed randomly by subdividing the each color texture image of size 512 · 512. Thus, obtained input color texture images may overlap. For gray scale texture classification, the color information is discharged by gray scale transformation. For color textures the red, green and blue components are decomposed from the original color texture. Step-2: This step involves both feature extraction and classification. Fig. 2 shows the proposed structure for classification of color textures. Feature extraction is the key for pattern recognition so that it is arguably the most important component of designing the intelligent system based on pattern recognition since even the best classifier will perform poorly if the features are not chosen well. (i) Wavelet layer: this layer is responsible for feature extraction from textures. The feature extraction process has three stages: (1) Wavelet decomposition: for wavelet decomposition of the color textures, the pyramid structure was used at depth m = 1. As we mentioned before, the gray scale textures are generated by computing the luminance of the color texture images, hereby discarding the color information.
On the other hand, for color texture images, each color component (RGB) is wavelet transformed separately. Wavelet decomposition was applied to the texture images using four different wavelet filters. These filters are Bior4.4, coif4, sym4 and db7, respectively. (2) Wavelet entropy: an entropy-based criterion describes information-related properties for an accurate representation of a given signal. Entropy is a common concept in many fields, mainly in image processing and signal processing. We calculated the norm entropy as defined in Eq. (18) of the textures at the terminal node obtained from wavelet decomposition. In norm entropy, P is the power and must be such that 1 6 P < 2. (3) Wavelet energy: beside wavelet entropy features, we use the energy values of each subband of the wavelet decomposed textures to the WT and ANFIS structure. Energy of each channel can be computed by using Eq. (19). (ii) ANFIS layer: this layer realizes the intelligent classification using features from wavelet layer. The training parameters and the structure of the ANFIS is shown in Table 1. The hybrid learning algorithm has been used in ANFIS structure because it is highly efficient in training (Guler & Ubeyli, 2005). A Gauss type membership functions and two membership functions is used in the ANFIS model.
4. Experiments and discussions Sixteen real world RGB color texture images of size 512 · 512 from different natural scenes are used in the experimental studies. Fig. 3 shows the color texture images (Internet: University of Oulu texture database, 2005). A data base of 1920 color image regions of 16 texture classes of size 128 · 128 was constructed randomly by subdividing the each color texture image. The 320 of the color texture sub images were used for training the ANFIS. This means that 20 sample sub images for each texture classes is used for training. Hundred sub images for each texture class were constructed for testing the ANFIS. This also means that totally 1600 sub images were used for test. The following feature vectors were constructed: (a) Intensity (gray scale) images were obtained from the RGB form, thereby discarding the color information. We obtain one-level wavelet decomposition, and use the HH, LH and HL where H and L stand for the high pass and low pass band in each of the horizontal
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
2125
Sub image (128x128)
Color texture image (512x512) Sub image (128x128)
Feature Extraction Block
Randomly selected sub image (128x128)
R
G
B
LL
LH
LL
LH
LL
LH
HL
HH
HL
HH
HL
HH
Feature Extraction Wavelet Entropy Wavelet Energy
error
+
∑
Classification Block
Adaptive Neuro-Fuzzy Inference System
- desired output
Fig. 2. The proposed color texture classification scheme.
and vertical orientations. The features were extracted which have been explained at Section 2.4. This process constructed a feature vector of size 6. (b) Each R, G and B component was wavelet transformed using one-level decomposition and the proposed feature extraction scheme is employed. This process con-
structed a feature vector of size 18 (3 color channels · 3 wavelet detail images · 2 feature values). (c) For comparison of the proposed method with the energy correlation signatures which was explained in Section 2.5. Each R, G and B color component was wavelet transformed using one-level decomposi-
2126
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
Fig. 3. Color texture images; from left to right and top to bottom: Grass, Flowers1, Flowers2, Bark1, Clouds, Fabric7, Leaves, Metal, Misc, Tile, Bark2, Fabric2, Fabric3, Food1, Water, and Food2.
tion and wavelet energy correlation signatures were calculated. This process constructed 18 features. (d) We conduct the same experiment as in experiment b, except that the RGB space is replaced by the K1K2K3 color space. The related formulations about the K1K2K3 color space is given in Appendix A. Each K1, K2 and K3 component was wavelet transformed using one-level decomposition and the proposed feature extraction scheme is employed. This process also constructed a feature vector of size 18 (3 color channels · 3 wavelet detail images · 2 feature values). The experiments are conducted based on the methodology which is illustrated in Fig. 2 and the experimental results are presented in Tables 2–6, respectively. The tables are designed as if the rows indicate the four different wavelet filter types and the columns indicate the color texture images. The correct classification rates are indicated for all color texture types and the related wavelet filters. The p parameter which is used for norm entropy function is fixed
to the 1,5. Table 2 indicates the experimental results for gray scale texture images. As we mentioned earlier, gray scale texture images were obtained after the color information is discharged by gray scale transformation. Flowers, Clouds and Misc are correctly classified for all wavelet filter types when the experimental studies carried out with all feature sets (feature vector 1, feature vector 2, etc.). The correct classification rate is 100% for these texture images. This high correct classification rate is obtained for these color texture types because of their homogeneity structure. On the other hand, the similar high correct classification rates are not obtained for the rest of the gray scale texture types. Another important property for Table 2 is that Bior4.4 wavelet filter is gained much more accuracy than the other wavelet filter types. The overall correct classification rates for gray scale texture images can be seen at Table 6. One observes from Table 6 that features and the subsequent classification performance is significantly improved when color information is added to the texture property. For example, for Bior4.4 wavelet filter type, the correct classification rate is almost 90% for feature vector 1 (gray
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128 Table 1 ANFIS architecture and training parameters
Table 6 Total correct classification rates for feature vectors (%)
Architecture The number of layers Type of input membership functions Training parameters Learning rule
Momentum constant Sum-squared error Epochs number to sum squared error
2127
Feature set 1
Feature set 2
Feature set 3
Feature set 4
90.4375 85.7500 88.0625 81.5000
96.2500 95.5000 93.1250 93.3750
93.5000 93.2500 92.1250 92.0000
97.6250 97.0625 95.0000 94.8750
5 Output: 1 Gauss
Bior 4.4 Coif 4 Sym 4 db7
Hybrid learning algorithm (back-propagation for nonlinear parameters (ai, ci) and Least square errors for Linear parameters (pi, qi, ri, si, ssi, ppi, ui)) 0.98 0.00001 1
images. The overall correct classification rates are nearly 95% for all wavelet filter types. Hundred percent correct classification rate is obtained for Flowers, Clouds, Misc, Fabric2 and Water texture images. Moreover, for Bior4.4, 6 color texture images are recognized with a success rate of 100%. On the other hand, the similar high correct classification rates are obtained with the other wavelet filters type (Table 3). Our goal is investigating the color texture classification improvements. Thus we compare our proposal with the wavelet energy correlation signatures. We also use the ANFIS structure for classification of the wavelet energy correlation features. The structure of the ANFIS and the parameter variables are given in Table 1. Table 4 shows the experimental results of the wavelet energy correlation
level texture features), this correct classification rate is increased to almost 95% when the color information is also added (Feature vector 2). Table 3 indicates the classification results of the proposed method for color texture Table 2 Correct classification rates for feature vector 1 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
84 80 81 81
100 100 100 100
78 73 74 70
81 70 80 53
100 100 100 100
89 86 85 81
89 85 86 71
92 90 91 89
100 100 100 100
94 91 90 88
85 76 84 83
90 71 88 87
85 84 82 81
96 91 81 75
99 95 98 84
85 80 89 61
Table 3 Correct classification rates for feature vector 2 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
94 91 88 91
100 100 100 100
92 88 91 90
98 97 88 86
100 100 100 100
94 94 89 90
96 97 89 95
92 97 91 90
100 100 100 100
94 89 90 88
93 94 90 87
100 100 100 100
96 90 85 85
100 100 99 99
100 100 100 100
91 91 90 93
Table 4 Correct classification rates for feature vector 3 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
92 88 86 88
100 100 100 100
90 85 88 90
89 93 89 82
100 100 100 100
90 86 88 86
91 94 88 91
90 92 91 92
100 100 100 100
90 88 90 90
90 89 90 88
100 100 90 100
86 89 85 84
100 97 99 91
100 99 100 98
88 92 90 92
Table 5 Correct classification rates for feature vector 4 (%)
Bior 4.4 Coif 4 Sym 4 db7
Grass
Flowers
Flowers1
Bark
Clouds
Fabric7
Leaves
Metal
Misc
Tile
Bark
Fabric2
Fabric3
Food1
Water
Food
95 93 90 92
100 100 100 100
95 90 91 93
99 100 91 89
100 100 100 100
96 95 93 91
100 98 95 96
93 94 95 94
100 100 100 100
96 95 93 90
96 96 90 91
100 100 100 100
100 95 91 90
99 100 99 100
100 100 100 100
93 97 92 92
2128
A. Sengur / Expert Systems with Applications 34 (2008) 2120–2128
signatures. As it is seen from the Table 6, for the feature vector 3, the overall correct classification rate is higher then the feature vector 1. Flowers, Clouds and Misc are also correctly classified (100%) for all wavelet filter types. The others are classified different success rates and correct classification rates are also satisfactory. But our proposal generates much more accurate results. The maximum correct classification rates are obtained for the correlation signatures when Bior4.4 and coif 4 wavelet types were used. And finally, Table 5 indicates the experimental results when we have changed our color space. The best results are obtained with the K–L color space (I1I2I3 color space). Grass, Flowers, Clouds, Misc, Fabric 2 and Water texture images are also classified 100% success rate. The overall correct classification rates are almost 96% for all wavelet filter types for this feature set. Note that the first component of the K–L transform is the intensity, which indeed proved to be an important feature. The other two axes then represent the image information which is statistically uncorrelated with intensity. These features can be interpreted as the extra information which is not present in texture or color separately, and which can be denoted as color texture information. 5. Conclusions In this paper, we have discussed the effect of the wavelet domain features and ANFIS classifier on the texture classification problem. The main aim of the study is combining the color and texture information to improve the classification of the texture images. We proposed a system which uses the wavelet domain entropy and energy values of the red, green and blue component of the RGB texture images. Among the three wavelet based methods that we examined, our proposed system for color texture provides best classification result. Experimental studies and subsequent results using a set of real world colored texture images, are showed the usefulness of the wavelet entropy and energy for color texture analysis. The results show that color is an important component for improving the classification results for texture analysis problem. In this study several important parameters such as wavelet decomposition level, wavelet filter type and norm entropy parameter value are constant. Anyway to select the best decomposition level is an important issue, furthermore the selecting best wavelet filter type and the best p parameter value for norm entropy will be studied in the future. On the other hand, proposed color texture classification scheme can be used for color texture segmentation. As we know that color texture segmentation is more important for computer vision applications and content based image retrieval. For instance, for segmentation of the color texture images, the wavelet entropy and wavelet energy values are computed for a small window centered on each pixel of the image, resulting in one dimensional feature vector per
pixel. Each pixel is then assigned to a particular image region by neural network (supervised) or clustering techniques (unsupervised). Appendix A. Color Transformation The K–L space (Karhunen–Loeve transform): 1 0 10 1 K1 0:333 0:333 0:333 R B C B CB C 0 0:500 A@ G A @ K2 A ¼ @ 0:500 0
K3
0:500
1:000
0:500
B
References Arivazhagan, S., Ganesan, L., & Angayarkanni, V. (XXXX). Color texture classification using wavelet transform. In Proceedings of the sixth international conference on computational intelligence and multimedia applications (ICCIMA’05). Caelli, T., & Reye, D. (1993). On the classification of image regions by color texture and shape. Pattern Recognition, 26(4), 461–470. Chen, C. C., & Chen, C. C. (1999). Filtering methods for texture discrimination. Pattern Recognition Letters, 20, 783–790. Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713–718. Crouse, M. S., Nowak, R. D., & Baraniuk, R. G. (1998). Wavelet based statistical signal processing using hidden Markov model. IEEE Transactions on Signal Processing, 46(4), 886–902. Daubechies, I. (1988). Orthogonal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41, 909–996. Guler, I., & Ubeyli, D. E. (2005). Adaptive neuro-fuzzy inference system for classification of EEG signals using wavelet coefficients. Journal of Neuroscience Methods, 148, 113–121. Haralick, R. M., Shanmugan, K., & Dinstein, I. (1973). Texture for image classification. IEEE Transactions on Systems Man and Cybernetics, 3(3), 610–621. Jang, J.-S. R. (1992). Self-learning fuzzy controllers based on temporal back propagation. IEEE Transactions on Neural Network, 3(5), 714–723. Karkanis, S. A., Iakovidis, D. K., Maroulis, D. E., Karras, D. A., & Tzivras, M. (2003). Computer-aided tumor detection in endoscopic video using color wavelet features, Information Tech. in Biomedicine. IEEE Transactions, 7(3), 141–152. Panjwani, D. K., & Healey, G. (1995). Markov random field models for unsupervised segmentation of textured color images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 939–954. Phan, F., & Micheli-Tzanakou, E. (2000). Supervised and unsupervised pattern recognition: feature extraction and computational intelligence. CRC Press. Qing, Xu, Jie, Yang, & Siyi, Ding (2005). Color texture analysis using the wavelet-based hidden Markov model. Pattern Recognition Letters, 26(11), 1710–1719. Reed, T. R., & du Buf, J. M. H. (1993). A review of recent texture segmentation and feature extraction techniques. CVGIP: Image Understanding, 57(3), 359–372. Sengur, A., Turkoglu, I., & Ince, M. C. (2007). Wavelet packet neural networks for texture classification. Expert Systems with Applications, 32(2), 527–533. Sklansky, J. (1978). Image segmentation and feature extraction. IEEE Transactions on Systems Man and Cybernetics, 8, 237–247. Internet: University of Oulu texture database (2005). http://www.outex. oulu.fi/outex.php. Van de Wouwer, G., Scheunders, P., Livens, S., & Van Dyck, D. (1999). Wavelet correlation signatures for color texture characterization. Pattern Recognition, 32(3), 443–451.