- Email: [email protected]
An effective method for color image retrieval based on texture
Computer Standards & Interfaces 34 (2012) 31–35
Contents lists available at ScienceDirect
Computer Standards & Interfaces j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / c s i
An effective method for color image retrieval based on texture WangXing-yuan ⁎, ChenZhi-feng, YunJiao-jiao Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China
a r t i c l e
i n f o
Article history: Received 14 June 2009 Received in revised form 1 November 2010 Accepted 2 May 2011 Available online 7 May 2011 Keywords: Color image retrieval Texture feature extraction Co-occurrence matrix
a b s t r a c t This paper presents an effective color image retrieval method based on texture, which uses the color cooccurrence matrix to extract the texture feature and measure the similarity of two color images. Due to the color information such as components and distribution is also taken into consideration, the feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level co-occurrence matrix method and color histogram method, and it enhances the retrieval accuracy which is measured in terms of the recall and precision in the meanwhile. © 2011 Elsevier B.V. All rights reserved.
1. Introduction With the development of the multimedia technique, digital library and multimedia database including kinds of image databases increase rapidly. Traditional retrieval methods based on key words and description text can't fulfill our requirements, therefore some scholars put forward the content-based image retrieval (CBIR) technique [1]. In a CBIR system, images are automatically indexed by summarizing their visual contents through automatically extracted primitive features, such as shape, texture, color, size and so on. At present, CBIR has become the hotspot of research both at home and abroad, and many effective retrieval algorithms have been presented [2–10]. Ref. [9] described a new co-occurrence matrix based the approach for multispectral texture analysis and Ref. [10] analyzed three different approaches to color texture based on the classification of images. In this paper, we propose a color image retrieval method based on texture using color co-occurrence matrix. We firstly extract the texture feature according to the co-occurrence matrix information, and then estimate the similarity of two color images. Color information, such as components and distribution, are the important factors of color image. The feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level cooccurrence matrix method and color histogram method, and it enhances the retrieval accuracy which is measured in terms of the recall and precision in the meanwhile. This paper is organized as follows. Section 2 gives a brief introduction of the texture feature extraction. Section 3 describes the general concept of gray-level co-occurrence matrix. The proposed
⁎ Corresponding author. E-mail address: [email protected] (X. Wang). 0920-5489/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.csi.2011.05.001
color co-occurrence matrix is elaborated in Section 4. Section 5 presents our proposed color image retrieval method based on color co-occurrence matrix. Section 6 gives some experimental results comparing our method with gray-level co-occurrence matrix method and color histogram method in terms of retrieval accuracy. Finally, we draw a conclusion in Section 7. 2. Texture feature extraction An important feature of an image is texture. The texture descriptor provides measures of properties such as smoothness, coarseness, and regularity. The three principle approaches used in image processing to describe the texture of a region are statistical, structural, and spectral. Statistical approaches yield characterizations of textures as smooth, coarse, grainy and so on. Gray-level co-occurrence matrix is one of the most commonly used statistical approaches to extract texture feature of an image, such as energy, inertia, entropy and uniformity. In this paper, we propose a color co-occurrence matrix to describe texture feature of an image. 3. Gray-level co-occurrence matrix The gray-level co-occurrence matrix presents the joint probability that a pair of gray pixels with the position (Δx, Δy) will occur at the same time, where Δx and Δy are determined by a specified displacement δ between the pair of pixels and angle θ, and they are subject to Δx = δ cos θ and Δy = δsinθ. There is no fixed method to determine the values of δ and θ, and we usually consider the neighbors of a pixel. Because the distribution of gray intensities will be different at every orientation, 8 connected neighbors seem to be the best for computing gray-level co-occurrence matrix, and there will be δ = 1; θ = 0 ∘, 45 ∘, 90 ∘, 135 ∘ for the 8 connected neighbors.
32
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
Let L be the number of distinct gray levels of an image, thus the gray-level co-occurrence matrix CM(Δx, Δy)(i, j)or CM(δ, θ)(i, j) will be a L × L matrix whose element mij is the number of times that a point with gray level i appears (Δx, Δy) distance to a point with gray level j. 3.1. Definition 1
4.2. Definition 3 Divide the blocks of image T into color connectivity region set S = {Ri}(1 ≤ i ≤ M) according to the criterion of 8-connectivity, where M is the number of color connectivity regions if M
(i) ∪ Ri = T; i=1
Suppose an image with gray levels in the range [0, L − 1] is denoted by f(x, y), thus f(x, y) ∈ [0, L − 1]. For an arbitrary region R of the image, define S as a set of point pairs with specific spatial correlation. Thus the gray-level co-occurrence matrix can be defined as follow: CMðδ;θÞ ði; jÞ = cardf½ðx1 ; y1 Þ; ðx2 ; y2 Þ∈S j f ðx1 ; y1 Þ = i&f ðx2 ; y2 Þ = jg: ð1Þ Then we normalize it, accordingly, formula (1) can be further written as follow: CMðδ;θÞ ði; jÞ =
cardf½ðx1 ; y1 Þ; ðx2 ; y2 Þ∈S j f ðx1 ; y1 Þ = i&f ðx2 ; y2 Þ = jg ; cardðSÞ ð2Þ
where i ∈ [0, L − 1], j ∈ [0, L − 1], x 2 = x 1 + d cos θ, y 2 = y 1 + d sin θ, card(S) denotes the sum of all values in formula (1). For instance, considering an image region with eight gray levels, there will be f(x, y) ∈ [0, 7]. The corresponding gray-level co-occurrence matrix CM(1, 90∘) of the region shown in Fig. 1(a) can be obtained as shown in Fig. 1(b). By choosing the values of δ and θ, a separate co-occurrence matrix is obtained. From each co-occurrence matrix a number of textural features and the spatial information of relative position of different pixels can be extracted. For a region with coarse texture, the high values of its corresponding gray-level co-occurrence matrix are near the main diagonal, because the differences (i − j) are smaller there. In contrast, for a region with smooth texture, the high values of its corresponding gray-level co-occurrence matrix scatter everywhere, because the differences (i − j) are bigger there. 4. Color co-occurrence matrix 4.1. Definition 2
Obviously, S is unique. In Ref. [9], M. Hauta-Kasari et al. used the method that the spectral and spatial domains processed together based on the training set (the gray-level co-occurrence matrices were computed from the training set) in the multispectral texture segmentation. Different from this method, we note that the color connectivity regions of an image will not be regular rectangle, and the sizes of them are different. Therefore, for each color connectivity region Ri(1 ≤ i ≤ M) in set S, we obtain co-occurrence matrix for four orientation (horizontal 0 ∘, vertical 90 ∘and diagonal 45 ∘and 135 ∘). Because of taking the information of color components and spatial distribution into consideration, the matrix obtained not only reflects the texture correlation but also represents the color components of an image to some extent. 5. Texture retrieval algorithm For the purpose of effectively retrieving more similar images from the large digital image database, this paper firstly extracts the texture feature according to the co-occurrence matrix information, and then estimates the similarity of two color images. 5.1. Texture feature extraction For each color connectivity region of a color image, extract the cooccurrence matrix of the following color components: ➀ Red and green components of RGB color model. Because of linear correlation between the three components, there is no longer a need to consider the blue component. ➁ H component of HSV color model. ③ Gray component I, which is defined as follow:
I = 0:21R + 0:72G + 0:07B:
Divide a color image T into N × N blocks. Calculate the dominant color C(i, j) using mosaic dominate color extraction algorithm [8] for each block. For two arbitrary block T(i, j) and T(k, l)(| i − k| = 1 and |j − l| = 1) which satisfy the criterion of 8-connectivity, if their corresponding dominate color C(i, j) and C(k, l) are correlative, blocks T(i, j) and T(k, l) are color connectivity.
(a) Image region
(ii) Ri ∩ Rj = ϕ(i ≠ j, 1 ≤ i, j ≤ M); (iii) Ri and Rj (i ≠ j, 1 ≤ i, j ≤ M) are not color connected.
The texture feature extraction algorithm based on co-occurrence matrix is outlined in the following pseudo-code: (i) Divide a color image T into N × N blocks, and calculate the color connectivity region set S = {Ri}(1 ≤ i ≤ M) according to the criterion as definition 2 and 3 defined. (ii) Extract the R,G, B, and Icomponent of color image T, and each component is quantized into D = 8.
(b) Gray-level co-occurrence matrix
Fig. 1. Image region and corresponding gray-level co-occurrence matrix.
ð3Þ
Fig. 2. Query image.
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
(a)
(b)
(c)
33
(d)
(e)
Fig. 3. Retrieval results using proposed method.
(iii) For each color connectivity region, calculate the normalized cooccurrence matrix for each R, G, B, and I component, and then extract the following statistical values from each matrix: D
D
2
E = ∑ ∑ ½mði; jÞ ;
ð4Þ
i=1 j=1
D
D
2
I = ∑ ∑ ði−jÞ mði; jÞ;
ð5Þ
5.2. Similarity measure function For two given color images A and B, suppose their corresponding color co-occurrence matrices are: FA =[FRA,FGA,FHA,FIA]= {fA1,fA2, …,fA16} and FB =[FRB, FGB, FHB,FIB] = {fB1,fB2,…, fB16}. We integrate the similarity results of the four components by combining the associated similarity values. A combining similarity function is as follow: Sð A; BÞ = w1 DE ðFRA ; FRB Þ + w2 DE ðFGA ; FGB Þ + w3 jFHA −FHB j + w4 jFIA −FIB j;
i=1 j=1
D
ð10Þ D
P = − ∑ ∑ mði; jÞ log½mði; jÞ;
ð6Þ
i=1 j=1
D
D
H= ∑ ∑
i=1 j=1
Cor =
where DE denotes the Euclidean distance of the vector space, wi (1 ≤ i ≤ 4) are weights for each component and they are subject to 4
0 b wi b 1 and ∑ wi = 1.
mði; jÞ ; 1 + ði−jÞ2
ð7Þ
D 1 D ∑ ∑ ði−μi Þ j−μj Mði; jÞ σi σj i = 1 j = 1
ð8Þ
Where E denotes energy, I represents inertia, P denotes entropy, and H represents uniformity. They are the commonly used texture descriptors and can effectively reflect the texture feature [7]. μi and σi are the horizontal mean and variance, μj and σj are the vertical statistics [10]. Thus region Ri can be denoted by the feature vector : Fi =[FRi,FGi,FHi,FIi]={fi1,fi2,…,fi16}. (iv) Suppose the number of blocks in each color connectivity region is Bt, calculate the normalized feature vector of image T, which is defined as follow: F = ½FR ; FG ; FH ; FI = ff1 ; f2 ; …; f16 g where
i=1
One of the difficulties involved in integrating similarities using different components is the difference in the range of associated similarity value. h Such pffiffiffii as the normalized Euclidean distance is within the range of 0; 2 and the absolute value distance is within the range of [0, 2]. In order to handle this problem, we normalize the four similarity values to be within the same range of [0, 1]. Accordingly, formula (10) can be further written as follow: Sð A; BÞ = w1
pffiffiffi pffiffiffi 2−DE ðFRA ; FRB Þ 2−DE ðFGA ; FGB Þ pffiffiffi pffiffiffi + w2 2 2
+ w3
2−FHA −FHB 2−FIA −FIB + w4 : 2 2
S(A, B) reflects the similarity of texture feature of two color images. If S(A, B) = 1, the two images are completely similar, and if S(A, B) = 0, the two images are completely dissimilar. The bigger S(A, B), the smaller difference between the two images. 5.3. Color image retrieval
M
fk =
∑ fik NBt2
i=1
max ðfik Þ
ðk = 1; 2; …; 16Þ:
ð9Þ
1≤i≤m
(a)
(b)
(c)
Actually, the main part is usually located in the center of the whole image for most images. Based on this, we propose a two-level color image retrieval method, which takes account of the color-spatial
(d)
Fig. 4. Retrieval results using gray-level co-occurrence matrix algorithm.
(e)
34
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
(a)
(b)
(c)
(d)
(e)
Fig. 5. Retrieval results using histogram method.
feature, the main part and the retrieval speed in this paper. The detailed steps are as follows.
(1) First set the threshold Ti for the ith (i = 1, 2) level retrieval and the number m of images which are expected to obtain in the process of retrieval. (2) For each color image B in the image database, calculate the similarity S(A, B) between query color image A and target color image B (as shown in formula (9)). If S(A, Β) ≥ T1, mark the current image B as selected, otherwise go to the next target image. (3) Suppose n to be the total number of retrieved images which is marked as selected in step (2). If n b m, go to step (4); otherwise, let S denote the set of retrieved images in step (2), then carry out the next retrieval described as follows: ➀ Divide the query image A into 64 blocks on average and consist of 5 regions. We mark the center, bottom, left, right and top region as regions 0, 1, 2, 3 and 4 respectively. As different regions have different importance, we should set different weights for different regions. Let wi(i = 0, 1, 2, 3, 4) denote the weight of region i. Generally, w0 should be bigger than wj (j = 1, 2, 3, 4) owing to the fact that the main part is usually located in the center region, viz., region 0. Afterwards, the overall similarity between two images is obtained by linearly combining of these five region similarity values. ➁ For each image B in the set S, divide it into 5 regions like processing of the query image, then calculate the similarity between query image A and target image B for each region. If S (A, Β) ≤ T2, mark the current image B as selected, otherwise go to the next target image in the set S.
In order to test the proposed method, we compare it with general texture retrieval method based on gray-level co-occurrence matrix and color histogram. Let Fig. 2 be the query image, the retrieval results are shown in Figs. 3, 4, and 5, respectively. Comparing between the retrieval results we can find that for pure texture images, the retrieval results have similar properties such as smoothness, coarseness, regularity and distribution. In addition, the color information is employed in our proposed method, thus the retrieval images have similar color to some extent. In order to evaluate the performance of the proposed method, we compare it with general texture retrieval method based on gray-level co-occurrence matrix. Images belonging to above five types are chosen from the database as queries. Table 1 presents the precision results averaged over 50 queries. From Table 1 we can see that for most of any type of images, the retrieval precision obtained by our proposed method is superior to the gray-level co-occurrence matrix method, this is because the color information of the color image such as components and distribution is considered when calculating the co-occurrence matrix. Let Fig. 6 be the query image. We adjust wi(i = 0, 1, 2, 3, 4) according to the main part which we focus on of the query image. Let w0 = 0.8 and w1 = w2 = w3 = w4 = 0.05. Fig. 7 shows the results of color image retrieval method. From Fig. 5, we can see that the precision of the proposed method is better. In order to test the color image retrieval method, we compare the average precision of our method with that of general texture retrieval method based on gray-level co-occurrence matrix and the method in Ref. [10]. Table 2 presents the comparison results. From Table 2, we can see that our proposed method is superior to the gray-level cooccurrence matrix method and the method in Ref. [10]. 7. Conclusion
(4) Output the retrieval results. 6. Experimental results The experimental simulation is done on a PC with AMD Processor 3000+/2.00 GHz and Visual C++ 6.0 software. The image database contains 10,000 color images and they are gathered from public sources and represent natural scenes such as landscapes, flowers, cars, textures, people, etc.
In this paper a texture image retrieval method based on color co-occurrence matrix is presented. We firstly obtain the color connectivity region set for a color image and then extract the co-
Table 1 Precision comparisons between gray-level co-occurrence matrix algorithm and proposed method. Query image type
Gray-level co-occurrence matrix method
Proposed method
Landscapes Flowers Cars Textures People
92.0% 70.9% 78.2% 72.3% 65.8%
90.3% 83.3% 84.0% 81.5% 75.6%
Fig. 6. Query images.
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
35
Fig. 7. Retrieval results using color image retrieval.
Table 2 Average precision comparison between proposed method and another two methods.
Average precision
Gray-level co-occurrence matrix method
Method in Ref. [10]
Proposed method
75.84%
77.10%
82.94%
occurrence matrix for four orientation (horizontal 0 ∘, vertical 90 ∘ and diagonal 45 ∘ and 135 ∘) for each connectivity region. Due to the color information of the color image such as components and distribution is taken into consideration, the feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level co-occurrence matrix method and color histogram method. Especially for texture images, it provides better retrieval performance. Acknowledgments This research is supported by the National Natural Science Foundation of China (No: 60573172, 60973152), the Superior University Doctor Subject Special Scientific Research Foundation of China (No: 20070141014) and the Natural Science Foundation of Liaoning province (No: 20082165).
[8] L.S. Davis, M. Clearman, J.K. Aggarwal, An empirical evaluation of generalized cooccurrence Matrices, IEEE Trans. Pattern Anal. Mach. Intell. 3 (2) (1981) 214–221. [9] M. Hauta-Kasari, J. Parkkinen, R. Jaasklainen, et al., Generalized co-occurrence matrix for multispectral texture analysis, Proceedings of ICPR'96, vol. 2, 1996, pp. 785–789. [10] V. Arvis, C. Bebain, M. Berducat, et al., Classification of textures by multispectral cooccurrence matrix, Image Analysis Stereology 23 (1) (2004) 63–72.
Xingyuan Wang was born in Liaoning, China, in 1964. He received his B.S. degree in Application Physics and his M.S. degree in Optics from Tianjin University, Tianjin, China, in 1987 and 1992, respectively, and his Ph.D. degree in Computer Software and Theory from Northeastern University, Shenyang, China, in 1999. From 1999 to 2001, he was a Postdoctoral Fellow with the Department of Automation, Northeastern University. He is currently a Professor with the Faculty of Electronic Information & Electrical Engineering, Dalian University of Technology, Dalian, China. His research interests include biomedical information, computer graphics, image processing, and chaos control and synchronization.
ZhiFeng Chen is a postgraduate studying at Dalian University of Technology, P. R. China. Now he is interested in image processing and signal processing.
References [1] Y.J. Zhang, Visual Information Retrieval Based on Content, TsingHua University Press, Beijing, 2003 102–126. [2] J.D. Sun, X.M. Zhang, J.T. Cui, Image retrieval based on color distribution entropy, Pattern Recognition Letters 27 (10) (2006) 1122–1126. [3] D. Zhong, I. Defée, DCT histogram optimization for image database retrieval, Pattern Recognition Letters 26 (14) (2005) 2272–2281. [4] H. Nezamabadi-pour, E. Kabir, Image retrieval using histograms of uni-color and bi-color blocks and directional changes in intensity gradient, Pattern Recognition Letters 25 (14) (2004) 1547–1557. [5] L.T. Chen, C.C. Chang, Color image retrieval technique based on color features and image bitmap, Inf. Process. Manage. 43 (2) (2007) 461–472. [6] W.Y. Ma, H.J. Zhang, Benchmarking of image features for content-based retrieval. Proceedings of the 32nd Asilomar Conference on Signals, Systems and Computers. CA, Monterey 1 (1998) 253–257. [7] Yang Y B. Research and applications on the key techniques of content- based image retrieval. PhD Thesis. Nanjing: Nanjing University, 2003.
Jiaojiao Yun received B.S. degree in Information Management and Information Technology from Inner Mongolian Agricultural University, Hohhot, China, in 2008. She was born in Inner Mongolia, China, in 1985. Currently, she is working towards her M.S. degree at the Faculty of Electronic Information & Electrical Engineering, Dalian University of Technology, Dalian, China. Her research interests include image processing and fractal image compression.
Contents lists available at ScienceDirect
Computer Standards & Interfaces j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / c s i
An effective method for color image retrieval based on texture WangXing-yuan ⁎, ChenZhi-feng, YunJiao-jiao Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China
a r t i c l e
i n f o
Article history: Received 14 June 2009 Received in revised form 1 November 2010 Accepted 2 May 2011 Available online 7 May 2011 Keywords: Color image retrieval Texture feature extraction Co-occurrence matrix
a b s t r a c t This paper presents an effective color image retrieval method based on texture, which uses the color cooccurrence matrix to extract the texture feature and measure the similarity of two color images. Due to the color information such as components and distribution is also taken into consideration, the feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level co-occurrence matrix method and color histogram method, and it enhances the retrieval accuracy which is measured in terms of the recall and precision in the meanwhile. © 2011 Elsevier B.V. All rights reserved.
1. Introduction With the development of the multimedia technique, digital library and multimedia database including kinds of image databases increase rapidly. Traditional retrieval methods based on key words and description text can't fulfill our requirements, therefore some scholars put forward the content-based image retrieval (CBIR) technique [1]. In a CBIR system, images are automatically indexed by summarizing their visual contents through automatically extracted primitive features, such as shape, texture, color, size and so on. At present, CBIR has become the hotspot of research both at home and abroad, and many effective retrieval algorithms have been presented [2–10]. Ref. [9] described a new co-occurrence matrix based the approach for multispectral texture analysis and Ref. [10] analyzed three different approaches to color texture based on the classification of images. In this paper, we propose a color image retrieval method based on texture using color co-occurrence matrix. We firstly extract the texture feature according to the co-occurrence matrix information, and then estimate the similarity of two color images. Color information, such as components and distribution, are the important factors of color image. The feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level cooccurrence matrix method and color histogram method, and it enhances the retrieval accuracy which is measured in terms of the recall and precision in the meanwhile. This paper is organized as follows. Section 2 gives a brief introduction of the texture feature extraction. Section 3 describes the general concept of gray-level co-occurrence matrix. The proposed
⁎ Corresponding author. E-mail address: [email protected] (X. Wang). 0920-5489/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.csi.2011.05.001
color co-occurrence matrix is elaborated in Section 4. Section 5 presents our proposed color image retrieval method based on color co-occurrence matrix. Section 6 gives some experimental results comparing our method with gray-level co-occurrence matrix method and color histogram method in terms of retrieval accuracy. Finally, we draw a conclusion in Section 7. 2. Texture feature extraction An important feature of an image is texture. The texture descriptor provides measures of properties such as smoothness, coarseness, and regularity. The three principle approaches used in image processing to describe the texture of a region are statistical, structural, and spectral. Statistical approaches yield characterizations of textures as smooth, coarse, grainy and so on. Gray-level co-occurrence matrix is one of the most commonly used statistical approaches to extract texture feature of an image, such as energy, inertia, entropy and uniformity. In this paper, we propose a color co-occurrence matrix to describe texture feature of an image. 3. Gray-level co-occurrence matrix The gray-level co-occurrence matrix presents the joint probability that a pair of gray pixels with the position (Δx, Δy) will occur at the same time, where Δx and Δy are determined by a specified displacement δ between the pair of pixels and angle θ, and they are subject to Δx = δ cos θ and Δy = δsinθ. There is no fixed method to determine the values of δ and θ, and we usually consider the neighbors of a pixel. Because the distribution of gray intensities will be different at every orientation, 8 connected neighbors seem to be the best for computing gray-level co-occurrence matrix, and there will be δ = 1; θ = 0 ∘, 45 ∘, 90 ∘, 135 ∘ for the 8 connected neighbors.
32
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
Let L be the number of distinct gray levels of an image, thus the gray-level co-occurrence matrix CM(Δx, Δy)(i, j)or CM(δ, θ)(i, j) will be a L × L matrix whose element mij is the number of times that a point with gray level i appears (Δx, Δy) distance to a point with gray level j. 3.1. Definition 1
4.2. Definition 3 Divide the blocks of image T into color connectivity region set S = {Ri}(1 ≤ i ≤ M) according to the criterion of 8-connectivity, where M is the number of color connectivity regions if M
(i) ∪ Ri = T; i=1
Suppose an image with gray levels in the range [0, L − 1] is denoted by f(x, y), thus f(x, y) ∈ [0, L − 1]. For an arbitrary region R of the image, define S as a set of point pairs with specific spatial correlation. Thus the gray-level co-occurrence matrix can be defined as follow: CMðδ;θÞ ði; jÞ = cardf½ðx1 ; y1 Þ; ðx2 ; y2 Þ∈S j f ðx1 ; y1 Þ = i&f ðx2 ; y2 Þ = jg: ð1Þ Then we normalize it, accordingly, formula (1) can be further written as follow: CMðδ;θÞ ði; jÞ =
cardf½ðx1 ; y1 Þ; ðx2 ; y2 Þ∈S j f ðx1 ; y1 Þ = i&f ðx2 ; y2 Þ = jg ; cardðSÞ ð2Þ
where i ∈ [0, L − 1], j ∈ [0, L − 1], x 2 = x 1 + d cos θ, y 2 = y 1 + d sin θ, card(S) denotes the sum of all values in formula (1). For instance, considering an image region with eight gray levels, there will be f(x, y) ∈ [0, 7]. The corresponding gray-level co-occurrence matrix CM(1, 90∘) of the region shown in Fig. 1(a) can be obtained as shown in Fig. 1(b). By choosing the values of δ and θ, a separate co-occurrence matrix is obtained. From each co-occurrence matrix a number of textural features and the spatial information of relative position of different pixels can be extracted. For a region with coarse texture, the high values of its corresponding gray-level co-occurrence matrix are near the main diagonal, because the differences (i − j) are smaller there. In contrast, for a region with smooth texture, the high values of its corresponding gray-level co-occurrence matrix scatter everywhere, because the differences (i − j) are bigger there. 4. Color co-occurrence matrix 4.1. Definition 2
Obviously, S is unique. In Ref. [9], M. Hauta-Kasari et al. used the method that the spectral and spatial domains processed together based on the training set (the gray-level co-occurrence matrices were computed from the training set) in the multispectral texture segmentation. Different from this method, we note that the color connectivity regions of an image will not be regular rectangle, and the sizes of them are different. Therefore, for each color connectivity region Ri(1 ≤ i ≤ M) in set S, we obtain co-occurrence matrix for four orientation (horizontal 0 ∘, vertical 90 ∘and diagonal 45 ∘and 135 ∘). Because of taking the information of color components and spatial distribution into consideration, the matrix obtained not only reflects the texture correlation but also represents the color components of an image to some extent. 5. Texture retrieval algorithm For the purpose of effectively retrieving more similar images from the large digital image database, this paper firstly extracts the texture feature according to the co-occurrence matrix information, and then estimates the similarity of two color images. 5.1. Texture feature extraction For each color connectivity region of a color image, extract the cooccurrence matrix of the following color components: ➀ Red and green components of RGB color model. Because of linear correlation between the three components, there is no longer a need to consider the blue component. ➁ H component of HSV color model. ③ Gray component I, which is defined as follow:
I = 0:21R + 0:72G + 0:07B:
Divide a color image T into N × N blocks. Calculate the dominant color C(i, j) using mosaic dominate color extraction algorithm [8] for each block. For two arbitrary block T(i, j) and T(k, l)(| i − k| = 1 and |j − l| = 1) which satisfy the criterion of 8-connectivity, if their corresponding dominate color C(i, j) and C(k, l) are correlative, blocks T(i, j) and T(k, l) are color connectivity.
(a) Image region
(ii) Ri ∩ Rj = ϕ(i ≠ j, 1 ≤ i, j ≤ M); (iii) Ri and Rj (i ≠ j, 1 ≤ i, j ≤ M) are not color connected.
The texture feature extraction algorithm based on co-occurrence matrix is outlined in the following pseudo-code: (i) Divide a color image T into N × N blocks, and calculate the color connectivity region set S = {Ri}(1 ≤ i ≤ M) according to the criterion as definition 2 and 3 defined. (ii) Extract the R,G, B, and Icomponent of color image T, and each component is quantized into D = 8.
(b) Gray-level co-occurrence matrix
Fig. 1. Image region and corresponding gray-level co-occurrence matrix.
ð3Þ
Fig. 2. Query image.
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
(a)
(b)
(c)
33
(d)
(e)
Fig. 3. Retrieval results using proposed method.
(iii) For each color connectivity region, calculate the normalized cooccurrence matrix for each R, G, B, and I component, and then extract the following statistical values from each matrix: D
D
2
E = ∑ ∑ ½mði; jÞ ;
ð4Þ
i=1 j=1
D
D
2
I = ∑ ∑ ði−jÞ mði; jÞ;
ð5Þ
5.2. Similarity measure function For two given color images A and B, suppose their corresponding color co-occurrence matrices are: FA =[FRA,FGA,FHA,FIA]= {fA1,fA2, …,fA16} and FB =[FRB, FGB, FHB,FIB] = {fB1,fB2,…, fB16}. We integrate the similarity results of the four components by combining the associated similarity values. A combining similarity function is as follow: Sð A; BÞ = w1 DE ðFRA ; FRB Þ + w2 DE ðFGA ; FGB Þ + w3 jFHA −FHB j + w4 jFIA −FIB j;
i=1 j=1
D
ð10Þ D
P = − ∑ ∑ mði; jÞ log½mði; jÞ;
ð6Þ
i=1 j=1
D
D
H= ∑ ∑
i=1 j=1
Cor =
where DE denotes the Euclidean distance of the vector space, wi (1 ≤ i ≤ 4) are weights for each component and they are subject to 4
0 b wi b 1 and ∑ wi = 1.
mði; jÞ ; 1 + ði−jÞ2
ð7Þ
D 1 D ∑ ∑ ði−μi Þ j−μj Mði; jÞ σi σj i = 1 j = 1
ð8Þ
Where E denotes energy, I represents inertia, P denotes entropy, and H represents uniformity. They are the commonly used texture descriptors and can effectively reflect the texture feature [7]. μi and σi are the horizontal mean and variance, μj and σj are the vertical statistics [10]. Thus region Ri can be denoted by the feature vector : Fi =[FRi,FGi,FHi,FIi]={fi1,fi2,…,fi16}. (iv) Suppose the number of blocks in each color connectivity region is Bt, calculate the normalized feature vector of image T, which is defined as follow: F = ½FR ; FG ; FH ; FI = ff1 ; f2 ; …; f16 g where
i=1
One of the difficulties involved in integrating similarities using different components is the difference in the range of associated similarity value. h Such pffiffiffii as the normalized Euclidean distance is within the range of 0; 2 and the absolute value distance is within the range of [0, 2]. In order to handle this problem, we normalize the four similarity values to be within the same range of [0, 1]. Accordingly, formula (10) can be further written as follow: Sð A; BÞ = w1
pffiffiffi pffiffiffi 2−DE ðFRA ; FRB Þ 2−DE ðFGA ; FGB Þ pffiffiffi pffiffiffi + w2 2 2
+ w3
2−FHA −FHB 2−FIA −FIB + w4 : 2 2
S(A, B) reflects the similarity of texture feature of two color images. If S(A, B) = 1, the two images are completely similar, and if S(A, B) = 0, the two images are completely dissimilar. The bigger S(A, B), the smaller difference between the two images. 5.3. Color image retrieval
M
fk =
∑ fik NBt2
i=1
max ðfik Þ
ðk = 1; 2; …; 16Þ:
ð9Þ
1≤i≤m
(a)
(b)
(c)
Actually, the main part is usually located in the center of the whole image for most images. Based on this, we propose a two-level color image retrieval method, which takes account of the color-spatial
(d)
Fig. 4. Retrieval results using gray-level co-occurrence matrix algorithm.
(e)
34
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
(a)
(b)
(c)
(d)
(e)
Fig. 5. Retrieval results using histogram method.
feature, the main part and the retrieval speed in this paper. The detailed steps are as follows.
(1) First set the threshold Ti for the ith (i = 1, 2) level retrieval and the number m of images which are expected to obtain in the process of retrieval. (2) For each color image B in the image database, calculate the similarity S(A, B) between query color image A and target color image B (as shown in formula (9)). If S(A, Β) ≥ T1, mark the current image B as selected, otherwise go to the next target image. (3) Suppose n to be the total number of retrieved images which is marked as selected in step (2). If n b m, go to step (4); otherwise, let S denote the set of retrieved images in step (2), then carry out the next retrieval described as follows: ➀ Divide the query image A into 64 blocks on average and consist of 5 regions. We mark the center, bottom, left, right and top region as regions 0, 1, 2, 3 and 4 respectively. As different regions have different importance, we should set different weights for different regions. Let wi(i = 0, 1, 2, 3, 4) denote the weight of region i. Generally, w0 should be bigger than wj (j = 1, 2, 3, 4) owing to the fact that the main part is usually located in the center region, viz., region 0. Afterwards, the overall similarity between two images is obtained by linearly combining of these five region similarity values. ➁ For each image B in the set S, divide it into 5 regions like processing of the query image, then calculate the similarity between query image A and target image B for each region. If S (A, Β) ≤ T2, mark the current image B as selected, otherwise go to the next target image in the set S.
In order to test the proposed method, we compare it with general texture retrieval method based on gray-level co-occurrence matrix and color histogram. Let Fig. 2 be the query image, the retrieval results are shown in Figs. 3, 4, and 5, respectively. Comparing between the retrieval results we can find that for pure texture images, the retrieval results have similar properties such as smoothness, coarseness, regularity and distribution. In addition, the color information is employed in our proposed method, thus the retrieval images have similar color to some extent. In order to evaluate the performance of the proposed method, we compare it with general texture retrieval method based on gray-level co-occurrence matrix. Images belonging to above five types are chosen from the database as queries. Table 1 presents the precision results averaged over 50 queries. From Table 1 we can see that for most of any type of images, the retrieval precision obtained by our proposed method is superior to the gray-level co-occurrence matrix method, this is because the color information of the color image such as components and distribution is considered when calculating the co-occurrence matrix. Let Fig. 6 be the query image. We adjust wi(i = 0, 1, 2, 3, 4) according to the main part which we focus on of the query image. Let w0 = 0.8 and w1 = w2 = w3 = w4 = 0.05. Fig. 7 shows the results of color image retrieval method. From Fig. 5, we can see that the precision of the proposed method is better. In order to test the color image retrieval method, we compare the average precision of our method with that of general texture retrieval method based on gray-level co-occurrence matrix and the method in Ref. [10]. Table 2 presents the comparison results. From Table 2, we can see that our proposed method is superior to the gray-level cooccurrence matrix method and the method in Ref. [10]. 7. Conclusion
(4) Output the retrieval results. 6. Experimental results The experimental simulation is done on a PC with AMD Processor 3000+/2.00 GHz and Visual C++ 6.0 software. The image database contains 10,000 color images and they are gathered from public sources and represent natural scenes such as landscapes, flowers, cars, textures, people, etc.
In this paper a texture image retrieval method based on color co-occurrence matrix is presented. We firstly obtain the color connectivity region set for a color image and then extract the co-
Table 1 Precision comparisons between gray-level co-occurrence matrix algorithm and proposed method. Query image type
Gray-level co-occurrence matrix method
Proposed method
Landscapes Flowers Cars Textures People
92.0% 70.9% 78.2% 72.3% 65.8%
90.3% 83.3% 84.0% 81.5% 75.6%
Fig. 6. Query images.
X. Wang et al. / Computer Standards & Interfaces 34 (2012) 31–35
35
Fig. 7. Retrieval results using color image retrieval.
Table 2 Average precision comparison between proposed method and another two methods.
Average precision
Gray-level co-occurrence matrix method
Method in Ref. [10]
Proposed method
75.84%
77.10%
82.94%
occurrence matrix for four orientation (horizontal 0 ∘, vertical 90 ∘ and diagonal 45 ∘ and 135 ∘) for each connectivity region. Due to the color information of the color image such as components and distribution is taken into consideration, the feature obtained not only reflects the texture correlation but also represents the color information. As a result, our proposed method is superior to the gray-level co-occurrence matrix method and color histogram method. Especially for texture images, it provides better retrieval performance. Acknowledgments This research is supported by the National Natural Science Foundation of China (No: 60573172, 60973152), the Superior University Doctor Subject Special Scientific Research Foundation of China (No: 20070141014) and the Natural Science Foundation of Liaoning province (No: 20082165).
[8] L.S. Davis, M. Clearman, J.K. Aggarwal, An empirical evaluation of generalized cooccurrence Matrices, IEEE Trans. Pattern Anal. Mach. Intell. 3 (2) (1981) 214–221. [9] M. Hauta-Kasari, J. Parkkinen, R. Jaasklainen, et al., Generalized co-occurrence matrix for multispectral texture analysis, Proceedings of ICPR'96, vol. 2, 1996, pp. 785–789. [10] V. Arvis, C. Bebain, M. Berducat, et al., Classification of textures by multispectral cooccurrence matrix, Image Analysis Stereology 23 (1) (2004) 63–72.
Xingyuan Wang was born in Liaoning, China, in 1964. He received his B.S. degree in Application Physics and his M.S. degree in Optics from Tianjin University, Tianjin, China, in 1987 and 1992, respectively, and his Ph.D. degree in Computer Software and Theory from Northeastern University, Shenyang, China, in 1999. From 1999 to 2001, he was a Postdoctoral Fellow with the Department of Automation, Northeastern University. He is currently a Professor with the Faculty of Electronic Information & Electrical Engineering, Dalian University of Technology, Dalian, China. His research interests include biomedical information, computer graphics, image processing, and chaos control and synchronization.
ZhiFeng Chen is a postgraduate studying at Dalian University of Technology, P. R. China. Now he is interested in image processing and signal processing.
References [1] Y.J. Zhang, Visual Information Retrieval Based on Content, TsingHua University Press, Beijing, 2003 102–126. [2] J.D. Sun, X.M. Zhang, J.T. Cui, Image retrieval based on color distribution entropy, Pattern Recognition Letters 27 (10) (2006) 1122–1126. [3] D. Zhong, I. Defée, DCT histogram optimization for image database retrieval, Pattern Recognition Letters 26 (14) (2005) 2272–2281. [4] H. Nezamabadi-pour, E. Kabir, Image retrieval using histograms of uni-color and bi-color blocks and directional changes in intensity gradient, Pattern Recognition Letters 25 (14) (2004) 1547–1557. [5] L.T. Chen, C.C. Chang, Color image retrieval technique based on color features and image bitmap, Inf. Process. Manage. 43 (2) (2007) 461–472. [6] W.Y. Ma, H.J. Zhang, Benchmarking of image features for content-based retrieval. Proceedings of the 32nd Asilomar Conference on Signals, Systems and Computers. CA, Monterey 1 (1998) 253–257. [7] Yang Y B. Research and applications on the key techniques of content- based image retrieval. PhD Thesis. Nanjing: Nanjing University, 2003.
Jiaojiao Yun received B.S. degree in Information Management and Information Technology from Inner Mongolian Agricultural University, Hohhot, China, in 2008. She was born in Inner Mongolia, China, in 1985. Currently, she is working towards her M.S. degree at the Faculty of Electronic Information & Electrical Engineering, Dalian University of Technology, Dalian, China. Her research interests include image processing and fractal image compression.