Central pixel selection strategy based on local gray-value distribution by using gradient information to enhance LBP for texture classification

Expert Systems With Applications 120 (2019) 319–334

Contents lists available at ScienceDirect

Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa

Central pixel selection strategy based on local gray-value distribution by using gradient information to enhance LBP for texture classification Zhibin Pan∗, Xiuquan Wu, Zhengyi Li School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, 710049, PR China

a r t i c l e

i n f o

Article history: Received 1 March 2018 Revised 20 November 2018 Accepted 29 November 2018 Available online 30 November 2018 Keywords: Local binary pattern (LBP) Central pixel selection (CPS) strategy Local gray-value distribution Central pixel class Texture classification

a b s t r a c t Local binary pattern (LBP) has been successfully used in computer vision and pattern recognition applications, such as biomedical image analysis, remote sensing and image retrieval. However, the current LBP-based features, which assign a fixed sampling radius for all pixels in a single scale, completely ignore the fact that different central pixels actually have different local gray-value distributions and the proper sampling radius should be different for pixels. In this paper, we propose a novel and effective central pixel selection (CPS) strategy by using gradient information to classify central pixels of a texture image into different classes based on their local gray-value distributions. Then, we introduce this CPS strategy into the LBP framework and assign an adaptive sampling radius for each central pixel according to the class it belongs to. As a preprocessing step of LBP framework, this CPS strategy can also be integrated into any other LBP variants so as to extract more effective local texture features. Extensive experiments on five representative texture databases of Outex, UIUC, CUReT, UMD and ALOT validate the efficiency of the proposed central pixel selection (CPS) strategy, which can achieve almost 16% improvement over the original LBP and 1%–10% improvement compared with the best classification accuracy among other benchmarked state-of-the-art LBP variants. © 2018 Elsevier Ltd. All rights reserved.

1. Introduction Texture classification, as one of the major tasks in texture analysis, receives considerable attention and has a wide range of applications, such as biomedical image analysis, remote sensing and image retrieval. The main challenge of texture classification is to deal with the intra-class changes or variances, such as rotation, illumination, view point and scale. A good texture classification method should be highly discriminative, computationally efficient and robust to the intra-class changes or variances. Ojala, Maenpaa, Pietikainen, and Viertola (2002) proposed to use the Local Binary Pattern (LBP) operator, which is invariant to monotonic gray-value changes and has low computational complexity, to extract local texture feature information. The generation process of LBP is shown in Fig. 1, in which a central pixel gc is firstly compared with its neighbors. The LBP binary code (0 or 1) of this central pixel gc can be obtained, then this binary LBP code is transformed into a decimal number to represent the local structural pattern. With the aim of achieving rotation invariance as well as reducing the dimensions of the final feature histograms,

∗

Corresponding author. E-mail address: [email protected] (Z. Pan).

https://doi.org/10.1016/j.eswa.2018.11.041 0957-4174/© 2018 Elsevier Ltd. All rights reserved.

Ojala et al. (2002) also proposed the rotation invariant uniform LBP pattern, which is represented by LBPriu2 . Because of its low computational complexity and high efficiency, LBP has been applied to many areas such as face image analysis, texture classification, face recognition, and biomedical image analysis (Lu, Jiang, & Kot, 2018; Pan, Wu, Li, & Zhou, 2017; Wang et al., 2017). To improve the original LBP, lots of LBP variants (Brahnam, Jain, Nanni, & Lumini, 2014; Guo, Wang, Zhou, & You, 2015; Nanni, Lumini, & Brahnam, 2012; Pan, Li, Fan, & Wu, 2017) have been proposed in the literatures. To achieve rotation invariance, Zhao, Huang, and Jia (2012) proposed local binary count (LBC), which counts the occurrence number of binary code “1” as the LBC code. Local ternary pattern (LTP) (Tan & Triggs, 2010) extended original LBP to 3-valued codes. As a result, LTP was more robust to noise. In Local Vector Quantization Pattern (LVQP) (Pan, Fan, & Zhang, 2015), vector quantization technique was used to quantize the difference vector between the center pixel gc and its neighborhood pixels, and each different structural pattern consequently best matched a unique codeword via searching a pattern codebook that was trained in advance. Guo, Zhang, and Zhang (2010) combined the conventional LBP with the measures of intensity difference together with central grayvalue and named it as completed LBP (CLBP). Liu, Long, Fieguth, Lao, and Zhao (2014) proposed the binary invariant and noise tol-

320

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

variants in detail. Section 4 discusses experimental results and the conclusion is given in Section 5. 2. Related work Fig. 1. The generation process of original LBP.

2.1. Local binary pattern (LBP) LBP is extracted by comparing the central pixel gc with its neighbors gp , and its original coding method can be described as follows:

erant (BRINT), in which the arc-based averaging was used to fix the sampling points in neighborhood to be a constant of eight to achieve fast and efficient classification and strong robustness. Liao, Law, and Chung (2009) observed that some LBP patterns with high frequency of occurrence were more robust and preserved more information than the LBP patterns with low frequency of occurrence and proposed the learning-based Dominant LBP (DLBP). Liu et al. (2016) proposed the median robust extended local binary pattern (MRELBP), which compared regional image medians rather than raw pixel intensities to achieve robustness to changes of scale, rotation and illumination. Basically, LBP and its variants all belong to a class of methods known as Histograms of Equivalent Patterns (Fernández, Álvarez, & Bianconi, 2013), which describes a comprehensive and unifying framework with a clear and unambiguous mathematical definition for texture analysis. Besides those LBPbased methods, a large number of CNN-based texture representation methods (Cusano, Napoletano, & Schettini, 2016; Liu, Fieguth, Guo, Wang, & Pietikäinen, 2017; Napoletano, 2017) have been proposed in recent years since the record breaking image classification result (Krizhevsky, Sutskever, & Hinton, 2012) achieved in 2012. A key to the success of CNNs is their ability to leverage large labeled datasets to learn high quality features. Although the original LBP and lots of LBP variants have achieved impressive classification accuracy on representative texture databases, their working mechanisms still need further investigations. The current existing LBP-based features, which assign a fixed sampling radius for all pixels in a single scale to extract local texture feature, completely ignore the fact that different central pixels actually have different local gray-value distributions and the proper sampling radius should be different for pixels. Specifically, this fixed sampling radius may be too big to capture features in a fast-changing local area or too small to describe features in a smoothly distributed local area. Therefore, it is intuitive that adaptive sampling radius should be considered and assigned to each central pixel according to its own local gray-value distribution. In this way, the extracted local texture feature can become more robust to the intra-class changes existed in texture images. Based on this consideration, in this paper we propose a novel central pixel selection (CPS) strategy based on local gray-value distribution of central pixel, which classifies central pixels in a texture image into different classes. Considering the feature extraction process of LBP, it is reasonable that small sampling radius should be assigned to the fast-changing local area while big sampling radius should be assigned to the smoothly distributed local area. It is to say that the adaptive sampling radius for each central pixel gc is directly related to its local gray-value distribution. Inspired by this analysis, we use this CPS strategy to divide central pixels with different local gray-value distributions into different classes before the feature extraction step of LBP. By introducing CPS strategy into LBP framework and utilizing this CPS strategy as a preprocessing step of LBP, adaptive sampling radius can be assigned to each central pixel so as to extract more effective local features and improve classification performance. The rest of this paper is organized as follows. Section 2 briefly reviews LBP and its related literatures. Section 3 presents the proposed CPS strategy and its application in LBP framework and LBP

LBPP,R (gc ) =

P−1 



s ( g p − gc ) 2 , s ( x ) = p

p=0

1, 0,

x≥0 x<0

(1)

where gc is the value of the central pixel and gp (p = 0,1,…, P−1) represents the value of the sampled neighbor pixel on a circle of radius R, and P is the number of sampled neighbors. The neighbors that do not fall at the integer positions should be estimated by bilinear interpolation. Ojala et al. (2002) found that certain local binary patterns occupied the vast majority of LBP and named them as uniform patterns. A uniformity measure of U is introduced to count the occurrence number of spatial transition (i.e., bitwise 1/0 or 0/1 changes), and its definition is as follows

U (LBPP.R ) = |s(gP−1 − gc ) − s(g0 − gc )| +

P−1 

|s(g p − gc ) − s(gP−1 − gc )|

(2)

p−1

Uniform patterns of LBP are defined as those patterns that satisfy the condition of U ≤ 2. Following this definition, the rotation invariant uniform LBP is defined as: riu2 LBPP,R ( gc ) =

⎧  ⎨P−1

s(g p − gc ), ⎩Pp=0+ 1,

i f U (LBPP,R (gc )) ≤ 2

(3)

otherwise

The rotation invariant and uniform LBP pattern is usually named as LBPriu2 . 2.2. LBP variants To improve the discriminative power of the original LBP, many LBP variants proposed to combine several different and complementary types of descriptors. Guo et al. (2010) proposed the completed modeling of LBP (CLBP) which was represented by a joint of three complementary components: the signs (CLBP_S), the magnitudes (CLBP_M) and the central pixel (CLBP_C). After that, extended local binary pattern (ELBP) (Liu, Zhao, Long, Kuang, & Fieguth, 2012) also introduced four LBP-like descriptors— center intensity based LBP (ELBP-CI), neighborhood intensity based LBP (ELBP-NI), radial difference based LBP (ELBP-RD) and angular difference based LBP (ELBP-AD). Other important examples include the local binary circumferential and radial derivative pattern (CRDP) (Wang, Bichot, Li, & Li, 2017), the median robust extended LBP (MRELBP) (Liu et al., 2016) and the Training-Based Gradient LBP (Ji, Ren, Liu, & Pu, 2017). However, although these methods have good discrimination capability with combination of complementary features, their higher feature dimension will lead to great challenges in computational cost and storage. In addition, learning-based methods or learning-based texton dictionary were also proposed to improve the original LBP. Such methods include the Dominant LBP (Liao et al., 2009) which learned the LBP patterns with high frequency of occurrence to build dominant texture feature. The sorted consecutive LBP (scLBP) (Ryu, Hong, & Yang, 2015) combined sorted consecutive patterns

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

321

Fig. 2. The process of gradient-based pixel classification using central pixel selection (CPS) strategy.

and the dictionary learning based on kd-tree (Bentley, 1975) to encode types of patterns. The scale selective LBP (SSLBP) (Guo et al., 2015) also learned the dominant patterns across different scales for texture classification. Nevertheless, despite their high classification accuracy, these methods need to use a learning stage to find the optimal parameters which will lead to increasing computational cost and storage. 3. Proposed method 3.1. Central pixel selection (CPS) strategy by using gradient Generally, gradients can be used to extract information of changes of a local region from images. In the gradient image, each pixel represents a gradient value that measures the local change around it. Based on this consideration, we utilize gradient to evaluate local gray-value change or distribution of each pixel. Small gradient value indicates that this pixel has a smooth local distribution while big gradient value indicates that this pixel is in a fastchanging local area. In this paper, we propose a central pixel selection (CPS) strategy to classify pixels into different classes according to their gradient values. In this way, pixels belonging to the same class have similar local gray-value distributions and they should be treated equally. However, pixels of different classes should be treated differently, which can enhance the classification accuracy performance effectively. To generate gradient image, Sobel operator (Kanopoulos, Vasanthavada, & Baker, 2002), which has been widely used in image processing and computer vision, can be used for computing the gradient. If we denote I as the original image, Gx and Gy as two images which correspond to the horizontal and vertical gradient of the original image I respectively, the computations of gradient are as follows:



Gx =

−1 −2 −1

0 0 0



+1 +2 ∗ I, +1



Gy =

+1 0 −1

+2 0 −2



+1 0 ∗I −1

(4)

where “∗ ” denotes the 2-dimensional convolution operation. For pixels in the original image I, their final gradient image can be defined as:



G=

G2x + G2y

(5)

To realize our proposed CPS strategy, the first step is to compute gradient image G from original image I by using Sobel operator. The second step is to sort all pixels of original image I in a descending order according to their gradient values in the gradient image G. The final step is to divide these pixels into M different classes with (M-1) thresholds. The whole process of central pixel classification is shown in Fig. 2.

Fig. 3. Pixels in the original image are separated into two classes of pixels_BigG and pixels_SmallG by CPS strategy using gradient thresholding.

As explained above, if pixels in the original image I are classified into the same class, it implies that they have the similar local gray-value distributions and can be treated equivalently to extract local features. But if pixels are separated into different classes, it means that their local gray-value distributions are different from each other. As a result, these pixels should be treated differently. As an example shown in Fig. 3, pixels in the original image are separated into two classes by the CPS strategy, and the classification threshold here is selected as the mean value of the gradients of all the pixels over the whole image. In the separated image of Class1 and Class2 , pixels that belong to this class will keep their gray-values unchanged, but if pixels don not belong to this class, their gray-values will be set to 0. As analyzed above, Class1 is made up of pixels with big gradients (pixels_BigG) and Class2 consists of pixels with small gradients (pixels_SmallG). It is clear that when extracting local features of LBP in original image, sampling radius R_BigG for pixels_BigG in Class1 should be smaller while the sampling radius R_SmallG for pixels_SmallG in Class2 should be bigger. 3.2. Introducing CPS strategy into LBP framework LBP, as an effective operator, is widely used to extract local feature of a central pixel. However, in the LBP framework, all central pixels in a texture image are assigned a fixed sampling radius when extracting local features. This completely ignores the fact that different central pixels in fact have different local gray-value distributions. Intuitively, when describing and extracting the local feature of a central pixel, the proper sampling radius for the central pixel is related to its local gray-value distributions. The smaller the gradient value of a pixel is, the bigger its sampling radius

322

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

Fig. 4. Classification accuracy results respectively using pixels with small gradient (pixels_SmallG) and pixels with big gradient (pixels_BigG) on (a) Outex_TC12_0 0 0 database and (b) Outex_TC12_001database.

should be. That is to say, pixels with small gradient values (pixels_SmallG) should have bigger sampling radius while pixels with big gradient values (pixels_BigG) should have smaller sampling radius. In our observation experiment, we choose two databases of Outex_TC12_0 0 0 and Outex_TC12_001 (Ojala et al., 2002). We use CPS strategy to divide pixels of a texture image into two classes: pixels with small gradient (pixels_SmallG) and pixels with big gradient (pixels_BigG). Then, we independently test the LBP performance of pixels from these two classes. The classification accuracy results on five sampling radiuses from R = 1 to R = 5 can be observed in Fig. 4. The results in Fig. 4 indicate that pixels_BigG achieves much better classification results compared with pixels_SmallG when the sampling radius R is rather small (R = 1 or R = 2). On the contrary, when the sampling radius R becomes big (R = 3, R = 4 or R = 5), pixels_SmallG achieves higher classification accuracy than pixels_BigG. It testifies that proper sampling radius for pixels in different classes needs to be different. More specifically, the adaptive sampling radius for pixels_BigG should be smaller, such as R = 1 or R = 2. Correspondingly, the adaptive sampling radius for

pixels_SmallG should be bigger, such as R = 3, R = 4 or R = 5. Furthermore, when the sampling radius R increases to 5, the classification accuracies of both pixels_SmallG and pixels_BigG decrease greatly. The reason is that the relevance between a central pixel and its neighbor pixels at R = 5 becomes weaker. So even though the proper sampling radius for pixels_SmallG needs to be bigger, it should have an upper bound and we set it as R = 5. Taking these analyses into consideration, we can integrate the novel CPS strategy into LBP framework now. In the LBP framework, the first step is to use our proposed CPS strategy to classify all the pixels in an original texture image I into M different classes by gradient thresholding, where M denotes the number of classes. After all pixels are classified, the second step is to assign different adaptive sampling radiuses to different classes. Pixels that are grouped into the same class will share the same sampling radius, but the sampling radius differs among classes. Based on this CPS strategy, all pixels will use their own adaptive sampling radiuses to extract more effective local features in the following LBP generation step. Once local features of all pixels of an original texture image I are extracted by LBP operator, the final step is to cascade M feature

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

323

Fig. 5. The process of using CPS strategy and its application in LBP framework with M = 2.

histograms independently obtained from M classes to construct the final feature vector of an image for texture classification. Moreover, experimental results of our own observation experiments indicate that the increasement of the number of classes M is not helpful for the improvement of classification performance. On the contrary, it leads to overwhelmingly increased histogram lengths and high computational cost. Therefore, in this paper, the number of pixel class M is set to be 2. In this case, pixels in a texture image are divided into 2 classes, and the threshold is taken as the mean value of gradients of all pixels in an original image, and the whole process is shown in Fig. 5. 3.3. Dissimilarity measure In this paper, we utilize the chi-square statistics (Ojala, Pietik, Inen, & Topi, 2002) as the dissimilarity between two histograms. If H = {hi } and K = {ki } (i = 1, 2, …, B) denote two histograms, then the chi-square statistics can be calculated as follows:

dχ 2 (H, K ) =

B  (hi − ki )2 hi + ki

(6)

in the new CPS strategy manner. The subscript CPS refers to the use of CPS strategy in LBP framework or its variants. According to the results of observation experiment in Fig. 4, the adaptive sampling radius for pixels_BigG (R_BigG) should be smaller, while the adaptive sampling radius for pixels_SmallG (R_SmallG) on the contrary should be bigger. Besides, an upper bound of R should also be set when selecting sampling radius and in our experiments the maximum sampling radius of R is up to 5. And when applying the proposed CPS strategy, the number of classes M is set to be 2. In the case of M = 2, pixels are divided into 2 classes of pixels_BigG and pixels_SmallG, and the threshold is taken as the average value of gradients of all the pixels in an image. Accordingly, the two parameter pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] represent the sampling radius and sampling numbers for pixels of these 2 classes respectively. We select R_SmallG from the set of {3, 4, 5}, and R_BigG from the set of {1, 2, 3}. Meanwhile, the sampling numbers of both P_BigG and P_SmallG are fixed to be 8. 4.1. Analysis of filter size of the Sobel operator

i=1

All the texture classification methods in this paper use the nearest neighborhood classifier for classification. 4. Experimental results To investigate the effectiveness of our proposed new CPS strategy for LBP framework, we carried out a series of experiments on five representative texture databases: the Outex database (Ojala et al., 2002), UIUC database (Lazebnik, Schmid, & Ponce, 2005), Columbia-Utrecht Reflection and Texture (CUReT) database (Dana, Nayar, Ginneken, & Koenderink, 1997), the University of Maryland (UMD) (Xu, Ji, & Fermuller, 2006) database and the Amsterdam Library of Textures (ALOT) database (Targhi, Geusebroek, & Zisserman, 2008). As a preprocessing module, the proposed CPS strategy can also be directly applied in any other LBP variants. Therefore, in this paper, we select five representative LBP variants: the LTP algorithm (Tan & Triggs, 2010), the CLBP algorithm (Guo et al., 2010), the CRDP method (Wang, Bichot, et al., 2017), the BRINT descriptor (Liu et al., 2014), and the MRELBP descriptor (Liu et al., 2016) to evaluate the effectiveness of using CPS strategy. These five methods will be compared in both the original way and

As introduced in Section 3.1, the Sobel operator is used to measure the local gray-value distributions of central pixels by computing their gradient values. The size of filter mask in the Sobel operator is usually an odd number, because masks of even sizes do not have a center of symmetry and they are inconvenient to implement. Therefore, the filter masks can be of size 3 × 3, 5 × 5, 7 × 7 and so on. Generally, the smallest filter mask of size 3 × 3 is adopted by researchers thanks to its effectiveness and low computational complexity. In our observation experiment, the size of filter mask of the Sobel operator is enlarged into 5 × 5, because the upper bound of the sampling radius is set as R = 5 in our proposed method. Then, we evaluate the experimental results of the original LBP by using the proposed CPS strategy manner with two different filter masks of size 3 × 3 and size 5 × 5 on Outex database (Ojala et al., 2002). Experimental results are listed in Table 1 as below. Experimental results of Table 1 show that the filter size of Sobel operator used in our proposed CPS strategy almost makes no difference to the final classification results. In fact, the local grayvalue distributions of central pixels obtained by Sobel operator with filter masks of size 3 × 3 are almost consistent with those ob-

324

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334 Table 1 Classification accuracy (%) of LBPCPS Method

LBPCPS

(3×3)

LBPCPS

(5×5)

(3×3)

and LBPCPS

Parameter pair(s)(R, P)

[(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

(5×5)

on TC10 and TC12.

TC10

96.33 96.30 96.09 98.13 94.45 96.72 96.17 96.09 96.09 97.58

TC12

Histogram length

“t”

“h”

91.09 91.48 89.77 92.43 86.92 92.11 92.57 89.77 92.69 94.47

89.98 91.64 90.74 92.48 86.97 91.60 91.81 90.74 92.64 92.41

20 20 20 20 20 20 20 20 20 20

Fig. 6. Samples of the 24 classes of texture images from the Outex database.

tained by Sobel operator with filter masks of size 5 × 5. Therefore, by using our proposed CPS strategy, they achieve similar classification accuracy results under the condition of the same parameter selection pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] in most cases. However, the time complexity of Sobel operator with filter masks of size 5 × 5 is much higher than that of Sobel operator with filter masks of size 3 × 3. As a result, Sobel operator with filter masks of size 3 × 3 is suggested as best choice and adopted in our experiments of this paper. 4.2. Experimental results on Outex database The Outex database normally includes two test suites: Outex_TC_0 0 010 (TC10) and Outex_TC_0 0 012 (TC12) as shown in Fig. 6. Both TC10 and TC12 contain 24 classes of texture images captured under three illuminations (“inca”, “tl84” and “horizon” ) and nine rotation angles (0°, 5°, 10°, 15°, 30°, 45°, 60°, 75° and 90°). There are twenty images for each rotation angle under a given illumination condition. The images of rotation angle 0° are used as the training data, and the other eight rotation angles are used for test in TC10 database with illumination “inca”. For TC12 dataset, all the texture images captured under illumination “tl84” or “horizon” are used as the test data. Table 2 lists the experimental results of LBP and LBPCPS on Outex database. The parameter pair (R, P) of LBP is the sampling radius and its corresponding sampling numbers. Since the CPS strategy divides pixels into 2 classes and assigns different sampling radiuses to different classes, the parameter pair (R, P) extends to two parameter pairs that are represented by [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] in the case of M = 2. It is clear that the CPS strategy makes LBP achieve much better classification accuracy. When M = 2, pixels in original image are divided into 2 classes, the parameter pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] = [(3, 8); (4, 8)] can reach the best accura-

cies of 98.13%, 92.43% and 92.48% while the best accuracies of LBP are 96.48%, 86.85% and 84.03%. Furthermore, with the CPS strategy, the sampling numbers of P_BigG and P_SmallG are both 8 and the final histogram length is therefore fixed to be 20, which achieves a rather better balance between high classification accuracy and low dimensionality. Tables 3–7 list the experimental results of five LBP variants and their using of CPS strategy on Outex database. Here we choose LTP, CLBP, CRDP, BIRNT and MRELBP for comparison, which have impressive classification accuracy performances. When M = 2, we have five parameter pairs selection of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] for each method, and these five parameter pairs correspond to five different sampling radius selection schemes. Introducing CPS strategy to LBP variants means that adaptive sampling radius will be assigned to each pixel in the following LBP feature extraction step, in which way the classification performances of these methods will all be strengthened. As shown in Tables 3–7, the CPS strategy efficiently enhances the classification performance of these LBP variants when it is used as a preprocessing step before extracting LBP features. Since BRINT and MRELBP fix the sampling numbers P to be 8 for different sampling scales, their histogram lengths are also fixed under different scales. To better evaluate the effectiveness of introducing CPS strategy, we compare BRINT and MRELBP in both single scale and multiple scale. Multiple scale means to cascade their feature histograms from different single scales, and parameter pairs of [(R1, P1) + (R2, P2)] represent the cascade of two single scales of R1 and R2. As shown in Table 6, when the histograms length are the same, the best accuracies of BRINT are 98.31%, 96.02% and 96.00% in the multiple scale of R = 1 and R = 3 while the best accuracies of BRINTCPS can reach 99.22%, 96.13% and 96.74%. And the same comparison can be made for MRELBP as shown in Table 7.

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

325

Table 2 Classification accuracy (%) of LBP and LBPCPS on TC10 and TC12. Method

LBP

LBPCPS

Parameter pair(s) (R, P)

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

TC10

TC12

84.81 89.40 95.07 96.48 94.35 96.33 96.30 96.09 98.13 94.45

Histogram length

“t”

“h”

65.46 82.26 85.04 86.85 82.11 91.09 91.48 89.77 92.43 86.92

63.68 75.20 80.78 84.03 80.72 89.98 91.64 90.74 92.48 86.97

10 18 26 26 26 20 20 20 20 20

Table 3 Classification accuracy (%) of LTP and LTPCPS on TC10 and TC12. Method

Parameter pair(s) (R, P)

TC10

TC12 “t”

“h”

LTP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8); (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

94.56 96.11 98.64 99.30 99.04 96.93 96.74 96.51 97.76 94.82

72.35 85.20 92.59 89.95 89.75 89.26 89.12 88.03 89.58 86.81

71.93 85.87 91.52 88.84 87.95 86.92 90.32 90.49 90.07 87.29

LTPCPS

Histogram length

20 36 52 52 52 40 40 40 40 40

Table 4 Classification accuracy (%) of CLBP and CLBPCPS on TC10 and TC12. Method

CLBP

CLBPCPS

Parameter pair(s) (R, P)

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

TC10

96.51 98.70 98.93 99.17 98.78 98.20 98.67 98.91 99.38 99.32

TC12

Histogram length

“t”

“h”

90.09 93.56 95.28 93.68 91.11 93.73 94.86 94.65 95.63 94.93

92.13 93.66 94.42 92.22 90.28 94.77 95.44 95.12 95.53 94.81

200 648 1352 1352 1352 400 400 400 400 400

Table 5 Classification accuracy (%) of CRDP and CRDPCPS on TC10 and TC12. Method

Parameter pair(s) (R, P)

TC10

TC12 “t”

“h”

CRDP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

95.57 97.66 98.67 95.49 92.14 95.73 98.18 98.15 98.39 97.50

90.72 93.59 94.58 85.97 81.64 89.68 95.30 94.51 95.05 93.29

92.92 93.84 92.36 89.81 85.37 89.54 94.49 93.87 94.40 93.06

CRDPCPS

Although multiple scale enhances the classification performance of BRINT and MRELBP, it can not extract and combine local feature information effectively as the CPS strategy does. In fact, multiple scale LBP scheme and CPS strategy are two different ideas. Multiple scale LBP scheme directly cascades multiple feature histograms extracted independently from different single scales. As

Histogram length

200 648 1352 1352 1352 400 400 400 400 400

we analyzed above, there is a limitation in single scale that all the pixels are assigned the same sampling radius without considering their local gray-value distributions. This means that directly using feature histogram extracted from single scale can not guarantee that adaptive sampling radius is assigned to each pixel, which makes it difficult to extract local features efficiently.

326

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334 Table 6 Classification accuracy (%) of BRINT and BRINTCPS on TC10 and TC12. Method

BRINT

Parameter pair(s) (R, P)

Single scale

Multiple scale BRINTCPS

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

TC10

91.87 96.43 96.04 94.11 95.16 97.34 98.31 97.99 98.83 98.91 98.10 99.22

TC12

Histogram length

“t”

“h”

86.46 93.38 94.47 91.11 92.11 94.14 96.02 94.77 96.23 95.39 94.81 96.13

88.50 93.98 94.40 93.20 92.29 94.38 96.00 95.23 96.46 96.67 95.09 96.74

144 144 144 144 144 288 288 288 288 288 288 288

Table 7 Classification accuracy (%) of MRELBP and MRELBPCPS on TC10 and TC12. Method

MRELBP

Parameter pair(s) (R, P)

Single scale

Multiple scale MRELBPCPS

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

Fig. 7. Samples of the 25 classes of texture images from the UIUC database.

4.3. Experimental results on UIUC database The UIUC texture database that includes 25 classes and 40 images in each class is shown in Fig. 7. The database contains texture images under significant viewpoint variations, and the resolution of each image is 640 × 480. In our texture classification experiments, N training images are randomly chosen from each class while the remaining (40 − N) images are used as the test images. In this paper, we set N to be 5, 10, 15 and 20 respectively. Meanwhile,

TC10

91.17 92.50 98.23 97.03 93.78 92.79 96.12 97.89 98.54 97.97 98.62 97.97

TC12

Histogram length

“t”

“h”

84.95 90.25 97.73 94.54 89.68 89.49 95.53 96.46 96.76 95.79 96.09 95.93

87.01 90.46 96.25 93.38 89.28 90.39 95.02 96.50 97.25 95.07 95.97 96.18

200 200 200 200 200 400 400 400 400 400 400 400

the random selection is implemented 100 times independently to compute the average classification accuracy, which is used as the final experimental results. Table 8 lists the experimental results of LBP and LBPCPS on UIUC database. It can be observed that our proposed CPS strategy significantly enhances the classification performance of the original LBP. When M = 2, pixels in an original image are divided into 2 classes, the parameter pairs [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] = [(3, 8); (5, 8)] can reach the best accuracies of 85.95%, 82.79%, 77.85% and 67.30% while the best accuracies of LBP are only 70.12%, 66.41%, 60.32% and 49.89%. The results demonstrate that proposed CPS strategy, as a preprocessing step before feature extraction, is very discriminative and powerful for texture classification. Tables 9–13 list the experimental results of five LBP variants and their using of CPS strategy on UIUC database. As shown in Table 9, the best classification accuracies of LTP are 85.77%, 82.85%, 77.57%, 66.59% in the original form. However, with the application of our proposed CPS strategy, LTP achieves much better classification accuracy. More precisely, the best classification accuracies of LTPCPS can reach 90.58%, 87.88%, 83.36%, 74.30%, which achieve an average improvement of more than 6%. The same findings can also be found in Tables 10 and 11, which shows that the CPS strategy efficiently enhances the classification accuracy performance of CLBP and CRDP. As for BRINT and MRELBP, we also compare these two methods both in single scale and in multiple scale as shown in Tables 12 and 13. The best accuracies of BRINT are 88.57%, 86.08%, 82.38% and 73.21% in the multiple scale of [(1, 8) + (2, 8)] while BRINTCPS can reach 92.25%, 90.18%, 87.29% and 79.26% with the same histogram length. As for MRELBP, its best accuracies are 92.96%, 90.98%, 87.80% and 80.28% in the single scale of R = 4, but when introducing CPS strategy, the MRELBPCPS can reach 95.17%, 93.63%,

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

327

Table 8 Classification accuracy (%) of LBP and LBPCPS on UIUC. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

LBP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

55.06 60.53 64.70 68.29 70.12 79.24 80.33 82.43 85.36 85.95

52.32 56.46 60.80 64.14 66.41 75.93 77.10 79.45 82.40 82.79

47.69 51.53 54.83 58.16 60.32 70.59 71.57 74.08 77.45 77.85

40.14 41.69 44.72 47.65 49.89 59.33 60.41 63.39 65.89 67.30

10 18 26 34 34 20 20 20 20 20

LBPCPS

Table 9 Classification accuracy (%) of LTP and LTPCPS on UIUC. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

LTP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

68.54 78.73 81.91 82.59 85.77 85.22 85.59 88.00 88.99 90.58

65.01 75.66 78.86 76.69 82.85 82.33 82.94 85.12 86.25 87.88

60.44 70.07 73.05 73.97 77.57 77.82 77.89 80.62 82.32 83.36

51.29 59.65 61.82 62.75 66.59 67.89 67.41 71.11 71.76 74.30

20 36 52 52 52 40 40 40 40 40

LTPCPS

Table 10 Classification accuracy (%) of CLBP and CLBPCPS on UIUC. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

CLBP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

87.48 91.14 90.94 90.53 92.30 91.23 92.02 92.50 92.70 93.71

85.92 89.35 89.20 88.95 90.22 89.39 90.13 91.21 91.07 92.29

82.51 86.14 85.90 85.09 86.46 86.50 86.98 88.23 87.95 89.08

74.84 78.93 78.11 76.73 78.03 78.84 79.77 80.73 80.57 81.56

200 648 1352 1352 1352 400 400 400 400 400

CLBPCPS

Table 11 Classification accuracy (%) of CRDP and CRDPCPS on UIUC. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

CRDP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

79.01 83.19 84.39 83.74 76.25 88.10 89.33 90.06 90.38 91.15

76.39 80.1 81.02 80.46 72.55 86.05 87.13 87.93 88.24 89.30

71.49 75.23 76.03 75.2 66.62 82.34 83.46 84.22 84.44 85.54

61.91 65.01 65.53 63.93 54.92 73.52 74.78 75.78 75.84 76.87

200 648 1352 1352 1352 400 400 400 400 400

CRDPCPS

Table 12 Classification accuracy (%) of BRINT and BRINTCPS on UIUC. Method BRINT

Single scale

Multiple scale BRINTCPS

Parameter pair (R, P)

20

15

10

5

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

79.20 83.71 86.68 88.35 88.57 84.66 87.76 89.26 89.83 90.81 91.28 92.25

76.12 81.68 83.93 85.73 86.08 81.74 84.56 86.98 88.28 89.06 89.52 90.18

70.95 77.15 79.76 81.27 82.38 77.08 80.45 83.22 84.85 85.88 86.34 87.29

61.77 67.98 70.53 71.87 73.21 67.63 70.83 75.28 77.11 77.63 78.13 79.26

144 144 144 144 144 288 288 288 288 288 288 288

328

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334 Table 13 Classification accuracy (%) of MRELBP and MRELBPCPS on UIUC. Method MRELBP

Single scale

Multiple scale MRELBPCPS

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

86.79 85.99 92.79 92.96 92.61 87.55 92.36 93.97 92.98 93.52 95.31 95.17

84.74 84.23 90.99 90.98 90.73 85.66 90.43 92.13 91.48 91.98 93.70 93.63

81.08 80.50 87.75 87.80 87.28 81.87 87.32 89.52 88.45 88.81 90.46 90.93

73.18 72.15 79.60 80.28 79.49 73.72 79.63 82.89 81.75 82.18 83.79 83.66

200 200 200 200 200 400 400 400 400 400 400 400

Table 14 Classification accuracy (%) of LBP and LBPCPS on CUReT. Method

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

LBP

(1, 8) (2, 16) (3, 24) (4, 32) (5, 32) [(1, 8) ; [(2, 8) ; [(2, 8) ; [(3, 8) ; [(3, 8) ;

81.82 85.90 87.74 87.86 86.59 94.24 94.17 94.46 94.03 95.18

75.58 79.94 82.48 82.25 81.36 90.31 90.18 90.30 89.27 90.78

68.00 72.82 75.80 75.66 74.97 84.61 84.11 84.41 82.44 84.10

58.45 63.46 67.16 62.27 66.18 75.87 75.22 75.12 72.24 74.45

10 18 26 34 34 20 20 20 20 20

LBPCPS

(3, (3, (4, (4, (5,

8)] 8)] 8)] 8)] 8)]

Fig. 8. Samples of texture images from the CUReT database.

90.93% and 83.66%, which demonstrates the effectiveness of our proposed CPS strategy.

Fig. 9. Samples of the 25 classes of texture images from the UMD database.

4.4. Experimental results on CUReT database The CUReT database contains 61 texture classes and each class has 205 images taken at different viewpoints and illumination orientations as shown in Fig. 8. For each texture image in the database, there are 118 images shot from where the viewing angle is less than 60°. Out of the 118 images, we choose 92 images and each of them has a sufficiently large visible region of texture. A central 200 × 200 region is cropped from each of these chosen images and is converted to gray scale. In our experiment, N (N = 6, 12, 23, 46) training images are randomly chosen from each class and the remaining (92 − N) images are used for test. Experimental results of LBP and LBPCPS on CUReT database are listed in Table 14. By applying CPS strategy, LBP achieves much better classification accuracy. When M = 2, pixels in an original image are divided into 2 classes, and the parameter pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] = [(1, 8); (3, 8)] can reach the best

accuracies of 94.24%, 90.31%, 84.61% and 75.87%, which achieves an improvement of 6.5%, 7.83%, 8.81% and 8.26% respectively over the best accuracies of LBP. Tables 15–19 list the experimental results of five LBP variants on CUReT database. Similar conclusions can be made as for Outex and UIUC databases, and the proposed CPS strategy efficiently enhances the classification performance of LBP variants. Besides, the parameter selection pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] = [(1, 8); (3, 8)] can reach their best accuracies in most cases. 4.5. Experimental results on UMD database The UMD texture database as shown in Fig. 9 includes 25 classes and 40 images in each class. This database consists of high

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

329

Table 15 Classification accuracy (%) of LTP and LTPCPS on CUReT. Method

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

LTP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; [(2, 8) ; [(2, 8) ; [(3, 8) ; [(3, 8) ;

85.40 91.09 93.86 94.05 93.85 95.28 95.52 95.79 95.36 95.89

77.44 85.47 88.57 88.88 88.38 90.95 91.29 91.82 90.89 91.71

68.19 77.75 81.12 81.33 80.57 84.29 84.63 85.15 83.75 84.74

57.06 66.90 70.56 70.75 69.74 74.32 74.80 75.24 73.25 74.05

20 36 52 52 52 40 40 40 40 40

LTPCPS

(3, (3, (4, (4, (5,

8)] 8)] 8)] 8)] 8)]

Table 16 Classification accuracy (%) of CLBP and CLBPCPS on CUReT. Method

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

CLBP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; [(2, 8) ; [(2, 8) ; [(3, 8) ; [(3, 8) ;

95.95 96.30 96.42 95.56 94.94 97.64 97.83 97.76 97.33 97.33

91.99 92.57 92.73 91.49 90.43 94.50 94.76 94.67 93.73 93.88

85.39% 86.32% 86.80% 85.11 83.48 89.01 88.84 88.88 87.47 87.75

75.02 76.27 76.91 75.02 73.53 79.39 78.89 78.85 76.82 77.25

200 648 1352 1352 1352 400 400 400 400 400

CLBPCPS

(3, (3, (4, (4, (5,

8)] 8)] 8)] 8)] 8)]

Table 17 Classification accuracy (%) of CRDP and CRDPCPS on CUReT. Method

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

CRDP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

94.17 94.15 91.49 85.08 82.40 93.99 95.74 95.73 95.42 95.42

89.85 89.68 86.52 78.97 75.39 88.64 91.24 91.20 91.04 91.23

83.30 83.41 79.69 71.58 66.65 80.72 84.37 84.09 83.93 84.25

73.38 73.57 70.16 61.06 55.77 68.97 74.17 73.87 73.26 73.66

200 648 1352 1352 1352 400 400 400 400 400

CRDPCPS

Table 18 Classification accuracy (%) of BRINT and BRINTCPS on CUReT. Method BRINT

Single scale

Multiple scale BRINTCPS

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, [(1, 8) + (3, [(1, 8) ; (3, [(2, 8) ; (3, [(2, 8) ; (4, [(3, 8) ; (4, [(3, 8) ; (5,

95.02 93.95 94.12 95.94 95.48 96.52 97.34 97.12 96.38 96.89 96.43 96.57

90.95 89.26 89.40 91.63 91.12 92.81 94.05 93.65 92.63 93.23 92.28 92.51

84.44 82.43 82.17 84.78 84.20 86.63 88.20 87.64 85.84 86.40 85.28 85.90

74.20 72.38 71.97 74.19 73.68 76.75 78.53 77.59 75.33 75.81 74.40 75.07

144 144 144 144 144 288 288 288 288 288 288 288

8)] 8)] 8)] 8)] 8)] 8)] 8)]

resolution (1280 × 960) images, and all images are captured under arbitrary rotations, significant viewpoint changes and scale differences. In our experiment, N images (N = 5, 10, 15, 20, respectively) per class are randomly selected for training and the remaining (40−N) images are for testing. Following the previous setting of CUReT, we average the final experimental results over 100 times independent random selections. Table 20 presents the classification results obtained by the original LBP and the LBPCPS descriptor on the UMD database. As shown in Table 19, the new LBPCPS descriptor achieves significantly and consistently better performances compared to the original LBP

methods. The best classification accuracies of LBPCPS in the case of [(3, 8); (5, 8)] are 91.57%, 90.20%, 88.03%, and 81.03%, which could have 5.32%, 6.21%, 7.56% and 8.45% improvement over the best accuracy of the original LBP: 86.25%, 83.99%, 80.47% and 72.58%, corresponding to the number of training numbers for each class set as 20, 15, 10 and 5, respectively. Tables 21–25 present the experimental results of five LBP variants on UMD database, respectively. Similar conclusions to those on the UIUC and CUReT database can be found on this database. The proposed CPS strategy efficiently enhances the classification performance of LBP variants and yields the best results in most

330

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334 Table 19 Classification accuracy (%) of MRELBP and MRELBPCPS on CUReT. Method MRELBP

Single scale

Multiple scale MRELBPCPS

Parameter pair(s) (R, P)

46

23

12

6

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, [(1, 8) + (3, [(1, 8) ; (3, [(2, 8) ; (3, [(2, 8) ; (4, [(3, 8) ; (4, [(3, 8) ; (5,

94.11 94.20 95.87 94.14 93.03 95.08 96.56 97.36 97.39 97.03 97.04 97.42

89.89 89.96 91.89 89.33 87.59 90.97 93.02 94.08 93.97 93.26 93.23 94.07

83.59 83.47 85.40 82.33 80.58 84.43 87.12 88.05 88.17 86.82 86.64 87.73

73.27 72.84 75.69 72.60 70.32 74.54 77.32 78.42 77.81 76.87 76.60 78.19

200 200 200 200 200 400 400 400 400 400 400 400

8)] 8)] 8)] 8)] 8)] 8)] 8)]

Table 20 Classification accuracy (%) of LBP and LBPCPS on UMD. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

LBP

(1, 8) (2, 16) (3, 24) (4, 32) (5, 32) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

84.09 84.67 86.25 86.13 85.66 90.39 90.10 90.73 91.48 91.57

81.80 82.28 83.99 84.09 83.17 89.07 88.79 89.59 90.20 90.20

77.89 78.73 80.47 80.17 79.19 86.79 86.21 87.11 87.60 88.03

70.28 70.81 72.25 72.17 72.04 81.03 79.81 81.04 81.09 81.03

10 18 26 34 34 20 20 20 20 20

LBPCPS

Table 21 Classification accuracy (%) of LTP and LTPCPS on UMD. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

LTP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

84.24 87.73 89.30 89.78 90.43 91.02 90.45 91.22 92.08 92.62

82.48 86.33 87.37 88.07 89.08 89.71 89.49 90.54 91.95 91.50

79.14 83.43 84.75 85.64 85.78 88.01 87.51 88.57 89.29 89.63

71.71 76.45 77.57 78.18 78.81 82.40 81.63 83.33 83.97 84.61

20 36 52 52 52 40 40 40 40 40

LTPCPS

Table 22 Classification accuracy (%) of CLBP and CLBPCPS on UMD. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

CLBP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

92.16 92.82 92.87 93.14 93.16 92.94 93.30 93.50 93.54 93.44

91.56 92.16 92.14 92.36 92.61 92.15 92.32 92.87 92.74 92.82

90.31 90.62 90.48 91.00 90.85 90.89 90.92 91.35 91.21 91.61

86.95 87.04 87.55 86.81 86.76 87.56 87.34 88.31 87.85 88.22

200 648 1352 1352 1352 400 400 400 400 400

CLBPCPS

Table 23 Classification accuracy (%) of CRDP and CRDPCPS on UMD. Method

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

CRDP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

91.82 91.91 92.10 92.07 92.06 92.99 92.93 93.32 93.24 93.49

90.93 91.13 91.16 91.18 91.05 92.37 92.21 92.84 92.61 92.33

89.45 89.52 89.24 88.72 88.87 90.68 90.83 91.41 91.21 91.42

85.15 84.79 84.11 84.03 83.26 87.03 86.71 87.67 87.16 87.13

200 648 1352 1352 1352 400 400 400 400 400

CRDPCPS

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

331

Table 24 Classification accuracy (%) of BRINT and BRINTCPS on UMD. Method BRINT

Single scale

Multiple scale BRINTCPS

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

91.61 91.87 92.34 92.72 92.96 92.90 93.32 93.13 93.30 93.51 93.48 93.44

90.76 90.92 90.74 91.84 91.92 92.88 92.88 92.44 92.68 92.95 93.04 92.85

89.16 89.38 90.10 90.30 90.32 91.71 90.62 91.20 91.93 91.64 91.72 91.47

85.55 84.91 85.42 85.15 84.63 88.18 87.17 87.66 87.39 87.84 87.64 87.62

144 144 144 144 144 288 288 288 288 288 288 288

Table 25 Classification accuracy (%) of MRELBP and MRELBPCPS on UMD. Method MRELBP

Single scale

Multiple scale MRELBPCPS

Parameter pair(s) (R, P)

20

15

10

5

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

92.74 93.58 94.30 94.11 94.15 93.47 94.20 94.10 93.99 94.30 94.23 94.29

91.06 92.72 93.85 93.38 93.85 92.79 93.67 93.98 93.77 93.90 93.71 93.96

87.62 91.36 92.87 92.31 92.06 91.31 92.82 92.99 92.87 92.87 93.07 92.68

87.62 88.10 89.31 87.89 87.29 88.14 89.53 90.13 90.25 90.02 90.06 83.98

200 200 200 200 200 400 400 400 400 400 400 400

Fig. 10. Samples of texture images from the ALOT database.

cases. Once again, we verified the effectiveness of our proposed CPS strategy on this high-resolution image database. 4.6. Experimental results on ALOT database The ALOT database consists of 250 rough texture classes and 100 images per class, as shown in Fig. 10. It is systematically collected with varied viewing angles, illumination angles and illumination colors for each material which is similar in spirit to the CUReT collection. Although the number of view-illumination directions per material is only half the BRDF bi-directional reflectance distribution function) resolution of Columbia-Utrecht Reflectance and Texture (CUReT) database, the ALOT extents the number of materials almost by a factor of 5, and it improves upon image resolution and color quality. In our experiment, all images are converted to gray-scale images. To get statistically significant experimental results, half of the samples per class are randomly selected

for training and the remaining half are for testing. The partition is implemented 100 times independently and the average accuracy over 100 random splits is our final classification accuracy. Table 26 gives the classification results obtained by the original LBP and the LBPCPS descriptor on ALOT database. As shown in Table 26, the new LBPCPS descriptor achieves significantly and consistently better performances than the original LBP methods. The best classification accuracy of LBPCPS in the case of [(2, 8); (4, 8)] is 91.03%, which has a 6.79% improvement over the best accuracy of 84.24% of the original LBP. The experimental results demonstrate that proposed CPS strategy, as a preprocessing step before feature extraction, is very discriminative and powerful for texture classification. Tables 27–31 show the experimental results of five LBP variants and their using of CPS strategy on ALOT database, respectively. As shown in Table 27, the best classification accuracy of LTPCPS in the case of [(2, 8); (4, 8)] is 92.97%, which is a little higher

332

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334 Table 26 Classification accuracy (%) of LBP and LBPCPS on ALOT. Method

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

LBP

(1, 8) (2, 16) (3, 24) (4, 32) (5, 32) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

80.35 84.07 84.24 83.60 82.00 90.88 90.72 91.03 90.39 90.41

10 18 26 34 34 20 20 20 20 20

LBPCPS

Table 27 Classification accuracy (%) of LTP and LTPCPS on ALOT. Method

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

LTP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

85.11 89.94 91.22 91.39 90.96 92.72 92.52 92.97 92.21 92.46

20 36 52 52 52 40 40 40 40 40

LTPCPS

Table 28 Classification accuracy (%) of CLBP and CLBPCPS on ALOT. Method

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

CLBP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

91.89 93.79 93.98 93.73 93.53 94.37 94.45 94.56 94.38 94.30

200 648 1352 1352 1352 400 400 400 400 400

CLBPCPS

Table 29 Classification accuracy (%) of CRDP and CRDPCPS on ALOT. Method

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

CRDP

(1, 8) (2, 16) (3, 24) (4, 24) (5, 24) [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

88.12 89.65 89.47 88.46 80.54 92.43 92.77 92.89 92.48 92.66

200 648 1352 1352 1352 400 400 400 400 400

CRDPCPS

Table 30 Classification accuracy (%) of BRINT and BRINTCPS on ALOT. Method BRINT

Single scale

Multiple scale BRINTCPS

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

89.21 90.77 90.71 89.99 89.34 91.32 92.74 93.17 93.08 93.47 93.22 93.66

144 144 144 144 144 288 288 288 288 288 288 288

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

333

Table 31 Classification accuracy (%) of MRELBP and MRELBPCPS on ALOT. Method MRELBP

Single scale

Multiple scale MRELBPCPS

Parameter pair(s) (R, P)

Classification accuracy

Histogram length

(1, 8) (2, 8) (3, 8) (4, 8) (5, 8) [(1, 8) + (2, 8)] [(1, 8) + (3, 8)] [(1, 8) ; (3, 8)] [(2, 8) ; (3, 8)] [(2, 8) ; (4, 8)] [(3, 8) ; (4, 8)] [(3, 8) ; (5, 8)]

91.42 90.82 93.94 93.39 92.17 91.90 94.12 94.65 94.65 94.35 94.72 94.57

200 200 200 200 200 400 400 400 400 400 400 400

than the best result of 91.39% of LTP in the case of (R, P) = (4,24). However, the histogram length of the new LTPCPS is 40, which is much smaller than 52 of LTP. The same findings can also be found in Tables 28 and 29, and the CPS strategy efficiently enhances the classification performance of CLBP and CRDP and results in a relative small histogram length. As for BRINT and MRELBP, we also compare these two methods both in single scale and in multiple scales as shown in Tables 30 and 31. With the using of our proposed CPS strategy, both BRINTCPS and MRELBPCPS achieve the best classification accuracies with low histogram lengths. According to the experimental results on five typical texture databases above, we find that on each database both the original LBP and other five representative LBP variants can achieve much better classification performance by using our proposed CPS strategy than that in their original way. Moreover, the best selection of sampling radius parameters pairs of [(R_BigG, P_BigG); (R_SmallG, P_SmallG)] differs in different databases. These experimental results significantly demonstrate the effectiveness of the proposed CPS strategy. The key reasons that the proposed CPS strategy efficiently enhances the classification accuracy performance of LBP variants lie in two factors. The first factor is that we take into account the fact that different central pixels in fact have different local gray-value distributions, and they should be treated differently. The second factor is that we classify all central pixels in an image into different classes based on their local gray-value distributions, and assign an adaptive sampling radius for the pixels in different classes. Finally, as introduced in Section 3.1, to realize our proposed CPS strategy, we need a total of three steps: convolution using Sobel operator, pixel sorting and pixel classification. As a result, compared to the original LBP method, the inevitably increased time complexity of convolution is O((W−2)(H−2)) and pixel sorting is O(WH) by using bucket sort, where W × H denotes the image size. Also, the increased space complexity is O(WH) by dividing central pixels of an original image into two different classes. However, the proposed method increases classification accuracy of the LBP by approximately 16%. Therefore, a preprocessing step such as CPS strategy is useful and practical for more effective local feature extraction.

the sampling radiuses for different central pixels need to be adaptive. In this paper, we introduced CPS strategy into LBP framework and used it as a preprocessing step before extracting LBP features. By applying CPS strategy, central pixels in an original image can be classified into different classes before local feature extraction of LBP, which ensures that adaptive sampling radiuses can be assigned to central pixels according to the classes they belong to. As a preprocessing step, CPS strategy can also be applied to any other LBP variants. Experimental results on four representative texture databases of Outex, UIUC, CUReT and UMD show that our proposed CPS strategy significantly enhances the classification accuracy performances of LBP and its variants of LTP, CLBP, CRDP, BRINT and MRELBP. Although the proposed CPS strategy could greatly enhance the classification performance of LBP-based methods on four public texture databases, it is unavoidable to increase the final histogram length due to cascading M feature histograms independently obtained from M classes. It will be more useful if the increasement of histogram length can be limited. Another task is to explore whether the proposed CPS strategy suits for other texture operators, such as the Weber Local Descriptor (WLD) (Chen et al., 2010). Furthermore, the current work has focused on texture classification, we wish to exploit the proposed CPS strategy for new applications such as face recognition and object recognition. With the advance in CNN architectures, CNN-based texture representations have quickly demonstrated their strengths in texture classification, whereas CNN model training is time consuming. Moreover, hand-crafted descriptors, for example the LBP-like methods such as LBPCPS , still have merits in the cases when real-time computation is a priority or when robustness to image degradation is needed. Therefore, considering the strong classification ability of CNN-based texture representations and the low computational cost of hand-crafted LBP descriptors, the challenge of LBP/CNN hybridization is our future research direction.

CRediT authorship contribution statement Zhibin Pan: Conceptualization, Methodology, Writing - revising. Xiuquan Wu: Methodology, Writing - original draft, Writing - revising. Zhengyi Li: Software, Validation.

5. Conclusion In this paper, we proposed a new central pixel selection (CPS) strategy, which classifies pixels of an original texture image into different classes based on their local gray-value distributions by using gradient information. So far, conventional LBP and its variants only assign a fixed sampling radius for all central pixels in an original image without taking into account the fact that different central pixels have different local gray-value distributions, and

Acknowledgments This work is supported in part by the Open Research Fund of Key Laboratory of Spectral Imaging Technology, Chinese Academy of Sciences (Grant No. LSIT201606D) and the Open Project Program of the National Laboratory of Pattern Recognition (NLPR) (Grant No. 20180 0 030).

334

Z. Pan, X. Wu and Z. Li / Expert Systems With Applications 120 (2019) 319–334

References Bentley, J. L. (1975). Multidimensional binary search trees used for associative searching. Communications of the Acm, 18(9), 509–517. Brahnam, S., Jain, L. C., Nanni, L., & Lumini, A. (2014). Local binary patterns: New variants and applications. Berlin Heidelberg: Springer. Cusano, C., Napoletano, P., & Schettini, R. (2016). Evaluating color texture descriptors under large variations of controlled lighting conditions. Journal of the Optical Society of America A: Optics and Image Science, and Vision, 33(1), 17–30. Chen, J., et al. (2010). WLD: A robust local image descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1705–1720. Dana, K. J., Nayar, S. K., Ginneken, B. V., & Koenderink, J. J. (1997). Reflectance and texture of real-world surfaces authors. ACM Transactions on Graphics, 18, 1–34. Fernández, A., Álvarez, M. X., & Bianconi, F. (2013). Texture description through histograms of equivalent patterns. Journal of Mathematical Imaging and Vision, 45(1), 76–102 pp. Guo, Z., Zhang, L., & Zhang, D. (2010). A completed modeling of local binary pattern operator for texture classification. IEEE Transactions on Image Processing, 19, 1657–1663. Guo, Z., Wang, X., Zhou, J., & You, J. (2015). Robust texture image representation by scale selective local binary patterns (SSLBP). IEEE Transactions on Image Processing, 25(2), 687–699. Ji, L., Ren, Y., Liu, G., & Pu, X. (2017). Training-based gradient lbp feature models for multiresolution texture classification. IEEE Transactions on Cybernetics, (99), 1–14. Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. In NIPS (pp. 1097–1105). Kanopoulos, N., Vasanthavada, N., & Baker, R. L. (2002). Design of an image edge detection filter using the sobel operator. IEEE Journal of Solid-State Circuits, 23(2), 358–367. Lazebnik, S., Schmid, C., & Ponce, J. (2005). A sparse texture representation using local affine regions. IEEE Transactions on Pattern Analysis & Machine Intelligence, 27, 1265. Liu, L., Fieguth, P., Guo, Y., Wang, X., & Pietikäinen, M. (2017). Local binary features for texture classification: Taxonomy and experimental study. Pattern Recognition, 62, 135–160. Liao, S., Law, M. W., & Chung, A. C. (2009). Dominant local binary patterns for texture classification. IEEE Transactions on Image Processing, 18(5), 1107. Liu, L., Long, Y., Fieguth, P. W., Lao, S., & Zhao, G. (2014). BRINT: Binary rotation invariant and noise tolerant texture classification. IEEE Transactions on Image Processing, 23, 3071–3084. Liu, L., Lao, S., Fieguth, P., Guo, Y., Wang, X., & Pietikainen, M. (2016). Median robust extended local binary pattern for texture classification. IEEE Transactions on Image Processing, 25(3), 1368–1381.

Liu, L., Zhao, L., Long, Y., Kuang, G., & Fieguth, P. (2012). Extended local binary patterns for texture classification. Image & Vision Computing, 30(2), 86–99. Lu, Z., Jiang, X. D., & Kot, A. (2018). Feature fusion with covariance matrix regularization in face recognition. Signal Processing, 144, 296–305. Nanni, L., Lumini, A., & Brahnam, S. (2012). Survey on lbp based texture descriptors for image classification. Expert Systems with Applications, 39, 3634–3641. Napoletano, P. (2017). Hand-crafted vs learned descriptors for color texture classification. Computational color imaging: 6th International workshop, CCIW 2017, Milan, Italy, March 29–31, 2017, proceedings, 259–271 10213. Ojala, T., Maenpaa, T., Pietikainen, M., & Viertola, J. (2002). Outex - New framework for empirical evaluation of texture analysis algorithms. International Conference on Pattern Recognition, 1, 701–706. Ojala, T., Pietik, Inen, M., & Topi (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence, 24, 971–987. Pan, Z., Li, Z., Fan, H., & Wu, X. (2017). Feature based local binary pattern for rotation invariant texture classification. Expert Systems with Applications, 88. Pan, Z., Fan, H., & Zhang, L. (2015). Texture classification using local pattern based on vector quantization. IEEE Transactions on Image Processing, 24, 5379–5388. Pan, Z., Wu, X., Li, Z., & Zhou, Z. (2017). Local adaptive binary patterns using diamond sampling structure for texture classification. IEEE Signal Process Letter, 24(6), 828–832. Ryu, J., Hong, S., & Yang, H. S. (2015). Sorted consecutive local binary pattern for texture classification. IEEE Transactions on Image Processing, 24(7), 2254–2265. Targhi, A. T., Geusebroek, J. M., & Zisserman, A. (2008). Texture classification with minimal training images. International conference on pattern recognition. IEEE. Tan, X., & Triggs, B. (2010). Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing, 19(6), 1635. Wang, K., Bichot, C. E., Li, Y., & Li, B. (2017). Local binary circumferential and radial derivative pattern for texture classification. Pattern Recognition, 67, 213–229. Wang, Q., Zheng, Y., Yang, G., Jin, W., Chen, X., & Yin, Y. (2017). Multi-scale rotation-invariant convolutional neural networks for lung texture classification. IEEE Journal of Biomedical & Health Informatics, 99, 1 -1. Xu, Y., Ji, H., & Fermuller, C. (2006). A projective invariant for textures. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 1932–1939. Zhao, Y., Huang, D. S., & Jia, W. (2012). Completed local binary count for rotation invariant texture classification. IEEE Transactions on Image Processing, 21, 4492–4497.