Fabric defect inspection based on lattice segmentation and template statistics

ARTICLE IN PRESS

JID: INS

[m3Gsc;October 23, 2019;22:1]

Information Sciences xxx (xxxx) xxx

Contents lists available at ScienceDirect

Information Sciences journal homepage: www.elsevier.com/locate/ins

Fabric defect inspection based on lattice segmentation and template statistics Liang Jia a, Chen Chen b,∗, Shoukun Xu a, Ju Shen c a

School of Information Science & Engineering, Chang Zhou University, China Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA c University of Dayton, Dayton, OH, USA b

a r t i c l e

i n f o

Article history: Received 18 August 2017 Revised 16 January 2019 Accepted 15 October 2019 Available online xxx Keywords: Fabric defect inspection Lattice segmentation Patterned texture

a b s t r a c t Automated fabric inspection is desirable for quality control of fabric industry. However, it is challenging due to some unpredictable fabric defects which may only occur during the production. Hence, methods aiming at automated fabric inspection are commonly developed in absence of defective samples. This paper proposes a novel automated fabric inspection method based on lattice segmentation and template statistics (LSTS) focusing on the patterned fabric images containing repetitive texture primitives. The proposed method attempts to infer the placement rule of texture primitives and divide the image into noneoverlapping lattices as texture primitives which represent the given image by hundreds of lattices instead of millions of pixels. The defects are then efficiently identified by comparing the lattice similarity w. r. t. the benchmarks named template statistics. The template statistics are learnt from defect-free samples through a modular framework in which multiple feature extraction methods like Gabor filters and local binary pattern can be flexibly combined according to their inspection efficiencies. The performance of the proposed method is evaluated in the databases of dot-, box- and star-patterned fabric images. By comparing the resultant and ground-truth images, an overall detection rate of 0.977 is achieved, which is competitive with the state-of-the-arts. © 2019 Published by Elsevier Inc.

1. Introduction Fabric (textile) is one of the most common industry products in our daily life. Due to the massive amount of fabric manufacturing, defects occurring during the fabric manufacturing process may cause tremendous loss. Therefore, fabric quality control is critical in this process. Traditionally, quality control depends on the human inspection whose success rate approximates 60–75% [35], and the just passable success rate leads to the profit loss up to 45–65% [35]. On the other hand, the accuracy of a state-of-the-art automated fabric inspection is higher than 90% [5]. The efficiency and the economic superiority of automated fabric inspection are self-evident. Accordingly, there are numerous reported methods aiming at implementing automated fabric inspection and these methods could be categorized from different perspectives, e.g., the perspectives of texture feature representations, wallpaper groups and training strategy. From the perspective of texture feature representations, there are commonly four classes [42,22,14]: structural analysis (SA) [6,3,34], statistical methods [29,19,45], spectral methods [38,39,37] and model-based methods [1,2,21]. Besides the fab∗

Corresponding author. E-mail addresses: [email protected] (C. Chen), [email protected] (S. Xu).

https://doi.org/10.1016/j.ins.2019.10.032 0020-0255/© 2019 Published by Elsevier Inc.

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 2

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

ric defect inspection, these four classes are adoptable for other surface defect inspection, such as steel and wood detection [42]. SA assumes the surface texture is generated by repeating texture primitives (texels) which could be overlapped or nonoverlapped [22,14]. The repetition may follow some placement rule [6,3] or be completely random [34]. For fabric texture, placement rule is commonly assumed to dominate the repetition of texture primitives [22,14] and texture primitives are defined differently throughout reported methods [6,3]. In [3], texture primitive is defined as texture blobs enclosed by the rectangular regions forming a grid overlapping the binarized image. The placement rule inference is omitted in [3] because texture primitives are assumed to be arranged in a grid. Recently, Jia and Liang [17] introduced an SA method whose framework is similar to the one in [3] except that the placement rule is inferred dynamically based on the locations of texture blobs rather than directly assuming it is conformed to a rigid grid [3]. As stated in [22] and [14], SA commonly requires the texture pattern be regular. From the perspective of training strategy, there are two classes [42]: methods based on the supervised or unsupervised training. The unsupervised fabric-inspection methods differ a little from the unsupervised machine learning methods. With the unsupervised fabric-inspection methods, the defective fabric samples are totally absent in the training data, because some defect types are unpredictable and may only occur during the production [43]; while the unsupervised machine learning methods require both defect-free and defective samples for training. The unpredictability of defects differentiates the fabric-inspection methods from the traditional machine learning methods. From the perspective of wallpaper groups [23], there are two classes [22]: methods capable or incapable of processing fabric images categorized as groups other than p1 group. Most of the aforementioned methods only aim at plain or twill fabric categorized as p1 group [23]. A few are able to process fabrics of groups other than p1 group [22], e.g., wavelet-pre-processed golden image subtraction (WGIS) [24], Bollinger bands(BB) [25], regular bands (RB) [26], image decomposition (ID) [27], motif-based (MB) method [23] and Elo rating (ER) method [40]. Besides the band regularity and template matching, image decomposition [28,36,44] provides a possibility to directly segment defects by completely removing the repeated texture primitives. The method named ID [27] integrates the image decomposition method described in [28] to inspect defects. Among the methods capable of processing images other than p1 group, there’s a special method named MB [23] which is designed based on motifs defined in wallpaper groups. MB is categorized as a separate one in [22]. Its distinguishing characteristic is the preprocessing step named lattice segmentation which is rare in literatures of fabric inspection. Lattice segmentation method divides the fabric image into none-overlapping regions called lattices (windows or blocks) which consist of the same textures. Analogously, the lattice approximates the texture primitives in SA, and lattice segmentation method infers a consistent placement rule for images of the same fabric patterns to guarantee the segmented lattices share roughly the same texture. If the lattice is defined as the same-shaped regions containing the same group of motifs whose spatial relationships are fixed, then the methods with or without lattice segmentation may form two categories: lattice-based or none-lattice-based methods. MB is a representative of lattice-based method. Actually, these two categories generalize the categorization of motif-based and none-motif-based methods introduced in [22]. This paper proposed a lattice-based method with a modular framework in which multiple feature extraction methods can be flexibly combined. The contributions of this paper can be summarized as follows. 1) A novel modular framework is developed in consideration of lattices enclosing periodically-changed textures which represent the texture primitives. For a texture primitive type and the corresponding lattices of defect-free samples, the feature vectors of lattices are computed w. r. t. a feature extraction method, and the resulting feature vectors are respectively averaged based on their statistics. The resulting averages serve as benchmarks for distinguishing defective fabrics from the defect-free ones. Essentially, each texture primitive type of defect-free samples is modeled by averages based on multiple feature extraction methods. The feature extraction methods serve as replaceable modules in the framework and the modeling and inspection process run in parallel when multiple feature extraction methods coexist. Thus, the framework leads to a high adaptability due to the flexible combination of the underlying feature extraction methods. The remainder of this paper is organized as follows. In Section 2, the algorithms involved in the proposed method are briefly introduced. In Section 3, the novel lattice segmentation and the modular framework are outlined. In Section 4, the effective combination of feature extraction methods and the corresponding performance is evaluated. Lastly, Section 5 concludes the paper. 2. Related works The proposed method is built on several reported algorithms which could be categorized to two classes according to their functionalities. The two classes are lattice segmentation and feature extraction. Lattice segmentation techniques for inferring placement rules are introduced briefly in Section 2.1, and the feature extraction methods serving as the combinable modules are concisely described in Section 2.2. 2.1. Lattice segmentation Lattice segmentation aims to capture the texture primitives by modeling the patterned image with some lattice placement rule (also called lattice structure), e.g., a grid containing rectangles. Lattice segmentation methods may be roughly categorized to two classes [13]: local feature-based and global feature-based methods. The two classes mainly differ at the phases of Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 3

capturing texture primitives. The former gets the information about the texture primitives before inferring the placement rule, while the latter infers the rule first and then obtains the texture primitives based on the rule. Generally, global featurebased methods are suitable for images containing large texture primitives of sophisticated appearances [13]. Consequently, global feature-based methods are commonly adopted in fabric inspection, e.g., lattice segmentation method in [16–18,23]. Lattice segmentation method proposed in [16] makes strict assumptions about the images, which results in the incapability of processing some fabric images, e.g., the dot-patterned fabric images described in [40]. However, lattice segmentation method in [16] has been improved in [18], and dot-patterned fabric images can be successfully segmented. In this paper, the lattice segmentation method in [18] is adopted for segmenting a grayscale image to none-overlapped rectangles called lattices. 2.2. Feature extraction There are numerous feature extraction methods adoptable for fabric inspection. In this paper, six feature extraction methods are involved, i.e., image raw moment (IRM) [30], histogram of gradients (HOG) [7], Zernike moments (ZM) [30], graylevel co-occurrence matrix (GLCM) [15], local binary pattern (LBP) [31] and Gabor filters [8]. IRM denoted by μpq provides a global description about the spatial distribution of the pixel values in a given image. It is estimated by summing pixel values weighted by the powers of the horizontal (p) and vertical (q) spatial deviations to the image center. For a given image, different combinations of p and q may result in distinct μpq s. IRM feature vector is built by concatenating μpq s. HOG samples edge orientations from the constituent regions of a given image. Namely, the gradients of the image are computed, and then the image is divided into rectangular or radial regions called cells. Gradient histogram of each cell is estimated by voting the gradient magnitudes to the histogram bins representing gradient directions. The histograms within a window which overlaps multiple cells are then concatenated and normalized to yield the window histogram and all window histograms are concatenated to form the final feature vector. ZM represents a function f (e.g., image) through the coefficients Zpq s of Zernike polynomials Vpq which are the orthogonal basis functions. The p and q denote the polynomial indices. Zpq s are computed as the inner product of f and Vpq . The resulting Zpq s are concatenated as the feature vector. For an image of p different pixel values, GLCM constructs a p–by–p co-occurrence matrix. The elements in matrix are the counts of paired pixels according to predefined spatial relationships. The feature vector is built by concatenating the contrast, correlation, energy and homogeneity of the co-occurrence matrix. LBP creates a code for each pixel by encoding the intensity changes in the neighborhood of each pixel. The histogram of the codes attached to pixels serves as the feature vector. Gabor feature computation convolves the image with Gabor filters parameterized by scale and orientation θ . The resulting image w. r. t. orientation θ is then rotated by –θ and the pixels of the image are accumulated row by row, and then the energies and amplitudes of the responses are concatenated as the feature vector. 3. The LSTS method Suppose there are numerous images of a certain fabric and they are all sampled from defect-free fabrics, when a new fabric of the same pattern is sampled, how to automatically and accurately determine whether the fabric is defective or defect-free? The proposed method can be used to solve this problem. If the fabric pattern consists of periodic-repeated texture primitives, then it is possible to segment the image sample to none-overlapped lattices enclosing the texture primitive. Under certain conditions, the textures enclosed by the resulting lattices may change periodically. Namely, there are different types of the texture primitives segmented from the image. For each texture primitive type, its characteristics may be reflected by the feature extraction methods introduced in Section 2.2. The feature statistics like means can be estimated on basis of feature vectors computed based on the defect-free samples. Therefore, each texture primitive type can be modeled by the feature statistics learnt from defect-free samples. The learnt statistics are called template statistics which then serve as the benchmark for discriminating the defective fabrics from the defect-free ones. Hence, the proposed method is based on lattice segmentation and template statistics (LSTS). There are two main issues of the aforementioned process, i.e., segmenting fabric image to regions corresponding to different classes of texture primitives and building the template statistics which lead to efficient defect inspection. The two issues are tackled by lattice segmentation and the modular framework. Lattice segmentation and the framework are discussed in Sections 3.1 and 3.2 respectively. The flowchart of LSTS is illustrated in Fig. 1. 3.1. Lattice segmentation Lattice segmentation is built on two assumptions [18]: the fabric image contains periodic-repeated texture primitives; the contrast is strong enough to yield a binary image for distinguishing the periodicity of texture primitive. If these two assumptions are met, after preprocessing like image decomposition [4,9–12,20,37,41] and binarization, the background pixels may concentrate between two adjacent texture primitives in the binary image regardless of the texture forms. If we count the background pixels along rows or columns, between any two adjacent texture primitives there are local maximums which form “peaks”. The regular locations of peaks suggest the separators of texture primitives, and the resulting regions segmented by separators are called lattices. Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 4

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

Fig. 1. Flowchart of LSTS.

3.2. Modular framework If there are multiple types of texture primitives enclosed by the segmented lattices, the distributions of feature vectors may vary drastically from lattice to lattice. Hence, the feature vectors of lattices should be discriminatively collected and analyzed. To achieve the discriminability, it is necessary to know the number of different types of texture primitives. This number is named lattice period which may be learnt from defect-free samples. According to the learnt lattice period, it is simple to infer the placement rule of certain texture primitive type. Therefore, lattices enclosing texture primitives of certain type can be discriminatively collected. The feature vectors related with a certain texture primitive type can then be employed to generate a general feature vector representing this texture primitive type. This general feature vector is named template statistics. A specific texture primitive type can thus be represented by template statistics. Then, it is possible to estimate the limits of distances between the feature vectors of defect-free lattices and the template statistics. The limits of the resulting distances are named distance limit. During the inspection, the feature vectors of a lattice are extracted and compared with all template statistics. The lattice is marked as the defective one if the corresponding distance difference exceeds the limits. The detailed descriptions are made in the following subsections. 3.2.1. Lattice period Suppose the texture primitives enclosed by the lattices are periodically different and the texture primitives of every t lattices are the same, then t is defined as lattice period. Namely, the texture primitives enclosed by the lattices can be categorized to t different types. To recover the lattice period of a fabric image, a period estimation based on Fourier transformation reported in [18] is employed. The estimation process is explained as follows. If the differences of the texture primitives enclosed by the lattices in the same row or column can be measured, they can be arranged in the same order of the lattices. The series of the ordered differences can thus be envisaged as a one-dimensional signal of sample rate one. For every t values of this signal, if the value approximates 0, the lattice period t can be easily found by finding the frequency of the maximal spectrum based on the representation of the signal in Fourier domain. 3.2.2. Template statistics According to the lattice period, the lattices can be discriminatively collected. Thus, for each texture primitive type, there exists a collection of lattices. For a specific feature extraction method and a texture primitive type, the components in feature vectors are the same. Thus, component-wise statistics can be computed. For a feature vector, the components of low standard deviations have stable values for defect-free samples. Such components are ideal for building the benchmark to discriminate defective lattices. For each sample, the means of these components are concatenated as the representative vector for the texture primitive type. However, a single representative vector may not be sufficient to represent a certain texture primitive type, e.g., the fabric quality may vary during sampling, which may lead to slightly-different texture primitives of the same type. Thus, to model the texture primitive type in a fine granularity, the representative vectors of the same texture primitive type are categorized through clustering algorithms. Categorized representative vectors in the same cluster are averaged. Therefore, for a texture primitive type and a feature extraction method, the type corresponds to averaged vectors Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 5

from multiple clusters. The component value variations of these vectors are expected to be small for defect-free samples. Generally, the aforementioned procedure is implemented in four steps: Step 1 finding component statistics, Step 2 sorting component statistics, Step 3 finding stable feature fields and Step 4 finding template statistics. The four steps are introduced sequentially as follows. Step 1 Finding Component Statistics The statistics of feature vector components should be computed for each texture primitive type throughout samples I1 , I2 …IN , which requires the collection of lattices enclosing the texture primitives of the same type. Since the lattice period t can be learnt in advance, the lattice arrangement in every t rows is the same. Every t lattices in a single row enclose the texture primitives of the same type. Hence, for every t rows, every t lattices can be extracted to form a matrix containing lattices of the same type. Therefore, there are at most t such matrices, i.e., C1 , C2 …Ct of the kth texture primitive type. An illustration of extracting lattices of the same type can be found in Fig. 6 of [18]. Suppose there are feature extraction methods f1 , f2 …fT , for each method fj , it is straightforward to compute the feature vectors based on C1 , C2 …Ct of the kth texture primitive type, then component statistics can be computed. If fj, 1 (L), fj, 2 (L) …fj, F (L) denotes the components of the feature vector involving lattice L based on method fj where F indicates the component number, then the component mean is defined as follows.

⎡

⎤



meanL∈Cl f j,1 (L )   ⎢ ⎥ 1  ⎢meanL∈Cl f j,2 (L ) ⎥ (i, j,k ) rmean = ⎢ ⎥, . .. t ⎣ ⎦ 1≤l≤t   meanL∈Cl f j,F (L )

(1)

where i, j, k correspond to the indices of ith defect-free sample Ii , the jth method fj and the kth texture primitive type. The (i, j,k )

(i, j,k )

definitions of rmin , rmax

(i, j,k )

and rstd

are similar.

Step 2 Sorting Component Statistics Although the defect-free samples contain the same texture primitive types, their arrangement order may differ from sample to sample. An illustration of the different orders can be found in Fig. 9 of [18]. Since the component statistics of formula (1) are respectively computed based on each sample, the statistics across all samples may not map to the same texture primitive type. Thus, the statistics need to be sorted. It is possible that the orders of all samples are the same. In this case, the sorting is unnecessary. To detect such a case, it is useful to compute the sum of the distances between the component means of samples before and after the sorting. If the sum decreases after sorting, keep the sorted result, otherwise, cancel the sorting. The sum is defined as follows.

d( j) =

N−1 

 ‘   (i , j,1 ) (i‘ +1, j,1 )  rmean − rmean 

2

N−1

i‘ =1

,

(2)

(i‘ , j,k )

where rmean is defined by formula (1) and index k is fixed to 1, i.e., the sum is computed w. r. t. a fixed texture primitive type for reducing the computation load. The sorting starts by finding the global order through component means of all samples. For each texture primitive type, component means can be clustered and a cluster of maximal size can be identified. For a specific component mean, it is simple to compute the distances between the mean and the centers of maximal clusters of all texture primitive types. The cluster of the center closest to the mean reveals its true type. The clustering is conducted through Adaptive K-means [17]. u) Specifically, if S (j,k denotes a center of cluster comprising the texture primitives of the kth type based on the jth method, then the cluster of the least distance is found based on

   (i, j,k)  (u )  u = arg minu  r − S , j,k   mean ∗

(3)

2

(k ) (k ) where u denotes the cluster label and i is the index of the sample. A disorder is detected when u∗ = umax where umax denotes the label of the maximal cluster. The type leading to the minimum difference reveals the true type of the disordered component mean, and the two means of the two types are exchanged.

Step 3 Finding Stable Feature Fields For the components of a feature vector, the ideal components are expected to stay stable when underlying sample is defect-free, and to change drastically when the sample is defective. Unstable components can be removed to shorten the feature vector to achieve an expected vector length defined by a hyper parameter named least feature number. The stableness s(j, k) (l) of the lth component is defined as the ratios of the component standard deviations and means, i.e.,

N

(i, j,k ) i=1 rstd (i, j,k ) i=1 rmean

s( j,k) (l ) =  N

(l ) (l )

.

(4)

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS 6

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

Fig. 2. Process of learning difference limits.

The shortening is implemented by decreasingly sorting components based on stableness, and remove the components from the tail until the number of component is no larger than least feature number. Step 4 Finding Template Statistics For each texture primitive type, the corresponding component means can be categorized to subtypes through clustering methods. Suppose the number of subtypes is defined as a hyper parameter named cluster number limit denoted by L, then the clustering may be conducted through K-means in which K is set to L. The averages of the component means in the resulting clusters then serve as the representatives of the type. Specifically, for the kth texture primitive type, there are (i, j,k ) ( j,k ) component means rmean s of labels i s based on the clustering. The labels  have L distinct values for L subtypes. The representative of the kth type is defined as follows. i∗ , j,k

r j,k,





( ) ( )  rmean · δ i∗ −  ,  =  ( j,k ) − 1≤i∗ ≤N 1≤i≤N δ i j,k

(5)

where δ (x ) = 1 if x = 0 and δ (x ) = 0 otherwise. Since rj, k,  s are computed based on the component statistics and they serve like templates for discriminating defective lattices, they are named templated statistics. 3.2.3. Difference limits A difference limit between defect-free lattices of texture primitive type may serve to classify lattices in inspection. However, to learn such a limit, the lattices of the defect-free images have to be discriminatively collected based on texture primitive types represented by templated statistic. Namely, a lattice is related with some templated statistic. Equivalently, a templated statistic is related with some lattices. Therefore, the difference limit for a templated statistic can be estimated based on the related lattices. If the kth type is represented by rj, k,  , rj, k, 2 …, then the r j,k∗ ,∗ most similar to a lattice L is identified by





k∗ , ∗ = arg mink,  f j∗ (L ) − r j,k,  , 2

(6)

where f j∗ (L ) represents the feature vector of stable components based on formula (4). Thus, each lattice is uniquely related with a pair of k∗ and ∗ . For rj, k,  , there may be some lattice of k∗ = k and ∗ = . If L( j,k, ) denotes the multiset of the lattices related with rj, k,  , then the difference limit of rj, k,  is defined as follows. ( j,k, ) dmax = max

L∈L( j,k, )

 ∗   f j (L ) − r j,k,  . 2

(7)

If rj, k,  is never related with any lattice based on formula (6), then rj, k,  is useless in inspection and it is removed from the template statistics. For example, Fig. 2 illustrates the steps of finding the limits involving feature extraction methods f1 , f2 …fT . Only the steps involving f1 and fT are visualized in the figure. A lattice is randomly selected from the lattices shown on the left-most figure, the selected lattice is highlighted by white border, and it is enlarged on the right side of it. For Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 7

Fig. 3. Process of inspecting defects.

the selected lattice, its feature vectors related with the multiple feature extraction methods are visualized by the polygonal lines next to the enlarged lattice. Then for each feature extraction method fj , the corresponding feature vector is compared with the rj, k,  s illustrated by polygonal lines surrounded by dots. Since there are two types of texture primitives, rj, k,  s are listed in two rows for each feature extraction method, thus the polygonal lines in each row visualized the rj, k,  s of a specific type. The comparisons are based on formula (6). The resulting distances, pairs of k∗ and ∗ are visualized in gray mosaics arranged in the same order of the corresponding lattices, and their grayness reflects the distance value. The larger the distance, the darker the mosaic is. Two digits on the mosaics represent the values of k∗ and ∗ of the corresponding lattices. Some rj, k,  may not be related with any lattices and thus removed, e.g., rT, 2, 3 vanished in the dotted frame on the right of Fig. 2. Finally, for each rj, k,  , a difference limit is computed based on formula (7), e.g., for four rT, k,  s, four limits are shown on the right-most figure accordingly. 3.2.4. Defect inspection For a given fabric image, none-overlapped lattices are segmented through the method described in Section 3.1 as shown on the left of Fig. 3. For each feature extraction method, the defects are identified in parallel. For instance, the steps involving feature extraction method f1 are visualized as sequential subfigures linked by arrows in Fig. 3. For each fj , the feature vectors are computed and their stable components are the same as the ones learnt based on formula (4). The vectors of stable components are visualized as polygonal lines arranged in square in the middle left of Fig. 3. For each feature vector of a specific feature extraction method, the most similar template statistic is identified according to formula (6). Then, the difference between the feature vector and the identified template statistic is compared with the corresponding limit learnt based on formula (7). The defects are identified as the lattices enclosing texture primitives of differences are beyond the limit. The defects are visualized as the rectangles with white borders in the middle right of Fig. 3. Since the limits are exceeded only if the underlying lattice contains the most defective texture primitive, it is possible to miss the lattices of less defective texture primitives. Empirically, such lattices are commonly around the identified ones. Thus, it is beneficial to explore the lattices surrounding the identified ones. The exploration process is described as follows. Compute the differences between the lattices and their most similar template statistics, and then explore the map of the differences. The map is formed by arranging the differences in the same order of the corresponding lattices. Starting from an identified defective lattice, it is easy to find the difference values of the defect-free lattices adjacent to the defective ones through the map. Then, the found values are compared with those of the defective ones, and mark the corresponding lattice as the defective lattice if their values exceed the identified ones. Repeat this process based on the new-identified defective lattices until no lattice is found to have the value exceeding the identified ones. Specifically, let D (L ) denote the difference values of the defect-tree lattices found in the 8-connected neighborhood of a defective lattice L. Then, the corresponding values are compared with a threshold dynamically updated during the exploration. The threshold is defined as

d p+1 = α · max (d, d p ), d∈D (L p )

(8)

where p is the times of the exploration, dp denotes the threshold found in the pth exploration, and α is a hyper parameter named threshold factor ranging from 0 to 1. As the exploration continues, the threshold becomes larger and larger until no adjacent lattices can be identified as defective. The lattices identified by the exploration are illustrated on the second right column in Fig. 3. Compared with the illustrations before the exploration, there are a few new-identified lattices after the exploration. Finally, all identified defective lattices are collected as the inspection result shown in the right-most Fig. 3. Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 8

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

4. Performance evaluation The performance evaluation and the comparison are conducted based on the patterned fabric image databases provided by Industrial Automation Research Laboratory from Dept. of Electrical and Electronic Engineering of Hong Kong University. The fabric image database employed in experiments consists of 247 images of size 256-by-256. Among these images, 166 images are in 24-bit depth and the rest are binary images. All images are categorized to three fabric patterns: dot pattern (30 defect-free, 30 defective and 30 ground-truth images), star pattern (25 defect-free, 25 defective and 25 ground-truth images) and box pattern (30 defect-free, 26 defective and 26 ground-truth images). For each defective image, there is a binary ground-truth image whose foreground pixels denote the defects in pixel level. There are five common defect types for all three fabric patterns, i.e., Broken end, Hole, Netting Multiple, Thick Bar and Thin Bar. Besides these five types, dot pattern has an additional defect type named Knots. The number of the defective images associated with each type and a specific pattern can be found in the first columns from Tables 2–4. For the proposed method, its merit is the capability of flexibly combining different feature extraction methods. Although more feature extraction methods means higher detection rate, it comes at the cost of consuming more processing time. Therefore, the performances of the possible combinations should be evaluated to find an appropriate combination which can balance the detection rate and the runtime, and then the performance of the selected combination is optimized through the parameter tuning. Finally, the optimized performance is compared with those of the state-of-the-art methods, i.e., waveletpre-processed golden image subtraction (WGIS) [24], Bollinger bands (BB) [25], regular bands (RB) [26] and Elo rating (ER) method [40]. The settings of the methods involved in comparisons and relevant performance evaluation metrics are described in Section 4.1. Section 4.2 illustrates the experiment results to reveal the performances of all possible combinations involving the feature extraction methods discussed in Section 2.2. Section 4.3 shows the analyses on the effects of parameters on the performance of the selected combination. In Section 4.4, the performance optimized by using the fine-tuned parameters is compared with those of BB, RB, ER and WGIS. In Section 4.6, the runtime of all methods involved in the experiments are analyzed and compared. All experiments are performed on a laptop computer with an Intel Core i7-6700HQ 260-GHz processor and 8.00 GB of memory. The software consists of Windows 10 and Matlab 8.4. 4.1. Evaluation settings Four methods introduced in Section 1, i.e., BB, RB, ER and WGIS are implemented for comparisons. The parameters and the performance calculations about the four methods are exactly the same as the settings described in [16] and [17]. Because the fabric inspection is equivalent to image bi-classification, i.e., defective fabric and defect-free fabric, it is helpful to measure the performance of deep neural networks adept at classification. Therefore, besides the four methods, a neural network is developed. Because the proposed method classifies defects at the lattice level, the network is thus trained based on the lattices. In test, the network replaces “Step Inspect Defect” in testing phase of Fig. 1, and the defective fabric images are segmented to lattices and each lattice is classified by the network. The lattices for network training are obtained by segmenting lattices from the defective fabric images. As a result, there are 12,449 defect-free lattices and 1728 defective lattices. According to Pareto’s principle, the defect-free lattices are divided to 9960 (80%) lattices for training and 2489 (20%) for testing. For defective lattices, we randomly select 86 (5%), 172 (10%), 259 (15%) … 1382 (80%) defective lattices, and obtain 16 groups containing 5%–80% of the original defective lattices. Because all groups contain fewer defective lattices than the defect-free lattices for training, the data augmentation is conducted to make the groups have the same number of defective and defect-free lattices. The data augmentation consists of random horizontal and vertical reflection, scale (0.9–1.2), shear (−30°~ 30°), translation (−5 pixel ~ 5 pixel), and random rotation (0–360°). Therefore, the number of defect-free and defective lattices is the same throughout all 16 groups. For example, there are 9,874 transformed lattices generated based on the group of 86 (5%) defective samples, and the sum of the transformed lattices and the group is up to 9960(9874 + +86) which is the same as the number of the defect-free lattices for training. Each of the 16 groups thus respectively serves as an individual training dataset D to reflect the performance w. r. t. the different number of the available defective lattices. To determine the optimal architecture of network, a NASNet-like [33] search space S is defined, i.e., a vector space S = R5 of five dimensions: filter number F ranging from 3 to 64, network depth N ranging from 1 to 3, initial learning rate ranging from 0.001 to 0.05, momentum ranging from 0.8 to 0.95 and L2 regularization ranging from 1e-10 to 0.01. The first two control the architecture and the last three affect the training procedure. The general architecture of the network in S is as below. As shown in Fig. 4, there are N convolution cells adjacent to max pooling layers. Each cell sequentially stacks a convolutional layer, a batch normalization layer and a ReLU layer. Each filter in any convolutional layer is of size 3-by-3, the stride and padding are universally set to 1, and the number of filters in convolution cell corresponds to the first dimension of S, i.e., filter number F. The second dimension of S, i.e., network depth N coincides with N in Fig. 4 and represents the number of convolution cells. Bayesian optimization (BO) [32] is emloyed to probe into the search space S. The kernel function employed by BO is squared exponential kernel. The scalar objective function f(x) of BO yields validation loss based on x ∈ S and training set, i.e., neural network is generated based on the architecture (shown in Fig. 4) parameterized by the first two components of x, then it is trained by using stochastic gradient descent with momentum parameterized by the last three components of x Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 9

Fig. 4. Network architecture.

Fig. 5. Validation accuracy based on varying number of defective lattices.

based on 80% data of D, and the trained network is validated based on the rest 20% data of D. During training, the batch size is set to 256, the epoch number is 3, and the loss function is cross entropy loss. The optimal architecture of D is chosen as the one leading to the highest validation accuracy throughout all points sampled by BO. The optimal architecture then replaces the Step Inspect Defect in testing phase of Fig. 1, and the defective fabric images are segmented to lattices and each lattice is classified by the network. The searching, training and test are all implemented by Matlab 8.4. There are five metrics: accuracy (ACC), true positive rate (TPR), false positive rate (FPR), positive predictive value (PPV) and negative predictive value (NPV) adopted for measuring the performances of the involved methods. The definitions of the five metrics can be found in [27]. The five metrics are defined at pixel level, while the outputs of LSTS are lattices, so binary images are synthetized based on the output lattices for computing the pixel-based metric values. The synthesis is implemented as follows. For a resulting lattice yielded by LSTS or neural network w. r. t. a specific defective image, the region in the corresponding ground-truth image overlapping the lattice is copied to a blank binary image if the lattice covers any foreground pixels denoting the defects in the ground truth image, otherwise the region overlapped by the lattice in the binary image is set to foreground to indicate the false detection. Requiring as few defective lattices as possible, the average ACC, TPR and FPR of each fabric pattern is then estimated for selecting the architecture of performance comparable with the proposed method. ACCs, TPRs and FPRs based on the varying defective lattices are shown in Fig. 5. Since there is a large leap of star-pattern TPR w. r. t. the training set of 30% defective lattices, the final architecture is decided to be the one sampled by BO based on this particular training set. The five dimensions of the final architecture are filter number 50, network depth 3, initial learning rate 0.0 0 05, momentum 0.8459 and L2 regularization 0.0065. 4.2. Feature selection based on TPR, FPR and runtime As introduced in Section 3, the proposed method has a flexible structure combining different feature extraction methods, which leads to distinct results. The results obtained through different combinations could be measured by different metrics described in Section 4.1. Among these metrics, we focus on TPR, FPR and runtime. The chosen metrics are the averages of the TPRs, FPRs and runtime obtained by applying the proposed method to the whole dataset 40 times, and the involved feature extraction methods are the ones discussed in Section 2.2, i.e., IRM, HOG, ZM, GLCM, LBP and Gabor. The hyper parameters introduced by the proposed method, i.e., least feature number, cluster number limit and threshold factor are fixed to 5, 10 and 0.9 respectively. The chosen metrics related with all possible combinations are visualized in Figs. 6 and 7, i.e., TPR and FPR of each combination is represented as a pie whose sectors correspond to the combination components as illustrated in Fig. 6, while TPR and runtime of each combination is similarly represented in Fig. 7. For each fabric pattern, there are two subfigures in Fig. 6 and Fig. 7, i.e., the one on the left showing the complete distribution of all possible combinations and the other on the right showing a zoomed version of the complete distribution. According to Section 3.2.5, the detection results of feature extraction methods involved in a combination are merged, which means TPR and FPR of a combination are not equivalent to the sums of TPRs and the sums of FPRs of the individual method involved in the combination, e.g., the FPRs of ZM and LBP both approximate 0.8 in the case of Dot Pattern in Fig. 6, however, their combination has FPR lower than 0.6. The 6 feature extraction methods described in Section 2.2 are adopted for constructing different combinations adopted by the proposed method. As shown in Fig. 6, no single feature extraction method could achieve high TPR while maintaining low FPR. Conversely, the combination of all feature extraction methods Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 10

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

Fig. 6. TPRs and FPRs of different combinations.

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 11

Fig. 7. TPRs and runtime of different combinations.

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS 12

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx Table 1 TPR FPR and runtime of feature extraction method combination.

Feature Extraction Method IRM

× × × × × × × × ×

HOG

ZM

× × × ×

× × ×

× × × × × ×

× × × × × × ×

× × × × × ×

× × × × ×

Star Pattern

Dot Pattern

LBP

Gabor

TPR

FPR

Time

TPR

FPR

Time

TPR

FPR

Time

× × ×

×

× ×

0.79 0.79 0.75 0.80 0.62 0.80 0.76 0.80 0.76 0.80 0.76 0.59 0.64 0.60 0.47 0.80 0.80 0.80 0.77 0.81 0.64 0.81

0.02 0.03 0.03 0.03 0.03 0.02 0.02 0.02 0.03 0.04 0.03 0.06 0.05 0.10 0.11 0.03 0.02 0.04 0.03 0.04 0.05 0.04

12.1 16.3 14.5 16.9 14.5 14.5 12.8 15.1 17.0 19.2 17.6 18.5 19.1 16.2 16.9 18.6 16.8 20.9 19.2 21.4 20.8 23.2

0.95 0.95 0.92 0.94 0.90 0.95 0.93 0.95 0.92 0.95 0.85 0.89 0.88 0.86 0.69 0.95 0.95 0.95 0.93 0.95 0.90 0.95

0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.08 0.09 0.09 0.09 0.07 0.09 0.10 0.09 0.09 0.09 0.09 0.10

19.6 22.7 19.9 22.6 20.4 20.8 18.1 20.7 21.2 23.8 21.2 23.4 23.3 22.2 20.7 26.2 24.3 27.4 24.7 27.4 26.9 30.9

0.70 0.70 0.68 0.53 0.59 0.72 0.69 0.55 0.69 0.55 0.49 0.59 0.33 0.57 0.44 0.71 0.72 0.72 0.69 0.56 0.59 0.72

0.01 0.01 0.01 0.02 0.01 0.01 0.01 0.01 0.01 0.02 0.01 0.01 0.03 0.01 0.02 0.02 0.01 0.02 0.01 0.02 0.01 0.02

35.1 37.7 32.4 39.3 35.6 35.9 30.8 37.5 33.4 40.0 35.0 38.0 39.6 37.2 32.9 44.7 42.9 45.6 40.5 47.1 45.1 52.6

× × × × × × × × × × ×

Box Pattern

GLCM

× × × × ×

× × × ×

× × × × × × × × × × × ×

× × × × × × × × ×

× × × × × × × × ×

always has the highest TPR and nearly lowest FPR throughout all fabric patterns. Generally, FPRs of combinations involving 4 or more feature extraction methods are almost always lower than 0.1, which suggests the number of feature extraction methods is larger than 4 to maintain low FPR while achieving high TPR. The runtime of all possible combinations w. r. t. three fabric patterns are shown in Fig. 7. The runtime refers to the time consumed for processing the whole dataset. For Box and Star Patterns, the runtime are all below 40 s. The runtime of Dot Pattern exceeds 50 s. The TPRs in the zoomed versions in Fig. 7 are dispersed, which allows a clearer observation of TPR. For Dot Pattern, the highest TPR approximates 0.07. Most combinations of at least four components are around 0.07. For Box and Star Patterns, the combinations of four or more components are also close to their highest TPRs. However, the runtime of the combinations around the highest TPRs are also close to the maximal runtime. Generally, the high runtime is basically unavoidable if high TPRs are achieved. Because the TPRs are overlapped densely in Figs. 6 and 7, TPR, FPR and runtime of all possible combinations of at least four feature extraction methods are listed in Table 1. As shown in the table, the highest TPRs of three fabric patterns are 0.82, 0.95 and 0.72. There is only one combination achieving the optimal TPRs for all patterns, i.e., the combinations involving all feature extraction methods which is shown in the bottom of Table 1. Another noticeable fact is that the combinations of HOG, GLCM and Gabor, highlighted with the dark background, have TPRs either optimal or very close to optimal throughout all fabric patterns. The three feature extraction methods are essential for the proposed method to achieve TPRs close to the optimal value. The combinations achieving optimal values always consist of HOG, GLCM and Gabor, while those with low TPRs always lack one or more such feature extraction methods. Because the TPR of Dot Pattern is lowest among all patterns, the combination leading to high TPR of Dot Pattern is preferable and the combination of IRM, HOG, GLCM and Gabor balances TPR, FPR and runtime throughout all fabric patterns based on Table 1. Thus, it is concluded that the efficient combination of feature extraction methods for the datasets is IRM, HOG, GLCM and Gabor. Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 13

Fig. 8. ACCs, TPRs and FPRs of all fabric patterns w. r. t. order p.

Fig. 9. ACCs, TPRs and FPRs of all fabric patterns w. r. t. order q.

4.3. Tuning the parameters There are two types of framework parameters: parameters of the feature extraction methods and parameters introduced by the framework itself. For the first type, there are four feature extraction methods adopted in the framework, i.e., image raw moment (IRM), histogram of gradients (HOG), gray-level co-occurrence matrix (GLCM) and Gabor filters. The computation of each feature extraction method involves some hyper parameters except HOG. The original hyper parameter of HOG is cell size populated adaptively by using half of the most frequent lattice size found in training, while the rest parameters include orders of IRM, offset of GLCM, and scale & strength parameters of Gabor filters. For the second type of framework parameters, there are three hyper parameters. Namely, least feature number, cluster number limit and threshold factor. The corresponding default values are predefined for these hyper parameters to conduct tests. For a specific parameter, the tests are conducted by varying its values while fixing the rest parameters to their default values. For each test, the proposed method is run through all datasets, and the resulting lattices are employed for computing ACC, TPR and FPR. The computed measurements for all tests involving the specific parameter are then individually visualized as curves in figures. These visualized measurements for the aforementioned parameters are discussed sequentially as follows. Order p of IRM: this parameter determines the order of the horizontal deviation involved in IRM feature computation. Order p can be any positive integers, it’s restricted within the range [1,25] in the test, and its default value is 3. Fig. 8 illustrates the test results related with order p. TPRs of dot and box patterns are sensitive to the change of order p, and their optimal values occur at 22 for dot pattern and 3 for box pattern. Although TPRs fluctuate as order p changes, there’s no value leading to severe drop of all TPRs for order p. Hence, the proposed method may maintain a stable performance when order p changes, and the default value 3 leads to acceptable performance. Order q of IRM: this parameter sets the order of the vertical deviation involved in IRM feature computation. Order q has the same range and default value as order p. Fig. 9 illustrates the test results related with order q. TPRs of dot and box patterns are sensitive to the change of order q, and their optimal values occur at 6 for dot pattern and 10 for box pattern. Although TPRs fluctuate as order q changes, there’s no value leading to severe drop of all TPRs for order q. Hence, the proposed method may maintain a stable performance when order q changes, and the default value 3 leads to acceptable performance. Offset of GLCM: this parameter determines the linear spatial relationships of paired pixels involved in GLCM computation. The spatial relationship refers to a line segment of fixed length and inclination. Since the inclination cannot be precisely represented in a limited discrete space formed by pixels, it is approximated by offsets. An offset contains the least Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 14

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

Fig. 10. ACCs, TPRs and FPRs of all fabric patterns w. r. t. offset.

Fig. 11. ACCs, TPRs and FPRs of all fabric patterns w. r. t. Gabor scale.

Fig. 12. ACCs, TPRs and FPRs of all fabric patterns w. r. t. Gabor orientation.

horizontal and vertical deviations from the image center, whose inverse tangent approximates the inclination. The distance is adaptively set as the hypotenuse of the triangle consisting of the deviations. The test results related with offset are illustrated in Fig. 10. Dot pattern is the only pattern of drastically-changed TPRs. The optimal value of offset corresponds to 330°for dot pattern, and the default value 0 leads to TPR a little higher than 0.6. Generally, dot pattern is sensitive to offset of GLCM; and the performance of the proposed method is stable for the rest patterns regardless of offset changes. Scale of Gabor filters: this parameter controls the shape of the Gaussian surface adopted in Gabor feature computation. It can be any positive real number. It is restricted to integers in the range [1,24] and it’s 1 by default. Fig. 11 illustrates the detection results of scales. All patterns are insensitive to scale changes except the scale value 4 for dot pattern. The value 4 causes a noticeable drop of TPR of dot pattern. Hence, the changes of scale hardly affect the performance of the proposed method for all fabric patterns. Orientation of Gabor filters: this parameter controls the orientation diversity of the Gabor filters. It can be any positive integer. It is restricted to integers in the range [2,25] and it’s 4 by default. Fig. 12 illustrates the results of all patterns w. r. t. the orientation. All patterns except dot pattern are relatively insensitive to the changes of orientation. The TPRs of Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 15

Fig. 13. Least feature numbers of all fabric patterns.

Fig. 14. Cluster number limits of all fabric patterns.

dot pattern tend to increase as the orientation increases, and it reaches its optimal value at orientation 25. Generally, the changes of orientation barely affect the performance of the proposed method for all patterns except dot pattern. Least feature number: this parameter specifies the higher bound of the component number involved in estimating stableness defined by formula (4). It can be any positive integer. In test, it is restricted to integers in the range [1,25] and it’s 5 by default. Fig. 13 illustrates the results of all patterns w. r. t. least feature number. As shown in the figure, ACC and FPR are nearly invariant throughout all fabric patterns and the TPR variations of Box and Star Pattern are relatively small compared with the case of Dot Pattern. TPRs of Dot Pattern slowly decrease as the parameter increases and TPR has high values at 3, 6, 10, 16 and 20. However, since the FPR of star pattern should be as low as possible, we choose 8 before 9 where FPR of star pattern begins to increase in Fig. 13. Hence, the optimal value for the least feature number is chosen as 8. Cluster number limit: this parameter specifies the higher bound of the cluster number. It can be any positive integer. In test, it is restricted to integers in the range [5,26] and it’s 10 by default. Fig. 14 illustrates the results of all patterns w. r. t. cluster number limit. As shown in the figure, ACC decreases and FPR increases slowly as the parameter increases, while TPR varies intensely throughout all patterns as illustrated in Fig. 14. Generally, TPR increases as FPR increases, which means more and more lattices are labeled defective but only a few of them are truly defective when nK is getting large. Hence, an ideal value of the parameter should balance TPR and FPR and lead to relatively high ACC. When the cluster number limit is 5, ACCs for dot and box patterns are the highest, and ACC for star pattern is suboptimal. When the cluster number limit is 5, the TPRs are not very high compared with other values like 16, while the FPRs are the lowest throughout all patterns as shown in Fig. 14. Hence, the optimal value for the cluster number limit is chosen as 5. Threshold factor: this parameter adjusts the dynamic threshold defined by formula (8). It can be any real number lying in (0, 1]. In test, it is restricted within [0.7, 0.95] and it’s 0.9 by default. Fig. 15 illustrates the results of all patterns w. r. t. threshold factor. As shown in the figure, the variations of ACC, TPR and FPR of different fabric patterns are distinct. For Dot Pattern, TPR and FPR both decrease while ACC increases as α becomes large. There are two local maximums of TPR when α > 0.82 where 0.82 leads to a large drop of TPR, the local maximums occur at α = 0.85 and 0.91. For Box Pattern, all metrics are relative stable except that a drop appears at α = 0.88. For Star Pattern, the variations of all metrics become negligible when α > 0.73. From the view of minimizing FPRs, 0.93 seems a promising candidate because it leads to the lowest FPR of dot pattern as shown in Fig. 15. Hence, a reasonable value for the threshold factor is chosen as 0.93. In short, all patterns except dot pattern are roughly insensitive to the changes of hyper parameters involved in defect inspection. For dot pattern, the fluctuations of the performances of all feature extraction methods except GLCM are relatively Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS 16

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

Fig. 15. Threshold factors of all fabric patterns. Table 2 Numerical results of dot pattern. Dot pattern Defect type

ACC

TPR

FPR

PPV

NPV

Method

Broken End (5)

0.92 0.79 0.88 0.90 0.82 1.00 0.99 0.79 0.97 0.97 0.87 1.00 0.98 0.78 0.96 0.97 0.81 1.00 0.98 0.81 0.96 0.97 0.95 1.00 0.92 0.71 0.87 0.97 0.65 1.00 0.99 0.79 0.97 0.99 0.73 1.00 0.96 0.79 0.94 0.96 0.81 1.00

0.58 0.83 0.00 0.25 0.18 1.00 0.79 0.84 0.00 0.38 0.47 0.98 0.84 0.84 0.00 0.20 0.80 1.00 0.82 0.76 0.00 0.19 0.05 0.97 0.50 1.00 0.00 0.83 0.67 1.00 0.69 1.00 0.00 0.76 0.33 1.00 0.70 0.88 0.00 0.44 0.42 0.99

0.01 0.20 0.00 0.01 0.16 0.00 0.00 0.21 0.00 0.02 0.12 0.00 0.01 0.22 0.00 0.00 0.19 0.00 0.01 0.19 0.00 0.00 0.02 0.00 0.00 0.33 0.00 0.00 0.36 0.00 0.01 0.21 0.00 0.00 0.26 0.00 0.01 0.23 0.00 0.01 0.19 0.00

0.74 0.25 N/A 0.58 0.17 1.00 0.87 0.1 N/A 0.42 0.11 1.00 0.81 0.13 N/A 0.68 0.14 1.00 0.79 0.14 N/A 0.91 N/A 1.00 0.94 0.30 N/A 0.97 0.22 1.00 0.75 0.11 N/A 0.83 0.04 1.00 0.82 0.17 N/A 0.73 N/A 1.00

0.92 0.94 0.88 0.90 0.93 1.00 0.99 0.99 0.97 0.98 0.98 1.00 0.99 0.99 0.96 0.97 0.99 1.00 0.99 0.99 0.96 0.97 0.96 1.00 0.92 1.00 0.87 0.97 0.93 1.00 0.99 1.00 0.97 0.99 0.98 1.00 0.97 0.99 0.94 0.96 0.96 1.00

LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN

Hole (5)

Knots (5)

Netting Multiple (5)

Thick Bar (5)

Thin Bar (5)

Overall

small. Hence, the general performance of the proposed method is stable in most cases of changing the parameters of the underlying feature extractions. 4.4. Results The defect inspection results of the proposed method (LSTS) and the neural network (NN) are numerically illustrated from Tables 2–4. All parameters of LSTS are fixed to their default values except least feature number, cluster number limit Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

17

Table 3 Numerical results of box pattern. Box pattern Defect type

ACC

TPR

FPR

PPV

NPV

Method

Broken End (5)

0.99 0.96 0.98 0.98 0.89 1.00 0.99 0.98 0.99 0.99 0.98 1.00 0.97 0.97 0.99 0.99 0.83 1.00 0.97 0.95 0.98 0.99 0.83 0.83 0.99 0.97 0.99 0.99 0.83 1.00 0.98 0.97 0.99 0.99 0.87 0.97

0.73 0.04 0.00 0.49 0.75 0.79 0.85 0.08 0.00 0.10 0.00 0.87 0.68 0.06 0.02 0.10 0.31 0.76 0.84 0.08 0.16 0.58 1.00 0.80 0.72 0.03 0.02 0.43 0.65 0.51 0.76 0.06 0.04 0.34 0.54 0.75

0.01 0.02 0.00 0.01 0.10 0.00 0.00 0.02 0.00 0.00 0.02 0.00 0.03 0.02 0.00 0.00 0.16 0.00 0.03 0.02 0.00 0.01 0.17 0.17 0.01 0.02 0.00 0.00 0.17 0.00 0.02 0.02 0.00 0.00 0.12 0.03

0.88 0.04 N/A 0.56 0.12 1.00 0.85 0.03 N/A 0.47 0.00 1.00 0.46 0.04 N/A 0.27 0.04 1.00 0.80 0.08 N/A 0.68 0.14 0.83 0.52 0.02 N/A 0.73 0.03 1.00 0.70 0.04 N/A 0.54 0.07 0.97

1.00 0.98 0.98 0.99 1.00 1.00 1.00 0.99 0.99 0.99 0.99 1.00 1.00 0.99 0.99 0.99 0.99 1.00 1.00 0.97 0.98 0.99 1.00 0.83 1.00 0.99 0.99 0.99 1.00 1.00 1.00 0.98 0.99 0.99 1.00 0.97

LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN

Hole (5)

Netting Multiple (5)

Thick Bar (6)

Thin Bar (5)

Overall

and threshold factor. Throughout all tests in this section, the three parameters employ the optimal values found in the Section 4.3. In each table, a column illustrates the measurements of a specific metric like ACC, and a row illustrates the numeric results of a method. The marginal columns of each table respectively list the defect types and method names, and the rows are marked by defect types. For each defect type listed in the left-most column of a table, there is an integer in parentheses; this integer indicates the sample numbers of the defect type. For each fabric pattern, there’s a relevant discussion describing and analyzing the corresponding results. For dot-patterned images, the numerical summaries are illustrated in Table 2. According to the overall performances shown in Table 2, NN outperforms all other methods throughout all defect types, and its average ACC surprisingly achieves 1.00. This means NN can perfectly process any defect types of dot pattern. LSTS and RB both achieve the second best ACC (0.96) while LSTS has much better TPR (0.70) than RB (0.44). The FPR of both methods are the same (0.01). BB has the second highest overall TPR, WGIS also has a large overall FPR. High FPR means the method introduces too many false responses. For box-patterned images, the relevant results are illustrated in Table 3. According to the overall performances shown in Table 3, LSTS has ACC (0.98) very close to the best value (0.99), and its overall TPR (0.76) is the best. LSTS achieves a slightly better ACC and TPR than NN. For the defect type Thin Bar TPR of NN is 0.51, which means NN is particularly not good at processing defective samples of Thin Bar. For different defect types, LSTS achieves the best TPR for all types except Thick Bar; while WGIS achieves the best TPR for Thick Bar. However, if FPRs are taken into account, WGIS is far worse than LSTS. In short, LSTS achieves the optimal overall TPR throughout all methods. Its general FPR is very close to the best value and it’s particularly suitable for processing defect types Hole and Netting Multiple of box pattern. For star-patterned images, the results are illustrated in Table 4. According to the overall summaries shown in Table 4, LSTS achieves the best TPR and NPV among all methods. Its overall ACC (0.99) and FPR (0.01) approximate the optimal values. All methods except BB and NN have good overall FPRs even including WGIS. The overall TPR (0.95) of LSTS is higher (better) than NN by 0.05. Accordingly, except Thin Bar, for all defect types, TPRs of LSTS are the best. FPRs of LSTS are either optimal or suboptimal, and the suboptimal FPRs (0.01) are very close to the optimal values (0.0). With LSTS, the optimal TPRs and suboptimal FPRs lead to the optimal ACCs throughout all defect types except Thin Bar. Except Thin Bar, Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS 18

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx Table 4 Numerical results of star pattern. Star pattern Defect type

ACC

TPR

FPR

PPV

NPV

Method

Broken End (5)

0.99 0.84 0.99 0.97 0.97 1.00 1.00 0.85 0.99 0.97 0.98 1.00 0.99 0.83 0.98 0.96 0.97 1.00 0.99 0.82 0.97 0.95 0.92 1.00 0.99 0.85 0.99 0.97 0.93 1.00 0.99 0.84 0.98 0.96 0.95 1.00

0.98 0.31 0.02 0.32 0.50 0.94 1.00 0.33 0.03 0.43 0.13 0.93 0.98 0.22 0.06 0.45 0.23 0.88 0.94 0.1 0.13 0.21 0.75 0.80 0.83 0.29 0.02 0.33 0.56 0.94 0.95 0.25 0.05 0.35 0.43 0.90

0.01 0.16 0.00 0.02 0.02 0.00 0.00 0.15 0.00 0.03 0.01 0.00 0.01 0.16 0.00 0.03 0.02 0.00 0.01 0.16 0.00 0.03 0.08 0.00 0.00 0.15 0.00 0.03 0.07 0.00 0.01 0.16 0.00 0.03 0.04 0.00

0.58 0.01 N/A 0.11 0.11 0.84 0.98 0.01 N/A 0.08 0.04 1.00 0.74 0.02 N/A 0.20 N/A 1.00 0.83 0.02 N/A 0.13 0.24 1.00 0.72 0.02 N/A 0.10 0.09 1.00 0.77 0.02 N/A 0.12 N/A 0.97

1.00 0.99 0.99 1.00 1.00 1.00 1.00 1.00 0.99 1.00 1.00 1.00 1.00 0.98 0.98 0.99 0.99 1.00 1.00 0.96 0.97 0.97 1.00 1.00 1.00 0.99 0.99 0.99 1.00 1.00 1.00 0.98 0.98 0.99 1.00 1.00

LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN LSTS BB ER RB WGIS NN

Hole (5)

Netting Multiple (5)

Thick Bar (5)

Thin Bar (5)

Overall

NN achieves the best FPRs for all defect types, and its TPRs are close to LSTS. In brief, maintaining good FPRs very close to the optimal value, LSTS achieves overwhelming ACCs and TPRs for most defect types. LSTS is particularly appropriate for processing star-patterned fabric images regardless of the defect types. Generally, the results show that the performance of LSTS is good for dot-patterned images, superior for box-patterned images and overwhelming for star-patterned images compared with any other methods in the experiments. For dot-patterned images, NN achieves overwhelming results; while for box- and star-patterned images, LSTS outperforms NN. However, LSTS requires no knowledge about defective samples; while NN does require data of defective lattices. For fabric images adopted in the experiments, NN needs at least 30% defective lattices segmented from the defective samples to achieve equal or better results than LSTS.

4.5. Runtime analysis Runtime of the proposed method (LSTS) is illustrated in Table 5. The illustrated runtime is the average time of processing a single image w. r. t. the specific defect types and fabric patterns. WGIS achieves the lowest runtime throughout all types and patterns. Although WGIS has moderate ACCs as discussed in the previous section, its runtime is surprisingly low compared with any other methods in the experiments. The second best runtime is achieved by ER; however, the TPRs of ER are almost the lowest throughout all defect types and fabric patterns. The runtime of LSTS is the fourth optimal. The runtime of LSTS and NN vary from pattern to pattern; while the runtime is roughly the same throughout all fabric patterns for any other method. The reason may be that lattice segmentation consumes different time for distinct patterns, e.g., segmenting lattices from dot-patterned images is more difficult than from box- and star-patterned images because of its inconspicuous patterns. Another possible reason may be that the number of template statistics differs for patterns, and more template statistics means more comparisons for searching the most appropriate one matching a given lattice. For example, compared with the box- and star-patterned images whose segmented lattices consisting of consistent textures, dot-patterned images are segmented to lattices of two different types of textures which may double the searching time . Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

ARTICLE IN PRESS

JID: INS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

19

Table 5 Runtime of different methods (sec). Fabric pattern

Defect type

LSTS

BB

ER

RB

WGIS

NN

Dot Pattern

Broken End Hole Knots Netting Multiple Thick Bar Thin Bar Average Broken End Hole Netting Multiple Thick Bar Thin Bar Average Broken End Hole Netting Multiple Thick Bar Thin Bar Average

1.1215 1.1073 1.1683 1.1454 1.2175 1.1858 1.1520 0.5347 0.5316 0.5320 0.6416 0.5149 0.5510 0.7679 0.8139 0.8144 0.7787 0.8478 0.8045

1.2900 1.2921 1.2882 1.2904 1.2977 1.2932 1.2919 1.2849 1.2927 1.2923 1.2938 1.3040 1.2935 1.2873 1.2870 1.2882 1.2885 1.2792 1.2860

0.2745 0.2756 0.2756 0.2809 0.2761 0.2735 0.2760 0.2784 0.2749 0.2769 0.2801 0.2939 0.2808 0.2770 0.2786 0.2750 0.2822 0.2727 0.2771

1.2626 1.2629 1.2618 1.2628 1.2695 1.2668 1.2644 1.2607 1.2700 1.2623 1.2668 1.2871 1.2694 1.2676 1.2628 1.2567 1.2627 1.2626 1.2625

0.0180 0.0179 0.0182 0.0178 0.0179 0.0178 0.0179 0.0176 0.0177 0.0179 0.0181 0.0180 0.0179 0.0180 0.0184 0.0188 0.0183 0.0183 0.0184

1.7145 1.5646 1.6434 1.6598 1.7709 1.7103 1.6773 0.7661 0.7587 0.7446 0.7396 0.7265 0.7767 1.1940 1.1764 1.1745 1.1592 1.2141 1.1837

Box Pattern

Star Pattern

Generally, the runtime of LSTS falls in the middle range of the runtime of all methods in the experiments. Unlike the runtime of any other methods, the runtime of LSTS changes from pattern to pattern, which should be noticed in the practical applications. 5. Conclusion This paper presents a novel method which identifies fabric defects by comparing the similarity of the lattices segmented from the fabric image. The lattice segmentation infers the placement rule of texture blobs generated by thresholding the structural image obtained by image decomposition technique. The fabric image is then divided into none-overlapping rectangular regions called lattices based on the inferred placement rule. The texture enclosed by a lattice is then represented by the feature vectors obtained by multiple feature extraction methods which can be flexibly combined thanks to the modular design of the proposed method. The effective combination is determined according to the efficiency analysis of some possible combinations. The benchmarks serving for comparing lattice similarity are named template statistics which are learnt from the defect-free samples. There are three key parameters of the proposed method, i.e., least feature number, cluster number and threshold factor, and their impacts on the performance are studied. The detection results of the proposed method is compared with the methods BB, RB, ER, WGIS and NN based on the databases of patterned fabric images. The comparisons indicate the proposed method outperforms the other five methods in most cases and it’s particularly suitable for processing star-patterned fabric images. A runtime analysis is conducted and the proposed method is found to have the moderate runtime among all methods in the experiments. It should be noticed that the runtime of the proposed method varies for different fabric patterns. The proposed method may be applicable for identifying defects of both fabric surfaces and other two dimensional surfaces like ceramic tile of repetitive textures. Declaration of Competing Interest None. Appendix: Abbreviations in this paper ACC stands for accuracy. It is a metric for evaluating algorithm performances and its definition can be found in [27]. BB stands for Bollinger bands and it refers to a fabric inspection method described in [25]. BO stands for Bayesian optimization. ER stands for Elo rating and it refers to a fabric inspection method described in [40]. FPR stands for false positive rate. It is a metric for evaluating algorithm performances and its definition can be found in [27]. GLCM stands for gray-level co-occurrence matrix. It refers to a feature extraction method described in [15]. The method is briefly introduced in Section 2.2. HOG stands for histogram of gradients. It refers to a feature extraction method described in [7]. The method is briefly introduced in Section 2.2. ID stands for image decomposition. It refers to a fabric inspection method described in [27]. Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS 20

ARTICLE IN PRESS

[m3Gsc;October 23, 2019;22:1]

L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

IRM stands for image raw moment. It refers to a feature extraction method described in [30]. The method is briefly introduced in Section 2.2. ILS stands for isotropic lattice segmentation. It refers to a lattice segmentation method introduced in [17]. LBP stands for local binary pattern. It refers to a feature extraction method described in [31]. The method is briefly introduced in Section 2.2. LSTS is short for lattice segmentation and template statistics; it refers to the proposed method. MB stands for motif-based method. It refers to a fabric inspection method described in [23]. NN is short for neural network. It refers to the neural network introduced in Section 4.1. NPV stands for negative predictive value. It is a metric for evaluating algorithm performances and its definition can be found in [27]. PPV stands for positive predictive value. It is a metric for evaluating algorithm performances and its definition can be found in [27]. RB stands for regular bands. It refers to a fabric inspection method described in [26]. TPR stands for true positive rate. It is a metric for evaluating algorithm performances and its definition can be found in [27]. WGIS stands for wavelet-pre-processed golden image subtraction. It refers to a fabric inspection method described in [24] ZM stands for Zernike moments. It refers to a feature extraction method described in [30]. The method is briefly introduced in Section 2.2.

Reference [1] O. Alata, C. Ramananjarasoa, Unsupervised textured image segmentation using 2-D quarter plane autoregressive model with four prediction supports, Pattern Recognit. Lett. 26 (2005) 1069–1081. [2] A. Bouhamidi, K. Jbilou, An iterative method for Bayesian Gauss–Markov image restoration, Appl. Math. Model. 33 (2009) 361–372. [3] A. Bodnarova, M. Bennamoun, K. Kubik, Defect defection in textile materials based on aspects on the HVS, in: Proc. IEEE Int’l Conf. Systems, Man, and Cybernetics, 1998, pp. 4423–4428. [4] A. Buades, Ba. Coll, J.-M. Morel, A non-local algorithm for image denoising, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005, CVPR’05, IEEE, 2005, pp. 60–65. Vol. 2. [5] C.H. Chan, G.K.H. Pang, Fabric defect detection by Fourier analysis, IEEE Trans. Ind. Appl. 36 (5) (20 0 0) 1267–1276. [6] J. Chen, A.K. Jain, A Structural Approach To Identify Defects In Textured Images, IEEE Int. Conf. Syst. Man Cybern. (1998) 29–32. [7] N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1 (2005) 886–893. [8] J.G. Daugman, Uncertainty relation for resolution in space, spatial frequency and orientation optimized by two-dimensional visual cortical filters, J. Opt. Soc. Am. A 2 (1985) 1160–1169. [9] J. Duan, Z. Pan, B. Zhang, et al., Fast algorithm for color texture image inpainting using the non-local CTV model, J. Glob. Optim. 62 (4) (2015) 853–876. [10] J. Duan, Z. Qiu, W. Lu, et al., An edge-weighted second order variational model for image decomposition, Digit. Signal Process. 49 (2016) 162–181. [11] J Duan, C Tench, I Gottlob, et al., New variational image decomposition model for simultaneously denoising and segmenting optical coherence tomography images, Phys. Med. Biol. 60 (22) (2015) 8901–8922. [12] G. Gilboa, S. Osher, Nonlocal operators with applications to image processing, Multiscale Model. Simul. 7 (3) (2008) 1005–1028. [13] J. Han, S.J. McKenna, Lattice estimation from images of patterns that exhibit translational symmetry, Image Vis. Comput. 32 (1) (2014) 64–73. [14] K. Hanbaya, M.F. Talub, Ö.F. Özgüven, Fabric defect detection systems and methods—a systematic literature review, Int. J. Light Electron Opt. 127 (24) (2016) 11960–11973. [15] R.M. Haralick, K. Shanmugan, I. Dinstein, Textural features for image classification, IEEE Trans. Syst. Man Cybern. (1973) 610–621 SMC-3. [16] L. Jia, C. Chen, J. Liang, Z. Hou, Fabric defect inspection based on lattice segmentation and Gabor filtering, Neurocomputing 238 (2017) 84–102. [17] L. Jia, J. Liang, Fabric defect inspection based on isotropic lattice segmentation, J. Frankl. Inst. 354 (13) (2017) 5694–5738. [18] L. Jia, J. Zhang, S. Chen, et al., Fabric defect inspection based on lattice segmentation and lattice templates, J. Frankl. Inst. 3545 (2018) 7764–7798. [19] A. Latif-Amet, A. Ertüzün, A. Erc, An efficient method for texture defect detection: sub-band domain co-occurrence matrices, Image Vis. Comput. 18 (20 0 0) 543–553. [20] W. Lu, J. Duan, Z. Qiu, Implementation of high-order variational models made easy for image processing, Math. Methods Appl. Sci. 39 (14) (2016) 4208–4233. [21] J.P. Monaco, A. Madabhushi, Class-specific weighting for Markov random field estimation: application to medical image segmentation, Med. Image Anal. 16 (2012) 1477–1489. [22] H.Y.T. Ngan, G.K.H. Pang, N.H.C. Yung, Automated fabric defect detection—a review, Image Vis. Comput. 29 (7) (2011) 442–458. [23] H.Y.T. Ngan, G.K.H. Pang, N.H.C. Yung, Motif-based defect detection for patterned fabric, Pattern Recognit. 41 (2008) 1878–1894. [24] H.Y.T. Ngan, G.K.H. Pang, N.H.C. Yung, et al., Wavelet based methods on patterned fabric defect detection, Pattern Recognit. 38 (4) (2005) 559–576. [25] H.Y.T. Ngan, G.K.H. Pang, Novel method for patterned fabric inspection using bollinger bands, Opt. Eng. 45 (8) (2006) 087202-1–087202-15. [26] H.Y.T. Ngan, G.K.H. Pang, Regularity analysis for patterned texture inspection, IEEE Trans. Autom. Sci. Eng. 6 (1) (2009) 131–144. [27] M.K. Ng, H.Y.T. Ngan, X. Yuan, et al., Patterned fabric inspection and visualization by the method of image decomposition, IEEE Trans. Autom. Sci. Eng. 11 (3) (2014) 943–947. [28] M. Ng, X.M. Yuan, W.X. Zhang, A coupled variational image decomposition and restoration model for blurred cartoon-plus-texture images with missing pixels, IEEE Trans. Image Process. 22 (6) (2013) 2233–2246. [29] H.-F. Ng, Automatic thresholding for defect detection, Pattern Recognit. Lett. 27 (14) (2006) 1644–1649. [30] M.S. Nixon, A.S. Aguado, in: Feature Extraction & Image Processing for Computer Vision, Academic Press, 2012, pp. 1–632. [31] T. Ojala, M. Pietikäinen, T. Mäenpää, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell. 24 (7) (2002) 971–987. [32] C.E. Rasmussen, C.K.I. Williams, Gaussian Processes for Machine Learning, MIT Press, Cambridge, Massachusetts, 2006. [33] E. Real, A. Aggarwal, Y. Huang, Regularized evolution for image classifier architecture search, arXiv:1802.01548v4 [cs.NE], Jun. 2018. [34] KY. Song, J. Kittler, M. Petrou, Defect detection in random colour textures, Image Vis. Comput. 14 (1996) 667–683. [35] K. Srinivasan, P.H. Dastoor, P. Radhakrishnaiah, S. Jayaraman, FDAS: a knowledge-based framework for analysis of defects in woven textile structures, J. Text. Inst. 83 (1992) 431–448. [36] JL. Starck, M. Elad, D. Donoho, Redundant multiscale transforms and their application for morphological component separation, Adv. Imaging Electron Phys. 132 (2004) 287–348.

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032

JID: INS

ARTICLE IN PRESS L. Jia, C. Chen and S. Xu et al. / Information Sciences xxx (xxxx) xxx

[m3Gsc;October 23, 2019;22:1] 21

[37] M. Tabassian, R. Ghaderi, R. Ebrahimpour, Knitted fabric defect classification for uncertain labels based on Dempster–Shafer theory of evidence, Expert Syst. Appl. 38 (2011) 5259–5267. [38] L. Tong, W.K. Wong, C.K. Kwong, Differential evolution-based optimal Gabor filter model for fabric inspection, Neurocomputing 173 (2016) 1386–1401. [39] D.M. Tsai, C.Y. Heish, Automated surface inspection for directional textures, Image Vis. Comput. 18 (1999) 49–62. [40] C.S.C. Tsang, H.Y.T. Ngan, G.K.H. Pang, Fabric inspection based on the Elo rating method, Pattern Recognit. 51 (2016) 378–394. [41] L.A. Vese, S.J. Osher, Modeling textures with total variation minimization and oscillating patterns in image processing, J. Sci. Comput. 19 (1–3) (2003) 553–572. [42] X. Xie, A review of recent advances in surface defect detection using texture analysis techniques, Electron. Lett. Comput. Vis. Image Anal. 7 (3) (2008) 1–22. [43] X. Xie, M. Mirmehdi, TEXEMS: texture exemplars for defect detection on random textured surfaces, IEEE Trans. Pattern Anal. Mach. Int. 29 (8) (2007) 1454–1464. [44] L. Xu, Q. Yan, Y. Xia, J. Jia, Structure extraction from texture via relative total variation, ACM Trans. Graph. 31 (6) (2012) Article 139. [45] Y.F. Zhang, R.R. Bresee, Fabric defect detection and classification using image analysis, Text. Res. J. 65 (1995) 1–9.

Please cite this article as: L. Jia, C. Chen and S. Xu et al., Fabric defect inspection based on lattice segmentation and template statistics, Information Sciences, https://doi.org/10.1016/j.ins.2019.10.032