- Email: [email protected]
Complex networks-based texture extraction and classification method for mineral flotation froth images
Minerals Engineering 83 (2015) 105–116
Contents lists available at ScienceDirect
Minerals Engineering journal homepage: www.elsevier.com/locate/mineng
Complex networks-based texture extraction and classification method for mineral flotation froth images Degang Xu, Xiao Chen, Yongfang Xie ⇑, Chunhua Yang, Weihua Gui College of Information Science and Engineering, Central South University, Changsha 410083, PR China
a r t i c l e
i n f o
Article history: Received 3 September 2014 Revised 20 August 2015 Accepted 21 August 2015
Keywords: Froth image Texture feature Complex network Minkowski distance Bubble size
a b s t r a c t With recent improvements in instrumentation and computer infrastructure, machine vision technologies have produced innovative approaches for controlling and monitoring mineral flotation process. In order to efficiently analyze froth image, froth texture is used to accurately and rapidly extract froth characteristics based on the statistics of image pixels. The characterization and identification of texture require a method that can express the context surrounding each pixel by combining local and global texture characteristics. To extract the distinctive froth texture features in different production states, a novel complex networks-based texture extraction and classification method for froth imaging is proposed in this paper. A network model is constructed by expressing pixels as network nodes and similarities between pixels as network links. This method automatically sets the optimal algorithm parameters for the complex network modeling of the froth images according to bubble sizes by using the Minkowski distance. Energy and entropy measurements are used to quantify the properties of the connectivity and topology of the froth-image network model. Copper froth images at different production states extracted from the flotation monitoring system in a flotation plant are used to test the froth-image network model. The experimental results show that the proposed method accurately describes the froth image texture and also robustly classifies the different production states. Ó 2015 Published by Elsevier Ltd.
1. Introduction Froth flotation is one of the most commonly used mineral separation methods in the mineral processing industry. It is used to separate valuable minerals from unwanted materials or other materials based on the differences in wettability of the various mineral particles (Aldrich et al., 2010). The flotation process produces froths with different characteristics to carry ore particles. The visual features of the surface of the flotation froth, e.g. texture, color, size and shape, are used to indicate the production states during the process. As a consequence, during the flotation process, operators can accurately evaluate the current production state based on the visual features of the surface of the froth, and modify production strategies accordingly. This task has been implemented manually by using the operators’ naked-eye observations and their experiences (Xu et al., 2012). However, this mode of production is affected by subjective factors which are dependent on the individual operator. Thus the flotation process does not always operate at optimal production conditions. To keep process consistency, machine vision technology has been employed to characterize ⇑ Corresponding author. E-mail address: [email protected] (Y. Xie). http://dx.doi.org/10.1016/j.mineng.2015.08.017 0892-6875/Ó 2015 Published by Elsevier Ltd.
the froth surface and facilitate the control strategy for the mineral flotation process. In the worldwide, this technology has been implemented on many industrial sites (Backes and Bruno, 2010). The visual mechanisms have inspired many recognition methods used in computer vision. Many mathematical methods by using pattern recognition and computer vision concepts have also been proposed. Image texture is one of the key visual features of the froth image. An image texture is a set of metrics which are calculated in image processing and designed to quantify the perceived texture of an image. Thus the information about the spatial arrangement of color or intensities in an image can be obtained. Texture is the correlation of gray among adjacent pixels in an image and is widely used to identify objects or regions of interest in an image for pattern recognition. The froth texture describes the roughness of the froth surface, and the smooth or wrinkled surface of the froth reflects the state of production (He et al., 2013). Therefore, the investigation on froth image texture extraction can provide guidance for the optimal operation of the flotation process. There has been a lot of research about texture characterization that basically can be divided into four major categories: statistical, signal processing, model-based and dynamic. What have been intensively studied are statistical-based methods. A commonly used statistical-based method for texture analysis is gray-level
106
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
co-occurrence matrix (GLCM). However, the GLCM is not able to automatically discover all the statistical properties of the froth image (Gui et al., 2013). Some improved methods have been proposed for froth image texture extraction, such as spatial graylevel dependence matrix (SGLDM) and neighboring gray-level dependence matrix (NGLDM) (Moolman and Aldrich, 1996). Recently, Gui et al. proposed a color co-occurrence matrix method for froth image texture extraction and established the relationship between texture features and the corresponding concentrate grade (Gui et al., 2013). Other statistical-based methods include fuzzy texture spectrum (Cheng et al., 2009) and local binary pattern variance (LBPV) (Tang et al., 2011). Signal processing-based methods for froth image texture extraction have also regarded as the efficient methods. The wavelet method was firstly introduced into the extraction of flotation froth texture by Bartolacci et al. (2006). However, one defect of such approach is that appropriate wavelets must be chosen for the wavelet decomposition in order to meet some specifications because different wavelets (and corresponding analyzing functions) exist. Liu et al. (2010) proposed an improved method based on Gabor wavelet, which solved the problem for wavelets selection, but the time complexity of this method is high. Dynamic texture analysis also has attracted more interest. Liu and Lu (2002) extracted froth texture according to flotation process time. However, the proposed algorithms cannot be used to monitor the flotation process in real time. Chen et al. (2013) proposed an improved dynamic texture analysis method to extract froth texture in real time. Model-based methods are effective ways to analyze complex image, i.e., flotation froth image, for the existing models and their analytical methods can provide important guidance for image analysis. However, the model-based methods are rarely used in froth image texture extraction. Therefore, this paper investigates the complex network model-based methods for froth image texture extraction. Image texture is usually defined as a model formed by the repeated arrangements of elements or primitives according to certain direction, density and cycle. It is a common inherent characteristic of object surface. The basic texture primitives are reflected by the spatial variation in pixel intensities (gray values). Therefore, texture characterization and identification require a method that can express the context surrounding each pixel by combining local and global texture characteristics. Recently, a novel image texture extraction method based on complex networks was proposed by Backes et al. (2013). The method provides a novel idea for froth image texture extraction. Many problems of the real-world systems with given structures including those undergoing dynamic changes of the topology can be represented and solved by the complex networks theory (Costa et al., 2007). Complex networks theory is widely used to analyze topological characteristics and features extraction for digital images by building complex network models (Albert and Barabási, 2002; Newman, 2003). For flotation froth image, texture describes the variation of pixel gray values on froth surface, which is closely related to froth physical properties, e.g., shape, size and its distribution. Although the bubbles in the froth image distribute randomly and irregularly, the statistical characteristics of its physical properties in different images may presents obvious similarities or differences, which is reflected by texture. The research of complex network lies at the intersection between graph theory and statistical mechanics. The problem is represented as a complex network followed by the analysis of its topological features obtained by a set of measurements (Gonçalves et al., 2014). The degree descriptors of complex network are rotation invariant and scale invariant. Moreover, these descriptors are robust to the variation of each froth structures since they are statistical characteristics of the whole image. Given the above analysis, froth image texture can be modeled by using complex network and represented by network connectivity. Thus,
different froths can be characterized and distinguished accurately by texture feature descriptors based on degree descriptors. However, the method proposed by Backes et al. (2013) aims at the texture extraction for common digital images. Compared with these images, forth image texture are usually more complex and difficult to extract in terms of different bubble sizes, color and lighting conditions. In addition, two parameters of this method need to be determined by subjective experience, which increases the uncertainty and inaccuracy. Therefore, this paper presents a novel complex networks-based texture method for froth image extraction and classification, which automatically sets the optimal algorithm parameters for texture extraction according to the bubble size. The extracted features are used to identify different classes of froth images. Experimental results are compared with traditional texture identification methods. The remainder of this paper is organized as follows. In Section 2, the complex networks-based froth image texture extraction and classification method for the flotation process is presented. A method to efficiently classify the production state of flotation cells is proposed based on the texture feature extracted from froth images in Section 3. In Section 4, the experimental results and a discussion of the comparison of the different methods is presented. Finally, the conclusions are presented in Section 5. 2. Froth image texture extraction and classification In the development of machine vision systems for froth images, froth textures can be used to accurately and rapidly determine froth characteristics based on the statistics of image pixels. The froth characteristics are important inputs to control systems in froth flotation process. Each image pixel can be represented as a node and similarities between pixels can be mapped as links between nodes. It can be observed that various types of textures are closely related with the node degree distribution, which is very distinct from those observed in random networks. Some measurements of the network connectivity are then utilized in order to obtain image feature vectors, which can be used for the texture characterization and classification. Then, the texture feature of an image can be represented, characterized and analyzed in terms of a complex network. This study utilizes complex networks to represent and characterize texture features in froth images. A new method to classify flotation production states based on froth images has also been developed. 2.1. Complex networks and its feature descriptors A complex network is represented by a graph consisting of a set of vertices and edges. The complex network can be described by its adjacency matrix for undirected and unweighted networks in this study. The adjacency matrix is a symmetric matrix with the elements consisted by zero and one, where one represents that two nodes are linked by an edge and zero means there is no edges
Fig. 1. The adjacency matrix for a network.
107
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
between two nodes. A complex network model and its adjacency matrix are shown in Fig. 1. Each complex network model has its specific topological features that describe the characteristics of the complex network. Therefore, the discrimination and analysis of the complex networks rely on the measurements that can express the most relevant topological features and their subsequent classification (Backes et al., 2013). The measurements used in this study are described in the following. (1) Degree and degree distribution The degree kðiÞ of the complex network node is the number of edges connected to the node in the complex network model. The degree distribution pðnÞ is the proportion of nodes with n degree in the complex network, as shown in Eq. (1). It is the basic topological feature of the complex network. For the complex network model of image, degree distribution is the statistics for the frequency and amplitude of change in pixel gray values. It allows studying image texture by using the complexity of the nodes connections in the network. The smoother the texture is, the lower the amplitude and frequency of the change in pixel gray values are, thus the more uniform the degree distribution is, and vice versa
pðnÞ ¼
hðnÞ N
ð1Þ
where N is the total number of nodes in the complex network and hðnÞ is the number of complex network nodes with n degree. (1) Energy The energy is the repeated conversion frequency of certain characteristics for the complex network in a particular frequency range, which is calculated by Eq. (2). For the complex network model of image, it reflects the change frequency of pixel gray values. The smoother the texture is, the lower the change frequency is, and the smaller the energy is, and vice versa.
Energy ¼
m X ðpðnÞÞ2
ð2Þ
n¼0
where pðnÞ is the distribution of the degree of the complex network and m ¼ maxðkðiÞÞ is the maximum degree of the complex network. (1) Entropy The entropy is a measure of the chaos of the complex network. It is expressed as the uniform of the distribution of the degree and described by Eq. (3). The complex network model of image reflects the amplitude change of pixel gray values, the smoother the texture is, the lower the change amplitude is; the more the uniform degree distribution is, the larger the entropy; vice versa. m X Entropy ¼ pðnÞ log2 pðnÞ
ð3Þ
n¼0
Usually, image texture is usually defined as a model formed by the repeated arrangements of elements or primitives according to certain direction and density. In this paper, complex networks are derived from the froth texture images by expressing pixels as network nodes and similarities between pixels as network edges. Then, measurements such as the node degree, energy and entropy are used in order to quantify properties of the connectivity and topology of the analyzed networks. Because such properties are directly related to the structure of the respective texture images, they can be used as features for characterizing and classifying textures. The basic texture primitives are reflected by the spatial variation in pixel intensities (gray values). Therefore, texture characterization and identification require a method that can express the context surrounding each pixel by combining local and global
texture characteristics. For flotation froth images, texture describes the variation of pixel gray values on froth surface, which is closely related to froth physical properties, e.g., shape, size and its distribution. The degree descriptors of complex network are rotation invariant and scale invariant. The degree distribution can reflect static characteristics (average bubble size and structure of the froth). The energy and entropy are likely to reflect dynamic change characteristics (i.e., froth mobility or froth velocity of the froth). These three metrics characterize froth surface conditions associated with different metal concentrations in ore. They are correlated with the physical appearance of flotation froth (color, spatial bubble size distributions and shapes) and some of the key performance indicator variables, such as froth health and the loading of the valuable metal or mineral recovery in the concentrate. Thus, different froths can be characterized and distinguished accurately by texture feature descriptors based on degree descriptors. The extracted features can be used to identify different froth images. 2.2. Digital image described by a complex network Complex networks theory is one of the most effective ways to analyze complex systems. According to this theory, any system composed of many component units or subsystems can be regarded as a complex network if their component units can be abstracted as nodes and the relations between units can be abstracted as edges (Tang et al., 2012). Digital images can also be considered as real-world complex systems, whose component units are large numbers of pixels. Images exhibit unique visual features in terms of the context surrounding each pixel which combines local and global characteristics. Therefore, for the digital image processing, the characteristics of each pixel and the relations with its neighbors have priorities. According to the above analysis, digital images can be represented by complex networks and analyzed by the complex network theory. Measurements such as the average and maximum values of degrees and the entropy, the energy and the average joint degree can be used for the identification of broad image classes. For a gray-level image containing M M pixels, it can be abstracted as a complex network with N ¼ M 2 nodes. The mapping relations between complex network nodes and image pixels are described in Eqs. (4)–(6)
i ¼ yi þ ðxi 1ÞM
ð4Þ
xi ¼ ½ðk 1Þ=M þ 1
ð5Þ
yi ¼ modððk 1Þ; MÞ þ 1
ð6Þ
where i is the i-th node of complex network model, 1 6 i 6 N; ðxi ; yi Þ is the Cartesian coordinates of the pixel associated to the i-th node, 1 6 x; y 6 M; ½ is the rounding function; mod(*) is the complementary function. The edges of complex network need to be established between nodes according to certain criteria. A complex network modeling approach for digital images has been developed by Backes et al. (2013). By this method edges between nodes are established based on the similarities between the pixels. The following criteria are used when establishing connections:
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðx x0 Þ2 þ ðy y0 Þ2 6 r
ð7Þ 0
0
;y Þj ðx x0 Þ2 þ ðy y0 Þ2 þ r 2 jIðx;yÞIðx L 6t 2 2r
ð8Þ
where ðx; yÞ and ðx0 ; y0 Þ are the Cartesian coordinates of two pixels in a gray image I. Iðx; yÞ and Iðx0 ; y0 Þ are their pixel values. r and t are the empirical threshold values for the search radius and the similarity
108
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
value. L is the maximum number of gray levels in the image, and Eq. (8) is normalized in the interval [0, 1] by the denominator. Eq. (7) gives the definition for the similar relation between two pixels with respect to spatial position. Pixels are surrounded by a circle whose center is the target pixel and radius r covers similar pixels with the target pixel based on spatial position. Then similar pixels are chosen by using Eq. (8) from the similar pixels based on spatial position. So the pixels satisfying these two criteria are similar pixels. After finding all the similar pixels in the image, a complex network model G = (V, E) is established by vertices (the pixels) V and edges E (the links connecting similar pixels). Fig. 3 shows a complex network model of a portion of a froth image (surrounded by a red circle) obtained by using the above method with the empirical values r = 4 and t = 0.315. The circles in the figure represent the vertices of the complex network and the numbers inside the circles are the pixel values. The connections between the circles represent the network edges. The detailed modeling processes with the complex network model (the color nodes) in Fig. 2 are illustrated in Fig. 3. 2.3. Froth texture feature extraction based on the complex network model The objective is to analyze mineral flotation froth produced in flotation process. It is widely accepted that mineral concentrations
and process status are closely related to the morphological features of flotation froths. Based on froth states, operators might make changes regarding one or more input parameters in order to achieve optimal performance. While the operational status of flotation process can be characterized by the features of froth bubbles (especially the size), which determines the froth load, the collision and the attachment efficiency. The differences in bubble sizes lead to the differences in the gray value change frequency, which affects texture features. However, the complex network modeling method used by Backes et al. (2013) primarily aimed at regular images. The model is not adequate to extract texture features of froth images because froth images are more complex and irregular than other digital images. The demand of quick analyses of forth images in flotation process monitoring is also important. Therefore, an improved method for texture feature extraction of froth images based on complex network is developed, which can extract distinctive texture features in different production states. The key idea of the method is to automatically select the optimal algorithm for the complex network model based on the characteristics of the froth, which is crucial to obtain an accurate image texture. According to the definition of similar pixels proposed by Backes et al. (2013), the similar pixels tend to accumulate in the area with a low gray-value change frequency, i.e., for froth images, it is probable that similar pixels accumulate on the same froth
Fig. 2. A complex network model of froth images.
Fig. 3. The process of modeling froth images based on complex network.
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
surface. To find the similar pixels as completely as possible, a wider search range for similar pixels should be applied in the images with large bubble compared with those with small bubble. The difference in the search range for the different types of images is determined by using the Minkowski distance which is described by Eq. (9). By replacing the Euclidean distance in Eqs. (7) and (8) with Eq. (9), the resulting Eqs. (10) and (11) are displayed as follows:
M p ðx; yÞ ¼
n X jxi yi jp
!1=p ð9Þ
i¼1
In Eq. (9), when p = 1, Mp is equal to the Manhattan distance, which is the distance between two points measured along axes at right angles. When p = 2, Mp is equal to the Euclidean distance corresponding to the shortest distance between two points. When p is equal to infinity, Mp is equal to the Chebyshev distance depicting the maximum difference between two points in a certain dimension.
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ðx x0 Þp þ ðy y0 Þp 6 r
ð10Þ 0
0
;y Þj ðx x0 Þp þ ðy y0 Þp þ rp jIðx;yÞIðx L 6t p 2r
ð11Þ
The parameters in Eqs. (10) and (11) are the same as those in Eqs. (7) and (8). Considering the real-time requirement of the algorithm, the search radius r must not be too large; therefore, r = 4 in this application. The parameter t is associated with the mean of the gray scale in the image, i.e., for an image with a large mean value of the gray scale, the difference in the gray value between similar pixels can be larger than those of an image with a small mean value of the gray scale. To address this issue, t ¼ m gvalue=L; where m gvalue is the mean value of the gray scale in the image and L is the maximum number of gray levels in the image. For an 8-bit digital image, L = 255 and the parameter p is determined from the size of the froth patches are shown in Eq. (12) as follows:
h p ¼ emax
m Size Sizemin Size
i
109
the network. The degree measurements are used to compose a set of texture descriptors. Because the degree distribution of nodes reflects the isolated texture primitives and the topology feature of the network also reflects the relation among nodes and their neighbors, which are basic elements for texture features. To facilitate the quantitative description of the texture feature, the energy and the entropy of the complex network are calculated by using the distribution of its degree. Network energy can be used to examine the power content with a certain frequency. Entropy describes the uniformity of the degree distribution. The combination of these two descriptors could describe image texture feature comprehensively. These properties are then used as the descriptors of the froth image texture feature, i.e., Texture = (Energy, Entropy). The texture extraction method of froth images is summarized as follows: Step 1: Determine the optimal parameters for modeling the complex network of froth images. When the bubble sizes in the images are measured, the optimal value for the parameter p in the Minkowski distance is automatically determined according to the bubble size by Eq. (12). Step 2: Use Eqs. (4)–(6) to set the nodes of the complex network model of froth images. Then use Eqs. (10) and (11) to find similar pixels and determine the edges of the complex network model. Step 3: Calculate the energy and the entropy of the complex network model for the texture feature of froth images. 2.4. The method for froth image extraction and classification In this section, a complex networks-based texture method for the froth image extraction and classification is presented. The method automatically selects the optimal algorithm parameters for the texture extraction according to bubble sizes. Network models are derived from froth texture images by denoting pixels as network nodes and similarities between pixels as network links. Based on this model, the flotation production state can be determined. The proposed method is divided into the following three steps:
ð12Þ
where m Size is the mean size of the froth patches in the images, measured by the watershed segmentation algorithm (Yang et al., 2009), max_Size and min_Size are the maximum and minimum sizes of the specific mineral froth, which are obtained from the database, and ½ is the top rounding function. The ratio in Eq. (12) can avoid the impact of different dimensions or measuring methods for the bubble size. According to Eq. (10), when p value increases and r value remains unchanged, the number of coordinates satisfying this inequality will increase. The number of pixels covered by the searching scope also increases, which ensures that the search range for similar pixels expands without changing the values of the parameters r and t with the increase of the bubble size. The complex network modeling method for froth images is given in terms of the optimal parameter value of the criteria. According to the mechanism of froth texture, the color or gray of image pixels and bubble size distribution are the two most important factors for froth texture, which are both taken into account in this modeling method. The threshold parameter t is associated with the gray of image pixels, which solves the problem of threshold selection for froth images with different intensities; the parameter p is associated with bubble size distribution, which achieves the automatic selection of optimal parameters. Therefore, the network model of froth images established by this method is more accurate and suitable for the texture feature extraction. After the complex network model is established, the texture feature can be extracted by analyzing the topological features of
(1) Segment froth images by using the watershed segmentation algorithm to determine bubble sizes. Determine the complex network modeling algorithm for the froth images by setting the optimal parameter value of the Minkowski distance in terms of the bubble sizes. (2) Establish a complex network model for the froth images and extract the energy and the entropy of the complex network model, which consist of the froth image texture descriptors. (3) Use the linear discriminant analysis classifier to determine the production state based on obtained froth image texture feature. A flowchart of the proposed method is shown in Fig. 4. The detailed process is shown as follows: Step 1. Image preprocessing. The purpose of this step is to improve the sample quality and consequently enhance the recognition accuracy. This step includes an image filter, the gray image and image enhancement. Step 2. Image modeling based on the complex network method. In this step, the bubble size is determined by using the watershed segmentation algorithm as the basis for the determination of the optimum value of parameter p in the Minkowski distance. Then, the froth image complex network modeling method is determined in terms of the optimum parameter value in the Minkowski distance. This step guarantees that the proposed method can automatically choose the optimal
110
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Offline froth images acquisition
Offline OfflinePhase Phase
Online acquisition for froth images
Online OnlinePhase Phasee
Gray image
Step 1 Image preprocessing
Image enhancement Image preprocessing
Image filter
Preprocessed image
watershed segmentation Step 2 Ccomplex network modeling
Use optimum complex network modeling method Count froth size
Determine optimum parameter value of Minkowski distance
Step Step33 Extract Extracttexture texturedescriptors descriptors
Optimum complex network modeling method
Establish the complex network model
Extract texture feature descriptors
Extract texture descriptors
Energy Entropy
Train LDA classifier
Production state recognition
LDA classifier
Final decision
Step 4 Production State Recognition
Fig. 4. Flowchart of complex networks-based texture extraction and classification method.
texture extraction algorithm for froth images of different characteristics. Step 3. Extract the texture descriptors. In this step, the froth image complex network model is established by using the optimum modeling method. The energy and the entropy of the complex network model are extracted as froth image texture descriptors. Step 4. Production state recognition. In this step, the linear discriminant analysis (LDA) classifier is used to recognize the production state based on the froth image texture feature. 3. Production state recognition based on the froth texture feature The texture feature of flotation froth images is correlated to froth properties and can be used as the indicators for the production states. The reason is that the accumulation of mineralized bubbles exhibit distinct texture characteristics. Usually, the smooth or
wrinkled froth surface reflects the changes in the production state. The mineralized bubbles with high water contents and low minerals enrichment degree represent rough texture, where the energy value is large and entropy value is small. While the mineralized bubbles with low water contents and high minerals enrichment degree represent smooth texture, where the energy value is small and entropy value is large. Therefore, the froth texture feature is closely related to the flotation production operation variables and the flotation performance indexes. A schematic diagram of a flotation process monitoring system for a copper flotation plant located in China is shown in Fig. 5. The monitoring system is mounted on the top of the rougher cell. The froth images are captured by the industrial camera and transmitted to the computer. Then, the visual features, including the texture feature, are extracted by the feature extraction algorithms for the froth image to guide production state recognition and optimal control of the flotation process. The froth images are captured in the industrial environment with disturbances, which seriously affect the veracity of the
111
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
(a) System schematic
(b) Device for the flotation cell
(c) Froth image collected from the cell
Fig. 5. Monitoring system for the copper flotation process.
recognition. Therefore, some image preprocessing is required. The preprocessing comprises three steps, i.e. adaptive median filtering algorithm for image filter; RGB weighted average algorithm for image gray and histogram equalization for image Enhancement. The production state recognition is performed by a LDA classifier. The LDA projects the high-dimensional samples onto the optimal discriminate space. The projected samples in the new space have the smallest distance within their class and the largest distance between classes, which are described by the intra-class scatter matrix and inter-class scatter matrix. A standard LDA is described as follows. Given a dataset X ¼ ½x1 ; x2 ; ; xn 2 Rmn of n samples, where each sample belongs to one of c classes ðC 1 ; C c Þ, a classification function is computed as follows.
gðxÞ ¼ W T x
ð13Þ
where W is the linear projection that minimizes the within-class scatter as follows:
Sw 2 Rmn ¼
c X ni ðli lÞðli lÞT
ð14Þ
i¼1
and maximizes the between-class scatter as follows: mn
Sb 2 R
¼
c X X
ðx li Þðx li Þ
T
ð15Þ
i¼1 x2C i
where l is the mean of all the samples, li is the mean of the samples in class Ci, and ni is the number of samples in class Ci. This projection is obtained by maximizing the Fisher criterion as follows: T
W opt ¼ argmaxW
jW Sb Wj jW T Sw Wj
ð16Þ
The optimal solution for the optimization problem is found by solving the generalized Eigen problem as follows:
Sb W ¼ KSw W
ð17Þ
The limitations of the standard LDA approach are that it is hard to be applied to the large-scale problems. In this paper, it is solved by dividing the problem into several sub-problems. The complexity, therefore, is reduced. The number of classes and the similarity between the training and the testing data influence the separability greatly. The data are separated into several sub-clusters. Multiple small (well-separable) subspaces are built according to the method
developed by Mencía and Johannes (2009). The intra-class scatter matrix and the inter-class scatter matrix for the different samples are calculated with Eqs. (18) and (19) as follows:
Sw ¼
c X X ðx xi Þðx xi Þ0
ð18Þ
i¼1 x2C i
Sb ¼
c X mi ðxi xÞðxi xÞ0
ð19Þ
i¼1
where c is the total number of classes, mi is the number of samples in class i, xi is the mean of the samples in class i, and x is the mean of P all the samples, x ¼ m1 ni¼1 mi xi . The best direction for the projection w is obtained by solving the generalized eigenvalue problem in Eq. (20) as follows:
J w ¼ argmax
jwT Sb wj jwT Sw wj
ð20Þ
Once the projection direction w is determined, the sample xi is projected onto the optimal discriminant space and the optimal projected yi is obtained by Eq. (21) as follows:
yi ¼ wT xi
ð21Þ
The classification result is obtained by using the nearest criterion based on the Euclidean distance d, which is shown in Eq. (22):
doptimal ¼ argmin½dðy; xi Þ
ð22Þ
where y is the optimal projected y of the test sample and xi is the centroid of the optimal projected y of the i-th class. In the production state recognition process, the eigenvector x used by the LDA is the froth image texture descriptor, i.e., Texture = (Energy, Entropy). The test sample therefore belongs to the production state whose distance is between its optimal projected distance and the shortest distance of all the samples. 4. Experiments and discussion 4.1. Froth image texture analysis The experimental data used in this study are the copper rougher froth images with resolution of 800 600 pixels from the database of the flotation process monitoring system. The copper rougher
112
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
behavior is divided into four categories which are identified as classes A, B, C and D according to the production states. The froths of the first two classes are smaller in size and those of the last two classes are larger in size. Fig. 6 shows representative images from each production state class. The froths from Class A have a higher degree of mineralization and better stability. Therefore, their shapes are irregular, most of which are slim and flat. The edges between the froth patches are not obvious. The texture of Class A is the smoothest. The mineral loads on the surface of the froth in Class B are lower than that in Class A. The froth patch sizes are uneven and the majority of the froth appears as small spots. The texture of Class B is relatively smooth, but irregular. The froths in Class C have solid mineral loads and moderate sizes. The texture of Class C froth is rougher than the froth in the first two classes. The mineral loads on the froth in Class D are reduced. The froth is unstable and prone to rupture and bubble coalescence. The froth spots are uneven and the majorities are large-sized patches. The texture of Class D is the roughest. Lighting conditions have a significant impact on the accuracy of the froth image texture. In the flotation process of this study, the main factor for lighting conditions is external illumination. Usually, there will be a directional component to the lighting and the balance between the directional and diffuse lighting will change with time of day and position in the circuit. These effects are mitigated to some degree by the use of a sealed box (see Fig. 5b) which reduces the lighting variability. The images of the same production state at the different times in a day were compared (Fig. 7). It can be seen that there are the small differences in the lighting, especially in the intensity. 4.2. Experimental procedure and discussion Five experiments are performed to validate the effectiveness and lighting conditions independent property of the proposed method. The system performances are also compared with the work by Backes et al. (2013), the GLCM and the local binary pattern variance (LBPV) methods. The texture feature descriptors used in the first two methods are the energy and the entropy extracted from the complex network model of froth images. The texture feature used in the GLCM is the mean values of the energy and the entropy in four directions (0 ; 45 ; 90 ; 135 ). Although LBPV has the same feature dimensions as LBP, LBPV adds additional contrast measures to the pattern histogram and usually produces significantly better results than LBP does (Guo et al., 2010). The texture feature derived by LBPV are the LBPV micro-modes with high dimensionality, which is not suitable for the comparison with other methods, so the Edge Micro-mode and Flat Micro-mode defined by Tang et al. (2011) are used as the texture feature of LBPV in the following experiment. The Edge Micro-mode is a kind of 8-neighborhood LBP micro-modes in which the numbers of 0
and 1 are equal. It can characterize the fine features such as bubble edges; the Flat Micro-mode is a kind of 8-neighborhood LBP micromodes in which all the 8 adjacent regions are 1. It can characterize the region features such as bubble surface. (1) Experiment I – Effectiveness of the proposed method The froth image texture feature extraction method is considered effective if the differences in froth image texture can be described by the topological features of the complex network model. The degree distribution of the network model of froth images collected from different production states should be discriminated. To validate the effectiveness of the proposed method, the bubble size statistics of the froth images in Fig. 6 are obtained by using the watershed segmentation algorithm. The results are shown in Fig. 8. The range of the copper rougher bubble size from the database is [5 mm2, 35 mm2]. Then, the optimal parameter p is determined by using Eq. (12) in terms of the mean of the bubble size. The result is p = 2 for the first two classes and p = 3 for the last two classes. This ensures a wider search range for the similar pixels in froth images with larger bubbles. Fig. 9 shows the degree histogram of the nodes in the complex network models for the froth images in Fig. 6. It reflects the degree distribution of each image. In Fig. 9, the horizontal axis is the degree of node and the vertical axis is the number of node with specific degree. It can be seen from Fig. 9 that the degree distributions trends for froth images under different production states are different. Therefore, the distinctive texture feature can be extracted from these complex network models according to their distinctive degree distributions. The result indicates the effectiveness of the proposed method. The statistical distribution of bubbles can be calculated based on the watershed segmentation method. Watershed transformation tends to over-segment the image since a basin is created for every local minimum (Aldrich et al., 2010). Applications of reported bubble size estimation approaches based on the watershed segmentation to the froth images collected at phosphorus oxide flotation processes revealed unsatisfactory performance. In fact, the degree distribution is the statistics for the frequency and amplitude of change in pixel gray values. Although the bubbles in the froth image distribute randomly and irregularly, the statistical characteristics of its physical properties in different images may presents obvious similarities or differences, which is reflected by texture. The method combines the distribution of pixel intensities along the image with the complexity of the spatial distribution of patterns and irregularities in the texture. The ability of such features to express from local (i.e. close node neighborhoods) to global (i.e. more distant neighborhoods) properties of the texture contributes further to integrating the local and global aspects often found in images. The descriptors are robust to the variation of each froth structures since they are statistical characteristics of the
Fig. 6. Representative froth images from four classes of the process production state.
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
113
Fig. 7. Froth images extracted at different time of a day.
Fig. 8. Bubble size statistics.
whole image. These are fundamental features for the description and discrimination of texture images, allowing a precise and robust result. (1) Experiment II – Lighting conditions robustness property of the proposed method The froth image texture feature extraction method is tested for the lighting conditions robustness when the production state recognition at different lighting conditions is similar. To validate the lighting conditions robustness property of this method, the following experiment is respectively done under different lighting conditions: the 200 images of copper flotation froth of the four classes shown in Fig. 6 are selected randomly. Fifty images are
selected per class, in which 20 images are used as training samples and the other 30 images are used as test samples. The training samples and test samples are randomly mixed after each test and the experiment is repeated for 10 times. The average accuracy of recognition thus obtained is shown in Table 1. The results show that the production state recognition accuracy under different lighting conditions is similar. It indicates the proposed method is robust to changes of the lighting conditions. (1) Experiment III – Comparison of the methods with respect to robustness The froth image texture feature extraction method is considered robust if it provides similar texture feature collected from the
114
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Fig. 9. Degree distribution of the complex network model of representative froth images.
feature II for LBPV is the LBPV Flat Micro-mode and the other methods are entropy.
Table 1 Average accuracy of recognition at different lighting conditions. Lighting conditions
Class A (%)
Class B (%)
Class C (%)
Class D (%)
Average (%)
Day time Evening time
85.1 86.7
93.7 95.9
82.4 82.6
87.2 87.8
87.1 88.3
Table 2 Variances of texture feature extracted by different methods. Method for texture feature extraction
Variances of texture feature I
Variances of texture feature II
LBPV GLCM Method proposed by Backes et al. Proposed method
0.064 0.025 0.017
0.079 0.033 0.014
0.006
0.005
similar production states. To compare the robustness of the methods, the 15 consecutive frames of the froth images are selected from the same flotation video in the same production state. The above four methods are then used to extract their texture feature. The variances of the texture feature extracted by these methods are calculated to measure the robustness of methods. The results are shown in Table 2. The relative deviation of texture feature for each image is calculated by using Eq. (23) for intuitively comparing the robustness. The results are shown in Fig. 10. The texture feature I for LBPV is the LBPV Edge Micro-mode and the other methods are energy and the texture
Relative Deviation ¼
jx xj 100% x
ð23Þ
where x is the extracted texture feature value; x is the average value of texture feature values. It can be seen from Table 2 and Fig. 10 that the variance and the relative deviations of texture feature extracted by the proposed method are the smallest. (1) Experiment IV – Comparison of the texture feature discrimination Discrimination of the texture feature means that the texture feature of the images under similar production states should be centralized, and those under different production states should be separated. The parameter R is the ratio of within-class distance and between-class distance for the texture feature to describe the discrimination using Eq. (24). The discriminative texture feature corresponds to small values of R as follows:
PC w¼1 N d ðSw Þ R ¼ PC p ¼ 1; q ¼ 1 J d ðSp ; Sq Þ p–q
ð24Þ
where C is the number of classes, N d ðSw Þ is the within-class distance of the texture feature of class w, according to Eq. (25) and Jd ðSp ; Sq Þ is the between-class distance of the texture feature of classes p and q, according to Eq. (26) as follows:
115
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Fig. 10. Relative deviation of froth image texture feature extracted by different methods.
Nd ðSw Þ ¼
Table 3 R value of the texture feature extracted by different methods. Method for texture feature extraction
R
LBPV GLCM Method proposed by Backes et al. Proposed method
0.35 0.26 0.32 0.19
m X m X 1 dðwi ; wj Þ mðm 1Þ i¼1 j¼1
J d ðSp ; Sp Þ ¼ d
m m 1X 1X pi ; q m i¼1 m i¼1 i
ð25Þ
! ð26Þ
where dðxi ; yj Þ is the Euclidean distance between two vectors xi and yj ; xi is the texture feature vector of sample i in class x, and m is the number of samples in one class.
Fig. 11. Different production state classes by different froth image texture extraction.
116
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Table 4 Average accuracy of recognition by different methods. Texture feature extraction method
Class A (%)
Class B (%)
Class C (%)
Class D (%)
Average (%)
LBPV GLCM Method by Backes et al. Proposed method
75.8 82.7 80.4 86.7
89.2 91.2 93.5 95.2
79.5 78.4 75.3 83.5
77.4 80.8 78.5 87.2
80.5 83.3 81.9 88.2
To compare the discrimination of the texture feature extracted by the above methods, 80 copper flotation froth images for four classes are randomly selected (including 20 images in each class). Then, their texture features are extracted and calculated. A summary of the results is shown in Table 3. The data in Table 3 show that the R value of the proposed method is less than that of the other methods. This result indicates that the proposed method has better texture feature discrimination compared with the traditional texture extraction methods. (1) Experiment V – Performance comparison of production state recognition To evaluate classification accuracy, these methods are applied to an actual flotation production process. Recognition performance is reflected by the recognition accuracy of the different production states. To compare the production state recognition performance of the above four methods, 200 copper flotation froth images of the four classes shown in Fig. 6 are selected randomly. Fifty images are selected per class, i.e., 20 images are used as training samples and the other 30 images are used as test samples. The texture features of the test samples obtained by the different methods are shown in Fig. 11. The training samples and test samples are randomly mixed after each test and the experiment is repeated 10 times. The average accuracy of recognition thus obtained is shown in Table 4. The data show that the texture feature overlaps in different ways for different production states extracted by the various methods. The proposed method has less overlap than the other methods. The recognition accuracy is also shown in Table 4. The recognition accuracy for the different classes and the average accuracy for all the classes by the proposed method are much larger than by the other methods. The froth image texture feature extracted by our method is available and efficient for classifying the production state. It provides more accurate guidance information for timely adjustment of the flotation operations, which is helpful in maintaining the flotation process in an optimal running state. 5. Conclusions This study develops a novel texture feature extraction method for froth images based on complex networks to extract froth image texture feature and to monitor the process production state. The proposed method applies the complex networks theory to froth image analysis and establishes a complex network model to extract texture feature. Compared with other froth image texture extraction methods, the proposed method can automatically select the optimum complex network modeling algorithm based on the bubble size by changing the parameter p in the Minkowski distance. The robustness of this method is superior to other methods
and the texture feature extracted by this method is more accurate and can better differentiate between classes of froth. Experimental results show that the proposed method can describe the froth image texture of different mineral production states and identify the flotation states accurately. The method can be used to monitor production states in real-time, enabling operators to adjust flotation operation to ensure the flotation process maintains an optimal state. Acknowledgements This work is supported by the National Natural Science Foundation of China (Grants 61104135, 61473319), the Key Project of the National Natural Science Foundation of China (Grant 61134006), the Fundamental Research Funds for the Central Universities, Graduate Innovation Fund of Central South University and the Special Foundation of the Doctoral Tutoring Classes of the Chinese Higher Doctoral Program (Grant 20120162110076). We would like to thank the anonymous referees for their valuable comments and suggestions. References Albert, R., Barabási, A., 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 47–97. Aldrich, C., Marais, C., Shean, B.J., Cilliers, J.J., 2010. Online monitoring and control of froth flotation systems with machine vision: a review. Int. J. Miner. Process. 96 (1–4), 1–13. Backes, A.R., Casanova, D., Bruno, O.M., 2013. Texture analysis and classification: a complex network-based approach. Inf. Sci. 219, 168–180. Backes, A.R., Bruno, O.M., 2010. Shape classification using complex network and multi-scale fractal dimension. Pattern Recognit. Lett. 31 (1), 44–51. Bartolacci, G., Pelletier, P., Tessier, J., Duchesne, C., Bossé, P.A., Fournier, J., 2006. Application of numerical image analysis to process diagnosis and physical parameter measurement in mineral processes. Part I: Flotation control based on froth textural characteristics. Miner. Eng. 19 (6–8), 734–747. Chen, Q., Zhu, J., Tang, Z., 2013. Flotation state classification applying dynamic textures analysis. Comput. Appl. Chem. 30 (10), 1117–1121 (In Chinese). Cheng, C., Yang, C., Zhou, K., 2009. Flotation froth image texture extraction method based on fuzzy texture spectrum. Miner. Eng. Res. 24 (4), 62–66 (In Chinese). Costa, L., Rodrigues, F.A., Travieso, G., Villas Boas, P.R., 2007. Characterization of complex networks: a survey of measurements. Adv. Phys. 56, 167–242. Gui, W., Liu, J., Yang, C., Chen, N., Liao, X., 2013. Color co-occurrence matrix based froth image texture extraction for mineral flotation. Miner. Eng. 46–47, 60–67. Gonçalves, W.N., Machado, B.B., Bruno, O.M., 2014. A complex network approach for dynamic texture recognition. Neurocomputing 153, 211–220. Guo, Z., Zhang, L., Zhang, D., 2010. Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit. 43, 706–719. He, M., Yang, C., Wang, X., Gui, W., Wei, L., 2013. Nonparametric density estimation of froth colour texture distribution for monitoring sulphur flotation process. Miner. Eng. 53, 203–212. Liu, J., Gui, W., Mu, X., 2010. Flotation froth image texture feature extraction based on Gabor wavelets. Chin. J. Sci. Instrum. 31 (8), 1699–1775. Liu, W., Lu, M., 2002. Research on digital image processing of coal flotation froth– the liner neighbor algorithm for extracting feature of digital coal flotation forth image. Department Chem. Environ. Eng. 3 (2), 120–123. Mencía, L., Johannes, E.F., 2009. Efficient classification for large-scale problems by multiple LDA subspaces. In: Proceeding of: VISAPP 2009 – Proceedings of the Fourth International Conference on Computer Vision Theory and Applications. Moolman, D.W., Aldrich, C., 1996. The interpretation between surface forth characteristics and industrial flotation performance. Miner. Eng. 9 (8), 837–854. Newman, M.E.J., 2003. The structure and function of complex networks. SIAM Rev. 45, 167–256. Tang, J., Jiang, B., Chang, C., Luo, B., 2012. Graph structure analysis based on complex network. Digital Signal Process. 22 (5), 713–725. Tang, Z., Zhu, C., Liu, J., 2011. Flotation froth image texture extraction based on LBPV. Appl. Res. Comput. 28 (10), 3934–3936 (In Chinese). Xu, C., Gui, W., Yang, C., Zhu, H., Lin, Y., Cao, S., 2012. Flotation process fault detection using output PDF of bubble size distribution. Miner. Eng. 26, 5–12. Yang, C., Zhou, K., Mou, X., Gui, W., 2009. Froth color and size measurement method for notation based on computer vision. Chin. J. Sci. Instrum. 4 (30), 717–721 (In Chinese).
Contents lists available at ScienceDirect
Minerals Engineering journal homepage: www.elsevier.com/locate/mineng
Complex networks-based texture extraction and classification method for mineral flotation froth images Degang Xu, Xiao Chen, Yongfang Xie ⇑, Chunhua Yang, Weihua Gui College of Information Science and Engineering, Central South University, Changsha 410083, PR China
a r t i c l e
i n f o
Article history: Received 3 September 2014 Revised 20 August 2015 Accepted 21 August 2015
Keywords: Froth image Texture feature Complex network Minkowski distance Bubble size
a b s t r a c t With recent improvements in instrumentation and computer infrastructure, machine vision technologies have produced innovative approaches for controlling and monitoring mineral flotation process. In order to efficiently analyze froth image, froth texture is used to accurately and rapidly extract froth characteristics based on the statistics of image pixels. The characterization and identification of texture require a method that can express the context surrounding each pixel by combining local and global texture characteristics. To extract the distinctive froth texture features in different production states, a novel complex networks-based texture extraction and classification method for froth imaging is proposed in this paper. A network model is constructed by expressing pixels as network nodes and similarities between pixels as network links. This method automatically sets the optimal algorithm parameters for the complex network modeling of the froth images according to bubble sizes by using the Minkowski distance. Energy and entropy measurements are used to quantify the properties of the connectivity and topology of the froth-image network model. Copper froth images at different production states extracted from the flotation monitoring system in a flotation plant are used to test the froth-image network model. The experimental results show that the proposed method accurately describes the froth image texture and also robustly classifies the different production states. Ó 2015 Published by Elsevier Ltd.
1. Introduction Froth flotation is one of the most commonly used mineral separation methods in the mineral processing industry. It is used to separate valuable minerals from unwanted materials or other materials based on the differences in wettability of the various mineral particles (Aldrich et al., 2010). The flotation process produces froths with different characteristics to carry ore particles. The visual features of the surface of the flotation froth, e.g. texture, color, size and shape, are used to indicate the production states during the process. As a consequence, during the flotation process, operators can accurately evaluate the current production state based on the visual features of the surface of the froth, and modify production strategies accordingly. This task has been implemented manually by using the operators’ naked-eye observations and their experiences (Xu et al., 2012). However, this mode of production is affected by subjective factors which are dependent on the individual operator. Thus the flotation process does not always operate at optimal production conditions. To keep process consistency, machine vision technology has been employed to characterize ⇑ Corresponding author. E-mail address: [email protected] (Y. Xie). http://dx.doi.org/10.1016/j.mineng.2015.08.017 0892-6875/Ó 2015 Published by Elsevier Ltd.
the froth surface and facilitate the control strategy for the mineral flotation process. In the worldwide, this technology has been implemented on many industrial sites (Backes and Bruno, 2010). The visual mechanisms have inspired many recognition methods used in computer vision. Many mathematical methods by using pattern recognition and computer vision concepts have also been proposed. Image texture is one of the key visual features of the froth image. An image texture is a set of metrics which are calculated in image processing and designed to quantify the perceived texture of an image. Thus the information about the spatial arrangement of color or intensities in an image can be obtained. Texture is the correlation of gray among adjacent pixels in an image and is widely used to identify objects or regions of interest in an image for pattern recognition. The froth texture describes the roughness of the froth surface, and the smooth or wrinkled surface of the froth reflects the state of production (He et al., 2013). Therefore, the investigation on froth image texture extraction can provide guidance for the optimal operation of the flotation process. There has been a lot of research about texture characterization that basically can be divided into four major categories: statistical, signal processing, model-based and dynamic. What have been intensively studied are statistical-based methods. A commonly used statistical-based method for texture analysis is gray-level
106
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
co-occurrence matrix (GLCM). However, the GLCM is not able to automatically discover all the statistical properties of the froth image (Gui et al., 2013). Some improved methods have been proposed for froth image texture extraction, such as spatial graylevel dependence matrix (SGLDM) and neighboring gray-level dependence matrix (NGLDM) (Moolman and Aldrich, 1996). Recently, Gui et al. proposed a color co-occurrence matrix method for froth image texture extraction and established the relationship between texture features and the corresponding concentrate grade (Gui et al., 2013). Other statistical-based methods include fuzzy texture spectrum (Cheng et al., 2009) and local binary pattern variance (LBPV) (Tang et al., 2011). Signal processing-based methods for froth image texture extraction have also regarded as the efficient methods. The wavelet method was firstly introduced into the extraction of flotation froth texture by Bartolacci et al. (2006). However, one defect of such approach is that appropriate wavelets must be chosen for the wavelet decomposition in order to meet some specifications because different wavelets (and corresponding analyzing functions) exist. Liu et al. (2010) proposed an improved method based on Gabor wavelet, which solved the problem for wavelets selection, but the time complexity of this method is high. Dynamic texture analysis also has attracted more interest. Liu and Lu (2002) extracted froth texture according to flotation process time. However, the proposed algorithms cannot be used to monitor the flotation process in real time. Chen et al. (2013) proposed an improved dynamic texture analysis method to extract froth texture in real time. Model-based methods are effective ways to analyze complex image, i.e., flotation froth image, for the existing models and their analytical methods can provide important guidance for image analysis. However, the model-based methods are rarely used in froth image texture extraction. Therefore, this paper investigates the complex network model-based methods for froth image texture extraction. Image texture is usually defined as a model formed by the repeated arrangements of elements or primitives according to certain direction, density and cycle. It is a common inherent characteristic of object surface. The basic texture primitives are reflected by the spatial variation in pixel intensities (gray values). Therefore, texture characterization and identification require a method that can express the context surrounding each pixel by combining local and global texture characteristics. Recently, a novel image texture extraction method based on complex networks was proposed by Backes et al. (2013). The method provides a novel idea for froth image texture extraction. Many problems of the real-world systems with given structures including those undergoing dynamic changes of the topology can be represented and solved by the complex networks theory (Costa et al., 2007). Complex networks theory is widely used to analyze topological characteristics and features extraction for digital images by building complex network models (Albert and Barabási, 2002; Newman, 2003). For flotation froth image, texture describes the variation of pixel gray values on froth surface, which is closely related to froth physical properties, e.g., shape, size and its distribution. Although the bubbles in the froth image distribute randomly and irregularly, the statistical characteristics of its physical properties in different images may presents obvious similarities or differences, which is reflected by texture. The research of complex network lies at the intersection between graph theory and statistical mechanics. The problem is represented as a complex network followed by the analysis of its topological features obtained by a set of measurements (Gonçalves et al., 2014). The degree descriptors of complex network are rotation invariant and scale invariant. Moreover, these descriptors are robust to the variation of each froth structures since they are statistical characteristics of the whole image. Given the above analysis, froth image texture can be modeled by using complex network and represented by network connectivity. Thus,
different froths can be characterized and distinguished accurately by texture feature descriptors based on degree descriptors. However, the method proposed by Backes et al. (2013) aims at the texture extraction for common digital images. Compared with these images, forth image texture are usually more complex and difficult to extract in terms of different bubble sizes, color and lighting conditions. In addition, two parameters of this method need to be determined by subjective experience, which increases the uncertainty and inaccuracy. Therefore, this paper presents a novel complex networks-based texture method for froth image extraction and classification, which automatically sets the optimal algorithm parameters for texture extraction according to the bubble size. The extracted features are used to identify different classes of froth images. Experimental results are compared with traditional texture identification methods. The remainder of this paper is organized as follows. In Section 2, the complex networks-based froth image texture extraction and classification method for the flotation process is presented. A method to efficiently classify the production state of flotation cells is proposed based on the texture feature extracted from froth images in Section 3. In Section 4, the experimental results and a discussion of the comparison of the different methods is presented. Finally, the conclusions are presented in Section 5. 2. Froth image texture extraction and classification In the development of machine vision systems for froth images, froth textures can be used to accurately and rapidly determine froth characteristics based on the statistics of image pixels. The froth characteristics are important inputs to control systems in froth flotation process. Each image pixel can be represented as a node and similarities between pixels can be mapped as links between nodes. It can be observed that various types of textures are closely related with the node degree distribution, which is very distinct from those observed in random networks. Some measurements of the network connectivity are then utilized in order to obtain image feature vectors, which can be used for the texture characterization and classification. Then, the texture feature of an image can be represented, characterized and analyzed in terms of a complex network. This study utilizes complex networks to represent and characterize texture features in froth images. A new method to classify flotation production states based on froth images has also been developed. 2.1. Complex networks and its feature descriptors A complex network is represented by a graph consisting of a set of vertices and edges. The complex network can be described by its adjacency matrix for undirected and unweighted networks in this study. The adjacency matrix is a symmetric matrix with the elements consisted by zero and one, where one represents that two nodes are linked by an edge and zero means there is no edges
Fig. 1. The adjacency matrix for a network.
107
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
between two nodes. A complex network model and its adjacency matrix are shown in Fig. 1. Each complex network model has its specific topological features that describe the characteristics of the complex network. Therefore, the discrimination and analysis of the complex networks rely on the measurements that can express the most relevant topological features and their subsequent classification (Backes et al., 2013). The measurements used in this study are described in the following. (1) Degree and degree distribution The degree kðiÞ of the complex network node is the number of edges connected to the node in the complex network model. The degree distribution pðnÞ is the proportion of nodes with n degree in the complex network, as shown in Eq. (1). It is the basic topological feature of the complex network. For the complex network model of image, degree distribution is the statistics for the frequency and amplitude of change in pixel gray values. It allows studying image texture by using the complexity of the nodes connections in the network. The smoother the texture is, the lower the amplitude and frequency of the change in pixel gray values are, thus the more uniform the degree distribution is, and vice versa
pðnÞ ¼
hðnÞ N
ð1Þ
where N is the total number of nodes in the complex network and hðnÞ is the number of complex network nodes with n degree. (1) Energy The energy is the repeated conversion frequency of certain characteristics for the complex network in a particular frequency range, which is calculated by Eq. (2). For the complex network model of image, it reflects the change frequency of pixel gray values. The smoother the texture is, the lower the change frequency is, and the smaller the energy is, and vice versa.
Energy ¼
m X ðpðnÞÞ2
ð2Þ
n¼0
where pðnÞ is the distribution of the degree of the complex network and m ¼ maxðkðiÞÞ is the maximum degree of the complex network. (1) Entropy The entropy is a measure of the chaos of the complex network. It is expressed as the uniform of the distribution of the degree and described by Eq. (3). The complex network model of image reflects the amplitude change of pixel gray values, the smoother the texture is, the lower the change amplitude is; the more the uniform degree distribution is, the larger the entropy; vice versa. m X Entropy ¼ pðnÞ log2 pðnÞ
ð3Þ
n¼0
Usually, image texture is usually defined as a model formed by the repeated arrangements of elements or primitives according to certain direction and density. In this paper, complex networks are derived from the froth texture images by expressing pixels as network nodes and similarities between pixels as network edges. Then, measurements such as the node degree, energy and entropy are used in order to quantify properties of the connectivity and topology of the analyzed networks. Because such properties are directly related to the structure of the respective texture images, they can be used as features for characterizing and classifying textures. The basic texture primitives are reflected by the spatial variation in pixel intensities (gray values). Therefore, texture characterization and identification require a method that can express the context surrounding each pixel by combining local and global
texture characteristics. For flotation froth images, texture describes the variation of pixel gray values on froth surface, which is closely related to froth physical properties, e.g., shape, size and its distribution. The degree descriptors of complex network are rotation invariant and scale invariant. The degree distribution can reflect static characteristics (average bubble size and structure of the froth). The energy and entropy are likely to reflect dynamic change characteristics (i.e., froth mobility or froth velocity of the froth). These three metrics characterize froth surface conditions associated with different metal concentrations in ore. They are correlated with the physical appearance of flotation froth (color, spatial bubble size distributions and shapes) and some of the key performance indicator variables, such as froth health and the loading of the valuable metal or mineral recovery in the concentrate. Thus, different froths can be characterized and distinguished accurately by texture feature descriptors based on degree descriptors. The extracted features can be used to identify different froth images. 2.2. Digital image described by a complex network Complex networks theory is one of the most effective ways to analyze complex systems. According to this theory, any system composed of many component units or subsystems can be regarded as a complex network if their component units can be abstracted as nodes and the relations between units can be abstracted as edges (Tang et al., 2012). Digital images can also be considered as real-world complex systems, whose component units are large numbers of pixels. Images exhibit unique visual features in terms of the context surrounding each pixel which combines local and global characteristics. Therefore, for the digital image processing, the characteristics of each pixel and the relations with its neighbors have priorities. According to the above analysis, digital images can be represented by complex networks and analyzed by the complex network theory. Measurements such as the average and maximum values of degrees and the entropy, the energy and the average joint degree can be used for the identification of broad image classes. For a gray-level image containing M M pixels, it can be abstracted as a complex network with N ¼ M 2 nodes. The mapping relations between complex network nodes and image pixels are described in Eqs. (4)–(6)
i ¼ yi þ ðxi 1ÞM
ð4Þ
xi ¼ ½ðk 1Þ=M þ 1
ð5Þ
yi ¼ modððk 1Þ; MÞ þ 1
ð6Þ
where i is the i-th node of complex network model, 1 6 i 6 N; ðxi ; yi Þ is the Cartesian coordinates of the pixel associated to the i-th node, 1 6 x; y 6 M; ½ is the rounding function; mod(*) is the complementary function. The edges of complex network need to be established between nodes according to certain criteria. A complex network modeling approach for digital images has been developed by Backes et al. (2013). By this method edges between nodes are established based on the similarities between the pixels. The following criteria are used when establishing connections:
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðx x0 Þ2 þ ðy y0 Þ2 6 r
ð7Þ 0
0
;y Þj ðx x0 Þ2 þ ðy y0 Þ2 þ r 2 jIðx;yÞIðx L 6t 2 2r
ð8Þ
where ðx; yÞ and ðx0 ; y0 Þ are the Cartesian coordinates of two pixels in a gray image I. Iðx; yÞ and Iðx0 ; y0 Þ are their pixel values. r and t are the empirical threshold values for the search radius and the similarity
108
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
value. L is the maximum number of gray levels in the image, and Eq. (8) is normalized in the interval [0, 1] by the denominator. Eq. (7) gives the definition for the similar relation between two pixels with respect to spatial position. Pixels are surrounded by a circle whose center is the target pixel and radius r covers similar pixels with the target pixel based on spatial position. Then similar pixels are chosen by using Eq. (8) from the similar pixels based on spatial position. So the pixels satisfying these two criteria are similar pixels. After finding all the similar pixels in the image, a complex network model G = (V, E) is established by vertices (the pixels) V and edges E (the links connecting similar pixels). Fig. 3 shows a complex network model of a portion of a froth image (surrounded by a red circle) obtained by using the above method with the empirical values r = 4 and t = 0.315. The circles in the figure represent the vertices of the complex network and the numbers inside the circles are the pixel values. The connections between the circles represent the network edges. The detailed modeling processes with the complex network model (the color nodes) in Fig. 2 are illustrated in Fig. 3. 2.3. Froth texture feature extraction based on the complex network model The objective is to analyze mineral flotation froth produced in flotation process. It is widely accepted that mineral concentrations
and process status are closely related to the morphological features of flotation froths. Based on froth states, operators might make changes regarding one or more input parameters in order to achieve optimal performance. While the operational status of flotation process can be characterized by the features of froth bubbles (especially the size), which determines the froth load, the collision and the attachment efficiency. The differences in bubble sizes lead to the differences in the gray value change frequency, which affects texture features. However, the complex network modeling method used by Backes et al. (2013) primarily aimed at regular images. The model is not adequate to extract texture features of froth images because froth images are more complex and irregular than other digital images. The demand of quick analyses of forth images in flotation process monitoring is also important. Therefore, an improved method for texture feature extraction of froth images based on complex network is developed, which can extract distinctive texture features in different production states. The key idea of the method is to automatically select the optimal algorithm for the complex network model based on the characteristics of the froth, which is crucial to obtain an accurate image texture. According to the definition of similar pixels proposed by Backes et al. (2013), the similar pixels tend to accumulate in the area with a low gray-value change frequency, i.e., for froth images, it is probable that similar pixels accumulate on the same froth
Fig. 2. A complex network model of froth images.
Fig. 3. The process of modeling froth images based on complex network.
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
surface. To find the similar pixels as completely as possible, a wider search range for similar pixels should be applied in the images with large bubble compared with those with small bubble. The difference in the search range for the different types of images is determined by using the Minkowski distance which is described by Eq. (9). By replacing the Euclidean distance in Eqs. (7) and (8) with Eq. (9), the resulting Eqs. (10) and (11) are displayed as follows:
M p ðx; yÞ ¼
n X jxi yi jp
!1=p ð9Þ
i¼1
In Eq. (9), when p = 1, Mp is equal to the Manhattan distance, which is the distance between two points measured along axes at right angles. When p = 2, Mp is equal to the Euclidean distance corresponding to the shortest distance between two points. When p is equal to infinity, Mp is equal to the Chebyshev distance depicting the maximum difference between two points in a certain dimension.
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ðx x0 Þp þ ðy y0 Þp 6 r
ð10Þ 0
0
;y Þj ðx x0 Þp þ ðy y0 Þp þ rp jIðx;yÞIðx L 6t p 2r
ð11Þ
The parameters in Eqs. (10) and (11) are the same as those in Eqs. (7) and (8). Considering the real-time requirement of the algorithm, the search radius r must not be too large; therefore, r = 4 in this application. The parameter t is associated with the mean of the gray scale in the image, i.e., for an image with a large mean value of the gray scale, the difference in the gray value between similar pixels can be larger than those of an image with a small mean value of the gray scale. To address this issue, t ¼ m gvalue=L; where m gvalue is the mean value of the gray scale in the image and L is the maximum number of gray levels in the image. For an 8-bit digital image, L = 255 and the parameter p is determined from the size of the froth patches are shown in Eq. (12) as follows:
h p ¼ emax
m Size Sizemin Size
i
109
the network. The degree measurements are used to compose a set of texture descriptors. Because the degree distribution of nodes reflects the isolated texture primitives and the topology feature of the network also reflects the relation among nodes and their neighbors, which are basic elements for texture features. To facilitate the quantitative description of the texture feature, the energy and the entropy of the complex network are calculated by using the distribution of its degree. Network energy can be used to examine the power content with a certain frequency. Entropy describes the uniformity of the degree distribution. The combination of these two descriptors could describe image texture feature comprehensively. These properties are then used as the descriptors of the froth image texture feature, i.e., Texture = (Energy, Entropy). The texture extraction method of froth images is summarized as follows: Step 1: Determine the optimal parameters for modeling the complex network of froth images. When the bubble sizes in the images are measured, the optimal value for the parameter p in the Minkowski distance is automatically determined according to the bubble size by Eq. (12). Step 2: Use Eqs. (4)–(6) to set the nodes of the complex network model of froth images. Then use Eqs. (10) and (11) to find similar pixels and determine the edges of the complex network model. Step 3: Calculate the energy and the entropy of the complex network model for the texture feature of froth images. 2.4. The method for froth image extraction and classification In this section, a complex networks-based texture method for the froth image extraction and classification is presented. The method automatically selects the optimal algorithm parameters for the texture extraction according to bubble sizes. Network models are derived from froth texture images by denoting pixels as network nodes and similarities between pixels as network links. Based on this model, the flotation production state can be determined. The proposed method is divided into the following three steps:
ð12Þ
where m Size is the mean size of the froth patches in the images, measured by the watershed segmentation algorithm (Yang et al., 2009), max_Size and min_Size are the maximum and minimum sizes of the specific mineral froth, which are obtained from the database, and ½ is the top rounding function. The ratio in Eq. (12) can avoid the impact of different dimensions or measuring methods for the bubble size. According to Eq. (10), when p value increases and r value remains unchanged, the number of coordinates satisfying this inequality will increase. The number of pixels covered by the searching scope also increases, which ensures that the search range for similar pixels expands without changing the values of the parameters r and t with the increase of the bubble size. The complex network modeling method for froth images is given in terms of the optimal parameter value of the criteria. According to the mechanism of froth texture, the color or gray of image pixels and bubble size distribution are the two most important factors for froth texture, which are both taken into account in this modeling method. The threshold parameter t is associated with the gray of image pixels, which solves the problem of threshold selection for froth images with different intensities; the parameter p is associated with bubble size distribution, which achieves the automatic selection of optimal parameters. Therefore, the network model of froth images established by this method is more accurate and suitable for the texture feature extraction. After the complex network model is established, the texture feature can be extracted by analyzing the topological features of
(1) Segment froth images by using the watershed segmentation algorithm to determine bubble sizes. Determine the complex network modeling algorithm for the froth images by setting the optimal parameter value of the Minkowski distance in terms of the bubble sizes. (2) Establish a complex network model for the froth images and extract the energy and the entropy of the complex network model, which consist of the froth image texture descriptors. (3) Use the linear discriminant analysis classifier to determine the production state based on obtained froth image texture feature. A flowchart of the proposed method is shown in Fig. 4. The detailed process is shown as follows: Step 1. Image preprocessing. The purpose of this step is to improve the sample quality and consequently enhance the recognition accuracy. This step includes an image filter, the gray image and image enhancement. Step 2. Image modeling based on the complex network method. In this step, the bubble size is determined by using the watershed segmentation algorithm as the basis for the determination of the optimum value of parameter p in the Minkowski distance. Then, the froth image complex network modeling method is determined in terms of the optimum parameter value in the Minkowski distance. This step guarantees that the proposed method can automatically choose the optimal
110
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Offline froth images acquisition
Offline OfflinePhase Phase
Online acquisition for froth images
Online OnlinePhase Phasee
Gray image
Step 1 Image preprocessing
Image enhancement Image preprocessing
Image filter
Preprocessed image
watershed segmentation Step 2 Ccomplex network modeling
Use optimum complex network modeling method Count froth size
Determine optimum parameter value of Minkowski distance
Step Step33 Extract Extracttexture texturedescriptors descriptors
Optimum complex network modeling method
Establish the complex network model
Extract texture feature descriptors
Extract texture descriptors
Energy Entropy
Train LDA classifier
Production state recognition
LDA classifier
Final decision
Step 4 Production State Recognition
Fig. 4. Flowchart of complex networks-based texture extraction and classification method.
texture extraction algorithm for froth images of different characteristics. Step 3. Extract the texture descriptors. In this step, the froth image complex network model is established by using the optimum modeling method. The energy and the entropy of the complex network model are extracted as froth image texture descriptors. Step 4. Production state recognition. In this step, the linear discriminant analysis (LDA) classifier is used to recognize the production state based on the froth image texture feature. 3. Production state recognition based on the froth texture feature The texture feature of flotation froth images is correlated to froth properties and can be used as the indicators for the production states. The reason is that the accumulation of mineralized bubbles exhibit distinct texture characteristics. Usually, the smooth or
wrinkled froth surface reflects the changes in the production state. The mineralized bubbles with high water contents and low minerals enrichment degree represent rough texture, where the energy value is large and entropy value is small. While the mineralized bubbles with low water contents and high minerals enrichment degree represent smooth texture, where the energy value is small and entropy value is large. Therefore, the froth texture feature is closely related to the flotation production operation variables and the flotation performance indexes. A schematic diagram of a flotation process monitoring system for a copper flotation plant located in China is shown in Fig. 5. The monitoring system is mounted on the top of the rougher cell. The froth images are captured by the industrial camera and transmitted to the computer. Then, the visual features, including the texture feature, are extracted by the feature extraction algorithms for the froth image to guide production state recognition and optimal control of the flotation process. The froth images are captured in the industrial environment with disturbances, which seriously affect the veracity of the
111
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
(a) System schematic
(b) Device for the flotation cell
(c) Froth image collected from the cell
Fig. 5. Monitoring system for the copper flotation process.
recognition. Therefore, some image preprocessing is required. The preprocessing comprises three steps, i.e. adaptive median filtering algorithm for image filter; RGB weighted average algorithm for image gray and histogram equalization for image Enhancement. The production state recognition is performed by a LDA classifier. The LDA projects the high-dimensional samples onto the optimal discriminate space. The projected samples in the new space have the smallest distance within their class and the largest distance between classes, which are described by the intra-class scatter matrix and inter-class scatter matrix. A standard LDA is described as follows. Given a dataset X ¼ ½x1 ; x2 ; ; xn 2 Rmn of n samples, where each sample belongs to one of c classes ðC 1 ; C c Þ, a classification function is computed as follows.
gðxÞ ¼ W T x
ð13Þ
where W is the linear projection that minimizes the within-class scatter as follows:
Sw 2 Rmn ¼
c X ni ðli lÞðli lÞT
ð14Þ
i¼1
and maximizes the between-class scatter as follows: mn
Sb 2 R
¼
c X X
ðx li Þðx li Þ
T
ð15Þ
i¼1 x2C i
where l is the mean of all the samples, li is the mean of the samples in class Ci, and ni is the number of samples in class Ci. This projection is obtained by maximizing the Fisher criterion as follows: T
W opt ¼ argmaxW
jW Sb Wj jW T Sw Wj
ð16Þ
The optimal solution for the optimization problem is found by solving the generalized Eigen problem as follows:
Sb W ¼ KSw W
ð17Þ
The limitations of the standard LDA approach are that it is hard to be applied to the large-scale problems. In this paper, it is solved by dividing the problem into several sub-problems. The complexity, therefore, is reduced. The number of classes and the similarity between the training and the testing data influence the separability greatly. The data are separated into several sub-clusters. Multiple small (well-separable) subspaces are built according to the method
developed by Mencía and Johannes (2009). The intra-class scatter matrix and the inter-class scatter matrix for the different samples are calculated with Eqs. (18) and (19) as follows:
Sw ¼
c X X ðx xi Þðx xi Þ0
ð18Þ
i¼1 x2C i
Sb ¼
c X mi ðxi xÞðxi xÞ0
ð19Þ
i¼1
where c is the total number of classes, mi is the number of samples in class i, xi is the mean of the samples in class i, and x is the mean of P all the samples, x ¼ m1 ni¼1 mi xi . The best direction for the projection w is obtained by solving the generalized eigenvalue problem in Eq. (20) as follows:
J w ¼ argmax
jwT Sb wj jwT Sw wj
ð20Þ
Once the projection direction w is determined, the sample xi is projected onto the optimal discriminant space and the optimal projected yi is obtained by Eq. (21) as follows:
yi ¼ wT xi
ð21Þ
The classification result is obtained by using the nearest criterion based on the Euclidean distance d, which is shown in Eq. (22):
doptimal ¼ argmin½dðy; xi Þ
ð22Þ
where y is the optimal projected y of the test sample and xi is the centroid of the optimal projected y of the i-th class. In the production state recognition process, the eigenvector x used by the LDA is the froth image texture descriptor, i.e., Texture = (Energy, Entropy). The test sample therefore belongs to the production state whose distance is between its optimal projected distance and the shortest distance of all the samples. 4. Experiments and discussion 4.1. Froth image texture analysis The experimental data used in this study are the copper rougher froth images with resolution of 800 600 pixels from the database of the flotation process monitoring system. The copper rougher
112
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
behavior is divided into four categories which are identified as classes A, B, C and D according to the production states. The froths of the first two classes are smaller in size and those of the last two classes are larger in size. Fig. 6 shows representative images from each production state class. The froths from Class A have a higher degree of mineralization and better stability. Therefore, their shapes are irregular, most of which are slim and flat. The edges between the froth patches are not obvious. The texture of Class A is the smoothest. The mineral loads on the surface of the froth in Class B are lower than that in Class A. The froth patch sizes are uneven and the majority of the froth appears as small spots. The texture of Class B is relatively smooth, but irregular. The froths in Class C have solid mineral loads and moderate sizes. The texture of Class C froth is rougher than the froth in the first two classes. The mineral loads on the froth in Class D are reduced. The froth is unstable and prone to rupture and bubble coalescence. The froth spots are uneven and the majorities are large-sized patches. The texture of Class D is the roughest. Lighting conditions have a significant impact on the accuracy of the froth image texture. In the flotation process of this study, the main factor for lighting conditions is external illumination. Usually, there will be a directional component to the lighting and the balance between the directional and diffuse lighting will change with time of day and position in the circuit. These effects are mitigated to some degree by the use of a sealed box (see Fig. 5b) which reduces the lighting variability. The images of the same production state at the different times in a day were compared (Fig. 7). It can be seen that there are the small differences in the lighting, especially in the intensity. 4.2. Experimental procedure and discussion Five experiments are performed to validate the effectiveness and lighting conditions independent property of the proposed method. The system performances are also compared with the work by Backes et al. (2013), the GLCM and the local binary pattern variance (LBPV) methods. The texture feature descriptors used in the first two methods are the energy and the entropy extracted from the complex network model of froth images. The texture feature used in the GLCM is the mean values of the energy and the entropy in four directions (0 ; 45 ; 90 ; 135 ). Although LBPV has the same feature dimensions as LBP, LBPV adds additional contrast measures to the pattern histogram and usually produces significantly better results than LBP does (Guo et al., 2010). The texture feature derived by LBPV are the LBPV micro-modes with high dimensionality, which is not suitable for the comparison with other methods, so the Edge Micro-mode and Flat Micro-mode defined by Tang et al. (2011) are used as the texture feature of LBPV in the following experiment. The Edge Micro-mode is a kind of 8-neighborhood LBP micro-modes in which the numbers of 0
and 1 are equal. It can characterize the fine features such as bubble edges; the Flat Micro-mode is a kind of 8-neighborhood LBP micromodes in which all the 8 adjacent regions are 1. It can characterize the region features such as bubble surface. (1) Experiment I – Effectiveness of the proposed method The froth image texture feature extraction method is considered effective if the differences in froth image texture can be described by the topological features of the complex network model. The degree distribution of the network model of froth images collected from different production states should be discriminated. To validate the effectiveness of the proposed method, the bubble size statistics of the froth images in Fig. 6 are obtained by using the watershed segmentation algorithm. The results are shown in Fig. 8. The range of the copper rougher bubble size from the database is [5 mm2, 35 mm2]. Then, the optimal parameter p is determined by using Eq. (12) in terms of the mean of the bubble size. The result is p = 2 for the first two classes and p = 3 for the last two classes. This ensures a wider search range for the similar pixels in froth images with larger bubbles. Fig. 9 shows the degree histogram of the nodes in the complex network models for the froth images in Fig. 6. It reflects the degree distribution of each image. In Fig. 9, the horizontal axis is the degree of node and the vertical axis is the number of node with specific degree. It can be seen from Fig. 9 that the degree distributions trends for froth images under different production states are different. Therefore, the distinctive texture feature can be extracted from these complex network models according to their distinctive degree distributions. The result indicates the effectiveness of the proposed method. The statistical distribution of bubbles can be calculated based on the watershed segmentation method. Watershed transformation tends to over-segment the image since a basin is created for every local minimum (Aldrich et al., 2010). Applications of reported bubble size estimation approaches based on the watershed segmentation to the froth images collected at phosphorus oxide flotation processes revealed unsatisfactory performance. In fact, the degree distribution is the statistics for the frequency and amplitude of change in pixel gray values. Although the bubbles in the froth image distribute randomly and irregularly, the statistical characteristics of its physical properties in different images may presents obvious similarities or differences, which is reflected by texture. The method combines the distribution of pixel intensities along the image with the complexity of the spatial distribution of patterns and irregularities in the texture. The ability of such features to express from local (i.e. close node neighborhoods) to global (i.e. more distant neighborhoods) properties of the texture contributes further to integrating the local and global aspects often found in images. The descriptors are robust to the variation of each froth structures since they are statistical characteristics of the
Fig. 6. Representative froth images from four classes of the process production state.
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
113
Fig. 7. Froth images extracted at different time of a day.
Fig. 8. Bubble size statistics.
whole image. These are fundamental features for the description and discrimination of texture images, allowing a precise and robust result. (1) Experiment II – Lighting conditions robustness property of the proposed method The froth image texture feature extraction method is tested for the lighting conditions robustness when the production state recognition at different lighting conditions is similar. To validate the lighting conditions robustness property of this method, the following experiment is respectively done under different lighting conditions: the 200 images of copper flotation froth of the four classes shown in Fig. 6 are selected randomly. Fifty images are
selected per class, in which 20 images are used as training samples and the other 30 images are used as test samples. The training samples and test samples are randomly mixed after each test and the experiment is repeated for 10 times. The average accuracy of recognition thus obtained is shown in Table 1. The results show that the production state recognition accuracy under different lighting conditions is similar. It indicates the proposed method is robust to changes of the lighting conditions. (1) Experiment III – Comparison of the methods with respect to robustness The froth image texture feature extraction method is considered robust if it provides similar texture feature collected from the
114
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Fig. 9. Degree distribution of the complex network model of representative froth images.
feature II for LBPV is the LBPV Flat Micro-mode and the other methods are entropy.
Table 1 Average accuracy of recognition at different lighting conditions. Lighting conditions
Class A (%)
Class B (%)
Class C (%)
Class D (%)
Average (%)
Day time Evening time
85.1 86.7
93.7 95.9
82.4 82.6
87.2 87.8
87.1 88.3
Table 2 Variances of texture feature extracted by different methods. Method for texture feature extraction
Variances of texture feature I
Variances of texture feature II
LBPV GLCM Method proposed by Backes et al. Proposed method
0.064 0.025 0.017
0.079 0.033 0.014
0.006
0.005
similar production states. To compare the robustness of the methods, the 15 consecutive frames of the froth images are selected from the same flotation video in the same production state. The above four methods are then used to extract their texture feature. The variances of the texture feature extracted by these methods are calculated to measure the robustness of methods. The results are shown in Table 2. The relative deviation of texture feature for each image is calculated by using Eq. (23) for intuitively comparing the robustness. The results are shown in Fig. 10. The texture feature I for LBPV is the LBPV Edge Micro-mode and the other methods are energy and the texture
Relative Deviation ¼
jx xj 100% x
ð23Þ
where x is the extracted texture feature value; x is the average value of texture feature values. It can be seen from Table 2 and Fig. 10 that the variance and the relative deviations of texture feature extracted by the proposed method are the smallest. (1) Experiment IV – Comparison of the texture feature discrimination Discrimination of the texture feature means that the texture feature of the images under similar production states should be centralized, and those under different production states should be separated. The parameter R is the ratio of within-class distance and between-class distance for the texture feature to describe the discrimination using Eq. (24). The discriminative texture feature corresponds to small values of R as follows:
PC w¼1 N d ðSw Þ R ¼ PC p ¼ 1; q ¼ 1 J d ðSp ; Sq Þ p–q
ð24Þ
where C is the number of classes, N d ðSw Þ is the within-class distance of the texture feature of class w, according to Eq. (25) and Jd ðSp ; Sq Þ is the between-class distance of the texture feature of classes p and q, according to Eq. (26) as follows:
115
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Fig. 10. Relative deviation of froth image texture feature extracted by different methods.
Nd ðSw Þ ¼
Table 3 R value of the texture feature extracted by different methods. Method for texture feature extraction
R
LBPV GLCM Method proposed by Backes et al. Proposed method
0.35 0.26 0.32 0.19
m X m X 1 dðwi ; wj Þ mðm 1Þ i¼1 j¼1
J d ðSp ; Sp Þ ¼ d
m m 1X 1X pi ; q m i¼1 m i¼1 i
ð25Þ
! ð26Þ
where dðxi ; yj Þ is the Euclidean distance between two vectors xi and yj ; xi is the texture feature vector of sample i in class x, and m is the number of samples in one class.
Fig. 11. Different production state classes by different froth image texture extraction.
116
D. Xu et al. / Minerals Engineering 83 (2015) 105–116
Table 4 Average accuracy of recognition by different methods. Texture feature extraction method
Class A (%)
Class B (%)
Class C (%)
Class D (%)
Average (%)
LBPV GLCM Method by Backes et al. Proposed method
75.8 82.7 80.4 86.7
89.2 91.2 93.5 95.2
79.5 78.4 75.3 83.5
77.4 80.8 78.5 87.2
80.5 83.3 81.9 88.2
To compare the discrimination of the texture feature extracted by the above methods, 80 copper flotation froth images for four classes are randomly selected (including 20 images in each class). Then, their texture features are extracted and calculated. A summary of the results is shown in Table 3. The data in Table 3 show that the R value of the proposed method is less than that of the other methods. This result indicates that the proposed method has better texture feature discrimination compared with the traditional texture extraction methods. (1) Experiment V – Performance comparison of production state recognition To evaluate classification accuracy, these methods are applied to an actual flotation production process. Recognition performance is reflected by the recognition accuracy of the different production states. To compare the production state recognition performance of the above four methods, 200 copper flotation froth images of the four classes shown in Fig. 6 are selected randomly. Fifty images are selected per class, i.e., 20 images are used as training samples and the other 30 images are used as test samples. The texture features of the test samples obtained by the different methods are shown in Fig. 11. The training samples and test samples are randomly mixed after each test and the experiment is repeated 10 times. The average accuracy of recognition thus obtained is shown in Table 4. The data show that the texture feature overlaps in different ways for different production states extracted by the various methods. The proposed method has less overlap than the other methods. The recognition accuracy is also shown in Table 4. The recognition accuracy for the different classes and the average accuracy for all the classes by the proposed method are much larger than by the other methods. The froth image texture feature extracted by our method is available and efficient for classifying the production state. It provides more accurate guidance information for timely adjustment of the flotation operations, which is helpful in maintaining the flotation process in an optimal running state. 5. Conclusions This study develops a novel texture feature extraction method for froth images based on complex networks to extract froth image texture feature and to monitor the process production state. The proposed method applies the complex networks theory to froth image analysis and establishes a complex network model to extract texture feature. Compared with other froth image texture extraction methods, the proposed method can automatically select the optimum complex network modeling algorithm based on the bubble size by changing the parameter p in the Minkowski distance. The robustness of this method is superior to other methods
and the texture feature extracted by this method is more accurate and can better differentiate between classes of froth. Experimental results show that the proposed method can describe the froth image texture of different mineral production states and identify the flotation states accurately. The method can be used to monitor production states in real-time, enabling operators to adjust flotation operation to ensure the flotation process maintains an optimal state. Acknowledgements This work is supported by the National Natural Science Foundation of China (Grants 61104135, 61473319), the Key Project of the National Natural Science Foundation of China (Grant 61134006), the Fundamental Research Funds for the Central Universities, Graduate Innovation Fund of Central South University and the Special Foundation of the Doctoral Tutoring Classes of the Chinese Higher Doctoral Program (Grant 20120162110076). We would like to thank the anonymous referees for their valuable comments and suggestions. References Albert, R., Barabási, A., 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 47–97. Aldrich, C., Marais, C., Shean, B.J., Cilliers, J.J., 2010. Online monitoring and control of froth flotation systems with machine vision: a review. Int. J. Miner. Process. 96 (1–4), 1–13. Backes, A.R., Casanova, D., Bruno, O.M., 2013. Texture analysis and classification: a complex network-based approach. Inf. Sci. 219, 168–180. Backes, A.R., Bruno, O.M., 2010. Shape classification using complex network and multi-scale fractal dimension. Pattern Recognit. Lett. 31 (1), 44–51. Bartolacci, G., Pelletier, P., Tessier, J., Duchesne, C., Bossé, P.A., Fournier, J., 2006. Application of numerical image analysis to process diagnosis and physical parameter measurement in mineral processes. Part I: Flotation control based on froth textural characteristics. Miner. Eng. 19 (6–8), 734–747. Chen, Q., Zhu, J., Tang, Z., 2013. Flotation state classification applying dynamic textures analysis. Comput. Appl. Chem. 30 (10), 1117–1121 (In Chinese). Cheng, C., Yang, C., Zhou, K., 2009. Flotation froth image texture extraction method based on fuzzy texture spectrum. Miner. Eng. Res. 24 (4), 62–66 (In Chinese). Costa, L., Rodrigues, F.A., Travieso, G., Villas Boas, P.R., 2007. Characterization of complex networks: a survey of measurements. Adv. Phys. 56, 167–242. Gui, W., Liu, J., Yang, C., Chen, N., Liao, X., 2013. Color co-occurrence matrix based froth image texture extraction for mineral flotation. Miner. Eng. 46–47, 60–67. Gonçalves, W.N., Machado, B.B., Bruno, O.M., 2014. A complex network approach for dynamic texture recognition. Neurocomputing 153, 211–220. Guo, Z., Zhang, L., Zhang, D., 2010. Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit. 43, 706–719. He, M., Yang, C., Wang, X., Gui, W., Wei, L., 2013. Nonparametric density estimation of froth colour texture distribution for monitoring sulphur flotation process. Miner. Eng. 53, 203–212. Liu, J., Gui, W., Mu, X., 2010. Flotation froth image texture feature extraction based on Gabor wavelets. Chin. J. Sci. Instrum. 31 (8), 1699–1775. Liu, W., Lu, M., 2002. Research on digital image processing of coal flotation froth– the liner neighbor algorithm for extracting feature of digital coal flotation forth image. Department Chem. Environ. Eng. 3 (2), 120–123. Mencía, L., Johannes, E.F., 2009. Efficient classification for large-scale problems by multiple LDA subspaces. In: Proceeding of: VISAPP 2009 – Proceedings of the Fourth International Conference on Computer Vision Theory and Applications. Moolman, D.W., Aldrich, C., 1996. The interpretation between surface forth characteristics and industrial flotation performance. Miner. Eng. 9 (8), 837–854. Newman, M.E.J., 2003. The structure and function of complex networks. SIAM Rev. 45, 167–256. Tang, J., Jiang, B., Chang, C., Luo, B., 2012. Graph structure analysis based on complex network. Digital Signal Process. 22 (5), 713–725. Tang, Z., Zhu, C., Liu, J., 2011. Flotation froth image texture extraction based on LBPV. Appl. Res. Comput. 28 (10), 3934–3936 (In Chinese). Xu, C., Gui, W., Yang, C., Zhu, H., Lin, Y., Cao, S., 2012. Flotation process fault detection using output PDF of bubble size distribution. Miner. Eng. 26, 5–12. Yang, C., Zhou, K., Mou, X., Gui, W., 2009. Froth color and size measurement method for notation based on computer vision. Chin. J. Sci. Instrum. 4 (30), 717–721 (In Chinese).