A P-t -SNE and MMEMPM based quality-related process monitoring method for a variety of hot rolling processes

Control Engineering Practice 89 (2019) 1–11

Contents lists available at ScienceDirect

Control Engineering Practice journal homepage: www.elsevier.com/locate/conengprac

A P-𝑡-SNE and MMEMPM based quality-related process monitoring method for a variety of hot rolling processes Chuanfang Zhang a , Kaixiang Peng a,b ,∗, Jie Dong a a

Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, PR China b National Engineering Research Center for Advanced Rolling Technology, University of Science and Technology Beijing, Beijing, 100083, PR China

ARTICLE

INFO

Keywords: Process monitoring Quality-related fault detection P-𝑡-SNE MMEMPM A variety of hot rolling processes

ABSTRACT A quality-related process monitoring method based on parametric 𝑡-distributed stochastic neighbour embedding (P-𝑡-SNE) and modified minimum error minimax probability machine (MMEMPM) is proposed for a variety of hot rolling processes. This work pays close attention to the connections between different types of strip steel products instead of conventional multimode methods, which could be useful in process modelling and monitoring. First, a parametric version of 𝑡-SNE is developed for streaming data in the hot rolling process. Then, a new space separation method based on parametric 𝑡-SNE and quality variables is implemented to extract internal shared information and external unique information among different types for quality-related fault detection. After that, a variety identification method is proposed to identify the online data in the hot rolling process. Finally, the performance of proposed quality-related process monitoring method is examined through a real hot rolling process. The efficiency and feasibility are demonstrated.

1. Introduction With extensive applications of integrated automation system and rapid developments of information technology, the process industry develops towards large-scale, complex and integrated, resulting a significant progress of process monitoring research (Chen, Ding, Peng, Yang, & Gui, 2018; Chen, Ding, Zhang, Li, & Z., 2016; Chen, Kruger, Meronk, & Leung, 2004; Ge, Song, Ding, & Huang, 2017; Huang & Kadali, 2008; Qin, 2003, 2012; Shardt, Hao, & Ding, 2015). As a typical representative of process industry, a hot rolling process consists of many units, which forms a series structure body from raw material to final product (Shardt et al., 2018). For plant engineers, the most concerned quality indices are final thickness and flatness of the strip. With increasing customization-oriented demands, strip steel production has gradually shifted from large batch manufacturing to multiple types and small batch production. High intellectual, high flexible manufacturing system based on multiple species and small batch production will be the mainstream in future. A 1700 mm strip hot rolling line in Ansteel Corporation in Liaoning Province of P. R. China has an annual output of about 3.5 million tonnes. For different products, steel grades and strip sizes (thickness and width) are different, the rolling system also works in different operating conditions. Besides, the uncertainties of raw materials, equipment states, external environments and disturbances have made it complicated and changeable. Such characteristic is defined as the variety characteristic in this paper. Noted that the

variety characteristic in the hot rolling process is unlike multimode in chemical processes (Yu & Qin, 2010). For chemical processes, the changes of raw material ratio mainly determine the multimode characteristic. Because of variety characteristic in the hot rolling process, it need to be concerned about the changes of strip forces in stands, strip tensions between stands, real-time strip temperatures, strip flow stresses, microstructure and mechanical properties. Thus, monitoring models in each type need to be established, which is time-consuming and inefficient. Quality monitoring for a variety of hot rolling processes has become an urgent problem for ensuring the quality and production efficiency of strip steel products. In the last decade, quality-related process monitoring (QPM) methods have gained more attention in both academic research and industrial applications (Ding, Yin, Peng, Hao, & Shen, 2013; Haghani, Jeinsch, & Ding, 2014; Peng, Zhang, Li, & Zhou, 2013; Peng, Zhang, You, Dong, & Wang, 2016). Compared with model-based QPM, datadriven ones have been raised and proved to be useful for quality control (Martin & Morris, 2000). Partial least squares (PLS) method is generally used as the fundamental technique in QPM and has been investigated intensively (Qin, 1998; Westerhuis, Kourti, & MacGregor, 1998). As an extension of PLS, kernel PLS (KPLS) based QPM methods were widely used. A hierarchical kernel PLS (HKPLS) was proposed for the batch processes, which gave more nonlinear information compared to hierarchical partial least squares (HPLS) and multiway PLS (MPLS) (Zhang &

∗ Corresponding author. E-mail address: [email protected] (K. Peng).

https://doi.org/10.1016/j.conengprac.2019.05.006 Received 11 December 2018; Received in revised form 3 May 2019; Accepted 4 May 2019 Available online 21 May 2019 0967-0661/© 2019 Elsevier Ltd. All rights reserved.

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

2. Preliminaries and problem formulation

Hu, 2011). A total kernel PLS (TKPLS) model was developed in Peng et al. (2013). TKPLS possesses the characteristics of both KPLS and total PLS (TPLS) and can monitor the hot rolling process effectively. Considering that multiple operating points are an inherent nature of process industry, traditional methods that assume single-operation condition cannot satisfy multimodal monitoring requirements, and thus, incorporating a multimodal model into an online monitoring scheme becomes a new issue (Feital, Kruger, Dutra, Pinto, & Lima, 2012; Ge & Song, 2009; Hong, Huang, & Ding, 2017; Jiang, Huang, & Yan, 2016; Sedghi, Sadeghian, & Huang, 2017; Zhang & Wang, 2012; Zhang, Zhou, Qin, & Chai, 2010; Zhao, Zhang, & Xu, 2004, 2006). A multiple principal component analysis model was proposed in Zhao et al. (2004), which adopted principal angles to measure the similarities of any two models. Based on Bayesian inference and two-step independent component analysis-principal component analysis feature extraction strategy, a new unsupervised method for multimode process monitoring was proposed (Ge & Song, 2009). For large-scale processes, a multiblock kernel partial least squares (MBKPLS) model was developed for decentralized fault diagnosis (Zhang et al., 2010). These multimode methods have been demonstrated to be generally effective, but they neglected the connections between each mode, and the common information or specific information in multimode. To solve such problem, a subspace separation based on locally linear embedded (LLE) was proposed in Zhang and Wang (2012). However, LLE attempts to preserve local geometry and requires each high-dimensional object to be associated with only a single location in the low-dimensional space. This makes it difficult to unfold many-to-one mappings in which a single ambiguous object really belongs in several disparate locations in the low-dimensional space (Hinton & Roweis, 2003). Fortunately, 𝑡-SNE is capable of capturing much of the local structure of samples in the original space very well, while also revealing global structure. The differences between conventional multimode methods and proposed are shown in Fig. 1. In a variety of hot rolling processes, it is not comprehensive to model and monitor a single type. Some process characteristics may remain invariant when the production plan changes, which is called the internal shared space in this paper, such as different types of products have the same production facilities and production procedures. Of course, each type has its divergent part, which indicates some process characteristics are changeable in the hot rolling process, e.g. significant changes of process variables in reheating furnaces, roughing mill, finishing mill, laminar cooling and coiler, etc. In this paper, it is called the external unique space of each type. In this paper, based on the previous work in subspace separation by Zhang and Wang (2012), a P-𝑡-SNE and MMEMPM based quality-related process monitoring method is proposed for a variety of hot rolling processes. Motivated by above observations, the main innovations and contributions of this paper are to: (1) develop a parametric version of 𝑡-SNE to overcome the out-ofsample extension problem, which can deal with streaming data in the hot rolling process; (2) propose a more efficient space separation based on P-𝑡-SNE and quality variables to overcome the drawback of LLE; (3) put forward a variety identification approach of online data based on MMEMPM for the integrality of proposed quality-related process monitoring method; (4) apply the proposed quality-related process monitoring method to a practical hot rolling process and compare the monitoring performance between the present and the existing ones. The rest of this paper is organized as follows. First, basics of 𝑡SNE and MEMPM are briefly reviewed, and the problem formulation is given in Section 2. Secondly, the proposed P-𝑡-SNE and MMEMPM based quality-related process monitoring method is presented detailly in Section 3. Then, the proposed method is applied to a practical hot rolling process in Section 4. Finally, conclusions and some outlooks are made in Section 5.

In this section, 𝑡-SNE and MEMPM are reviewed briefly, which will motivate the problem formulation for the proposed method. 2.1. 𝑡-SNE 𝑡-SNE is a nonlinear dimension reduction method based on manifold learning, which was proposed by Maaten and Hinton (2008). As an extension of SNE, 𝑡-SNE has the symmetric cost function and uses 𝑡distribution to replace Gaussian distribution in the projection space. Thus, the crowding problem (clusters in low-dimensional space gather together and cannot be distinguished) is solved by 𝑡-SNE. The gradient optimization of cost function becomes much easier. It minimizes the sum of Kullback–Leibler divergences over all data points. The cost function 𝐸 is defined as: ∑ 𝑝𝑖𝑗 ( ) ∑∑ (1) 𝐾𝐿 𝑃𝑖 ‖ 𝑝𝑖𝑗 log 𝐸= ‖𝑄𝑖 = 𝑞 𝑖

𝑖

𝑗

𝑖𝑗

where 𝑝𝑖𝑗 is the joint probability of 𝐱𝑖 and 𝐱𝑗 (data points in highdimensional space ∈ 𝑅𝐷 ), 𝑞𝑖𝑗 is the joint probability of 𝐲𝑖 and 𝐲𝑗 (mapped data points in projection space ∈ 𝑅𝑑 , with 𝑑 < 𝐷). The goal of 𝑡-SNE is to find mapped data point 𝐲𝑖 such that difference between 𝑝𝑖𝑗 and 𝑞𝑖𝑗 becomes small as measured by the cost function 𝐸. 𝑡-SNE relies on a gradient descent technique, and the gradient of 𝐸 is calculated as: ( )−1 ∑( )( ) 𝛿𝐸 ‖ ‖2 =4 𝑝𝑖𝑗 − 𝑞𝑖𝑗 𝐲𝑖 − 𝐲𝑗 1 + ‖𝐲𝑖 − 𝐲𝑗 ‖ (2) ‖ ‖ 𝛿𝐲𝑖 𝑗

As a canonical type of nonparametric dimension reduction method, 𝑡-SNE pays more attention to finding low-dimensional mapped data points such that the features of high-dimensional data points are preserved as much as possible. Unlike parametric methods, it provides a mapping of the given data points only, that is to say, 𝑡-SNE is short of an explicit out-of-sample extension. Because of this drawback, 𝑡-SNE cannot deal with online data of the hot rolling process and monitor it. To cope with this problem, 𝑡-SNE is modified into a parametric version, and furthermore, quality variables are fused in it for space separation and quality-related process monitoring in the hot rolling process. 2.2. MEMPM for binary classification A novel model for two-category classification tasks called the Minimax Probability Machine (MPM) was proposed by Lanckriet, Ghaoui, Bhattacharyya, and Jordan (2002). This model tries to minimize the probability of misclassification of future data points in a worst-case setting, i.e., under all possible choices of class-conditional densities with a given mean and covariance matrix. When compared with traditional generative models, MPM avoids making assumptions with respect to the data distribution. Such assumptions are often invalid and lack generality. However, MPM assumes that the accuracies of two classes are the same 𝛼. This assumption seems inappropriate, since there is no need for the accuracies of both classes to be identical. Minimum Error Minimax Probability Machine is a generalized form of MPM, which was proposed by Huang, Yang, King, Lyu, and Chan (2004). MEMPM eliminates this unreasonable constraint and a general model is proposed, which has more attractive performance for binary classification. The formulation of MEMPM model is written as follows: max 𝜃𝛼 + (1 − 𝜃) 𝛽 { } inf 𝐏𝐫 𝐚𝑇 𝐱 ≥ 𝑏 ≥ 𝛼 ̄ 𝐱) 𝐱∼(𝐱,𝛴 { 𝑇 } (inf ) 𝐏𝐫 𝐚 𝐲 ≤ 𝑏 ≥ 𝛽

𝛼,𝛽,𝐚≠0,𝑏

s.t.

(3)

̄ 𝐲 𝐲∼ 𝐲,𝛴

{ } { } ̄ 𝛴𝐱 and 𝐲 ∼ 𝐲, ̄ 𝛴𝐲 are random vectors with means where 𝐱 ∼ 𝐱, ̄ 𝐲, 𝐲̄ ∈ 𝑅𝑛 and 𝛴𝐱 , 𝛴𝐲 ∈ 𝑅𝑛×𝑛 . 𝛼 and covariance matrices. Note that 𝐱, 𝐱, and 𝛽 represent classification accuracies for class 𝐱 and 𝐲. 𝜃 ∈ (0, 1) is 2

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 1. Conventional multimode method and the proposed.

a weighting coefficient. 𝐚𝑇 𝐳 − 𝑏 = 0 (𝐚, 𝐳 ∈ 𝑅𝑛 , 𝑏 ∈ 𝑅) is the hyperplane that separates the two classes of points. MEMPM avoids making assumptions with respect to the data distribution, which make it very suitable in a variety of hot rolling processes, because process data in different types may behave according to different distributions. However, MEMPM was originally proposed for binary classification, and the hot rolling process has many types of products. How to effectively modify MEMPM for multiclass classification is still an ongoing research issue. One-versus-one and oneversus-all are common approaches for binary classifiers constructing a multiclass classifier, but one-versus-one has much higher computational complexity than one-versus-all. Thus, a modified approach is used to solve this issue.

mapping given by the ELM kernel is defined as: ∑ ( ) 𝐲 = 𝑓 (𝐱) = 𝐚𝑖 𝑘̃ 𝐱, 𝐱𝑖 |𝑝 → +∞ 𝑖 ( ) ∑ 𝑘 𝐱, 𝐱𝑖 |𝑝 → +∞ = 𝐚𝑖 √ √ ( ) 𝑖 𝑘 (𝐱, 𝐱 |𝑝 → +∞ ) 𝑘 𝐱𝑖 , 𝐱𝑖 |𝑝 → +∞ [ ( )] ∑ 𝐸 𝜎 (𝐰𝐱) 𝜎 𝐰𝐱𝑖 𝐚𝑖 √ = √ [ ( ) ( )] 𝑖 𝐸 [𝜎 (𝐰𝐱) 𝜎 (𝐰𝐱)] 𝐸 𝜎 𝐰𝐱𝑖 𝜎 𝐰𝐱𝑖

(4)

where 𝐚𝑖 are parameters corresponding to points in the projection ( ) space and 𝐱𝑖 is a sample in 𝐗′ . 𝑘̃ 𝐱, 𝐱𝑖 |𝑝 → +∞ is the normalized asymptotic ELM kernel with a single hidden layer of units 𝑝. 𝜎 (𝐰𝐱) is the activation of ELM with hidden weight 𝐰. For a particular case in Frénay and Verleysen (2011), the weights and the biases in hidden layer are randomly generated from an isotropic Gaussian distribution with variance 𝜎𝐸𝐿𝑀 and the analytical expression of asymptotic ELM kernel is as follows: ( ) 𝑘 𝐱, 𝐱𝑖 |𝑝 → +∞ ( ) 1 + 𝐱, 𝐱𝑖 2 (5) = arcsin √( ) )( 𝜋 ( ) 1 1 + 1 + (𝐱, 𝐱) + 1 + 𝐱𝑖 , 𝐱𝑖 2 2

3. Quality-related process monitoring for a variety of hot rolling processes This section contains a detailed description of the proposed qualityrelated process monitoring method. First, an extension of 𝑡-SNE based on extreme learning machine (ELM) kernel is discussed to overcome the drawback of 𝑡-SNE mentioned in Section 2.1. Afterward, using quality variables, a space separation method based on P-𝑡-SNE and quality variables is proposed for offline modelling. Then the details about variety identification based on MMEMPM is introduced for online data in the hot rolling process.

2𝜎𝐸𝐿𝑀

2𝜎𝐸𝐿𝑀

From experiments in Frénay and Verleysen (2011), it can be seen that 𝜎𝐸𝐿𝑀 does not affect the results. If it is chosen large enough, namely, ELM kernel can be considered as a kind of parameterless kernel. As a matter of fact, ELM kernel strongly reduces the computational cost and yields competitive performance to the Gaussian kernel. Then, the parameter 𝐚𝑖 is optimized by minimizing the sum squared error ( )‖2 ∑ ‖ 𝐲 − 𝑓 𝐱𝑖 ‖ and constitute a coefficient matrix 𝐀 in the following 𝑖‖ ‖ 𝑖 ‖ from:

3.1. Parametric version of 𝑡-SNE based on ELM kernel [ ] For a given preprocessed data set 𝐗 = 𝐱1 ; ...; 𝐱𝑛 ∈ 𝑅𝑛×𝑚 , where 𝑛 ′ and 𝑚 denote the number of samples and variables. A subset 𝐗′ ∈ 𝑅𝑛 ×𝑚 ′ ′ 𝑛 for training the mapping is randomly selected from 𝐗, then 𝐘 ∈ 𝑅 ×𝑟 ′ is obtained by original 𝑡-SNE, where 𝑛 and 𝑟 denote the number of samples and dimension of projection space. The parametric version of 𝑡-SNE is based on a normalized asymptotic ELM kernel. And the explicit

𝐀 = 𝐊†𝑡𝑟 𝐘′

(6)

where 𝐊†𝑡𝑟 is the pseudo-inverse of the Gram matrix with each entry [ ] ( ) 𝐊𝑡𝑟 𝑖,𝑗 = 𝑘̃ 𝐱𝑖 , 𝐱𝑗 |𝑝 → +∞ . 3

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

) ( When a new data 𝐱𝑛𝑒𝑤 is obtained, 𝐊𝑛𝑒𝑤 𝑛𝑒𝑤,𝑖 = 𝑘̃ 𝐱𝑛𝑒𝑤 , 𝐱𝑖 |𝑝 → +∞ is done for each entry in 𝐊𝑛𝑒𝑤 , then 𝐘𝑛𝑒𝑤 can be calculated as: ]

[

′ 𝐘𝑛𝑒𝑤 = 𝐊𝑛𝑒𝑤 𝐀 = 𝐊𝑛𝑒𝑤 𝐊−1 𝑡𝑟 𝐘

(7)

3.2. A space separation method based on P-𝑡-SNE and quality variables In a variety of hot rolling processes, different types have similarities in the internal shared space and dissimilarities in the external unique space. Global monitoring is applied to the internal shared space, which reflects states of production facilities and properties of production procedures in the hot rolling process. Local monitoring is implemented in the external unique space, which reflects the different microstructure and properties in steel slabs, strip forces and strip temperatures. The internal shared space and external unique spaces are quite different, but they can interact on each other, such as wear of working roll can affect the rolling force. The key of quality-related fault detection for a variety of hot rolling processes lies in how to separate the spaces. The P-𝑡-SNE discussed in Section 3.1 is used to obtain lowdimensional projection coordinates of 𝐗 as[ the internal shared space of ] a variety of hot rolling processes, 𝐗𝐼𝑆 = 𝐱𝐼𝑆,1 ; … ; 𝐱𝐼𝑆,𝑛 ∈ 𝑅𝑛×𝑟 = 𝐘. And the remaining part of 𝐗 represents the external unique space of each type. To make space separable, set 𝐘 = 𝐗𝐖, where 𝐖 is mapping matrix from high dimensional space to low [dimensional ] space. Considering the preprocessed quality variables 𝐐 = 𝐪1 ; … ; 𝐪𝑘 ∈ 𝑅𝑛×𝑘 , [ ] where 𝑘 is [the number variables, let 𝐓 = 𝐭1 , … , 𝐭𝑟 ∈ 𝑅𝑛×𝑟 ] of quality and 𝐏 = 𝐩1 , … , 𝐩𝑟 ∈ 𝑅𝑘×𝑟 be the scoring and loading matrix of 𝐐, 𝐓 = 𝐐𝐏. The relationship between 𝐗𝐼𝑆 and 𝐐 is represented by covariance, which makes the relation information between the internal shared space and quality variables be fully extracted. The model of space separation is to minimize the following loss function: 𝑚𝑖𝑛𝑒 (𝐖, 𝐏) =

𝑛 ∑ 𝑖=1

=

𝑛 ∑ 𝑖=1

= = 𝑠.𝑡.

Fig. 2. MMEMPM.

where 𝐗𝐸𝑈 ,𝑖 ∈ 𝑅𝑛𝑖 ×𝑚 is the external unique space in Type 𝑖, where 𝑖 = 1, … , 𝑐, 𝑐 is the number of types, and 𝑛1 + ⋯ + 𝑛𝑐 = 𝑛. Thus, the space separation result is as follows: [ ] 𝐗 = 𝐗𝐼𝑆 𝐖𝑇 + 𝐗𝐸𝑈 = 𝐘𝐖𝑇 + 𝐗𝐸𝑈 ,1 ; ...; 𝐗𝐸𝑈 ,𝑐 (13) After space separation, each space should be monitored for the safety and stability of hot rolling process. The proposed quality-related fault detection method is similar to that using T2 statistic and SPE statistic in KPLS, and the statistics are defined as 𝑇 = 𝐱𝐖Λ−1 𝐖𝑇 𝐱 T2 = 𝐱𝐼𝑆 Λ−1 𝐱𝐼𝑆 )‖2 ‖( )‖2 ‖( SPE = ‖ 𝐱 − 𝐱𝐼𝑆 𝐖𝑇 ‖ = ‖ 𝐱 − 𝐱𝐖𝐖𝑇 ‖ ‖ ‖ ‖ ‖ )‖2 ( ( )) ( ( ))𝑇 ‖ ( = ‖𝐱 𝐈 − 𝐖𝐖𝑇 ‖ = 𝐱 𝐈 − 𝐖𝐖𝑇 𝐱 𝐈 − 𝐖𝐖𝑇 ‖( ‖ ) ( ) 𝑇 = 𝐱 𝐈 − 𝐖𝐖𝑇 𝐈 − 𝐖𝐖𝑇 𝐱𝑇

( ) ‖𝐱𝐼𝑆,𝑖 − 𝐱𝑖 𝐖‖2 − 𝟏𝑇 cov 𝐗𝐼𝑆 , 𝐓 𝟏𝑟 ‖ ‖ 𝑟 ‖𝐲𝑖 − 𝐱𝑖 𝐖‖2 − 𝟏𝑇 cov (𝐗𝐖, 𝐐𝐏) 𝟏𝑟 ‖ ‖ 𝑟 𝑇

𝑡𝑟𝑎𝑐𝑒 (𝐘 − 𝐗𝐖) (𝐘 − 𝐗𝐖) − 𝟏𝑇𝑟 (𝐗𝐖)𝑇 𝐐𝐏𝟏𝑟 𝑡𝑟𝑎𝑐𝑒 (𝐘 − 𝐗𝐖) (𝐘 − 𝐗𝐖)𝑇 − 𝟏𝑇𝑟 𝐖𝑇 𝐗𝑇 𝐐𝐏𝟏𝑟 { 𝑇 𝐖 𝐖 = 𝐈𝑟 𝐏𝑇 𝐏 = 𝐈 𝑟

where Λ−1 is the diagonal matrix of the inverse of the eigenvalues associated with internal shared space 𝐗𝐼𝑆 on the training set, 𝐈𝑚 is an 𝑚 -dimensional identity matrix. Assuming each variable vector is multivariate normal, the detection threshold of T2 and SPE can be calculated as follows (Choi & Lee, 2005): ( ) 𝑟 𝑛2 − 1 2 T𝐼𝑆 = 𝐹 𝑛 (𝑛 − 𝑟) 𝑟,𝑛−𝑟,𝛼 (15) SPE𝐸𝑈 ,𝑖 = 𝑔𝑖 𝜒ℎ2 ,𝛼

(8)

where 𝟏𝑟 = [1, … , 1]𝑇 ∈ 𝑅𝑟×1 , 𝐈𝑟 is an 𝑟-dimensional identity matrix. Constraints in the above model can be incorporated into above loss function with the help of Lagrange multiplier method: ( ) 𝐿 = 𝑡𝑟𝑎𝑐𝑒 (𝐘 − 𝐗𝐖) (𝐘 − 𝐗𝐖)𝑇 − 𝟏𝑇𝑟 𝐖𝑇 𝐗𝑇 𝐐𝐏𝟏𝑟 + 𝜆1 𝐖𝑇 𝐖 − 𝐈𝑟 ( ) + 𝜆2 𝐏𝑇 𝐏 − 𝐈𝑟 (9)

𝑖

where 𝐹𝑟,𝑛−𝑟,𝛼 is an F-distribution with 𝑟 and 𝑛 − 𝑟 degrees of freedom, 𝑟 is the dimension of 𝐗𝐼𝑆 , 𝑛 is the number of samples, 𝛼 is the confidence level. 𝑔𝑖 𝜒ℎ2 ,𝛼 is a 𝜒 2 -distribution with the weighted parameter 𝑔𝑖 = 𝑖 𝑠2𝑖 ∕2𝜇𝑖 and ℎ𝑖 = 2𝜇𝑖2 ∕𝑠2𝑖 degrees of freedom, 𝜇𝑖 and 𝑠2𝑖 are the estimated mean and variance of the SPE𝐸𝑈 ,𝑖 . Consequently, offline modelling of the proposed method is completed.

Setting partial derivatives to zero gives the constraints: 𝜕𝐿 = 2𝐗𝑇 (𝐗𝐖 − 𝐘) − 𝐗𝑇 𝐐𝐏 + 2𝜆1 𝐖 = 0𝑚×𝑟 𝜕𝐖 𝜕𝐿 = −𝐐𝑇 𝐗𝐖 + 2𝜆2 𝐏 = 0𝑘×𝑟 𝜕𝐏 𝜕𝐿 = 𝐖𝑇 𝐖 − 𝐈𝑟 = 𝟎𝑟×𝑟 𝜕𝜆1 𝜕𝐿 = 𝐏𝑇 𝐏 − 𝐈𝑟 = 𝟎𝑟×𝑟 𝜕𝜆2

3.3. Variety identification based on MMEMPM

(10)

For a variety of hot rolling processes, when a new sample 𝐱𝑛𝑒𝑤 is obtained, the type of it needs to be checked. variety identification is a problem of classification. In this paper, the MMEMPM algorithm is introduced for variety identification. Consider 𝑐 types in the hot rolling process, where a training set [( ) ( )] 𝐱1 , y1 ; ....; 𝐱𝑛 , y𝑛 ∈ 𝑅𝑛×(𝑚+1) is an 𝑚-dimensional process sample, y𝑖 ∈ {1, … , 𝑐} represents the corresponding type. Selecting an optimal sequence of variables could provide better classification performance. Thus, before variety identification, stepwise multiple regression (Rencher, 2002) is used to determine the correlated variables. Oneversus-rest strategy is adopted to train 𝑐 − 1 binary MEMPM classifiers, while one-versus-one needs to train 𝑐 (𝑐 − 1)∕2 binary classifiers and one-versus-all needs to train 𝑐 binary classifiers. First, MEMPM1 is used to separate the training samples of Type 1 from the rest, that is to say,

From Eq. (10), the following equation can be obtained: 𝑇

𝐗 𝐐𝐐𝑇 𝐗𝐖 = 4𝜆1 𝜆2 𝐖 = 𝜆𝐖

(11)

The eigenvectors corresponding to the first 𝑟 minimum eigenvalues of matrix 𝐗𝑇 𝐐𝐐𝑇 𝐗 construct the mapping matrix 𝐖 from highdimensional space to low-dimensional space. Then the external unique space in each type can be obtained as: 𝐗𝐸𝑈 ,1 = 𝐗1 − 𝐗𝐼𝑆,1 𝐖𝑇 𝐗𝐸𝑈 ,2 = 𝐗2 − 𝐗𝐼𝑆,2 𝐖𝑇 ⋮ 𝐗𝐸𝑈 ,𝑐 = 𝐗𝑐 − 𝐗𝐼𝑆,𝑐 𝐖𝑇

(14)

(12)

4

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 3. Flowchart of the proposed quality-related process monitoring method.

training samples of Type 1 have positive labels and all the others with negative labels. Then, MEMPM2 is used to classify the training samples of Type 2 in the rest samples. Repeat such steps until all the rest samples are classified, which is shown in Fig. 2. Theoretically, an optimization problem is solved by the 𝑖th MEMPM as follows: ( ) 𝜃𝜅 2 𝛼𝑖 + (1 − 𝜃) 𝛽𝑖 max ( ) (16) 𝛽𝑖 ,𝐚𝑖 ≠0 𝜅 2 𝛼 + 1 𝑖 𝑇 ̄ =1 s.t. 𝐚𝑖 (𝐱̄ − 𝐲)

processes. The quality-related fault detection logic is: ⎧ ⎪T2𝑛𝑒𝑤 ≤ T2𝐼𝑆 and SPE𝑛𝑒𝑤 ≤ SPE𝐸𝑈 ,𝑖 , fault − f ree in Type 𝑖 ⎨ ⎪T2𝑛𝑒𝑤 > T2𝐼𝑆 or SPE𝑛𝑒𝑤 >SPE𝐸𝑈 ,𝑖 , faulty in Type 𝑖 ⎩

In summary, flow chart of the proposed quality process monitoring method is shown in Fig. 3 and the procedures are as follows: (1) Offline modelling Step 1: Training dataset 𝐗 and 𝐐 with different types are collected and preprocessed. Step 2: Internal shared space and 𝑐 external unique spaces are extracted by the proposed space separation method. Step 3: The detection thresholds of T2 and SPE are calculated in correspond spaces. (2) Online monitoring Step 4: A new online sample 𝐱𝑛𝑒𝑤 is collected and preprocessed. Step 5: The type of 𝐱𝑛𝑒𝑤 is identified by MMEMPM. Step 6: The statistics of T2𝑛𝑒𝑤 and SPE𝑛𝑒𝑤 are calculated. Step 7: The faulty condition is detected if T2𝑛𝑒𝑤 or SPE𝑛𝑒𝑤 exceeds the corresponding detection threshold.

For a new sample 𝐱𝑛𝑒𝑤 , if 𝐚𝑇𝑖 𝐱𝑛𝑒𝑤 − 𝑏𝑖 is positive, then 𝐱𝑛𝑒𝑤 belongs to Type 𝑖, otherwise, it belongs to the rest. But the above MEMPM classifiers are given in a linear configuration, it is possible to extend the linear version, for example, using Mercer kernels and thereby forming nonlinear decision boundaries. Kernelization of MEMPM (Huang et al., 2004) has better classification performance than linear MEMPM and comparison experiments are conducted in next section. After knowing the type of 𝐱𝑛𝑒𝑤 , T2 and SPE are calculated as follows for internal shared space and corresponding external unique space: T2𝑛𝑒𝑤 = 𝐱𝑛𝑒𝑤 𝐖Λ−1 𝐖𝑇 𝐱𝑛𝑒𝑤 ( )( )𝑇 𝑇 SPE𝑛𝑒𝑤 = 𝐱𝑛𝑒𝑤 𝐈 − 𝐖𝐖𝑇 𝐈 − 𝐖𝐖𝑇 𝐱𝑛𝑒𝑤

(18)

(17)

4. Application to the hot rolling process

T2𝑛𝑒𝑤

If or SPE𝑛𝑒𝑤 exceeds the detection threshold, then the fault occurs in the corresponding type, otherwise, the process is in normal condition and the products have no quality problems. It is worthwhile to note that the proposed is unapplicable to time-varying or dynamic

In this section, an example based on a practical hot rolling process is studied to show the monitoring performance of proposed method. The simulation environment in this paper is MATLAB V2016b on a computer CPU with the main frequency of 2.8 GHz and 8 GB memory. 5

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 4. Schematic layout of the hot rolling process.

Fig. 5. Variety identification results: (a) MMEMPM, (b) The kernel version. Table 1 Process and quality variables in FMA. Variable 𝐺𝑊 1 ∼ 𝐺𝑊 7 𝐺𝐷1 ∼ 𝐺𝐷7 𝐹𝑊 1 ∼ 𝐹𝑊 7 𝐹𝐷1 ∼ 𝐹𝐷7 𝐵2 ∼ 𝐵7 𝑄𝑇 𝑄𝐹

Table 2 Typical quality-related faults in FMA.

Type

Description

Unit

Fault no.

Type

Measured Measured Measured Measured Measured Quality Quality

Roll gap of 𝑖th stand on the work side, 𝑖 = 1, … , 7 Roll gap of 𝑖th stand on the drive side, 𝑖 = 1, … , 7 Rolling force of 𝑖th stand on the work side, 𝑖 = 1, … , 7 Rolling force of 𝑖th stand on the drive side, 𝑖 = 1, … , 7 Bending force of 𝑖th stand, 𝑖 = 2, … , 7 Finishing mill exit strip thickness Finishing mill exit strip flatness

mm mm MN MN MN mm I

1 (1.55 mm)

Thickness-related (± 0.040 mm) Thickness-related (± 0.050 mm) Thickness-related (± 0.050 mm)

2 (2.70 mm) 3 (3.95 mm)

4 (5.00 mm)

Flatness-related (20 I-Units)

Description Malfunction of gap control loop in the 4th stand Malfunction of gap control loop in the 5th stand Stiction of the cooling valve between the 2nd and the 3rd stands Fault of bending force measuring sensor in the 6th stand

Occurrence 1501st 1501st 1001st

1001st

4.1. Hot rolling process introduction Modern hot rolling process is a completely automated process industry, which is always characterized by high temperature, high speed, long process and variety. As the most important area in the hot rolling process, the finishing mill area (FMA) ensures continuous production, stability and high precision of the final products (Ma, Dong, Peng, & Zhang, 2017), as shown in Fig. 4. There are usually 7 stands in the FMA. In each set of stands, there are four rolls, two working rolls in

the middle and two supporting rolls on both sides. In each stand, a hydraulic system is equipped to offer rolling and bending forces so that the thickness of strip steel can be reduced, and an electromechanical system has to rotate the rolls so that the strip steel can be smoothly moved forward. Meanwhile, the rolling forces and bending forces can be measured in real-time by piezomagnetic sensors and straingauge sensors. Due to the high temperature and speed, the gap between two 6

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 6. Monitoring results of Fault 1: (a-b) KPLS, (c-d) LLESS, (e-f) Proposed.

stands is nearly unmeasurable. In practical FMA, the gap between two working rolls is approximately estimated by measuring the altitude difference between two supporting rolls. The finishing exit thickness and flatness are two important quality variables to determine the product quality, which are measured by an X-ray device and charge coupled device (CCD) in the last mill exit. The fault occurred in the previous stand cannot be controlled until X-ray device and CCD detect the abnormal values of thickness and flatness. Thus, how to establish realtime relationship among the process variables and quality variables, and monitor the process are hot issues. In this paper, process variables consist of roll gaps, rolling forces and roll bending forces (the first stand has no roll-bending force) of seven stands. The finishing exit thickness and flatness are the quality variables, as shown in Table 1.

4.2. Monitoring results of the proposed method In order to validate the effectiveness of proposed method, it is applied to FMA. According to different industrial demands, production types can be determined by different thicknesses of the steel strip. Four types are selected for modelling: the thickness equalling 1.55 mm, 2.70 mm, 3.95 mm and 5.00 mm are used. Various types of datasets corresponding to normal and faulty operations are collected for modelling and monitoring. The sampling interval is 0.01 s, 3000 normal training samples in each type are used for KPLS, LLE based subspace separation (LLESS) and proposed method modelling, 2000 testing samples in each type are used for online monitoring. Meanwhile, the confidence levels for monitoring are defined as 99%. 7

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 7. Monitoring results of Fault 2: (a-b) KPLS, (c-d) LLESS, (e-f) Proposed.

For different faults occurring in FMA, four types of faults in aforementioned types are considered as the detected objects in this section, which are shown in Table 2. Because the exit thicknesses exceed thickness guaranteed deviation in Type 1.55 mm (± 0.040 mm), Type 2.70 mm (± 0.050 mm), Type 3.95 mm (± 0.050 mm), Fault 1, Fault 2 and Fault 3 can be considered as thickness-related faults. The exit flatness exceed flatness guaranteed deviation in Type 5.00 mm (20 I-Units), and thus Fault 4 can be regard as a kind of flatnessrelated fault. Furthermore, the duration of aforementioned faults is 5 s, namely, 500 samples. For online monitoring, first of all, variety identification based on MMEMPM and its kernel version are used to identify the type of testing samples, with results shown in Fig. 5. Misclassifications exist in

Fig. 5(a), which indicates linear MEMPM cannot identify the last 2000 testing sample very well and its kernel version has more powerful identification ability. Then, the identified testing samples are monitored in the internal shared space and corresponding external unique space. The FDR and FAR indices are used to evaluate the performance of the three methods, which are defined by: ( ) samples 𝐽 > 𝐽𝑡ℎ |𝑓 ≠ 0 FDR = (19) total samples (𝑓 ≠ 0)

FAR = 8

( ) samples 𝐽 > 𝐽𝑡ℎ |𝑓 = 0 total samples (𝑓 = 0)

(20)

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 8. Monitoring results of Fault 3: (a-b) KPLS, (c-d) LLESS, (e-f) Proposed.

where 𝐽 and 𝐽𝑡ℎ are the test statistic and corresponding threshold, and 𝑓 represents the fault. A more sensitive monitoring scheme has the higher FDR and lower FAR, and it should be able to detect a fault at its early stages, and thus, it has the lower detection time delay. Table 3 shows the FDRs and FARs of KPLS, LLESS and the proposed. For each fault, the highest fault detection rate and lowest fault alarm rate are shown in bold type. Obviously, P-𝑡-SNE has the highest fault detection rates and the lowest fault alarm rates. The findings show that the proposed method outperforms the other methods. Fault 1 and Fault 2 represent the failure of hydraulic roll gap control structure in Type 1 (1.55 mm) and Type 2 (2.70 mm), which occur at the 1501st monitoring sample. The values of roll gaps in the 4th and the 5th stand are directly affected, and then the values of rolling forces in

the 4th and the 5th stand are also affected. Because of the influence of feedback control system, roll gaps and rolling forces will be changed in the following stands, and then finally the exit thicknesses are affected. The monitoring results of KPLS, LLESS and the proposed are shown in Fig. 6(a–f) and Fig. 7(a–f). KPLS and LLESS can detect the fault at the 1501st sample but the detection thresholds are not explicit in Fault 1 and Fault 2. In order to inspect the false alarm samples, it needs to zoom into the T2 and SPE monitoring results of the first 1500 samples. Evidently, the fault detection performances of KPLS, LLESS and P-𝑡SNE are almost the same, but KPLS and LLESS have much more fault alarm samples especially in the SPE statistic of Fault 2, as shown in Fig. 7(b,d). Thus, the proposed method is effective in eliminating false alarms when the process is under normal condition. 9

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

Fig. 9. Monitoring results of Fault 4: (a-b) KPLS, (c-d) LLESS, (e-f) Proposed.

Table 3 FDRs and FARs of the 4 faults in FMA. Fault no.

FDR

FAR

KPLS

Fault Fault Fault Fault

1. 2. 3. 4.

LLESS

Proposed

KPLS

LLESS

Proposed

T2

SPE

T2

SPE

T2

SPE

T2

SPE

T2

SPE

T2

SPE

1 1 0.926 1

1 1 1 0.978

1 1 0.934 1

1 1 1 0.998

1 1 1 1

1 1 1 1

0.083 0.053 0 0.033

0.076 0.207 0.149 0.022

0.075 0.001 0 0.023

0.061 0.183 0.116 0.015

0.004 0 0 0

0.001 0 0 0.005

Fault 3 is a fault of cooling valve between the 2nd and the 3rd

3rd stand which leads to the changes of rolling forces and roll gaps

stands in Type 3 (3.95 mm). The strip temperature is abnormal at the

in the following stands. Eventually, the product thickness is affected. 10

C. Zhang, K. Peng and J. Dong

Control Engineering Practice 89 (2019) 1–11

The fault occurs at the 1001st monitoring sample. Fig. 8(a–f) gives the monitoring results of KPLS, LLESS and P-𝑡-SNE. The FDRs of SPE are the same, but the T2 of the proposed method has higher FDR than KPLS and LLESS. The FARs of T2 are the same, but the SPE of P-𝑡-SNE has lower FAR than the other two methods. Thus, parametric t-SNE is effective in eliminating false alarms when the process is under normal condition. Fault 4 represents the fault of sampling value of the 6th bending force in Type 4 (5.00 mm). When it occurs at the 1001st monitoring sample, the value of 𝐵6 increases greatly. Then, with feedback regulation of automatic control system, 𝐵7 is changed correspondingly. Fault 4 is a kind of step transition, which has little impact on the thickness, but it will cause the change of exit strip shape, that means it is flatness-related. The monitoring results of Fault 4 are shown in Fig. 9(a-f). The FDRs of T2 are the same, but the SPE of parametric 𝑡SNE is superior to the other two method. Moreover, P-𝑡-SNE has lower FAR than KPLS and LLESS. The aforementioned monitoring results and analyses demonstrate that proposed method has superior monitoring performance.

Ding, S. X., Yin, S., Peng, K. X., Hao, H. Y., & Shen, B. (2013). A novel scheme for key performance indicator prediction and diagnosis with application to an industrial hot strip mill. IEEE Transactions on Industrial Informatics, 9, 2239–2247. Feital, T., Kruger, U., Dutra, J., Pinto, J. C., & Lima, E. L. (2012). Modeling and performance monitoring of multivariate multimodal processes. AIChE Journal, 59, 1557–1569. Frénay, B., & Verleysen, M. (2011). Parameter-insensitive kernel in extreme learning for non-linear support vector regression. Neurocomputing, 74, 2526–2531. Ge, Z. Q., & Song, Z. H. (2009). Multimode process monitoring based on Bayesian method. Journal of Chemometrics, 23, 636–650. Ge, Z. Q., Song, Z. H., Ding, S. X., & Huang, B. (2017). Data mining and analytics in the process industry: the role of machine learning. IEEE Access, 5, 20590–20616. Haghani, A., Jeinsch, T., & Ding, S. X. (2014). Quality-related fault detection in industrial multimode dynamic processes. IEEE Transactions on Systems, Man, and Cybernetics, 61, 6446–6453. Hinton, G. E., & Roweis, S. T. (2003). Stochastic neighbor embedding. In Advances in neural information processing systems, Vol. 15. Hong, X., Huang, B., & Ding, Y. (2017). Multi-model multivariate Gaussian process modelling with correlated noises. Journal of Process Control, 58, 11–22. Huang, B., & Kadali, R. (2008). Dynamic modeling, predictive control and performance monitoring. Landon: Springer-Verlag. Huang, K. Z., Yang, H. Q., King, I., Lyu, M. R., & Chan, L. W. (2004). The minimum error minimax probability machine. Journal Of Machine Learning Research, 5, 1253–1286. Jiang, Q., Huang, B., & Yan, X. F. (2016). GMM and optimal principal componentsbased Bayesian method for multimode fault diagnosis. Computers & Chemical Engineering, 84, 338–349. Lanckriet, G. R. G., Ghaoui, L. E., Bhattacharyya, C., & Jordan, M. I. (2002). Minimax probability machine. In Advances in neural information processing systems, Vol. 14. Ma, L., Dong, J., Peng, K. X., & Zhang, K. (2017). A novel data-based quality-related fault diagnosis scheme for fault detection and root cause diagnosis with application to hot strip mill process. Control Engineering Practice, 67, 43–51. Maaten, L. V. D., & Hinton, G. E. (2008). Visualizing data using 𝑡-SNE. Journal of Machine Learning Research, 9, 2579–2605. Martin, E. B., & Morris, A. J. (2000). Process monitoring and fault detection using multivariate SPC. Landon: Springer-Verlag. Peng, K. X., Zhang, K., Li, G., & Zhou, D. H. (2013). Contribution rate plot for nonlinear quality-related fault diagnosis with application to the hot strip mill process. Control Engineering Practice, 21, 360–369. Peng, K. X., Zhang, K., You, B., Dong, J., & Wang, Z. D. (2016). A qualitybased nonlinear fault diagnosis framework focusing on industrial multimode batch processes. IEEE Transactions on Industrial Electronics, 63, 1454–1463. Qin, S. J. (1998). Recursive PLS algorithms for adaptive data modeling. Computers and Chemical Engineering, 22, 503–514. Qin, S. J. (2003). Statistical process monitoring: basics and beyond. Journal of Chemometrics, 17, 480–502. Qin, S. J. (2012). Survey on data-driven industrial process monitoring and diagnosis. Annual Reviews in Control, 36, 220–234. Rencher, A. C. (2002). Methods of multivariate analysis. New York: Wiley. Sedghi, S., Sadeghian, A., & Huang, B. (2017). Mixture semisupervised probabilistic principal component regression model with missing inputs. Computers & Chemical Engineering, 103, 176–187. Shardt, Y. A. W., Hao, H. Y., & Ding, S. X. (2015). A new softsensor-based process monitoring scheme incorporating infrequent KPI measurements. IEEE Transactions on Industrial Electronics, 62, 3843–3851. Shardt, Y. A. W., Mehrkanoon, S., Zhang, K., Yang, X., Suykens, J., Ding, S. X., et al. (2018). Modelling the strip thickness in hot steel rolling mills using leastsquares support vector machines. The Canadian Journal of Chemical Engineering, 96, 171–178. Westerhuis, J. A., Kourti, T., & MacGregor, J. F. (1998). Analysis of multiblock and hierarchical PCA and PLS models. Journal of Chemometrics, 12, 301–321. Yu, J., & Qin, S. J. (2010). Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models. AIChE Journal, 54, 1811–1829. Zhang, Y. W., & Hu, Z. Y. (2011). On-line batch process monitoring using hierarchical kernel partial least squares. Chemical Engineering Research and Design, 89, 2078–2084. Zhang, Y. W., & Wang, C. (2012). Modeling and monitoring of multimodes process. In Advances in neural networks-ISNN 2012. Zhang, Y. W., Zhou, H., Qin, S. J., & Chai, T. Y. (2010). Decentralized fault diagnosis of large-scale processes using multiblock kernel partial least squares. IEEE Transactions on Industrial Informatics, 6, 3–10. Zhao, S. J., Zhang, J., & Xu, Y. M. (2004). Monitoring of processes with multiple operating modes through multiple principle component analysis models. Industrial and Engineering Chemistry Research, 43, 7025–7035. Zhao, S. J., Zhang, J., & Xu, Y. M. (2006). Performance monitoring of processes with multiple operating modes through multiple PLS models. Journal of Process Control, 16, 763–772.

5. Conclusions In this study, a P-𝑡-SNE and MMEMPM based quality-related process monitoring method is proposed for a variety of hot rolling processes. This method consists of two main parts: the space separation part and the variety identification part. Based on P-𝑡-SNE and quality variables, the training data from different types were used to extract the internal shared space and external unique spaces, then the T2 and SPE thresholds were developed for quality-related fault detection. The types of testing data were identified using the abilities of MMEMPM. The proposed method was applied to a practical hot rolling process to monitor four types faults in different types that affect the product’s thickness and flatness. Obviously, the proposed method can show accurate identification results, higher fault detection rate, and lower fault alarm rate. Future work considers topics with dynamics and working condition transitions in the hot rolling process to achieve optimal operating performance. Declaration of competing interest The authors declare that there is no conflict of interest in this paper. Acknowledgements This paper was supported by the Natural Science Foundation of China (NSFC) under Grants (61873024, 61773053, 61473033), by Fundamental Research Funds for the China Central Universities of USTB (FRF-GF-17-A4, FRF-BD-18-002A), PR China. Also thanks for the National Key R&D Program of China (No.2017YFB0306403) for funding. References Chen, Z. W., Ding, S. X., Peng, T., Yang, C. H., & Gui, W. H. (2018). Fault detection for non-Gaussian processes using generalized canonical correlation analysis and randomized algorithms. IEEE Transactions on Industrial Electronics, 65, 1559–1567. Chen, Z. W., Ding, S. X., Zhang, K., Li, Z., & Z., Hu. (2016). Canonical correlation analysis-based fault detection methods with application to alumina evaporation process. Control Engineering Practice, 46, 51–58. Chen, Q., Kruger, U., Meronk, M., & Leung, A. Y. T. (2004). Synthesis of 𝑇 2 and 𝑄 statistics for process monitoring. Control Engineering Practice, 12, 745–755. Choi, S. W., & Lee, I. M. (2005). Multiblock PLS-based localized process diagnosis. Journal of Process Control, 15, 295–306.

11