- Email: [email protected]
Comparative analysis of proteins from Bombyx mori and Antheraea pernyi cocoons for the purpose of silk identification
ISA Transactions xxx (xxxx) xxx
Contents lists available at ScienceDirect
ISA Transactions journal homepage: www.elsevier.com/locate/isatrans
Research article
A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems ∗
Hongtian Chen a , Jianping Wu a , Bin Jiang a , , Wen Chen b a
College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, 169 Shengtai West Road, Jiang Ning District, Nanjing, 211106, China b Division of Engineering Technology, Wayne State University, Detroit, MI, 48202, USA
article
info
Article history: Received 28 September 2018 Received in revised form 5 July 2019 Accepted 10 August 2019 Available online xxxx Keywords: Detection of incipient faults Modified neighborhood preserving embedding Industrial cyber–physical systems (ICPSs) Electrical drive systems
a b s t r a c t Industrial cyber–physical systems (ICPSs) are backbones of the Industrial 4.0 where control, physical entities, and monitoring are intensively interacted. Aiming to improve safety of a small-scale ICPS whose physical entity is an electrical drive system, this paper will develop a new detection strategy for incipient faults in neighborhood preserving embedding (NPE) framework that can provide stable solutions. The proposed modified NPE can not only extract local information effectively on data manifold of the ICPS but also solve the singularity problem caused by generalized eigenvalue decomposition skills. Additional advantages of this design for ICPSs include the enhanced fault detectability, inherent scalability, and accelerated computation efficiency. The proposed method is firstly evaluated by mathematical deviations and then is evaluated by its application to a small-scale ICPS. Three sets of experimental results show the efficacy of the proposed method in dealing with online detection of incipient faults in the ICPS. © 2019 ISA. Published by Elsevier Ltd. All rights reserved.
1. Introduction Industrial cyber–physical systems (ICPSs), regarded as a transdisciplinary topic, are receiving increasing interests from researchers and practitioners in recent years [1–4]. Because of potential security threats such as performance degradation of components, human factors, cyber attacks, etc., their safety and reliability issues are raising public concerns [5]. Specially, as shown in Fig. 1, smart sensors are the interactive components which connect the physical system and the cyber system of all ICPSs via sharing data and information in real time. Therefore, system monitoring or called fault detection (FD) for sensors equipped in ICPSs is necessary to enhance the safety. Except for closed-loop control and system monitoring, another function of equipped sensors in advanced ICPSs is information cooperation [6]. When a sensor fault appears, the online observations will deviate from their nominal values in an unexpected way, which leads to performance degradation even sudden stop of running equipment. Compared with traditional industrial systems, these adverse impacts caused by sensor faults will be serious in ICPSs because of rapid fault propagation and contamination via plenty of shared real-time information in networks [7]. If ∗ Corresponding author. E-mail address: [email protected] (B. Jiang).
detection of sensor faults can be achieved in large-scale industrial systems in their early phase, the safety of ICPSs will be guaranteed [8]. Conventionally, ICPSs such as intelligent factories and smart grids consist of numerous units and operate in complex mechanism [3]. For model-based FD methods, the first step is to establish system models of complex ICPS where physical knowledge and mathematical foundations are usually needed, and this work may be achieved at the cost of long-term endeavors from various experts. Thanks to the current development of data-processing and storage techniques, data-driven modeling [5] and state estimation [7] of ICPSs are proved to be promising via extracting redundancy information hidden in quantitative observations. These advanced strategies can provide more flexible solutions than model-based modeling to support FD applications to ICPSs. For the existing data-driven FD methods with applications to ICPSs, they can be generally classified into two following categories based on the characteristics of monitored systems: (1) multivariate statistical analysis (MVA)-based methods for statistic systems [6,9–11]; (2) state estimation and parameter identification-aided methods for dynamic systems [12–14]. Among the MVA-based FD methods, principal component analysis [10], partial least squares, canonical correlation analysis [15], independent component analysis [16], etc. [17], are applicable techniques. For the systems with dynamic characteristics, state estimation and parameter identification will show their salient advantage than MVA to deal with time-varying characteristics
https://doi.org/10.1016/j.isatra.2019.08.022 0019-0578/© 2019 ISA. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
2
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
In this section, detailed descriptions of industrial ICPSs and NPE algorithm are given firstly, and the ultimate objectives of this paper are then introduced.
systems (CPSs) as a key way of improving its national competitiveness while Germany regards cyber–physical production system as the core structure of Industrial 4.0. Regarded as a new avenue to achieve smart manufacturing, CPSs have become more popular globally [23]. Nowadays, wide applications of CPSs include, but not limited to, smart grids [3], medical treatments [4], and transportation networks [10], etc. Recently, CPSs have been a hot research topic. Known as ICPSs, the CPSs are paid more attention to recognition and decisionmaking ability of prediction. Different from traditional industrial systems based on the rule of manual control, ICPSs are focused on intelligent control. On this basis, researchers pointed out the urgency of building the interaction between cyber systems and physical systems. The integration of these two systems makes extensive support of perception, analysis, decision-making, control, and management [5,24]. In fact, ICPSs are the space fusion resulted from physical spaces and cyber spaces in industrial systems. For the two mentioned spaces, their relationship and constitutions are shown in Fig. 1. Typically, the physical space includes equipment, environment, and activities. Data acquired and stored from the physical space can be stored and assessed in the cyber space, which promotes the connection of this entire industrial system. The cyber space is composed of different subspaces such as the individual space, group space, computation space, deduction space, and decisionmaking space. All subspaces perform their own tasks as well as cooperate with each other to obtain useful information. Afterwards, the information is transferred into knowledge, and the comprehensive applications of knowledge to cyber space will play a guiding role to exact activities in physical space. Actually, an ICPS is a system that can transform data into knowledge. A remarkable feature of an ICPS is its ability of promoting generation and inheritance of knowledge by using a great amount of manufacturing data based on industrial operation platforms. The successful construction of an ICPS will take full advantage of knowledge mining of the system. Besides, physical structure based on cyber information can be carried out, aiming at revealing potential information of industrial data. In this way, the cyber systems may optimize the known states and predict the unknown states at the same time. As a transdisciplinary subject, ICPSs have many properties. The ability of perceiving and predicting changes along with uncertainties in the environment is a useful capability for industrial systems. At the same time, emulating new knowledge and experience from group activities is also of great significance for decision making [25]. In addition, the real-time handling capacity, smart computational request, and the adaptation to diverse conditions are also a wide spectrum of aspects that ICPSs cover. The advantages may also be disadvantages once faults occur. More precisely, as soon as faults happen in the system, either in the physical dynamics or in the cyber network, the automatic interaction of ICPSs among physical spaces and cyber spaces will speed up the fault-propagating process. At the same time, the data management and communication layer may be broken down, which means the industrial system security may become collapsed [25]. Accordingly, the study of fault detection in ICPSs to improve the reliability and credibility is of great importance, which is also receiving increasing attention in recent years [25, 26].
2.1. Industrial cyber–physical systems
2.2. Neighborhood preserving embedding
It is critical to find a transformation strategy for industrial development over the world. Thus, the USA defined cyber–physical
Neighborhood preserving embedding (NPE) is one of the nonlinear projection algorithms that is capable of persisting efficient
Fig. 1. The framework of ICPS.
[18]. In fact, some latent assumptions, such as Gaussian distribution on observed signals or linear time-invariant parameters on systems, are usually used for the above mentioned methods. Recently, to extract nonlinear features of samplings, manifold learning methods are paid increasing attention to detect faults in large-scale industrial systems [19]. For example, in [20], an integration of locality preserving projections and principal component analysis are designed to search for both global and local information hidden in industrial signals. In order to highlight partial information, neighborhood preserving embedding (NPE) and support vector data description are used to improve fault detectability [21]. As pointed out in [22], numerous manifold learning methods suffer from singularity problem caused by the singular generalized eigenvalue computation on very high-dimensional projections of original samplings. Motivated by the above observations, we will design a smallscale ICPS for an electrical drive system where real-time control and system monitoring are integrated in the cyber system. Specially, a newly modified NPE method will be developed to deal with the singularity problem, based on which an FD solution is then formulated. The effectiveness of the proposed method will be illustrated using three sets of experiments on one small-scale ICPS. This paper is organized as follows. In Section 2, ICPSs and NPE are firstly described and the objective of this study is followed. In Section 3, design of ICPSs is presented, in which the proposed FD method is discussed in details. In Section 4, complete experiments including fault injection and FD results are carried out, based on which some discussions are given. In Section 5, some concluding remarks together with further work are summarized. 2. Preliminaries
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
information in a feature matrix. The statistical features are extracted by reconstructing the relationship between one specified point and its neighbors [18,27]. In real-world applications, the NPE has a faster learning rate than many other manifold learning methods because of its linear approximation to constructing neighborhood [28]. Given input data set X = [x1 , x2 , . . . , xN ] ∈ RD×N with N being the total number of samplings in a high-dimensional space RD . The purpose of NPE is to determine the output representation Y = [y1 , y2 , . . . , yN ] ∈ Rd×N (d < D) of the initial set X , where the column vector yi of Y is the embedding for the initial xi . The NPE aims at seeking a projection matrix U ∈ RD×d so that Y = U T X . That is, original data set X can be mapped into Y with lower dimension, but there is no loss of both normal and faulty information. The algorithm procedure is stated as follows: (1) Constructing a neighborhood graph: The K -nearest neighbors method is accepted to find the neighbors of the ith node, which corresponds to the data point xi . When xj belongs to the adjacency of xi , a directed edge is put from xi to xj . (2) Finding the weight of each edge: Denote W as a weight matrix with the element wij representing the weight of each edge from node i to node j with 0 ≤ wij < 1. If there is no such edge, then wij = 0. (3) Computing the weight matrix: To grasp the contribution rate of each neighbor node ∑N to the observed node intuitively, with the constraint j=1 wij = 1, the optimal weight matrix can be calculated by minimizing the following reconstruction error [27]
φ (W ) = min
N ∑
∥ xi −
i=1
N ∑
wij xj ∥2 .
(1)
j=1
yyT =1
= min
yyT =1
N ∑
∥yi −
N ∑
wij yj ∥22
j=1
i=1 N ∑
∥uT (xi −
i=1
N ∑
wij xj )∥22
(2)
j=1
= min uT X (I − W )T (I − W )X T u yyT =1
where M = (I −W )T (I −W ). Then the optimization problem is simplified as follows arg min uT XMX T u.
(3)
uT XX T u=1
With the aid of Lagrange multiplier, the optimal projection directions are obtained via solving the following generalized eigenvector problem [27] XMX T u = λXX T u.
the generalized eigenvector problem given in Eq. (4), the rank of XX T may encounter unstable problem, i.e., rank(XX T ) = rank(X ) ≤ min(D, N).
(5)
As presented in Eq. (5), it is possible that the eigen-matrix of XX T is nonsingular and hereby results in the infeasibility of NPE. Thus, a modified version of NPE is proposed in this study to avoid the aforementioned situation. 2.3. Objective In this paper, a small-scale ICPS based on the electrical drive system is firstly designed. We will be providing a practical verification platform for fault detection applications. On such a verification platform with real-time calculation and transmission of observations, the ultimate objectives of this paper are summarized as follows
• Establishment of an integrated upper-layer computer for the ICPS including real-time control and supervisory units;
• Development of a stable solution to strengthen the robustness of the system, called modified NPE, for fault detection purpose in the supervisory unit; and • Analysis of the effectiveness of the modified NPE without the loss of discriminative information. 3. The proposed methodology for cyber–physical systems In this section, the design of our small-scale ICPS will be introduced into physical space and cyber space. More significantly, the improvement of modified NPE will be addressed in details including performance and characteristic analysis. 3.1. Design of a small-scale ICPS
(4) Calculating the projection manifold: Vectors of u1 , u2 , . . . , ud are the projection directions to formulate the matrix U. On the basis of another constraint, i.e. yyT = 1, the problem of arbitrary scaling factor is avoided. Meanwhile, an optimal mapping has been chosen to minimize the following cost function
Φ (Y ) = min
3
(4)
The eigenvectors associated with d smallest eigenvalues of Eq. (4) will form matrix U. Thus, primary high dimensional data set has been lessened and the projection manifold has been traced eventually. The original NPE method shows its advantages on dimensionality reduction and feature extraction. However, when NPE solves
Our small-scale ICPS, which is composed of physical space and cyber space, is actually an electrical traction platform. Detailed descriptions of all components are illustrated as follows. In physical space of this electrical traction system, to be more specific, the equipment mainly consists of a high voltage control panel, a permanent magnetic synchronous motor (PMSM) and a controller. Fig. 2 shows the detailed components of this platform. Sensors are deployed in different positions so that various data samples can be obtained including multiple output circuit signals, current, and speed. This system adopts stator flux-oriented vector control strategy so that it is capable of driving the motor to run at a preset speed. The cyber space of this small-scale electrical traction ICPS refers to the data points with different information. The whole collected sampling data sets can form the group space which can provide raw data for computation space, in which online fault detection algorithms is included. When it comes to computation space, all the industrial data acquired from sensors or actuators should be computed and managed in a particular way. It is worth mentioning that, by the use of this proposed method, the new feature can be extracted directly from original data sets. In addition, this hidden information is important in deduction space. Once receiving derived features and required performance by networks, the deduction space will deliver results to decision-making space where information will be transformed into knowledge and the knowledge will then be sent to physical space. As a significant component in the cyber space, the network mainly transfers data to computational unit as well as conveys the execution instructions to the actuator. Once faults are injected into this small-scale ICPS in an unpredicted way, the physical space will collect data with faulty
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
4
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 2. Detailed components of small-scale CPS.
information and share them with cyber space. After procedures such as computation and deduction, variable features obtained together with fault location will be fed back to physical space through network. This small-scale ICPS connects physical and cyber systems fluidly, and the decision-making behavior is based on the data analysis so that the fault in this small-scale ICPS can be detected reliably. 3.2. Modified neighborhood preserving embedding Aimed at simplifying the generalized eigenvalue problem of the NPE, one algebraic method has been proposed. More strictly speaking, the construction of a new matrix Ur can seek the relationship between optimization objection and constraints. A pseudo-code for computing the projection matrix is shown in Algorithm 1.
where E = (Er E˜ r ) = (e1 , e2 , . . . , er , er +1 , . . . , eD ) with column vectors normalized and orthogonal to each other, r = rank(Ur ), and Σ = diag(σ1 , σ2 , . . . , σr , 0, . . . , 0) which is composed of eigenvalues of Ur with σ1 ≤ σ2 ≤ · · · ≤ σr . At the same time, E ∈ RD×D , Er ∈ RD×r , E˜ r ∈ RD×(D−r) , and Σ ∈ RD×D . The column vectors of E form a basis for the vector space Ur . Denote V as the spanned set by Ur such that V = Span{e1 , e2 , . . . , eD }. Considering eigenvalues of E˜ r to be 0s , we can have E˜ r ∈ N(Ur ), where N(Ur ) is the null space of Ur . Thus, the column vectors of Er form the basis for Ur . Let R(X ) be the column space of X and R(XX T ) be the column space of XX T , then the following equality holds R(X ) = N(X T )⊥ = N(XX T )⊥
(7)
= R((XX T )T ) = R(XX T )
where X and XX T share the same column space, which means the basis for the vector space XX T is E = (Er E˜ r ), the same as Ur . Then, we will find the relationship between E˜ r and X with the 1/2 aid of Ur . Define that S = ∥Ur E˜ r ∥2l such that 2
S = E˜ r [X (I − W )T (I − W )X T + XX T ]E˜ rT = [E˜ r X (I − W )T ][E˜ r X (I − W )T ]T + (E˜ r X )(E˜ r X )T
= ∥[E˜ r X (I − W )T ]T ∥2l2 + ∥(E˜ r X )T ∥2l2
.
(8)
= 0 Then, one has
{
E˜ r X (I − W )T = 0 E˜ r X = 0
The detailed process of the algorithm is now elaborated. Define Ur = XMX T + XX T such that the eigenvalue decomposition (EVD) of Ur will be represented as follows Ur = XMX T + XX T
= EΣ ET
(6)
(9)
where E˜ r ∈ N(X ), N(X ) is the null space of X , and the column vectors of Er form the basis of X . Given the fact that U is the projection matrix which is comprised of the neighboring information of X , then the nonempty subset U is a subspace of the vector space X . Hence, any vector in U can be written as a linear combination of the basis of X . Suppose that u is the solution of Eq. (4), then u = Er l where l is the coefficient vector of Er and l ∈ Rr . Remark 1. The application of EVD is to extract direction of the designated matrix. Under this situation, we represent u with basis
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
of E˜ r so that the optimization can be changed. With mathematical calculation, one specific relationship between the optimal formula and constraints are tracked to alleviate the computational burden. Thus, the optimal problem of Eq. (3) can be transformed into this form: arg min lT ErT XX T Er l=c
lT ErT XMX T Er l.
(10)
We can further have an equivalent form of the above optimal problem 1/2
Σ −1/2 ErT XMX T Er Σ −1/2 Σ 1/2 l
arg min
lT Σ
s.t.
lT Σ 1/2 Σ −1/2 ErT XX T Er Σ −1/2 Σ 1/2 l = c
.
(11)
The purpose of extending this function is to find the essential relationship between optimal objection and constraints. By using an equivalent transformation, the connection/projection between X and Y can be found; meanwhile, the unstable problem caused by nonsingular problems can be solved. Several variables are represented as follows, including v = Σ 1/2 l, Ub = Σ −1/2 ErT XMX T Er Σ −1/2 , and Uc = Σ −1/2 ErT XX T Er Σ −1/2 . Now that Ub + Uc = I, Eq. (10) is equal to arg min v T (I − Uc )v.
(12)
v T Uc v=c
In order to calculate the optimal v , a Lagrange function is used to solve Eq. (12) as follows L(v, λ) = v T (I − Uc )v − λ(v T Uc v − c).
(13)
The above optimal problem is the same as the following generalized eigenvalue problem: (I − Uc )v = λUc v.
(14)
Designate the solution of (14) as V = (v1 , v2 , . . . , vd ) with λ1 ≤ λ2 ≤ · · · ≤ λd . The vector u can be obtained by u = Er l while the projection matrix U can be computed as U = Er Σ −1/2 V . Remark 2. Note that the local reconstruction of coefficients is incorporated in the intrinsic graph and when the local linear reconstruction error is minimized, the optimal projection matrix can be obtained. This obtained projection matrix can not only extract characteristic features on data manifold but also provide an available procedure to ensure the effectiveness of inner combination of neighbored nodes locally. In this process, Uc is of full rank; it means there exist no singular problems for this modified NPE algorithm. Remark 3. With the help of matrix theories, the generalized eigenvalue calculation with XMX T and XX T in NPE is transformed into the EVD problem and generalized eigenvalue problem. Because the relevant matrices are symmetric and positive semidefinite, this newly proposed algorithm will effectively decrease computation loads and avoid the instability caused by NPE for the small-scale ICPSs.
5
Table 1 Parameters of the experimental setup. Symbol
Values (Unit)
Descriptions
p Jm La Ra Dm Oa Te Ts TL
4(1) 0.425 × 10−3 (kg m2 ) 2.96 × 10−3 (H) 0.985 () 1 × 10−4 (1) 8.1 × 10−2 (Wb) 2.43 (N m) 1 × 10−4 (s) 3.645 (N m)
Pole pairs Moment of inertia Inductance of motor coil Resistance of motor coil Viscosity friction coefficient Magnet flux Electromagnetic torque Sampling time Load torque
and will cost plenty of time and memory. Moreover, matrix XX T has the possibility of being singular so that it may bring the interruption of calculation and loss of sensitivity of fault detection. What makes modified NPE attractive is its ability to overcome the aforementioned drawbacks of the NPE. The unstable problem of the NPE can be solved via transforming it into an EVD problem and a generalized eigenvalue problem. It is also evident that the similarities between neighbored pairs of data points can be preserved in the original high-dimensional space. According to matrix theories, the primary information is included in eigenvectors corresponding to the maximum eigenvalues. Eq. (6) shows that the essential information of XMX T + XX T is merely associated with eigenvalues of σ1 , σ2 , . . . , σr ; that is, eigenvectors of Er . Thus, it can make great impact on reduction of calculation of this proposed algorithm. We can know that the local information of xi and its neighborhoods are stored in weight matrix W . The column vectors of W are linear coefficients between data point xi and xj that characterize local geometry of different nodes. With the purpose of analyzing performance of the designed algorithm thoroughly, Mahalanobis distance is applied to measure the deviation of measurements, which can be computed as [29] T 2 = X T U Λ−1 U T X
(15)
where Λ is the covariance matrix of Y , and Λ = 1/(n − 1)Y T Y . Euclidean distance is used to indicate measurement deviations, which can be calculated uniquely by SPE = X (I − UU T )X T .
(16)
With the application of above two criteria of performance, T 2 not only shows the sensitivity of projection matrix, but also characterizes the FD performance of this proposed method while SPE takes the correlation among signals from sensors into account of this ICPS. The validity of the introduction of T 2 and SPE has been entirely stated in [30,31]. Two available statistics mentioned above can detect abnormalities when changes emerge in the monitored system. In addition, the selection of control limits of T 2 and SPE also plays a significant role in the fault detection results [19]. The productive and practical way to obtain the corresponding confident region and control limit is explained in details in [6,15]. 4. Experiments
3.3. Performance and characteristics In the considered small-scale ICPS, data set X acquired from the electrical drive system should be projected into Y by U. The above NPE algorithm has the capability of transforming data with dimension of D into a reduced set of dimension of d. Nevertheless, the dimension of each of matrices XM T X and XX T is D × D, which means when it comes to generalized eigenvalue problem, this initial algorithm is confronted with great amount of calculation
In this section, the fault injection and detection process for our experimental small-scale ICPS has been implemented, after which the detection results have been discussed. 4.1. Experimental platform with fault injection To evaluate the performance of this proposed algorithm, three sets of experiments with sensor faults have been conducted in
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
6
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 3. Fault detection results for f1 with NPE.
Fig. 4. Fault detection results for f1 with modified NPE.
Fig. 5. Fault detection results for f2 with NPE.
our small-scale ICPS. Different types of sensors equipped in the physical space can provide samples for the closed-loop control unit in physical space and analysis along with monitor in cyber space. Accurate detection of various sensor faults is urgently needed in this electrical traction ICPS. Table 1 gives main parameters of the permanent magnetic synchronous motor. With the aid of a Timebase module in QUARC to support a more accurate sampling time setting, the sampling time interval of the experiment is 0.1 ms. In accordance with electrical systems, seven signals have been selected, including ia and ib as current signals, va , vb , vc , vdc as voltage signals, and s as speed signal of the motor. Three sensor faults are injected into the process: (1) an incipient fault on ia sensor with the
fault amplitude is f1 = 0.1 A; (2) an incipient fault on s sensor with the form as f2 = (t − 4) r/min; (3) an intermittent fault on vdc sensor as f3 = 10 V. Samples containing both healthy and faulty data are collected from 49 s to 55 s. To obtain online data sets for validation, three incipient sensor faults are injected into this small-scale ICPS from 51 s. Once faults occur, they are always with diverse latent signal information, which means the underlying contents are contained in the above signals. 4.2. Detection of incipient faults Once faults are injected into this physical space, the cyber space is going to detect faults. The complete implementation
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
7
Fig. 6. Fault detection results for f2 with modified NPE.
of fault detection on our small-scale ICPS is proposed with offline modeling and online fault detection. Based on the proposed modified NPE algorithm, the projection matrix U aims to capture the local structure of original data space from off-line data to online data in a corresponding way. The off-line part can be summarized as follows:
of extracting features with unsatisfactory performance. In the following contents, we make a comparison study between NPE and modified NPE for the same faults, and the detection results indicate the advantages of our modified one. To demonstrate the superiority of our proposed method than traditional one, monitoring results for three sensor faults are given in Figs. 3 to 8. 4.3. Analysis and discussions
In our experiment, we first construct off-line models by acquiring off-line data with usual operating cases; they are used as the training set to calculate the projection matrix, as well as generation of T 2 , SPE statistics, and control limits. Thanks to the modified NPE process, the features with normal and faulty samples are extracted and memorized in the projection matrix and will be helpful to online procedure. For online fault detection, actual signals in both normal and abnormal conditions are collected and normalized as the testing set. Then, projection matrix calculated from off-line training set plays a significant role here to fetch the current T 2 and SPE statistics. Both off-line and online data sets share the common latent structure which reveals systemic information because the electrical traction systems work under the same control strategy when data sets are collected in both off-line and online phase. We now make a comparison between new statistics and control limits; once new statistics outweigh their limits, a fault is successfully detected. The detailed online procedure is as follows.
In practice, we usually collect off-line training data enough to make the distribution of off-line set similar to the distribution of online set. With the application of modified NPE method, the normal training matrix is augmented and 30 neighbors are selected for each sample in the constructed adjacency graph. Now that the actual CPS operates with noise and disturbance (i.e., uncertainties in the system), the original NPE is capable
Under the same false alarm ratio which can be determined based on the choice of thresholds, Fig. 4 indicates detection results for f1 using the modified NPE algorithm while Fig. 3 indicates the results using the original NPE algorithm. It is clearly shown that, by the use of the NPE algorithm, the index T 2 shows poor fault detectability. But for the modified NPE algorithm, both T 2 and SPE statistics can achieve satisfactory performance. Fig. 5 indicates the detection results for f2 using NPE, and the performance is unaccepted because both T 2 and SPE statistics are invalid for the detection. However, as shown in Fig. 6, by analyzing the detection results with modified NPE for incipient fault f2 , it is observed that T 2 statistic in this case has weak ability of recognizing fault because of its high false alarm rate. Nevertheless, SPE shows its capability of detecting fault accurately for f2 when it occurs. In fact, the dramatic change of rotation speed of the motor could result in vibration. As soon as this instability occurs, the closed-loop control unit in the physical system of the ICPS starts to work to attenuate the consequence fault f2 brings about, which means the validity of our algorithm would be affected. Considering the control unit, a comparison will be made among f1 , f2 and f3 . Different from f1 and f2 that are closedloop faults, f3 is an open-loop one. To have a good view of fault detection performance, logarithmic transformation is applied to generate the detection result of f3 . Based on detection results shown in Fig. 8, when f3 occurs, the proposed FD method can rapidly detect the occurrence of this fault; meanwhile, NPE algorithm applied to the same data set shows barely satisfactory performance if only T 2 statistic is considered. Thus, the effectiveness and superiority of our proposed method are illustrated on this experimental platform of a small-scale ICPS. In our experiments, sampling time interval is 1 × 10−4 s. By using the modified NPE algorithm, the calculation speed is 2.03 × 10−6 s, which means the computing process of fault detection can catch up with the sampling procedure totally. Obviously, it has been demonstrated that the modified NPE algorithm is effective in fault detection. The local structure and information of samplings can be converted into projection matrix; that is, modified NPE can realize its goal of preserving the local structure of the data set in a linear way to reveal the intrinsic relationship between the observed data and its neighbors.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
8
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 7. Fault detection results for f3 with NPE.
Fig. 8. Fault detection results for f3 with modified NPE.
5. Conclusions
References
In this paper, a small-scale ICPS has been designed for an electrical drive system. Focused on the extendibility and safety, a new FD method called modified NPE has been proposed for the online monitoring purpose. It was elaborated that by solving the singularity problem, the proposed method is more effective than traditional manifold learning methods for detection of sensor faults. Through the designed small-scale ICPS, the performance and effectiveness of this study are convincingly demonstrated. In addition, based on this work, further research on safety and reliability will be drawn on data-driven fault-tolerant control for ICPSs.
[1] Jiang Y, Yin S, Kaynak O. Data-driven monitoring and safety control of industrial cyber-physical systems: basics and beyond. IEEE Access 2018;6:47374–84. [2] Colombo AW, Karnouskos S, Shi Y, Yin S, Kaynak O. Industrial cyberphysical systems. In: 2017 IEEE international conference on mechatronics, Vol. 104, No. 5. 2017, p. 899–903. [3] Liu M, Shi Y, Gao H. Aggregation and charging control of PHEVs in smart grid: A cyber-hysical perspective. Proc IEEE 2016;104(5). [4] Zhang Y, Qiu M, Tsai CW, Hassan MM, Alamri A. Health CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Syst J 2017;11(1):88–95. [5] Jiang Y, Yin S. Recursive total principle component regression based fault detection and its application to vehicular cyber-physical systems. IEEE Trans Ind Inf 2017;14(4):1415–23. [6] Chen H, Jiang B, Lu N, Mao Z. Deep PCA based real-time incipient fault detection and diagnosis methodology for electrical drive in highspeed trains. IEEE Trans Veh Technol 2018;67(6):4819–30. [7] Pajic M, Weimer J, Bezzo N, Sokolsky O, Pappas GJ, Lee I. Design and implementation of attack-resilient cyber-physical systems: With a focus on attack-resilient state estimators. IEEE Control Syst 2017;37(2):66–81. [8] Zhang K, Shardt YAW, Chen Z, Peng K. Using the expected detection delay to assess the performance of different multivariate statistical process monitoring methods for multiplicative and drift faults. ISA Trans 2017;67:56–66. [9] Chen H, Jiang B, Lu N. An improved incipient fault detection method based on kullback-leibler divergence. ISA Trans 2018;79:127–36. [10] Chen H, Jiang B, Ding SX, Lu N, Chen W. Probability-relevant incipient fault detection and diagnosis methodology with applications to electric drive systems. IEEE Trans Control Syst Technol 2018. http://dx.doi.org/10. 1109/TCST.2018.2866976. [11] Qin SJ. Survey on data-driven industrial process monitoring and diagnosis. Annu Rev Control 2012;36(2):220–34. [12] Ding SX, Yang Y, Zhang Y. Data-driven realizations of kernel and image representations and their application to fault detection and control system design. Automatica 2014;50(10):2615–23.
Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work was supported by National Natural Science Foundation of China (61490703, 61374141, 61573180), Funding of Jiangsu Innovation Program for Graduate Education (KYLX 16_0378), Priority Academic Program Development of Jiangsu Higher Education Institutions, and Open Fund for Postgraduate Innovation Laboratory of Nanjing University of Aeronautics and Astronautics under Grant kfjj20180320. In addition, the authors are grateful for Associate Editor and anonymous reviewers for their constructive comments.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx [13] Luo H, Yang X, Kruger M, Ding SX, Peng K. A plug-and-play monitoring and control architecture for disturbance compensation in rolling mills. IEEE/ASME Trans Mechatronics 2018;23(1):200–10. [14] Kim J, Tong L, Thomas RJ. Subspace methods for data attack on state estimation: A data driven approach. IEEE Trans Signal Process 2015;63(5):1102–14. [15] Chen Z, Ding SX, Peng T, Yang C, Gui W. Fault detection for non-gaussian processes using generalized canonical correlation analysis and randomized algorithms. IEEE Trans Ind Electron 2018;65(2):1559–67. [16] Tong C, Lan T, Shi X. Double-layer ensemble monitoring of non-gaussian processes using modified independent component analysis. ISA Trans 2017;68:181–8. [17] He W, He Y, Zhang C. A new fault diagnosis approach for analog circuits based on spectrum image and feature weighted kernel Fisher discriminant analysis. Rev Sci Instrum 2018;89(7). 074702. [18] Wang Y, Wu Y. Complete neighborhood preserving embedding for face recognition. Pattern Recognit 2010;43(3):1008–15. [19] Chen H, Jiang B. A review of fault detection and diagnosis for the traction system in high-speed trains. IEEE Trans Intell Transp Syst 2019. http: //dx.doi.org/10.1109/TITS.2019.2897583. [20] Zhang M, Ge Z, Song Z, Fu R. Global-local structure analysis model and its application for fault detection and identification. Ind Eng Chem Res 2011;50:6837–48. [21] Jiang Q, Yan X. Probabilistic weighted NPE-SVDD for chemical process monitoring. Control Eng Pract 2014;28:74–89.
9
[22] Lunga D, Prasad S, Crawford MM, Ersoy O. Manifold-learning-based feature extraction for classification of hyperspectral data: A review of advances in manifold learning. IEEE Signal Process Mag 2014;31(1):55–66. [23] Seshia SA, Hu S, Li W, Zhu Q. Design automation of cyber-physical systems: challenges, advances, and opportunities. IEEE Trans Comput-Aided Des Integr Circuits Syst 2017;36(9):1421–34. [24] Tao F, Cheng J, Qi Q. IIHub: an industrial internet-of-things hub towards smart manufacturing based on cyber-physical system. IEEE Trans Ind Inf 2018;14(5):2271–80. [25] Pasqualetti F, Dorfler F, Bullo F. Attack detection and identification in cyber-physical systems. IEEE Trans Automat Control 2013;58(11):2715–29. [26] Massoumnia MA, Verghese GC, Willsky AS. Failure detection and identification. IEEE Trans Automat Control 1989;34(3):316–21. [27] He X, Cai D, Yan S, Zhang H. Neighborhood preserving embedding. In: Proceedings of the 10th IEEE international conference on computer vision. 2005, p. 1208–13. [28] Roweis ST, Saul LK. Nonlinear dimensionality reduction by locally linear embedding. Science 2000;290(5500):2323–6. [29] Ding SX. Data-driven design of fault diagnosis and Fault-Tolerant control systems. Springer-Verlag; 2014. [30] Yin S, Ding SX, Xie X, Luo H. A review on basic data-driven approaches for industrial process monitoring. IEEE Trans Ind Electron 2014;61(11):6418–28. [31] Gao Z, Cecati C, Ding SX. A survey of fault diagnosis and fault-tolerant techniques-part II: fault diagnosis with knowledge-based and hybrid/active approaches. IEEE Trans Ind Electron 2015;62(6):3768–74.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
Contents lists available at ScienceDirect
ISA Transactions journal homepage: www.elsevier.com/locate/isatrans
Research article
A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems ∗
Hongtian Chen a , Jianping Wu a , Bin Jiang a , , Wen Chen b a
College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, 169 Shengtai West Road, Jiang Ning District, Nanjing, 211106, China b Division of Engineering Technology, Wayne State University, Detroit, MI, 48202, USA
article
info
Article history: Received 28 September 2018 Received in revised form 5 July 2019 Accepted 10 August 2019 Available online xxxx Keywords: Detection of incipient faults Modified neighborhood preserving embedding Industrial cyber–physical systems (ICPSs) Electrical drive systems
a b s t r a c t Industrial cyber–physical systems (ICPSs) are backbones of the Industrial 4.0 where control, physical entities, and monitoring are intensively interacted. Aiming to improve safety of a small-scale ICPS whose physical entity is an electrical drive system, this paper will develop a new detection strategy for incipient faults in neighborhood preserving embedding (NPE) framework that can provide stable solutions. The proposed modified NPE can not only extract local information effectively on data manifold of the ICPS but also solve the singularity problem caused by generalized eigenvalue decomposition skills. Additional advantages of this design for ICPSs include the enhanced fault detectability, inherent scalability, and accelerated computation efficiency. The proposed method is firstly evaluated by mathematical deviations and then is evaluated by its application to a small-scale ICPS. Three sets of experimental results show the efficacy of the proposed method in dealing with online detection of incipient faults in the ICPS. © 2019 ISA. Published by Elsevier Ltd. All rights reserved.
1. Introduction Industrial cyber–physical systems (ICPSs), regarded as a transdisciplinary topic, are receiving increasing interests from researchers and practitioners in recent years [1–4]. Because of potential security threats such as performance degradation of components, human factors, cyber attacks, etc., their safety and reliability issues are raising public concerns [5]. Specially, as shown in Fig. 1, smart sensors are the interactive components which connect the physical system and the cyber system of all ICPSs via sharing data and information in real time. Therefore, system monitoring or called fault detection (FD) for sensors equipped in ICPSs is necessary to enhance the safety. Except for closed-loop control and system monitoring, another function of equipped sensors in advanced ICPSs is information cooperation [6]. When a sensor fault appears, the online observations will deviate from their nominal values in an unexpected way, which leads to performance degradation even sudden stop of running equipment. Compared with traditional industrial systems, these adverse impacts caused by sensor faults will be serious in ICPSs because of rapid fault propagation and contamination via plenty of shared real-time information in networks [7]. If ∗ Corresponding author. E-mail address: [email protected] (B. Jiang).
detection of sensor faults can be achieved in large-scale industrial systems in their early phase, the safety of ICPSs will be guaranteed [8]. Conventionally, ICPSs such as intelligent factories and smart grids consist of numerous units and operate in complex mechanism [3]. For model-based FD methods, the first step is to establish system models of complex ICPS where physical knowledge and mathematical foundations are usually needed, and this work may be achieved at the cost of long-term endeavors from various experts. Thanks to the current development of data-processing and storage techniques, data-driven modeling [5] and state estimation [7] of ICPSs are proved to be promising via extracting redundancy information hidden in quantitative observations. These advanced strategies can provide more flexible solutions than model-based modeling to support FD applications to ICPSs. For the existing data-driven FD methods with applications to ICPSs, they can be generally classified into two following categories based on the characteristics of monitored systems: (1) multivariate statistical analysis (MVA)-based methods for statistic systems [6,9–11]; (2) state estimation and parameter identification-aided methods for dynamic systems [12–14]. Among the MVA-based FD methods, principal component analysis [10], partial least squares, canonical correlation analysis [15], independent component analysis [16], etc. [17], are applicable techniques. For the systems with dynamic characteristics, state estimation and parameter identification will show their salient advantage than MVA to deal with time-varying characteristics
https://doi.org/10.1016/j.isatra.2019.08.022 0019-0578/© 2019 ISA. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
2
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
In this section, detailed descriptions of industrial ICPSs and NPE algorithm are given firstly, and the ultimate objectives of this paper are then introduced.
systems (CPSs) as a key way of improving its national competitiveness while Germany regards cyber–physical production system as the core structure of Industrial 4.0. Regarded as a new avenue to achieve smart manufacturing, CPSs have become more popular globally [23]. Nowadays, wide applications of CPSs include, but not limited to, smart grids [3], medical treatments [4], and transportation networks [10], etc. Recently, CPSs have been a hot research topic. Known as ICPSs, the CPSs are paid more attention to recognition and decisionmaking ability of prediction. Different from traditional industrial systems based on the rule of manual control, ICPSs are focused on intelligent control. On this basis, researchers pointed out the urgency of building the interaction between cyber systems and physical systems. The integration of these two systems makes extensive support of perception, analysis, decision-making, control, and management [5,24]. In fact, ICPSs are the space fusion resulted from physical spaces and cyber spaces in industrial systems. For the two mentioned spaces, their relationship and constitutions are shown in Fig. 1. Typically, the physical space includes equipment, environment, and activities. Data acquired and stored from the physical space can be stored and assessed in the cyber space, which promotes the connection of this entire industrial system. The cyber space is composed of different subspaces such as the individual space, group space, computation space, deduction space, and decisionmaking space. All subspaces perform their own tasks as well as cooperate with each other to obtain useful information. Afterwards, the information is transferred into knowledge, and the comprehensive applications of knowledge to cyber space will play a guiding role to exact activities in physical space. Actually, an ICPS is a system that can transform data into knowledge. A remarkable feature of an ICPS is its ability of promoting generation and inheritance of knowledge by using a great amount of manufacturing data based on industrial operation platforms. The successful construction of an ICPS will take full advantage of knowledge mining of the system. Besides, physical structure based on cyber information can be carried out, aiming at revealing potential information of industrial data. In this way, the cyber systems may optimize the known states and predict the unknown states at the same time. As a transdisciplinary subject, ICPSs have many properties. The ability of perceiving and predicting changes along with uncertainties in the environment is a useful capability for industrial systems. At the same time, emulating new knowledge and experience from group activities is also of great significance for decision making [25]. In addition, the real-time handling capacity, smart computational request, and the adaptation to diverse conditions are also a wide spectrum of aspects that ICPSs cover. The advantages may also be disadvantages once faults occur. More precisely, as soon as faults happen in the system, either in the physical dynamics or in the cyber network, the automatic interaction of ICPSs among physical spaces and cyber spaces will speed up the fault-propagating process. At the same time, the data management and communication layer may be broken down, which means the industrial system security may become collapsed [25]. Accordingly, the study of fault detection in ICPSs to improve the reliability and credibility is of great importance, which is also receiving increasing attention in recent years [25, 26].
2.1. Industrial cyber–physical systems
2.2. Neighborhood preserving embedding
It is critical to find a transformation strategy for industrial development over the world. Thus, the USA defined cyber–physical
Neighborhood preserving embedding (NPE) is one of the nonlinear projection algorithms that is capable of persisting efficient
Fig. 1. The framework of ICPS.
[18]. In fact, some latent assumptions, such as Gaussian distribution on observed signals or linear time-invariant parameters on systems, are usually used for the above mentioned methods. Recently, to extract nonlinear features of samplings, manifold learning methods are paid increasing attention to detect faults in large-scale industrial systems [19]. For example, in [20], an integration of locality preserving projections and principal component analysis are designed to search for both global and local information hidden in industrial signals. In order to highlight partial information, neighborhood preserving embedding (NPE) and support vector data description are used to improve fault detectability [21]. As pointed out in [22], numerous manifold learning methods suffer from singularity problem caused by the singular generalized eigenvalue computation on very high-dimensional projections of original samplings. Motivated by the above observations, we will design a smallscale ICPS for an electrical drive system where real-time control and system monitoring are integrated in the cyber system. Specially, a newly modified NPE method will be developed to deal with the singularity problem, based on which an FD solution is then formulated. The effectiveness of the proposed method will be illustrated using three sets of experiments on one small-scale ICPS. This paper is organized as follows. In Section 2, ICPSs and NPE are firstly described and the objective of this study is followed. In Section 3, design of ICPSs is presented, in which the proposed FD method is discussed in details. In Section 4, complete experiments including fault injection and FD results are carried out, based on which some discussions are given. In Section 5, some concluding remarks together with further work are summarized. 2. Preliminaries
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
information in a feature matrix. The statistical features are extracted by reconstructing the relationship between one specified point and its neighbors [18,27]. In real-world applications, the NPE has a faster learning rate than many other manifold learning methods because of its linear approximation to constructing neighborhood [28]. Given input data set X = [x1 , x2 , . . . , xN ] ∈ RD×N with N being the total number of samplings in a high-dimensional space RD . The purpose of NPE is to determine the output representation Y = [y1 , y2 , . . . , yN ] ∈ Rd×N (d < D) of the initial set X , where the column vector yi of Y is the embedding for the initial xi . The NPE aims at seeking a projection matrix U ∈ RD×d so that Y = U T X . That is, original data set X can be mapped into Y with lower dimension, but there is no loss of both normal and faulty information. The algorithm procedure is stated as follows: (1) Constructing a neighborhood graph: The K -nearest neighbors method is accepted to find the neighbors of the ith node, which corresponds to the data point xi . When xj belongs to the adjacency of xi , a directed edge is put from xi to xj . (2) Finding the weight of each edge: Denote W as a weight matrix with the element wij representing the weight of each edge from node i to node j with 0 ≤ wij < 1. If there is no such edge, then wij = 0. (3) Computing the weight matrix: To grasp the contribution rate of each neighbor node ∑N to the observed node intuitively, with the constraint j=1 wij = 1, the optimal weight matrix can be calculated by minimizing the following reconstruction error [27]
φ (W ) = min
N ∑
∥ xi −
i=1
N ∑
wij xj ∥2 .
(1)
j=1
yyT =1
= min
yyT =1
N ∑
∥yi −
N ∑
wij yj ∥22
j=1
i=1 N ∑
∥uT (xi −
i=1
N ∑
wij xj )∥22
(2)
j=1
= min uT X (I − W )T (I − W )X T u yyT =1
where M = (I −W )T (I −W ). Then the optimization problem is simplified as follows arg min uT XMX T u.
(3)
uT XX T u=1
With the aid of Lagrange multiplier, the optimal projection directions are obtained via solving the following generalized eigenvector problem [27] XMX T u = λXX T u.
the generalized eigenvector problem given in Eq. (4), the rank of XX T may encounter unstable problem, i.e., rank(XX T ) = rank(X ) ≤ min(D, N).
(5)
As presented in Eq. (5), it is possible that the eigen-matrix of XX T is nonsingular and hereby results in the infeasibility of NPE. Thus, a modified version of NPE is proposed in this study to avoid the aforementioned situation. 2.3. Objective In this paper, a small-scale ICPS based on the electrical drive system is firstly designed. We will be providing a practical verification platform for fault detection applications. On such a verification platform with real-time calculation and transmission of observations, the ultimate objectives of this paper are summarized as follows
• Establishment of an integrated upper-layer computer for the ICPS including real-time control and supervisory units;
• Development of a stable solution to strengthen the robustness of the system, called modified NPE, for fault detection purpose in the supervisory unit; and • Analysis of the effectiveness of the modified NPE without the loss of discriminative information. 3. The proposed methodology for cyber–physical systems In this section, the design of our small-scale ICPS will be introduced into physical space and cyber space. More significantly, the improvement of modified NPE will be addressed in details including performance and characteristic analysis. 3.1. Design of a small-scale ICPS
(4) Calculating the projection manifold: Vectors of u1 , u2 , . . . , ud are the projection directions to formulate the matrix U. On the basis of another constraint, i.e. yyT = 1, the problem of arbitrary scaling factor is avoided. Meanwhile, an optimal mapping has been chosen to minimize the following cost function
Φ (Y ) = min
3
(4)
The eigenvectors associated with d smallest eigenvalues of Eq. (4) will form matrix U. Thus, primary high dimensional data set has been lessened and the projection manifold has been traced eventually. The original NPE method shows its advantages on dimensionality reduction and feature extraction. However, when NPE solves
Our small-scale ICPS, which is composed of physical space and cyber space, is actually an electrical traction platform. Detailed descriptions of all components are illustrated as follows. In physical space of this electrical traction system, to be more specific, the equipment mainly consists of a high voltage control panel, a permanent magnetic synchronous motor (PMSM) and a controller. Fig. 2 shows the detailed components of this platform. Sensors are deployed in different positions so that various data samples can be obtained including multiple output circuit signals, current, and speed. This system adopts stator flux-oriented vector control strategy so that it is capable of driving the motor to run at a preset speed. The cyber space of this small-scale electrical traction ICPS refers to the data points with different information. The whole collected sampling data sets can form the group space which can provide raw data for computation space, in which online fault detection algorithms is included. When it comes to computation space, all the industrial data acquired from sensors or actuators should be computed and managed in a particular way. It is worth mentioning that, by the use of this proposed method, the new feature can be extracted directly from original data sets. In addition, this hidden information is important in deduction space. Once receiving derived features and required performance by networks, the deduction space will deliver results to decision-making space where information will be transformed into knowledge and the knowledge will then be sent to physical space. As a significant component in the cyber space, the network mainly transfers data to computational unit as well as conveys the execution instructions to the actuator. Once faults are injected into this small-scale ICPS in an unpredicted way, the physical space will collect data with faulty
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
4
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 2. Detailed components of small-scale CPS.
information and share them with cyber space. After procedures such as computation and deduction, variable features obtained together with fault location will be fed back to physical space through network. This small-scale ICPS connects physical and cyber systems fluidly, and the decision-making behavior is based on the data analysis so that the fault in this small-scale ICPS can be detected reliably. 3.2. Modified neighborhood preserving embedding Aimed at simplifying the generalized eigenvalue problem of the NPE, one algebraic method has been proposed. More strictly speaking, the construction of a new matrix Ur can seek the relationship between optimization objection and constraints. A pseudo-code for computing the projection matrix is shown in Algorithm 1.
where E = (Er E˜ r ) = (e1 , e2 , . . . , er , er +1 , . . . , eD ) with column vectors normalized and orthogonal to each other, r = rank(Ur ), and Σ = diag(σ1 , σ2 , . . . , σr , 0, . . . , 0) which is composed of eigenvalues of Ur with σ1 ≤ σ2 ≤ · · · ≤ σr . At the same time, E ∈ RD×D , Er ∈ RD×r , E˜ r ∈ RD×(D−r) , and Σ ∈ RD×D . The column vectors of E form a basis for the vector space Ur . Denote V as the spanned set by Ur such that V = Span{e1 , e2 , . . . , eD }. Considering eigenvalues of E˜ r to be 0s , we can have E˜ r ∈ N(Ur ), where N(Ur ) is the null space of Ur . Thus, the column vectors of Er form the basis for Ur . Let R(X ) be the column space of X and R(XX T ) be the column space of XX T , then the following equality holds R(X ) = N(X T )⊥ = N(XX T )⊥
(7)
= R((XX T )T ) = R(XX T )
where X and XX T share the same column space, which means the basis for the vector space XX T is E = (Er E˜ r ), the same as Ur . Then, we will find the relationship between E˜ r and X with the 1/2 aid of Ur . Define that S = ∥Ur E˜ r ∥2l such that 2
S = E˜ r [X (I − W )T (I − W )X T + XX T ]E˜ rT = [E˜ r X (I − W )T ][E˜ r X (I − W )T ]T + (E˜ r X )(E˜ r X )T
= ∥[E˜ r X (I − W )T ]T ∥2l2 + ∥(E˜ r X )T ∥2l2
.
(8)
= 0 Then, one has
{
E˜ r X (I − W )T = 0 E˜ r X = 0
The detailed process of the algorithm is now elaborated. Define Ur = XMX T + XX T such that the eigenvalue decomposition (EVD) of Ur will be represented as follows Ur = XMX T + XX T
= EΣ ET
(6)
(9)
where E˜ r ∈ N(X ), N(X ) is the null space of X , and the column vectors of Er form the basis of X . Given the fact that U is the projection matrix which is comprised of the neighboring information of X , then the nonempty subset U is a subspace of the vector space X . Hence, any vector in U can be written as a linear combination of the basis of X . Suppose that u is the solution of Eq. (4), then u = Er l where l is the coefficient vector of Er and l ∈ Rr . Remark 1. The application of EVD is to extract direction of the designated matrix. Under this situation, we represent u with basis
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
of E˜ r so that the optimization can be changed. With mathematical calculation, one specific relationship between the optimal formula and constraints are tracked to alleviate the computational burden. Thus, the optimal problem of Eq. (3) can be transformed into this form: arg min lT ErT XX T Er l=c
lT ErT XMX T Er l.
(10)
We can further have an equivalent form of the above optimal problem 1/2
Σ −1/2 ErT XMX T Er Σ −1/2 Σ 1/2 l
arg min
lT Σ
s.t.
lT Σ 1/2 Σ −1/2 ErT XX T Er Σ −1/2 Σ 1/2 l = c
.
(11)
The purpose of extending this function is to find the essential relationship between optimal objection and constraints. By using an equivalent transformation, the connection/projection between X and Y can be found; meanwhile, the unstable problem caused by nonsingular problems can be solved. Several variables are represented as follows, including v = Σ 1/2 l, Ub = Σ −1/2 ErT XMX T Er Σ −1/2 , and Uc = Σ −1/2 ErT XX T Er Σ −1/2 . Now that Ub + Uc = I, Eq. (10) is equal to arg min v T (I − Uc )v.
(12)
v T Uc v=c
In order to calculate the optimal v , a Lagrange function is used to solve Eq. (12) as follows L(v, λ) = v T (I − Uc )v − λ(v T Uc v − c).
(13)
The above optimal problem is the same as the following generalized eigenvalue problem: (I − Uc )v = λUc v.
(14)
Designate the solution of (14) as V = (v1 , v2 , . . . , vd ) with λ1 ≤ λ2 ≤ · · · ≤ λd . The vector u can be obtained by u = Er l while the projection matrix U can be computed as U = Er Σ −1/2 V . Remark 2. Note that the local reconstruction of coefficients is incorporated in the intrinsic graph and when the local linear reconstruction error is minimized, the optimal projection matrix can be obtained. This obtained projection matrix can not only extract characteristic features on data manifold but also provide an available procedure to ensure the effectiveness of inner combination of neighbored nodes locally. In this process, Uc is of full rank; it means there exist no singular problems for this modified NPE algorithm. Remark 3. With the help of matrix theories, the generalized eigenvalue calculation with XMX T and XX T in NPE is transformed into the EVD problem and generalized eigenvalue problem. Because the relevant matrices are symmetric and positive semidefinite, this newly proposed algorithm will effectively decrease computation loads and avoid the instability caused by NPE for the small-scale ICPSs.
5
Table 1 Parameters of the experimental setup. Symbol
Values (Unit)
Descriptions
p Jm La Ra Dm Oa Te Ts TL
4(1) 0.425 × 10−3 (kg m2 ) 2.96 × 10−3 (H) 0.985 () 1 × 10−4 (1) 8.1 × 10−2 (Wb) 2.43 (N m) 1 × 10−4 (s) 3.645 (N m)
Pole pairs Moment of inertia Inductance of motor coil Resistance of motor coil Viscosity friction coefficient Magnet flux Electromagnetic torque Sampling time Load torque
and will cost plenty of time and memory. Moreover, matrix XX T has the possibility of being singular so that it may bring the interruption of calculation and loss of sensitivity of fault detection. What makes modified NPE attractive is its ability to overcome the aforementioned drawbacks of the NPE. The unstable problem of the NPE can be solved via transforming it into an EVD problem and a generalized eigenvalue problem. It is also evident that the similarities between neighbored pairs of data points can be preserved in the original high-dimensional space. According to matrix theories, the primary information is included in eigenvectors corresponding to the maximum eigenvalues. Eq. (6) shows that the essential information of XMX T + XX T is merely associated with eigenvalues of σ1 , σ2 , . . . , σr ; that is, eigenvectors of Er . Thus, it can make great impact on reduction of calculation of this proposed algorithm. We can know that the local information of xi and its neighborhoods are stored in weight matrix W . The column vectors of W are linear coefficients between data point xi and xj that characterize local geometry of different nodes. With the purpose of analyzing performance of the designed algorithm thoroughly, Mahalanobis distance is applied to measure the deviation of measurements, which can be computed as [29] T 2 = X T U Λ−1 U T X
(15)
where Λ is the covariance matrix of Y , and Λ = 1/(n − 1)Y T Y . Euclidean distance is used to indicate measurement deviations, which can be calculated uniquely by SPE = X (I − UU T )X T .
(16)
With the application of above two criteria of performance, T 2 not only shows the sensitivity of projection matrix, but also characterizes the FD performance of this proposed method while SPE takes the correlation among signals from sensors into account of this ICPS. The validity of the introduction of T 2 and SPE has been entirely stated in [30,31]. Two available statistics mentioned above can detect abnormalities when changes emerge in the monitored system. In addition, the selection of control limits of T 2 and SPE also plays a significant role in the fault detection results [19]. The productive and practical way to obtain the corresponding confident region and control limit is explained in details in [6,15]. 4. Experiments
3.3. Performance and characteristics In the considered small-scale ICPS, data set X acquired from the electrical drive system should be projected into Y by U. The above NPE algorithm has the capability of transforming data with dimension of D into a reduced set of dimension of d. Nevertheless, the dimension of each of matrices XM T X and XX T is D × D, which means when it comes to generalized eigenvalue problem, this initial algorithm is confronted with great amount of calculation
In this section, the fault injection and detection process for our experimental small-scale ICPS has been implemented, after which the detection results have been discussed. 4.1. Experimental platform with fault injection To evaluate the performance of this proposed algorithm, three sets of experiments with sensor faults have been conducted in
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
6
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 3. Fault detection results for f1 with NPE.
Fig. 4. Fault detection results for f1 with modified NPE.
Fig. 5. Fault detection results for f2 with NPE.
our small-scale ICPS. Different types of sensors equipped in the physical space can provide samples for the closed-loop control unit in physical space and analysis along with monitor in cyber space. Accurate detection of various sensor faults is urgently needed in this electrical traction ICPS. Table 1 gives main parameters of the permanent magnetic synchronous motor. With the aid of a Timebase module in QUARC to support a more accurate sampling time setting, the sampling time interval of the experiment is 0.1 ms. In accordance with electrical systems, seven signals have been selected, including ia and ib as current signals, va , vb , vc , vdc as voltage signals, and s as speed signal of the motor. Three sensor faults are injected into the process: (1) an incipient fault on ia sensor with the
fault amplitude is f1 = 0.1 A; (2) an incipient fault on s sensor with the form as f2 = (t − 4) r/min; (3) an intermittent fault on vdc sensor as f3 = 10 V. Samples containing both healthy and faulty data are collected from 49 s to 55 s. To obtain online data sets for validation, three incipient sensor faults are injected into this small-scale ICPS from 51 s. Once faults occur, they are always with diverse latent signal information, which means the underlying contents are contained in the above signals. 4.2. Detection of incipient faults Once faults are injected into this physical space, the cyber space is going to detect faults. The complete implementation
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
7
Fig. 6. Fault detection results for f2 with modified NPE.
of fault detection on our small-scale ICPS is proposed with offline modeling and online fault detection. Based on the proposed modified NPE algorithm, the projection matrix U aims to capture the local structure of original data space from off-line data to online data in a corresponding way. The off-line part can be summarized as follows:
of extracting features with unsatisfactory performance. In the following contents, we make a comparison study between NPE and modified NPE for the same faults, and the detection results indicate the advantages of our modified one. To demonstrate the superiority of our proposed method than traditional one, monitoring results for three sensor faults are given in Figs. 3 to 8. 4.3. Analysis and discussions
In our experiment, we first construct off-line models by acquiring off-line data with usual operating cases; they are used as the training set to calculate the projection matrix, as well as generation of T 2 , SPE statistics, and control limits. Thanks to the modified NPE process, the features with normal and faulty samples are extracted and memorized in the projection matrix and will be helpful to online procedure. For online fault detection, actual signals in both normal and abnormal conditions are collected and normalized as the testing set. Then, projection matrix calculated from off-line training set plays a significant role here to fetch the current T 2 and SPE statistics. Both off-line and online data sets share the common latent structure which reveals systemic information because the electrical traction systems work under the same control strategy when data sets are collected in both off-line and online phase. We now make a comparison between new statistics and control limits; once new statistics outweigh their limits, a fault is successfully detected. The detailed online procedure is as follows.
In practice, we usually collect off-line training data enough to make the distribution of off-line set similar to the distribution of online set. With the application of modified NPE method, the normal training matrix is augmented and 30 neighbors are selected for each sample in the constructed adjacency graph. Now that the actual CPS operates with noise and disturbance (i.e., uncertainties in the system), the original NPE is capable
Under the same false alarm ratio which can be determined based on the choice of thresholds, Fig. 4 indicates detection results for f1 using the modified NPE algorithm while Fig. 3 indicates the results using the original NPE algorithm. It is clearly shown that, by the use of the NPE algorithm, the index T 2 shows poor fault detectability. But for the modified NPE algorithm, both T 2 and SPE statistics can achieve satisfactory performance. Fig. 5 indicates the detection results for f2 using NPE, and the performance is unaccepted because both T 2 and SPE statistics are invalid for the detection. However, as shown in Fig. 6, by analyzing the detection results with modified NPE for incipient fault f2 , it is observed that T 2 statistic in this case has weak ability of recognizing fault because of its high false alarm rate. Nevertheless, SPE shows its capability of detecting fault accurately for f2 when it occurs. In fact, the dramatic change of rotation speed of the motor could result in vibration. As soon as this instability occurs, the closed-loop control unit in the physical system of the ICPS starts to work to attenuate the consequence fault f2 brings about, which means the validity of our algorithm would be affected. Considering the control unit, a comparison will be made among f1 , f2 and f3 . Different from f1 and f2 that are closedloop faults, f3 is an open-loop one. To have a good view of fault detection performance, logarithmic transformation is applied to generate the detection result of f3 . Based on detection results shown in Fig. 8, when f3 occurs, the proposed FD method can rapidly detect the occurrence of this fault; meanwhile, NPE algorithm applied to the same data set shows barely satisfactory performance if only T 2 statistic is considered. Thus, the effectiveness and superiority of our proposed method are illustrated on this experimental platform of a small-scale ICPS. In our experiments, sampling time interval is 1 × 10−4 s. By using the modified NPE algorithm, the calculation speed is 2.03 × 10−6 s, which means the computing process of fault detection can catch up with the sampling procedure totally. Obviously, it has been demonstrated that the modified NPE algorithm is effective in fault detection. The local structure and information of samplings can be converted into projection matrix; that is, modified NPE can realize its goal of preserving the local structure of the data set in a linear way to reveal the intrinsic relationship between the observed data and its neighbors.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
8
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx
Fig. 7. Fault detection results for f3 with NPE.
Fig. 8. Fault detection results for f3 with modified NPE.
5. Conclusions
References
In this paper, a small-scale ICPS has been designed for an electrical drive system. Focused on the extendibility and safety, a new FD method called modified NPE has been proposed for the online monitoring purpose. It was elaborated that by solving the singularity problem, the proposed method is more effective than traditional manifold learning methods for detection of sensor faults. Through the designed small-scale ICPS, the performance and effectiveness of this study are convincingly demonstrated. In addition, based on this work, further research on safety and reliability will be drawn on data-driven fault-tolerant control for ICPSs.
[1] Jiang Y, Yin S, Kaynak O. Data-driven monitoring and safety control of industrial cyber-physical systems: basics and beyond. IEEE Access 2018;6:47374–84. [2] Colombo AW, Karnouskos S, Shi Y, Yin S, Kaynak O. Industrial cyberphysical systems. In: 2017 IEEE international conference on mechatronics, Vol. 104, No. 5. 2017, p. 899–903. [3] Liu M, Shi Y, Gao H. Aggregation and charging control of PHEVs in smart grid: A cyber-hysical perspective. Proc IEEE 2016;104(5). [4] Zhang Y, Qiu M, Tsai CW, Hassan MM, Alamri A. Health CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Syst J 2017;11(1):88–95. [5] Jiang Y, Yin S. Recursive total principle component regression based fault detection and its application to vehicular cyber-physical systems. IEEE Trans Ind Inf 2017;14(4):1415–23. [6] Chen H, Jiang B, Lu N, Mao Z. Deep PCA based real-time incipient fault detection and diagnosis methodology for electrical drive in highspeed trains. IEEE Trans Veh Technol 2018;67(6):4819–30. [7] Pajic M, Weimer J, Bezzo N, Sokolsky O, Pappas GJ, Lee I. Design and implementation of attack-resilient cyber-physical systems: With a focus on attack-resilient state estimators. IEEE Control Syst 2017;37(2):66–81. [8] Zhang K, Shardt YAW, Chen Z, Peng K. Using the expected detection delay to assess the performance of different multivariate statistical process monitoring methods for multiplicative and drift faults. ISA Trans 2017;67:56–66. [9] Chen H, Jiang B, Lu N. An improved incipient fault detection method based on kullback-leibler divergence. ISA Trans 2018;79:127–36. [10] Chen H, Jiang B, Ding SX, Lu N, Chen W. Probability-relevant incipient fault detection and diagnosis methodology with applications to electric drive systems. IEEE Trans Control Syst Technol 2018. http://dx.doi.org/10. 1109/TCST.2018.2866976. [11] Qin SJ. Survey on data-driven industrial process monitoring and diagnosis. Annu Rev Control 2012;36(2):220–34. [12] Ding SX, Yang Y, Zhang Y. Data-driven realizations of kernel and image representations and their application to fault detection and control system design. Automatica 2014;50(10):2615–23.
Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work was supported by National Natural Science Foundation of China (61490703, 61374141, 61573180), Funding of Jiangsu Innovation Program for Graduate Education (KYLX 16_0378), Priority Academic Program Development of Jiangsu Higher Education Institutions, and Open Fund for Postgraduate Innovation Laboratory of Nanjing University of Aeronautics and Astronautics under Grant kfjj20180320. In addition, the authors are grateful for Associate Editor and anonymous reviewers for their constructive comments.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.
H. Chen, J. Wu, B. Jiang et al. / ISA Transactions xxx (xxxx) xxx [13] Luo H, Yang X, Kruger M, Ding SX, Peng K. A plug-and-play monitoring and control architecture for disturbance compensation in rolling mills. IEEE/ASME Trans Mechatronics 2018;23(1):200–10. [14] Kim J, Tong L, Thomas RJ. Subspace methods for data attack on state estimation: A data driven approach. IEEE Trans Signal Process 2015;63(5):1102–14. [15] Chen Z, Ding SX, Peng T, Yang C, Gui W. Fault detection for non-gaussian processes using generalized canonical correlation analysis and randomized algorithms. IEEE Trans Ind Electron 2018;65(2):1559–67. [16] Tong C, Lan T, Shi X. Double-layer ensemble monitoring of non-gaussian processes using modified independent component analysis. ISA Trans 2017;68:181–8. [17] He W, He Y, Zhang C. A new fault diagnosis approach for analog circuits based on spectrum image and feature weighted kernel Fisher discriminant analysis. Rev Sci Instrum 2018;89(7). 074702. [18] Wang Y, Wu Y. Complete neighborhood preserving embedding for face recognition. Pattern Recognit 2010;43(3):1008–15. [19] Chen H, Jiang B. A review of fault detection and diagnosis for the traction system in high-speed trains. IEEE Trans Intell Transp Syst 2019. http: //dx.doi.org/10.1109/TITS.2019.2897583. [20] Zhang M, Ge Z, Song Z, Fu R. Global-local structure analysis model and its application for fault detection and identification. Ind Eng Chem Res 2011;50:6837–48. [21] Jiang Q, Yan X. Probabilistic weighted NPE-SVDD for chemical process monitoring. Control Eng Pract 2014;28:74–89.
9
[22] Lunga D, Prasad S, Crawford MM, Ersoy O. Manifold-learning-based feature extraction for classification of hyperspectral data: A review of advances in manifold learning. IEEE Signal Process Mag 2014;31(1):55–66. [23] Seshia SA, Hu S, Li W, Zhu Q. Design automation of cyber-physical systems: challenges, advances, and opportunities. IEEE Trans Comput-Aided Des Integr Circuits Syst 2017;36(9):1421–34. [24] Tao F, Cheng J, Qi Q. IIHub: an industrial internet-of-things hub towards smart manufacturing based on cyber-physical system. IEEE Trans Ind Inf 2018;14(5):2271–80. [25] Pasqualetti F, Dorfler F, Bullo F. Attack detection and identification in cyber-physical systems. IEEE Trans Automat Control 2013;58(11):2715–29. [26] Massoumnia MA, Verghese GC, Willsky AS. Failure detection and identification. IEEE Trans Automat Control 1989;34(3):316–21. [27] He X, Cai D, Yan S, Zhang H. Neighborhood preserving embedding. In: Proceedings of the 10th IEEE international conference on computer vision. 2005, p. 1208–13. [28] Roweis ST, Saul LK. Nonlinear dimensionality reduction by locally linear embedding. Science 2000;290(5500):2323–6. [29] Ding SX. Data-driven design of fault diagnosis and Fault-Tolerant control systems. Springer-Verlag; 2014. [30] Yin S, Ding SX, Xie X, Luo H. A review on basic data-driven approaches for industrial process monitoring. IEEE Trans Ind Electron 2014;61(11):6418–28. [31] Gao Z, Cecati C, Ding SX. A survey of fault diagnosis and fault-tolerant techniques-part II: fault diagnosis with knowledge-based and hybrid/active approaches. IEEE Trans Ind Electron 2015;62(6):3768–74.
Please cite this article as: H. Chen, J. Wu, B. Jiang et al., A modified neighborhood preserving embedding-based incipient fault detection with applications to small-scale cyber–physical systems. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.08.022.