- Email: [email protected]
Automated diagnostics for manufacturing machinery based on well-regularized deep neural networks
Author’s Accepted Manuscript Automated Diagnostics for Manufacturing Machinery Based on Well-Regularized Deep Neural Networks Robert DiBiano, Supratik Mukhopadhyay www.elsevier.com/locate/vlsi
PII: DOI: Reference:
S0167-9260(17)30182-7 http://dx.doi.org/10.1016/j.vlsi.2017.03.012 VLSI1324
To appear in: Integration, the VLSI Journal Received date: 23 April 2016 Revised date: 21 March 2017 Accepted date: 21 March 2017 Cite this article as: Robert DiBiano and Supratik Mukhopadhyay, Automated Diagnostics for Manufacturing Machinery Based on Well-Regularized Deep Neural Networks, Integration, the VLSI Journal, http://dx.doi.org/10.1016/j.vlsi.2017.03.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Automated Diagnostics for Manufacturing Machinery Based on Well-Regularized Deep Neural Networks Robert DiBiano AutoPredictiveCoding LLC Email: [email protected] Supratik Mukhopadhyay Louisiana State University Email: [email protected]
Corresponding Author: Supratik Mukhopadhyay Louisiana State University 164 B Coates Hall Baton Rouge, LA 70803 Email: [email protected] Phone: 225-578-1496 Fax: 225-578-1465 Abstract: This paper presents a bigdata framework based on regularized deep neural networks for automated diagnostics for manufacturing machinery based on emitted sound, vibration, and magnetic field data. More precisely, we present SpotCheck, a prototype system that uses well-regularized deep neural networks to analyze sound, vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machinery. It is completely automatic requiring no manual extraction of handcrafted features. It can operate with relatively small amounts of training data, but can take advantage of large volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data to improve performance as more data becomes available after deployment.
Keywords: Modeling and analytics for big data; Smart intelligent technologies to enable automated systems for M2M (machine to machine) communication
1. Introduction Regular monitoring and maintenance of machinery in a manufacturing plant are absolutely necessary for efficient functioning and to prevent disasters. Malfunctioning machinery can not only stand in the way of efficient operations, causing shutdown of essential processes and requiring expensive repairs resulting in millions of dollars in losses for a manufacturing plant, but can also result in potentially hazardous situations. Constant monitoring is needed to ensure that safety and manufacturing protocols are being adhered to. Violation of such protocols can lead to defective products and/or hazardous environments on the factory floor. For example, in the plastic manufacturing industry, the product needs to be dried at an appropriate vacuum level (pressure); excess or reduced vacuum level due to dryer malfunction will result in a brittle product. Similarly, in the drug manufacturing industry, tablets need to be coated at an appropriate vacuum level; otherwise the resulting product can be too soft due to excess moisture or can crumble due to too little moisture. In a (nuclear) power plant, malfunction of water pumps can lead to violent explosions. Thus reactive maintenance, i.e., repairs in the aftermath of a malfunction can be extremely expensive to a (manufacturing) plant. Proactive maintenance practices include regular manual checkups to ensure that the vital parts of a machine are working properly; in other words to ensure that the vacuum level, belt tension, oil state, oil level, bearing faults, etc. have the correct values in accordance with specifications of the manufacturing process. However, such checkups are usually invasive resulting in machine downtime causing financial loss to a plant. Besides, such manual checkups seldom predict far into the future. For efficient operation of a manufacturing plant, monitoring needs to be non-invasive and maintenance needs to be predictive. Recent advances in Artificial Intelligence (AI) [15] have provided us the opportunity to (remotely) noninvasively and inexpensively monitor and predictively maintain machinery in manufacturing plants. It may now be possible to completely automatically perform checkups of manufacturing machinery 24/7 without any machine downtime and automatically schedule maintenance predictively. In the last few years, significant advances in AI have greatly improved old applications such as machine translation, voice recognition, etc., and have created a host of new applications such as self-driving cars, personal digital assistants, etc. [16, 17]. In the heart of such developments lie neural networks, an AI technique loosely emulating the human brain. However, traditional neural networks need large volumes of training data [18], and this has limited their use for industrial applications where representative, labeled data might be difficult or costly to acquire. For example, to acquire failure data for a manufacturing machine, it has to be run on failure mode, a dangerous operation. But improved methods of regularization [19] allow deeper neural networks to potentially be trained with smaller amounts of labeled data, bringing industrial applications within reach. We present SpotCheck, a prototype system that uses deep, well-regularized neural networks to analyze sound, vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machine. It is completely automatic requiring no manual extraction of handcrafted features. It can operate with relatively smaller collection of labeled training data, but can potentially take advantage of larger volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data to improve performance as more data becomes available after deployment. 1.1 Novelty of SpotCheck We are moving towards an environment where sensors are collecting data from manufacturing machinery 24/7, and sending it to a bigdata system capable of assimilating it into a sophisticated learning algorithm. The SpotCheck system is a first step in that direction for industrial applications. Unlike existing approaches, SpotCheck provides a new, general method that uses only raw spectral features where physically possible, and falls back to a general purpose ensemble of statistical features extracted
automatically when low or irregular sampling rate makes spectral analysis impossible. This enables SpotCheck to work equally well for many different types of signals without any expert knowledge. SpotCheck can robustly generalize between different machines, sensors, and fault instances. It can operate with relatively lesser collection of labeled training data, but can potentially take advantage of larger volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data. We demonstrate below that SpotCheck outperforms existing approaches in several diagnosis/estimation problems for manufacturing machinery. While deep neural networks are not new, using them effectively for this type of industrial, raw labeled-data-sparse application has not been thoroughly studied yet and is novel.
2. Related Work In the last few years, significant advances have been made in the area of neural networks, old applications have been greatly improved, and a host of new applications have been developed [16, 17]. And deep neural networks still have yet to reach their full potential. Convolutional Neural Networks (CNNs) such as VGG-16 (Visual Geometry Group - 16) [10] and AlexNet [6] have been used for large scale image recognition tasks with great success. Similar classification tasks using other datatypes such as sound, light, radio, temperature, or electrical signals are less developed but also of great interest [8]. There are two standard approaches to machine signal analysis: statistical analysis of the raw signal, or analysis of the frequency spectrum. Sassi et al. [9] geometrically modeled common faults in ball bearings, and provided theoretical justification for using both spectral and first order statistical features to detect and analyze them. Most of the same principles apply to all machinery with rotating parts [8]. Although the type of signal measured is most commonly vibration [7], any time varying signal can be used. Wu and Liu [12] used spectral wavelet features, specifically Discrete Wavelet Transform (DWT) with a neural network to diagnose faults in internal combustion engines. Later in [13], the authors used another spectral technique, the Wavelet Packet Transform (WPT) for the same problem. The principle was sound, but their neural networks did not use modern regularization, and they used a handcrafted feature of Shannon entropy, and handpicked a subset of these features that were considered significant to the problem. It can detect four binary fault conditions with high accuracy, but not estimate engine parameters. It is also unclear whether it can detect the same fault conditions as accurately on different engines of the same type, since only one example of each engine, sensor, and fault was used for both training and testing. Tyagi [11] experimented with using neural nets and Support Vector Machines (SVMs) for rolling element bearing fault diagnosis based on vibrational data. He noted that the energy for some types of defects is low, and spread out over a wide range of frequencies. These types of responses are easily masked, and difficult to manually detect, but lend themselves well to neural networks that can interpret raw, high dimensional inputs. Tyagi used hand crafted statistical and spectral features, and in the absence of good regularization methods experimented with using SVMs instead of neural networks. His system was trained with several common defect types, and can predict binary bearing quality as normal or abnormal. Farokhzad et al. [3] use neural networks to detect fault conditions in water pumps. They used statistical and spectral features of vibrational signals. They detected three binary fault conditions, but were unable to test on more than one pump. Yadav et al. [14] use neural networks and handcrafted features extracted from the spectrogram for condition analysis on internal combustion engines. Their system makes a binary classification for 6 types of faults. It is an expert system, using handcrafted features and knowledge specific to this problem. Abad et al. [1] used neural networks with spectral analysis to detect five types of faults in rotating machinery via sound waves. As opposed to the works presented above that extract
handcrafted features manually from the data, SpotCheck automatically extracts features from any type of data (magnetic field emissions, sound, vibrations, etc.); SpotCheck can use essentially any time-varying signal characterizing machine behavior. Besides, most of the works above are concerned with binary classification whereas SpotCheck can approximate functions. Dandare and Dudul [2] detected faults in automobile engines based on emitted sound. They used spectral features, and showed that SVM gave much better performance than neural networks, presumably due to the good regularization built into SVMs, sparse training data, and a lack of modern regularization on their neural networks. This is supported by the fact that Tyagi [11] noted a much smaller improvement by SVMs over neural networks, even after his neural networks showed clear evidence of overfitting (0% training error with 23% testing error), suggesting they were not effectively regularized. Gharehbolagh et al. [4] recognize wheel and pinion faults using neural networks and handcrafted statistical and spectral analysis of sound. Some researchers have experimented with other signal types or attempted to estimate non binary parameters in a limited way. Refaat et al. [7] developed a fault detection system analyzing the electrical current signal through induction motors. They use handcrafted features and expert knowledge, and they can predict non-binary types of faults. Sadegh et al. [8] use spectral features, genetic algorithms, and neural networks - analyzing acoustic signals to estimate lubrication conditions in bearings. Unlike most previous methods, they do not limit themselves to binary fault conditions, but classify between three “lubrication regimes”, with 96% accuracy. Their system required heavy frequency band selection, and feature selection, for which they used genetic algorithms. However, they still used a master list of handcrafted wavelet features which must be determined manually, and handcrafted statistical features are used in all cases as well. They did not use different engines for training and testing purposes; thus they are likely to overfit to a few specific engines. The methods mentioned above have generally concentrated on diagnosis of existing faults, rather than machine diagnostics that could be used to analyze performance and predict future faults. Most are problem specific rather than general purpose, and all use some form of handcrafted features. Almost all use spectral features, usually combined with 1st order statistical features. They presumably use the same engine for training and testing sets, which fails to clarify whether the method is overfitting for that specific engine or finding useful generalizations that can be applied everywhere. The reported experiments are generally limited to binary classification of a few error conditions. There is no intra machine calibration, and expert knowledge is often used. In contrast, SpotCheck provides a new, general method that uses only raw spectral features where physically possible, and falls back to a general purpose ensemble of statistical features extracted automatically when low or irregular sampling rate makes spectral analysis impossible. In the latter case, we only use 1st order features, i.e., features that can be calculated directly from the histogram, which are therefore sampling rate and frequency independent. Such a system should work equally well for many different types of signals without any expert knowledge, and has been tested successfully on sound, magnetic field, and vibrations. We use modern deep neural networks with sophisticated regularization techniques to compensate for high input dimensionality of the dataset and limited training data, and are able to estimate even non binary parameters with a high degree of accuracy. Unlike most previous systems, SpotCheck was tested on multiple sensor, machine, and sensor-position combinations to verify that it robustly generalizes between different machines, sensors, and fault instances. There is no single standard method in use for industrial machine diagnostics, but generally they involve observing statistical or spectral data and trying to find patterns manually. These methods may require
invasive machinery, or carefully calibrated rangefinders to measure vibration without touching moving parts [5]. They generally will not operate on anything as chaotic and indirect as nearby sound or electromagnetic waves.
3. Bigdata in Machinery Diagnostics: Velocity, Volume, Variety Sound, magnetic field, and vibration emissions from manufacturing machinery running 24/7 generate a tremendous volume, velocity, and variety of unlabeled data. Sound data collected from a blower sampled at 44.1KHz at 16 bits per sample amounts to a megabyte every six seconds (velocity) represented as wav files (providing lossless compression); i.e., every 24 hours approximately 1.44 GB of data per sensor gets collected. Vibration and magnetic emissions from a vacuum pump collected at 5Hz amount to 40 bytes per second (with a floating point representation); this amounts to 1.7 MB per day. In a typical small factory with 20 blowers, one can easily expect around 28 GB of unlabeled sound data per day (volume). Labeled data, however, is costly to acquire; one needs to run machines in failure mode, a dangerous operation. Thus, a bigdata framework for manufacturing machinery diagnostics needs to employ techniques that can exploit the easily available unlabeled data rather than depending solely on labeled data. Techniques like Support Vector Machines [25] are unable to leverage large amounts of unlabeled data extracted from industrial machinery and depend on being trained with labeled data for classification. If large volumes of labeled training data are unavailable, machine learning techniques employed by bigdata frameworks to provide predictive analytics need to prevent overfitting. In addition, bigdata frameworks processing data from industrial machinery need to handle the variety in the type of data collected (e.g., sound vs. magnetic/vibration emissions) together with different representations (wav files vs. floating point). Besides, the processing of this data needs to be in real time as opposed to offline for predictive diagnosis. Hence, bigdata frameworks processing data emitted by manufacturing machinery need to leverage graphic processing units (GPUs) to exploit data parallelism.
4. Theoretical Background As we have noted, analyzing industrial machinery based on the spectral characteristics of the noise has worked in both research and real world [5] applications in the past. Obviously if there weren’t any spectral differences between different machine states, the method couldn’t work; but it does seem to hold for a wide variety of practical problems. The exact theoretical reasons why this works vary by the geometry of the device being analyzed, but [9] gives us good insight into the physics of rotating machinery, and is a good representative example. A properly functioning bearing has specific frequencies associated with it, and common fault types have their own characteristic frequencies. In [24], fundamental train frequency and ball spin frequency are defined as: [
( (
) ) (
(
)] )
(1) (2)
where ωi and ωo are rotation speed of the inner and outer races respectively, Bd denotes the ball diameter, Pd denotes the pitch diameter, and α denotes the contact angle. Characteristic frequencies associated with faults, ball pass frequency on outer and inner defects are defined as:
(
) (
)
(3)
(
) (
)
(4)
where Nb is number of rolling elements. So by looking for the correct frequencies in equations 1 and 2 we can detect proper function, and by looking for the frequencies in equations 3 and 4 we can identify specific defects. In practice, we would like a system that will discover these various characteristic frequencies automatically via deep learning using well-regularized neural networks. The two most important aspects of modern neural networks are regularization and data abstraction. Regularization allows a neural network, trained with the backpropagation algorithm, to interpolate between different training samples and draw generalizations to classify similar samples. Data abstraction allows a network to break data into progressively simpler sub-classes, allowing it to extrapolate learning to samples that do not closely resemble anything it was trained with. For a case where there is a high dimensional-feature space and limited training data, good regularization is key in providing good generalizations. The most common regularization method is to modify the cost function the neural network seeks to minimize to have a penalty for irregular decision boundaries or other phenomenon that increase overfitting. L2 regularization [19], probably the most common type, adds a penalty for high neuron weights, incentivizing sparser, simpler configurations. This is accomplished by adding the L2 norm of the weights multiplied by a parameter λ to the cost function. The parameter λ determines the rate at which the weights “decay” during training using backpropagation [19]. We also used dropout [20], a model averaging method that performs weight regularization automatically. In cases where more training data is available, data abstraction can be applied to find complex patterns in training data, as well as utilize large quantities of unlabeled data to improve performance. In the case of Deep Belief Networks (DBN), stacked autoencoders [23] or Restricted Boltzmann machines (RBMs) [22] are used during a pretraining step to effectively reduce input dimensionality, and deal with the vanishing gradient problem that historically prevented deep networks from efficiently training.
5. The SpotCheck System We now describe our method in detail. The algorithm used for SpotCheck is detailed below in Algorithm 1. Our signal classification algorithm windows a short period of the signal (where the signal is emitted sound, magnetic field, or vibration data). After calculating the magnitude of the power spectrum via FFT (Fast Fourier Transform) (line 4 in Algorithm 1), we normalize the signal volume (line 5). We bin the FFT results into a smaller, fixed number of bins (line 6). This allows us to compare signals sampled at different rates. Humans perceive many stimuli, including sound volume, on a logarithmic scale. For this reason, we also convert the power spectrum to a logarithmic scale (line 7). Finally, we clip off very low and very high frequency bands that are likely to contain artifacts, i.e., perceptual errors introduced by the recording device (line 8). For regularization we used a combination of L2 and dropout.
Algorithm 1:
1. Open datafile 2. Iterate through each window 3. IF (High Sampling Rate) 4. Take magnitude of FFT 5. Normalize “volume” 6. Re-bin to lower resolution 7. Convert to log scale 8. Clip any non-significant frequencies 9. ELSE (Low Sampling Rate) 10. normalize power 11. extract first order features 12. Input spectrum/features into function fitting neural network 13. Pretrain the neural network layer-wise with unlabeled data using Restricted Boltzmann Machines 14. Train the neural network with labeled data using backpropagation If the sampling rate is so low that the FFT cannot be reliably taken over a significant band of frequencies, instead of taking the FFT, first order features are taken directly from the (normalized) signal power instead (lines 9, 10, 11 in Algorithm 1). First order features are any features that can be computed entirely from the histogram, as shown in Figure 1, such as mean, median, standard deviation, kurtosis, skew, peak value, heterogeneity, crest factor, shape factor, or impulse factor. Raw (normalized) histogram bin counts can be used as features as well. This type of statistic is invariant to low or irregular sampling rate, because it doesn’t rely on the relationships between adjacent samples. The disadvantage is that these features are weak individually, and provide less information than spectral features. However collectively, they can still be very useful. Figure 2 shows that for our problem, they can almost be separated by a simple application of PCA, although it was not enough for accurate classification/estimation purposes. In this case, we used Bayesian regularization [21]. In either case, we use all available data to do a layer-wise pre-training step [18], using Restricted Boltzmann Machines (RBMs) [18] to reduce the effective dimensionality of the data in preparation for standard neural network training (line 13), before using standard backpropagation with regularization to fit the labeled data(line 14).
6. Datasets We collected vibrational and magnetic emission data from a series of 5 horsepower vacuum pumps (Novatec VPDB model Positive Displacement Bolted) operating at different levels and with different oil qualities, at around 3 samples per second. The specifications of the vibration and magnetic field sensors are as follows: vibration sensor: Type MEMS - 3 axis +8 to -8G, magnetic field sensor: Type MEMS - 3 axis: 0-4000 micro-Tesla. The pumps and the sensors were provided by our industrial partner ProphecySensorLytics who also helped in the collection of the data. Using this very sparse data we were able to measure vacuum level and oil quality after calibrating at each pump. We also collected audio data from the same pumps at different data rates at or above 8kHz and were able to remotely measure vacuum level and oil quality on different pumps with no calibration. Figure 3 shows an Audio power spectrum from 0 to 8 kHz.
Figure 1: First order features
Figure 2: Vacuum Levels(6,7,8,9,10,11,12”Hg) plotted by PCA, using 1st order features only
Sample Power Spectrum
Figure 3: Audio power spectrum from 0 to 8KHz
7. Results
We estimated vacuum level based on ~5Hz sampling from magnetic and vibration sensors with a window size of 10 seconds. Our network (comprising two hidden layers with forty neurons each with Bayesian regularization) was able to determine the vacuum level accurately with a Pearson product-moment correlation coefficient of .95, as shown in Figure 4. Within a few 10 second intervals, it is possible to predict the vacuum level accurately. The vibrational and magnetic sensors we used had a sampling rate of only around 3-5Hz, so spectral features were entirely unavailable and we fell back to 1st order statistics exclusively. A similar experiment was carried out for a single pump with audio data and a shorter 1 second window. Audio samples were taken at 4 vacuum levels of 6, 8, 10, and 12 “Hg (inches of Mercury), and for binary good vs bad oil quality. Our network was able to determine vacuum level into the correct category with an accuracy of 96.5% (closest to the correct category) and detect oil quality with an accuracy of 84.3%, based on only external audio data obtained from variable microphone positions. The results are shown in Figures 5 and 6 respectively. For sound waves where spectral data was available, we used only the raw Fourier spectrum, and no hand crafted or statistical features. We also experimented on estimating the vacuum level of a blower (Novatec SVP-10: Silenced Vacuum Pump; 10 hp) from its sound emission to check for abusive operation. The training data consists of forty samples of three seconds each. The vacuum levels in the ground truth were 0, -5, or -10. The test data (collected 4 days later) consisted of fifteen samples of three seconds each. Our network (using 4 hidden layers with 400 neurons per layer) was able to determine the vacuum levels for the test data with an accuracy of 97.45%. We expanded our experiment to cover multiple different pumps, pump locations, and relative sensor locations, and cover 8 vacuum levels ranging from 2 to 12.5”Hg. A sample sensor positioning for a 5 HP vacuum pump is provided in Figure 7. Two sensors n2 and n3 can be seen. Vacuum level on a pump, location, and sensor that had not been trained with was still predicted 85.2% accurately, and nearly all the misses were only one category removed, 10 instead of 11, or 12 instead of 12.5. In this run, we experimented with classification instead of function-approximation; the results are shown in Figure 8. For this run, we used a neural network containing one hidden layer comprising two thousand neurons and used L2 regularization and dropout. Rather than only estimate binary error conditions, we also estimated machine parameters with a function approximator network. We also showed that it is possible to draw generalized inference from a small number of training samples in this case using regularization. Our industrial partner ProphecySensorlytics deployed an Android mobile app called OnSpot based on SpotCheck in their factory test environment for real time machinery diagnostics. The app used the soundrecorder of the mobile device to collect sound data. A screenshot of the interface of the app is shown in Figure 9.
Figure 4: Target vs. prediction at different vacuum levels, estimated from magnetic plus vibration at ~3Hz
Figure 5: Target vs. prediction at different vacuum levels, estimated from audio data at >=8KHz
Figure 6: Target vs. prediction for oil quality, estimated from audio data at >=8KHz
Figure 7: A sample configuration showing placement of sensors for a 5 HP vacuum pump
Figure 8: Target vs. prediction at different vacuum levels based on audio, with different pumps, locations, and sensors
Figure 9: Screenshot of the OnSpot Mobile App
7.1 Comparison with Artificial Neural Network-based Approach of [1] We implemented the Artificial Neural Network (ANN)-based approach of [1] for diagnosis based on acoustic signals. We tested this implementation in estimating the vacuum level and oil quality from audio data collected from Novatec VPDB model 5HP vacuum pumps (as described in Section 6). We also tested it to estimate the vacuum level of the blower from the sound emission data collected from a Novatec SVP-10 (10 hp) pump (as described in Section 7 above). For vacuum level estimation from audio data collected from Novatec VPDB model 5HP vacuum pumps, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 93%. As mentioned above (see Section 7), for the same dataset,
our SpotCheck algorithm estimated vacuum levels with an accuracy of 96.5%. For oil quality estimation from audio data collected from Novatec VPDB model 5HP vacuum pumps, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 77.8%. As mentioned above (see Section 7), for the same dataset, our SpotCheck algorithm estimated oil quality with an accuracy of 84.3%. For vacuum level estimation for the blower for sound emission data collected from Novatec SVP-10 (10 hp) pump, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 69.4%. As mentioned above (see Section 7), for the same dataset, our SpotCheck algorithm estimated vacuum level for the blower with an accuracy of 97.45%. Thus, in all the three cases, the SpotCheck algorithm outperformed the Artificial Neural Network-based approach of [1].
Estimated Variable
Accuracy using ANN-based approach of [1]
Accuracy using SpotCheck
93%
96.5%
Oil quality
77.8%
84.3%
Vacuum level
69.4%
97.45%
Data
Novatec VPDB Model, 5HP Sound 8kHz Novatec VPDB Model 5HP Sound 8kHz Novatec SVP-10 10 HP Blower Sound 44.1kHz
Vacuum level
Figure 10: Comparison of SpotCheck with ANN-based approach of [1]
7.2 Comparison with SVM-based approach of [11] To compare the performance of SpotCheck with SVM-based approach, we implemented SVM using the library libsvm [26]. We used SVM to estimate vacuum level and oil quality from sound data acquired from Novatec VPDB model 5HP vacuum pump sampled at 8kHz and vacuum level from blower sound from Novatec SVP-10 (10 HP) pump sampled at 44.1 kHz. Following [11], we ran each experiment with three settings for the SVM kernel: linear, polynomial, and Radial Basis Function (RBF). The results (accuracy obtained in each case) are summarized in Figure 11. It can be seen that in every case SpotCheck outperforms SVM. The accuracies obtained using SVM in each case is 50% or less. It therefore seems that SVM is not suited for such a high-dimensional space.
Estimated Accuracy Accuracy Accuracy Accuracy Variable with with with with linear polynomial RBF SpotCheck kernel kernel kernel SVM SVM SVM
Data
Novatec VPDB Model 5HP Sound 8kHz Novatec VPDB Model 5HP Sound 8kHz Novatec SVP-10 10 HP Blower Sound 44.1kHz
Vacuum Level
15.99%
22.17%
22.17%
96.5%
Oil Quality
81.25%
50%
50%
84.3%
Vacuum Level
72.74%
19.77%
19.77%
97.45%
Figure 11: Comparison of SpotCheck with SVM-based Approach [11]
7.3 GPU-based Implementation of the SpotCheck Algorithm to Exploit Data Parallelism We used an NVIDIA Titan X with 64GB of RAM to provide hardware acceleration for the SpotCheck algorithm for exploiting data parallelism. Figure 12 compares the training and testing times for the SpotCheck implementation without GPU hardware acceleration and that with GPU hardware acceleration. GPU hardware provided a significant decrease in training time, particularly when dealing with the higher bandwidth 44.1 kHz sound spectra. For lower bandwidth sound data (8 kHz), the training time is sped up by about a factor of 2 while for higher bandwidth sound data, the training time is sped up by about a factor of 4. Hardware acceleration using GPUs exploits parallelism in the layer-wise unsupervised pretraining and the backpropagation algorithm to speed up training. Testing is real time for both with and without GPU-based hardware acceleration. During testing, however, the sequential nature of the feedforward neural network does not allow any speedup.
Data
Novatec VPDB model 5HP Sound 8kHz Novatec VPDB model 5HP Sound 8kHz Novatec SVP10 10 HP Blower Sound 44.1kHz
Estimated Training time variable with CPU (no acceleration) Vacuum 1518.78s Level
Training time Testing time with GPU with CPU (acceleration) (no acceleration) 616.68s 0.078s
Testing time Training with GPU Speedup (acceleration) 0.183s
~ 2.46x
Oil quality
387.30s
175.45s
0.047s
0.042s
~ 2.21x
Vacuum Level
649.34s
147.81s
0.231s
0.438s
~ 4.41x
Figure 12: Speedup Results from GPU implementation of SpotCheck
8. Conclusions We presented SpotCheck, a system for performing automated non-invasive diagnostics on manufacturing machinery based on the emitted sound, magnetic field, and vibrations data both for fault detection and to meter the day to day mode of operation of machinery. Spotcheck can generalize effectively from relatively small amounts of training data through use of modern regularization techniques. 8. Future Work We intend to expand our system to use large unlabeled datasets available in some industrial settings to improve data abstraction and take full advantage of pre-training, as well as apply it to more signal types, and estimate other quantities such as belt tension and oil level. 9. Acknowledgements We thank the engineers of our industrial partner ProphecySensorLytics for their help in collecting data References [1] Mohammad Reza Asadi Asad Abad, Ali Mohammad Borghei, Hojat Ahmadi, Saeid Minaei, Babak Beheshti, and Saeid Farokhzad. Fault diagnosis of a differential using acoustic condition monitoring and artificial neural network technique. Journal of Current Research in Science, 1(6):575, 2013. [2] Mr SN Dandare and SV Dudul. Novel technique for multiple fault detection in an automobile engine using sound signal. International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE), 1(6):pp–122, 2012. [3] Saeid Farokhzad, Hojjat Ahmadi, Ali Jaefari, MR Asadi Asad Abad, and M Ranjbar Kohan. Artificial neural network based classification of faults in centrifugal water pump. Vibroengineering, 14(4):1734–1744, 2012. [4] Ghasem Maghsoudi Gharehbolagh, Saeid Farokhzad, and Mohammad Reza Asadi Asad. Fault diagnosis of crown wheel and pinion in differential using acoustic signals with discrete wavelet transform and neural network. 2013. [5] Paul Goldman and A Muszynska. Application of full spectrum to rotating machinery diagnostics. Orbit, 20(1):17–21, 1999. [6] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012. [7] S. S. Refaat, H. Abu-Rub, M. S. Saad, E. M. Aboul-Zahab, and A. Iqbal. Ann-based for detection, diagnosis the bearing fault for three phase induction motors using current signal. In Industrial Technology (ICIT), 2013 IEEE International Conference on, pages 253–258, Feb 2013. [8] Hosseini Sadegh, Ahmadi Najafabadi Mehdi, and Akhlaghi Mehdi. Classification of acoustic emission signals generated from journal bearing at different lubrication conditions based on wavelet analysis in combination with artificial neural network and genetic algorithm. Tribology International, 95:426–434, 2016.
[9] Sadok Sassi, Bechir Badri, and Marc Thomas. A numerical model to predict damaged bearing vibrations. Journal of Vibration and Control, 13(11):1603–1628, 2007. [10] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. [11] Commander Sunil Tyagi. A comparative study of svm classifiers and artificial neural networks application for rolling element bearing fault diagnosis using wavelet transform preprocessing. [12] Jian-Da Wu and Chiu-Hong Liu. Investigation of engine fault diagnosis using discrete wavelet transform and neural network. Expert Systems with Applications, 35(3):1200 – 1213, 2008. [13] Jian-Da Wu and Chiu-Hong Liu. An expert system for fault diagnosis in internal combustion engines using wavelet packet transform and neural network. Expert Systems with Applications, 36(3, Part 1):4278 – 4286, 2009. [14] Sandeep Kumar Yadav and Prem Kumar Kalra. Automatic fault diagnosis of internal combustion engine based on spectrogram and artificial neural network. In Proceedings of the 10th WSEAS Int. conference on robotics, control and manufacturing technology. Citeseer, 2010. [15] Stuart J. Russell, Peter Norvig: Artificial Intelligence - A Modern Approach (3. internat. ed.). Pearson Education 2010, ISBN 978-0-13-207148-2, pp. I-XVIII, 1-1132 [16]
Sebastian Thrun: Toward robotic cars. Commun. ACM 53(4): 99-106 (2010)
[17] Vinay K. Chaudhri, Adam Cheyer, Richard Guilii, William Jarrold, Karen Myers, John Niekarsz: A Case Study in Engineering a Knowledge Base for an Intelligent Personal Assistant. SemDesk 2006 [18] Geoffrey E. Hinton, Simon Osindero, Yee Whye Teh: A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18(7): 1527-1554 (2006) [19] Bishop, Christopher M. Pattern recognition and machine learning (Corr. printing. ed.). New York: Springer. ISBN 978-0387310732, 2007. [20] Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov: Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(1): 1929-1958 (2014) [21] Burden F1, Winkler D.: Bayesian regularization of neural networks. Methods Mol Biol.; 458:2544, 2008. [22] G. Hinton, and R. Salakhutdinov . Reducing the dimensionality of data with neural networks. Science 313 (5786): 504-507 (July 2006) [23] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. The Journal of Machine Learning Research, 11(Dec):3371−3408, 2010. [24]
Taylor, J. I. The Vibration Analysis Handbook. Vibration Consultants, Tampa, FL, 1994.
[25] Corinna Cortes, Vladimir Vapnik: Support-Vector Networks. Machine Learning 20(3): 273-297 (1995) [26]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Highlights: This paper presents SpotCheck, a system based on well-regularized neural networks to analyze sound,
vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machinery.
PII: DOI: Reference:
S0167-9260(17)30182-7 http://dx.doi.org/10.1016/j.vlsi.2017.03.012 VLSI1324
To appear in: Integration, the VLSI Journal Received date: 23 April 2016 Revised date: 21 March 2017 Accepted date: 21 March 2017 Cite this article as: Robert DiBiano and Supratik Mukhopadhyay, Automated Diagnostics for Manufacturing Machinery Based on Well-Regularized Deep Neural Networks, Integration, the VLSI Journal, http://dx.doi.org/10.1016/j.vlsi.2017.03.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Automated Diagnostics for Manufacturing Machinery Based on Well-Regularized Deep Neural Networks Robert DiBiano AutoPredictiveCoding LLC Email: [email protected] Supratik Mukhopadhyay Louisiana State University Email: [email protected]
Corresponding Author: Supratik Mukhopadhyay Louisiana State University 164 B Coates Hall Baton Rouge, LA 70803 Email: [email protected] Phone: 225-578-1496 Fax: 225-578-1465 Abstract: This paper presents a bigdata framework based on regularized deep neural networks for automated diagnostics for manufacturing machinery based on emitted sound, vibration, and magnetic field data. More precisely, we present SpotCheck, a prototype system that uses well-regularized deep neural networks to analyze sound, vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machinery. It is completely automatic requiring no manual extraction of handcrafted features. It can operate with relatively small amounts of training data, but can take advantage of large volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data to improve performance as more data becomes available after deployment.
Keywords: Modeling and analytics for big data; Smart intelligent technologies to enable automated systems for M2M (machine to machine) communication
1. Introduction Regular monitoring and maintenance of machinery in a manufacturing plant are absolutely necessary for efficient functioning and to prevent disasters. Malfunctioning machinery can not only stand in the way of efficient operations, causing shutdown of essential processes and requiring expensive repairs resulting in millions of dollars in losses for a manufacturing plant, but can also result in potentially hazardous situations. Constant monitoring is needed to ensure that safety and manufacturing protocols are being adhered to. Violation of such protocols can lead to defective products and/or hazardous environments on the factory floor. For example, in the plastic manufacturing industry, the product needs to be dried at an appropriate vacuum level (pressure); excess or reduced vacuum level due to dryer malfunction will result in a brittle product. Similarly, in the drug manufacturing industry, tablets need to be coated at an appropriate vacuum level; otherwise the resulting product can be too soft due to excess moisture or can crumble due to too little moisture. In a (nuclear) power plant, malfunction of water pumps can lead to violent explosions. Thus reactive maintenance, i.e., repairs in the aftermath of a malfunction can be extremely expensive to a (manufacturing) plant. Proactive maintenance practices include regular manual checkups to ensure that the vital parts of a machine are working properly; in other words to ensure that the vacuum level, belt tension, oil state, oil level, bearing faults, etc. have the correct values in accordance with specifications of the manufacturing process. However, such checkups are usually invasive resulting in machine downtime causing financial loss to a plant. Besides, such manual checkups seldom predict far into the future. For efficient operation of a manufacturing plant, monitoring needs to be non-invasive and maintenance needs to be predictive. Recent advances in Artificial Intelligence (AI) [15] have provided us the opportunity to (remotely) noninvasively and inexpensively monitor and predictively maintain machinery in manufacturing plants. It may now be possible to completely automatically perform checkups of manufacturing machinery 24/7 without any machine downtime and automatically schedule maintenance predictively. In the last few years, significant advances in AI have greatly improved old applications such as machine translation, voice recognition, etc., and have created a host of new applications such as self-driving cars, personal digital assistants, etc. [16, 17]. In the heart of such developments lie neural networks, an AI technique loosely emulating the human brain. However, traditional neural networks need large volumes of training data [18], and this has limited their use for industrial applications where representative, labeled data might be difficult or costly to acquire. For example, to acquire failure data for a manufacturing machine, it has to be run on failure mode, a dangerous operation. But improved methods of regularization [19] allow deeper neural networks to potentially be trained with smaller amounts of labeled data, bringing industrial applications within reach. We present SpotCheck, a prototype system that uses deep, well-regularized neural networks to analyze sound, vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machine. It is completely automatic requiring no manual extraction of handcrafted features. It can operate with relatively smaller collection of labeled training data, but can potentially take advantage of larger volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data to improve performance as more data becomes available after deployment. 1.1 Novelty of SpotCheck We are moving towards an environment where sensors are collecting data from manufacturing machinery 24/7, and sending it to a bigdata system capable of assimilating it into a sophisticated learning algorithm. The SpotCheck system is a first step in that direction for industrial applications. Unlike existing approaches, SpotCheck provides a new, general method that uses only raw spectral features where physically possible, and falls back to a general purpose ensemble of statistical features extracted
automatically when low or irregular sampling rate makes spectral analysis impossible. This enables SpotCheck to work equally well for many different types of signals without any expert knowledge. SpotCheck can robustly generalize between different machines, sensors, and fault instances. It can operate with relatively lesser collection of labeled training data, but can potentially take advantage of larger volumes of unlabeled data when available, and scale to very large volumes of labeled or unlabeled data. We demonstrate below that SpotCheck outperforms existing approaches in several diagnosis/estimation problems for manufacturing machinery. While deep neural networks are not new, using them effectively for this type of industrial, raw labeled-data-sparse application has not been thoroughly studied yet and is novel.
2. Related Work In the last few years, significant advances have been made in the area of neural networks, old applications have been greatly improved, and a host of new applications have been developed [16, 17]. And deep neural networks still have yet to reach their full potential. Convolutional Neural Networks (CNNs) such as VGG-16 (Visual Geometry Group - 16) [10] and AlexNet [6] have been used for large scale image recognition tasks with great success. Similar classification tasks using other datatypes such as sound, light, radio, temperature, or electrical signals are less developed but also of great interest [8]. There are two standard approaches to machine signal analysis: statistical analysis of the raw signal, or analysis of the frequency spectrum. Sassi et al. [9] geometrically modeled common faults in ball bearings, and provided theoretical justification for using both spectral and first order statistical features to detect and analyze them. Most of the same principles apply to all machinery with rotating parts [8]. Although the type of signal measured is most commonly vibration [7], any time varying signal can be used. Wu and Liu [12] used spectral wavelet features, specifically Discrete Wavelet Transform (DWT) with a neural network to diagnose faults in internal combustion engines. Later in [13], the authors used another spectral technique, the Wavelet Packet Transform (WPT) for the same problem. The principle was sound, but their neural networks did not use modern regularization, and they used a handcrafted feature of Shannon entropy, and handpicked a subset of these features that were considered significant to the problem. It can detect four binary fault conditions with high accuracy, but not estimate engine parameters. It is also unclear whether it can detect the same fault conditions as accurately on different engines of the same type, since only one example of each engine, sensor, and fault was used for both training and testing. Tyagi [11] experimented with using neural nets and Support Vector Machines (SVMs) for rolling element bearing fault diagnosis based on vibrational data. He noted that the energy for some types of defects is low, and spread out over a wide range of frequencies. These types of responses are easily masked, and difficult to manually detect, but lend themselves well to neural networks that can interpret raw, high dimensional inputs. Tyagi used hand crafted statistical and spectral features, and in the absence of good regularization methods experimented with using SVMs instead of neural networks. His system was trained with several common defect types, and can predict binary bearing quality as normal or abnormal. Farokhzad et al. [3] use neural networks to detect fault conditions in water pumps. They used statistical and spectral features of vibrational signals. They detected three binary fault conditions, but were unable to test on more than one pump. Yadav et al. [14] use neural networks and handcrafted features extracted from the spectrogram for condition analysis on internal combustion engines. Their system makes a binary classification for 6 types of faults. It is an expert system, using handcrafted features and knowledge specific to this problem. Abad et al. [1] used neural networks with spectral analysis to detect five types of faults in rotating machinery via sound waves. As opposed to the works presented above that extract
handcrafted features manually from the data, SpotCheck automatically extracts features from any type of data (magnetic field emissions, sound, vibrations, etc.); SpotCheck can use essentially any time-varying signal characterizing machine behavior. Besides, most of the works above are concerned with binary classification whereas SpotCheck can approximate functions. Dandare and Dudul [2] detected faults in automobile engines based on emitted sound. They used spectral features, and showed that SVM gave much better performance than neural networks, presumably due to the good regularization built into SVMs, sparse training data, and a lack of modern regularization on their neural networks. This is supported by the fact that Tyagi [11] noted a much smaller improvement by SVMs over neural networks, even after his neural networks showed clear evidence of overfitting (0% training error with 23% testing error), suggesting they were not effectively regularized. Gharehbolagh et al. [4] recognize wheel and pinion faults using neural networks and handcrafted statistical and spectral analysis of sound. Some researchers have experimented with other signal types or attempted to estimate non binary parameters in a limited way. Refaat et al. [7] developed a fault detection system analyzing the electrical current signal through induction motors. They use handcrafted features and expert knowledge, and they can predict non-binary types of faults. Sadegh et al. [8] use spectral features, genetic algorithms, and neural networks - analyzing acoustic signals to estimate lubrication conditions in bearings. Unlike most previous methods, they do not limit themselves to binary fault conditions, but classify between three “lubrication regimes”, with 96% accuracy. Their system required heavy frequency band selection, and feature selection, for which they used genetic algorithms. However, they still used a master list of handcrafted wavelet features which must be determined manually, and handcrafted statistical features are used in all cases as well. They did not use different engines for training and testing purposes; thus they are likely to overfit to a few specific engines. The methods mentioned above have generally concentrated on diagnosis of existing faults, rather than machine diagnostics that could be used to analyze performance and predict future faults. Most are problem specific rather than general purpose, and all use some form of handcrafted features. Almost all use spectral features, usually combined with 1st order statistical features. They presumably use the same engine for training and testing sets, which fails to clarify whether the method is overfitting for that specific engine or finding useful generalizations that can be applied everywhere. The reported experiments are generally limited to binary classification of a few error conditions. There is no intra machine calibration, and expert knowledge is often used. In contrast, SpotCheck provides a new, general method that uses only raw spectral features where physically possible, and falls back to a general purpose ensemble of statistical features extracted automatically when low or irregular sampling rate makes spectral analysis impossible. In the latter case, we only use 1st order features, i.e., features that can be calculated directly from the histogram, which are therefore sampling rate and frequency independent. Such a system should work equally well for many different types of signals without any expert knowledge, and has been tested successfully on sound, magnetic field, and vibrations. We use modern deep neural networks with sophisticated regularization techniques to compensate for high input dimensionality of the dataset and limited training data, and are able to estimate even non binary parameters with a high degree of accuracy. Unlike most previous systems, SpotCheck was tested on multiple sensor, machine, and sensor-position combinations to verify that it robustly generalizes between different machines, sensors, and fault instances. There is no single standard method in use for industrial machine diagnostics, but generally they involve observing statistical or spectral data and trying to find patterns manually. These methods may require
invasive machinery, or carefully calibrated rangefinders to measure vibration without touching moving parts [5]. They generally will not operate on anything as chaotic and indirect as nearby sound or electromagnetic waves.
3. Bigdata in Machinery Diagnostics: Velocity, Volume, Variety Sound, magnetic field, and vibration emissions from manufacturing machinery running 24/7 generate a tremendous volume, velocity, and variety of unlabeled data. Sound data collected from a blower sampled at 44.1KHz at 16 bits per sample amounts to a megabyte every six seconds (velocity) represented as wav files (providing lossless compression); i.e., every 24 hours approximately 1.44 GB of data per sensor gets collected. Vibration and magnetic emissions from a vacuum pump collected at 5Hz amount to 40 bytes per second (with a floating point representation); this amounts to 1.7 MB per day. In a typical small factory with 20 blowers, one can easily expect around 28 GB of unlabeled sound data per day (volume). Labeled data, however, is costly to acquire; one needs to run machines in failure mode, a dangerous operation. Thus, a bigdata framework for manufacturing machinery diagnostics needs to employ techniques that can exploit the easily available unlabeled data rather than depending solely on labeled data. Techniques like Support Vector Machines [25] are unable to leverage large amounts of unlabeled data extracted from industrial machinery and depend on being trained with labeled data for classification. If large volumes of labeled training data are unavailable, machine learning techniques employed by bigdata frameworks to provide predictive analytics need to prevent overfitting. In addition, bigdata frameworks processing data from industrial machinery need to handle the variety in the type of data collected (e.g., sound vs. magnetic/vibration emissions) together with different representations (wav files vs. floating point). Besides, the processing of this data needs to be in real time as opposed to offline for predictive diagnosis. Hence, bigdata frameworks processing data emitted by manufacturing machinery need to leverage graphic processing units (GPUs) to exploit data parallelism.
4. Theoretical Background As we have noted, analyzing industrial machinery based on the spectral characteristics of the noise has worked in both research and real world [5] applications in the past. Obviously if there weren’t any spectral differences between different machine states, the method couldn’t work; but it does seem to hold for a wide variety of practical problems. The exact theoretical reasons why this works vary by the geometry of the device being analyzed, but [9] gives us good insight into the physics of rotating machinery, and is a good representative example. A properly functioning bearing has specific frequencies associated with it, and common fault types have their own characteristic frequencies. In [24], fundamental train frequency and ball spin frequency are defined as: [
( (
) ) (
(
)] )
(1) (2)
where ωi and ωo are rotation speed of the inner and outer races respectively, Bd denotes the ball diameter, Pd denotes the pitch diameter, and α denotes the contact angle. Characteristic frequencies associated with faults, ball pass frequency on outer and inner defects are defined as:
(
) (
)
(3)
(
) (
)
(4)
where Nb is number of rolling elements. So by looking for the correct frequencies in equations 1 and 2 we can detect proper function, and by looking for the frequencies in equations 3 and 4 we can identify specific defects. In practice, we would like a system that will discover these various characteristic frequencies automatically via deep learning using well-regularized neural networks. The two most important aspects of modern neural networks are regularization and data abstraction. Regularization allows a neural network, trained with the backpropagation algorithm, to interpolate between different training samples and draw generalizations to classify similar samples. Data abstraction allows a network to break data into progressively simpler sub-classes, allowing it to extrapolate learning to samples that do not closely resemble anything it was trained with. For a case where there is a high dimensional-feature space and limited training data, good regularization is key in providing good generalizations. The most common regularization method is to modify the cost function the neural network seeks to minimize to have a penalty for irregular decision boundaries or other phenomenon that increase overfitting. L2 regularization [19], probably the most common type, adds a penalty for high neuron weights, incentivizing sparser, simpler configurations. This is accomplished by adding the L2 norm of the weights multiplied by a parameter λ to the cost function. The parameter λ determines the rate at which the weights “decay” during training using backpropagation [19]. We also used dropout [20], a model averaging method that performs weight regularization automatically. In cases where more training data is available, data abstraction can be applied to find complex patterns in training data, as well as utilize large quantities of unlabeled data to improve performance. In the case of Deep Belief Networks (DBN), stacked autoencoders [23] or Restricted Boltzmann machines (RBMs) [22] are used during a pretraining step to effectively reduce input dimensionality, and deal with the vanishing gradient problem that historically prevented deep networks from efficiently training.
5. The SpotCheck System We now describe our method in detail. The algorithm used for SpotCheck is detailed below in Algorithm 1. Our signal classification algorithm windows a short period of the signal (where the signal is emitted sound, magnetic field, or vibration data). After calculating the magnitude of the power spectrum via FFT (Fast Fourier Transform) (line 4 in Algorithm 1), we normalize the signal volume (line 5). We bin the FFT results into a smaller, fixed number of bins (line 6). This allows us to compare signals sampled at different rates. Humans perceive many stimuli, including sound volume, on a logarithmic scale. For this reason, we also convert the power spectrum to a logarithmic scale (line 7). Finally, we clip off very low and very high frequency bands that are likely to contain artifacts, i.e., perceptual errors introduced by the recording device (line 8). For regularization we used a combination of L2 and dropout.
Algorithm 1:
1. Open datafile 2. Iterate through each window 3. IF (High Sampling Rate) 4. Take magnitude of FFT 5. Normalize “volume” 6. Re-bin to lower resolution 7. Convert to log scale 8. Clip any non-significant frequencies 9. ELSE (Low Sampling Rate) 10. normalize power 11. extract first order features 12. Input spectrum/features into function fitting neural network 13. Pretrain the neural network layer-wise with unlabeled data using Restricted Boltzmann Machines 14. Train the neural network with labeled data using backpropagation If the sampling rate is so low that the FFT cannot be reliably taken over a significant band of frequencies, instead of taking the FFT, first order features are taken directly from the (normalized) signal power instead (lines 9, 10, 11 in Algorithm 1). First order features are any features that can be computed entirely from the histogram, as shown in Figure 1, such as mean, median, standard deviation, kurtosis, skew, peak value, heterogeneity, crest factor, shape factor, or impulse factor. Raw (normalized) histogram bin counts can be used as features as well. This type of statistic is invariant to low or irregular sampling rate, because it doesn’t rely on the relationships between adjacent samples. The disadvantage is that these features are weak individually, and provide less information than spectral features. However collectively, they can still be very useful. Figure 2 shows that for our problem, they can almost be separated by a simple application of PCA, although it was not enough for accurate classification/estimation purposes. In this case, we used Bayesian regularization [21]. In either case, we use all available data to do a layer-wise pre-training step [18], using Restricted Boltzmann Machines (RBMs) [18] to reduce the effective dimensionality of the data in preparation for standard neural network training (line 13), before using standard backpropagation with regularization to fit the labeled data(line 14).
6. Datasets We collected vibrational and magnetic emission data from a series of 5 horsepower vacuum pumps (Novatec VPDB model Positive Displacement Bolted) operating at different levels and with different oil qualities, at around 3 samples per second. The specifications of the vibration and magnetic field sensors are as follows: vibration sensor: Type MEMS - 3 axis +8 to -8G, magnetic field sensor: Type MEMS - 3 axis: 0-4000 micro-Tesla. The pumps and the sensors were provided by our industrial partner ProphecySensorLytics who also helped in the collection of the data. Using this very sparse data we were able to measure vacuum level and oil quality after calibrating at each pump. We also collected audio data from the same pumps at different data rates at or above 8kHz and were able to remotely measure vacuum level and oil quality on different pumps with no calibration. Figure 3 shows an Audio power spectrum from 0 to 8 kHz.
Figure 1: First order features
Figure 2: Vacuum Levels(6,7,8,9,10,11,12”Hg) plotted by PCA, using 1st order features only
Sample Power Spectrum
Figure 3: Audio power spectrum from 0 to 8KHz
7. Results
We estimated vacuum level based on ~5Hz sampling from magnetic and vibration sensors with a window size of 10 seconds. Our network (comprising two hidden layers with forty neurons each with Bayesian regularization) was able to determine the vacuum level accurately with a Pearson product-moment correlation coefficient of .95, as shown in Figure 4. Within a few 10 second intervals, it is possible to predict the vacuum level accurately. The vibrational and magnetic sensors we used had a sampling rate of only around 3-5Hz, so spectral features were entirely unavailable and we fell back to 1st order statistics exclusively. A similar experiment was carried out for a single pump with audio data and a shorter 1 second window. Audio samples were taken at 4 vacuum levels of 6, 8, 10, and 12 “Hg (inches of Mercury), and for binary good vs bad oil quality. Our network was able to determine vacuum level into the correct category with an accuracy of 96.5% (closest to the correct category) and detect oil quality with an accuracy of 84.3%, based on only external audio data obtained from variable microphone positions. The results are shown in Figures 5 and 6 respectively. For sound waves where spectral data was available, we used only the raw Fourier spectrum, and no hand crafted or statistical features. We also experimented on estimating the vacuum level of a blower (Novatec SVP-10: Silenced Vacuum Pump; 10 hp) from its sound emission to check for abusive operation. The training data consists of forty samples of three seconds each. The vacuum levels in the ground truth were 0, -5, or -10. The test data (collected 4 days later) consisted of fifteen samples of three seconds each. Our network (using 4 hidden layers with 400 neurons per layer) was able to determine the vacuum levels for the test data with an accuracy of 97.45%. We expanded our experiment to cover multiple different pumps, pump locations, and relative sensor locations, and cover 8 vacuum levels ranging from 2 to 12.5”Hg. A sample sensor positioning for a 5 HP vacuum pump is provided in Figure 7. Two sensors n2 and n3 can be seen. Vacuum level on a pump, location, and sensor that had not been trained with was still predicted 85.2% accurately, and nearly all the misses were only one category removed, 10 instead of 11, or 12 instead of 12.5. In this run, we experimented with classification instead of function-approximation; the results are shown in Figure 8. For this run, we used a neural network containing one hidden layer comprising two thousand neurons and used L2 regularization and dropout. Rather than only estimate binary error conditions, we also estimated machine parameters with a function approximator network. We also showed that it is possible to draw generalized inference from a small number of training samples in this case using regularization. Our industrial partner ProphecySensorlytics deployed an Android mobile app called OnSpot based on SpotCheck in their factory test environment for real time machinery diagnostics. The app used the soundrecorder of the mobile device to collect sound data. A screenshot of the interface of the app is shown in Figure 9.
Figure 4: Target vs. prediction at different vacuum levels, estimated from magnetic plus vibration at ~3Hz
Figure 5: Target vs. prediction at different vacuum levels, estimated from audio data at >=8KHz
Figure 6: Target vs. prediction for oil quality, estimated from audio data at >=8KHz
Figure 7: A sample configuration showing placement of sensors for a 5 HP vacuum pump
Figure 8: Target vs. prediction at different vacuum levels based on audio, with different pumps, locations, and sensors
Figure 9: Screenshot of the OnSpot Mobile App
7.1 Comparison with Artificial Neural Network-based Approach of [1] We implemented the Artificial Neural Network (ANN)-based approach of [1] for diagnosis based on acoustic signals. We tested this implementation in estimating the vacuum level and oil quality from audio data collected from Novatec VPDB model 5HP vacuum pumps (as described in Section 6). We also tested it to estimate the vacuum level of the blower from the sound emission data collected from a Novatec SVP-10 (10 hp) pump (as described in Section 7 above). For vacuum level estimation from audio data collected from Novatec VPDB model 5HP vacuum pumps, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 93%. As mentioned above (see Section 7), for the same dataset,
our SpotCheck algorithm estimated vacuum levels with an accuracy of 96.5%. For oil quality estimation from audio data collected from Novatec VPDB model 5HP vacuum pumps, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 77.8%. As mentioned above (see Section 7), for the same dataset, our SpotCheck algorithm estimated oil quality with an accuracy of 84.3%. For vacuum level estimation for the blower for sound emission data collected from Novatec SVP-10 (10 hp) pump, the accuracy obtained using the Artificial Neural Network-based approach of [1] is 69.4%. As mentioned above (see Section 7), for the same dataset, our SpotCheck algorithm estimated vacuum level for the blower with an accuracy of 97.45%. Thus, in all the three cases, the SpotCheck algorithm outperformed the Artificial Neural Network-based approach of [1].
Estimated Variable
Accuracy using ANN-based approach of [1]
Accuracy using SpotCheck
93%
96.5%
Oil quality
77.8%
84.3%
Vacuum level
69.4%
97.45%
Data
Novatec VPDB Model, 5HP Sound 8kHz Novatec VPDB Model 5HP Sound 8kHz Novatec SVP-10 10 HP Blower Sound 44.1kHz
Vacuum level
Figure 10: Comparison of SpotCheck with ANN-based approach of [1]
7.2 Comparison with SVM-based approach of [11] To compare the performance of SpotCheck with SVM-based approach, we implemented SVM using the library libsvm [26]. We used SVM to estimate vacuum level and oil quality from sound data acquired from Novatec VPDB model 5HP vacuum pump sampled at 8kHz and vacuum level from blower sound from Novatec SVP-10 (10 HP) pump sampled at 44.1 kHz. Following [11], we ran each experiment with three settings for the SVM kernel: linear, polynomial, and Radial Basis Function (RBF). The results (accuracy obtained in each case) are summarized in Figure 11. It can be seen that in every case SpotCheck outperforms SVM. The accuracies obtained using SVM in each case is 50% or less. It therefore seems that SVM is not suited for such a high-dimensional space.
Estimated Accuracy Accuracy Accuracy Accuracy Variable with with with with linear polynomial RBF SpotCheck kernel kernel kernel SVM SVM SVM
Data
Novatec VPDB Model 5HP Sound 8kHz Novatec VPDB Model 5HP Sound 8kHz Novatec SVP-10 10 HP Blower Sound 44.1kHz
Vacuum Level
15.99%
22.17%
22.17%
96.5%
Oil Quality
81.25%
50%
50%
84.3%
Vacuum Level
72.74%
19.77%
19.77%
97.45%
Figure 11: Comparison of SpotCheck with SVM-based Approach [11]
7.3 GPU-based Implementation of the SpotCheck Algorithm to Exploit Data Parallelism We used an NVIDIA Titan X with 64GB of RAM to provide hardware acceleration for the SpotCheck algorithm for exploiting data parallelism. Figure 12 compares the training and testing times for the SpotCheck implementation without GPU hardware acceleration and that with GPU hardware acceleration. GPU hardware provided a significant decrease in training time, particularly when dealing with the higher bandwidth 44.1 kHz sound spectra. For lower bandwidth sound data (8 kHz), the training time is sped up by about a factor of 2 while for higher bandwidth sound data, the training time is sped up by about a factor of 4. Hardware acceleration using GPUs exploits parallelism in the layer-wise unsupervised pretraining and the backpropagation algorithm to speed up training. Testing is real time for both with and without GPU-based hardware acceleration. During testing, however, the sequential nature of the feedforward neural network does not allow any speedup.
Data
Novatec VPDB model 5HP Sound 8kHz Novatec VPDB model 5HP Sound 8kHz Novatec SVP10 10 HP Blower Sound 44.1kHz
Estimated Training time variable with CPU (no acceleration) Vacuum 1518.78s Level
Training time Testing time with GPU with CPU (acceleration) (no acceleration) 616.68s 0.078s
Testing time Training with GPU Speedup (acceleration) 0.183s
~ 2.46x
Oil quality
387.30s
175.45s
0.047s
0.042s
~ 2.21x
Vacuum Level
649.34s
147.81s
0.231s
0.438s
~ 4.41x
Figure 12: Speedup Results from GPU implementation of SpotCheck
8. Conclusions We presented SpotCheck, a system for performing automated non-invasive diagnostics on manufacturing machinery based on the emitted sound, magnetic field, and vibrations data both for fault detection and to meter the day to day mode of operation of machinery. Spotcheck can generalize effectively from relatively small amounts of training data through use of modern regularization techniques. 8. Future Work We intend to expand our system to use large unlabeled datasets available in some industrial settings to improve data abstraction and take full advantage of pre-training, as well as apply it to more signal types, and estimate other quantities such as belt tension and oil level. 9. Acknowledgements We thank the engineers of our industrial partner ProphecySensorLytics for their help in collecting data References [1] Mohammad Reza Asadi Asad Abad, Ali Mohammad Borghei, Hojat Ahmadi, Saeid Minaei, Babak Beheshti, and Saeid Farokhzad. Fault diagnosis of a differential using acoustic condition monitoring and artificial neural network technique. Journal of Current Research in Science, 1(6):575, 2013. [2] Mr SN Dandare and SV Dudul. Novel technique for multiple fault detection in an automobile engine using sound signal. International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE), 1(6):pp–122, 2012. [3] Saeid Farokhzad, Hojjat Ahmadi, Ali Jaefari, MR Asadi Asad Abad, and M Ranjbar Kohan. Artificial neural network based classification of faults in centrifugal water pump. Vibroengineering, 14(4):1734–1744, 2012. [4] Ghasem Maghsoudi Gharehbolagh, Saeid Farokhzad, and Mohammad Reza Asadi Asad. Fault diagnosis of crown wheel and pinion in differential using acoustic signals with discrete wavelet transform and neural network. 2013. [5] Paul Goldman and A Muszynska. Application of full spectrum to rotating machinery diagnostics. Orbit, 20(1):17–21, 1999. [6] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012. [7] S. S. Refaat, H. Abu-Rub, M. S. Saad, E. M. Aboul-Zahab, and A. Iqbal. Ann-based for detection, diagnosis the bearing fault for three phase induction motors using current signal. In Industrial Technology (ICIT), 2013 IEEE International Conference on, pages 253–258, Feb 2013. [8] Hosseini Sadegh, Ahmadi Najafabadi Mehdi, and Akhlaghi Mehdi. Classification of acoustic emission signals generated from journal bearing at different lubrication conditions based on wavelet analysis in combination with artificial neural network and genetic algorithm. Tribology International, 95:426–434, 2016.
[9] Sadok Sassi, Bechir Badri, and Marc Thomas. A numerical model to predict damaged bearing vibrations. Journal of Vibration and Control, 13(11):1603–1628, 2007. [10] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. [11] Commander Sunil Tyagi. A comparative study of svm classifiers and artificial neural networks application for rolling element bearing fault diagnosis using wavelet transform preprocessing. [12] Jian-Da Wu and Chiu-Hong Liu. Investigation of engine fault diagnosis using discrete wavelet transform and neural network. Expert Systems with Applications, 35(3):1200 – 1213, 2008. [13] Jian-Da Wu and Chiu-Hong Liu. An expert system for fault diagnosis in internal combustion engines using wavelet packet transform and neural network. Expert Systems with Applications, 36(3, Part 1):4278 – 4286, 2009. [14] Sandeep Kumar Yadav and Prem Kumar Kalra. Automatic fault diagnosis of internal combustion engine based on spectrogram and artificial neural network. In Proceedings of the 10th WSEAS Int. conference on robotics, control and manufacturing technology. Citeseer, 2010. [15] Stuart J. Russell, Peter Norvig: Artificial Intelligence - A Modern Approach (3. internat. ed.). Pearson Education 2010, ISBN 978-0-13-207148-2, pp. I-XVIII, 1-1132 [16]
Sebastian Thrun: Toward robotic cars. Commun. ACM 53(4): 99-106 (2010)
[17] Vinay K. Chaudhri, Adam Cheyer, Richard Guilii, William Jarrold, Karen Myers, John Niekarsz: A Case Study in Engineering a Knowledge Base for an Intelligent Personal Assistant. SemDesk 2006 [18] Geoffrey E. Hinton, Simon Osindero, Yee Whye Teh: A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18(7): 1527-1554 (2006) [19] Bishop, Christopher M. Pattern recognition and machine learning (Corr. printing. ed.). New York: Springer. ISBN 978-0387310732, 2007. [20] Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov: Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(1): 1929-1958 (2014) [21] Burden F1, Winkler D.: Bayesian regularization of neural networks. Methods Mol Biol.; 458:2544, 2008. [22] G. Hinton, and R. Salakhutdinov . Reducing the dimensionality of data with neural networks. Science 313 (5786): 504-507 (July 2006) [23] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. The Journal of Machine Learning Research, 11(Dec):3371−3408, 2010. [24]
Taylor, J. I. The Vibration Analysis Handbook. Vibration Consultants, Tampa, FL, 1994.
[25] Corinna Cortes, Vladimir Vapnik: Support-Vector Networks. Machine Learning 20(3): 273-297 (1995) [26]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Highlights: This paper presents SpotCheck, a system based on well-regularized neural networks to analyze sound,
vibrational, and magnetic emissions of industrial machinery to provide noninvasive machine diagnostics, both for fault detection and to meter the day to day mode of operation of the machinery.