Remaining useful life prediction for supercapacitor based on long short-term memory neural network

Journal of Power Sources 440 (2019) 227149

Contents lists available at ScienceDirect

Journal of Power Sources journal homepage: www.elsevier.com/locate/jpowsour

Remaining useful life prediction for supercapacitor based on long short-term memory neural network Yanting Zhou a, Yinuo Huang a, Jinbo Pang b, Kai Wang a, * a

School of Electrical Engineering, Qingdao University, Qingdao, 266000, China Collaborative Innovation Center of Technology and Equipment for Biological Diagnosis and Therapy in Universities of Shandong, Institute for Advanced Interdisciplinary Research (iAIR), Jinan University, Jinan, 250022, China

b

H I G H L I G H T S

G R A P H I C A L A B S T R A C T

� Long short-term memory neural network is employed for prediction. � Aging experiments at different temper­ ature and work voltage are conducted. � The proposed method is applied to the untrained offline data of life predication.

A R T I C L E I N F O

A B S T R A C T

Keywords: Remaining useful life Supercapacitor Long short-term memory neural network Root mean square error Overfitting

The remaining useful life prediction of supercapacitor is an important part of the supercapacitor management system. In order to improve the reliability of the entire supercapacitor bank, this paper proposes a life prediction method based on long short-term memory neural network. It is used to learn the long-term dependence of degraded capacity of supercapacitor. The Dropout algorithm is used to prevent overfitting and the neural network is optimized by the Adam algorithm. The supercapacitor data measured under different working con­ ditions is divided into training set and predictive set as the input of the neural network. The root mean square error of the predicted result is about 0.0261. At the same time, in order to verify the applicability of the algo­ rithm, it is also used for the life prediction of offline data, and the root mean square error is about 0.0338. The overall results show that long short-term memory neural network exhibits excellent performance for remaining useful life prediction of supercapacitor.

1. Introduction Supercapacitors have higher energy density than conventional electrolytic capacitors, and have higher power density than current energy storage components such as batteries. In addition, the

supercapacitor has the characteristics of high charge and discharge ef­ ficiency, large charge and discharge current, long cycle life and wide operating temperature range [1,2]. The electrochemical performance of supercapacitors is strongly related to the type, properties and surface structure of the electrode material. It should have high surface area,

* Corresponding author. E-mail address: [email protected] (K. Wang). https://doi.org/10.1016/j.jpowsour.2019.227149 Received 16 March 2019; Received in revised form 17 August 2019; Accepted 11 September 2019 Available online 13 September 2019 0378-7753/© 2019 Elsevier B.V. All rights reserved.

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

Fig. 1. (a) Architecture of the LSTM RNN method. (b) Standard recurrent neural network. (c) Recurrent neural network after applying Dropout algorithm.

machine (RVM) use support vectors or correlation vectors as data points to fit the data and predict. Dave Andre et al. [45] used the SVM method to predict the SOC and SOH of lithium batteries. The double filter composed of the standard Kalman filter and the unscented Kalman filter is coupled with the SVM. The results show that the estimation error is small. However, SVM computational complexity is high and consumes a lot of time in practical applications. RVM is based on the SVM method, which reduces the amount of computation and complexity. Li Hong et al. [46] proposed a multi-step prediction model based on average entropy and relevance vector machine, which is applied to battery health monitoring and remaining life prediction. However, the high sparsity of the RVM method leads to the instability of its output. Simple recurrent neural network (SIM RNN) is also used to predict the life of supercapacitor, but SIM RNN has the disadvantage of longterm dependent learning. If information is stored for a long time, the gradient will disappear and SIM RNN cannot continue learning [47,48]. Abdenour Souallhi et al. [49] proposed a new fuzzy neural network method to monitor the health of supercapacitor. Based on fuzzy logic and neural network, the equivalent series resistance ESR and capaci­ tance C of the supercapacitor are estimated and predicted, and the data provided by the accelerated aging test is processed in real time and time series. Long short-term memory recurrent neural network (LSTM RNN) is a deep learning network. It is used to solve the problem of long-term dependence. The memory unit is used to store information, and the forgotten gate filters redundant information, so that information can be stored for a long period of time without gradient disappearance. It is widely used in machine translation, speech recognition, natural lan­ guage processing, picture description, wave prediction, and environ­ mental prediction [50–54]. Zhang Yongzhi et al. [55] used LSTM RNN

large electrical conductivity and high chemical stability [3–12]. Therefore, the application of supercapacitors is very extensive [13–16]. Especially when supercapacitors are used as power supplies or auxiliary power systems for complex electronic systems in the form of modules, the Remaining useful life (RUL) will directly affect the reliability and safety of the entire system [17–23]. The supercapacitor RUL prediction method is divided into a model-based prediction method and a data-based prediction method. Model-based prediction methods usually combine different models and filtering methods to achieve data tracking and prediction [24–28]. Zhang Lijun et al. [29] proposed a method for predicting the remaining life of lithium-ion batteries based on exponential model and particle filter (PF), and then used extrapolation to quantify the uncertainty of life expectancy. Mohamed Ahwiadi et al. [30] proposed an enhanced mutated PF (EMPF) technology to improve PF performance. In EMPF technology, a new enhanced mutation method is used to actively explore the posterior probability density function to locate the high likelihood region. Asmae El Mejdoubi et al. [31] proposed a lithium battery pre­ diction model based on Rao-Blackwellization PF considering battery aging conditions, which can estimate the posterior value of the aging index and predict RUL. Dong Guangzhong et al. [32] used the Brownian motion (BM) degradation model and PF to achieve online short-term State of Health (SOH) estimation and long-term RUL prediction. How­ ever, due to the complex nature of supercapacitors, the model-driven approach is complex and difficult to implement [33–38]. Compared to model-based methods, data-based methods do not require complex mathematical models to simulate the internal aging mechanism of supercapacitor, relying primarily on large amounts of data [39–44]. Support vector machine (SVM) and relevance vector 2

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

for RUL prediction of lithium-ion batteries. The elastic mean squared backpropagation algorithm is used for adaptive optimization and Monte Carlo simulations are combined to generate a probabilistic RUL pre­ diction. Gated recurrent unit (GRU) is a network created on the basis of LSTM, which was proposed by Kyunghyun Cho et al., in 2014. As a variant of LSTM, GRU combines the input gate and the forgetting gate into a single update gate, and combines the memory unit and the hidden layer unit, so the structure is simpler than LSTM. Zhao Ruxiu et al. [56] studied the modeling of lithium-ion battery based on recurrent neural network. The dynamic response of the lithium battery is simulated using the RNN method consisting of GRU and deep feature selection (DFS) structure. The advantage of GRU is that it is a simpler model, so it’s easier to create a larger network. GRU is more likely to converge due to fewer parameters. However, LSTM RNN has better performance when the data set is large [57–59]. In this work, we present a prediction method of remaining useful life for supercapacitor based on long short-term memory neural network architecture optimized with both Adam and Dropout algorithm, which effectively solves the over-fitting problem. Also our approach designs the adaptive learning rate according to different parameters to improve the prediction accuracy. In brief, three network models are constructed based on long short-term memory recurrent neural network, gated recurrent unit, simple recurrent neural network to train complete supercapacitor discharge data and predict remaining useful life, and evaluate the performance of the proposed algorithm under different voltage and temperature conditions. The structure of this paper is as follows: Section 2 introduces the LSTM RNN model for supercapacitor RUL prediction. The super­ capacitor aging test platform and the aging affecting factors are pre­ sented in Section 3. Section 4 is the experimental results and error analysis of RUL prediction. Finally, the main conclusion of this paper is showed in Section 5.

cell state ct at the current time. The LSTM RNN uses an output gate to control how much of the unit state ct is output to the current output value ht of the LSTM RNN. � ft ¼ σ Wf ⋅ ½ht 1 ; xt � þ bf (2) Among them, Wf is the weight matrix of the forgetting gate, [ht-1, xt] means that the two vectors are connected into a longer vector, bf is the offset term of the forgetting gate, and σ is the sigmoid function. If the dimension entered is dx, the dimension of the hidden layer is dh, and the dimension of the cell state is dc (usually dc ¼ dh), then the weight matrix of the forgetting gate Wf dimension is dc � (dh þ dx). In fact, the weight matrix Wf is composed of two matrices: one is Wfh, which corresponds to the input ht-1, and its dimension is dc � dh; one is Wfx, which corresponds to the input xt, and its dimension is dc � dx. Wf can be written as: � � � � � ht 1 � � ht 1 � (3) Wf ¼ Wfh Wfx ¼ Wfh ht 1 þ Wfh xt xt xt The input gate is calculated as follows: it ¼ σ ðWi ⋅ ½ht 1 ; xt � þ bi Þ

In the above formula, Wi is the weight matrix of the input gate, and bi is the offset term of the input gate. Calculate the current input unit status based on the last output and this input: c’t ¼ tanhðWc ⋅½ht 1 ; xt � þ bc Þ

(5)

The unit state ct at the current time is multiplied by the last forgetting gate ft by the element state ct-1, and then multiplied by the input gate it by the element state of the current input unit c’t , and then the two products are added: (6)

ct ¼ ft ∘ct 1 þit ∘c’t

Symbol ∘ means multiplication by element. By the above formula, the LSTM RNN is combined with the current memory c’t and the long-term memory ct-1 to form a new unit state ct. The control of the forgotten gate can save information long ago and the control of the input gate can prevent the current insignificant content from entering the memory. The output gate controls the effect of longterm memory on the current output. The output gate is calculated as follows:

2. LSTM RNN for the prediction of remaining useful life of supercapacitors 2.1. LSTM RNN architecture SIM RNN cannot solve long-term dependency problems, but LSTM RNN is easier to learn long-term dependencies than SIM RNN. The hidden layer of the original RNN has only one state, h, which is very sensitive to short-term inputs. The LSTM RNN adds a state c to hold the long-term state. The newly added state c is called the cell state. The architecture of LSTM RNN is shown in Fig. 1. At time t, the LSTM RNN has three inputs: the input value xt of the current time network, the output value ht-1 at the previous time, and the unit state ct-1 at the previous time; The LSTM RNN has two outputs: the LSTM RNN output value ht at the current time, and the cell state ct at the current time. The key to LSTM RNN is to control long-term state c. This article uses three control switches for the LSTM RNN. The first switch, responsible for controlling to continue to save the long-term state c; The second switch is responsible for controlling the input of the immediate state to the long-term state c; The third switch is responsible for controlling whether the long-term state c is the output of the current LSTM RNN. The implementation of LSTM RNN in the algorithm uses the concept of a gate. The gate is actually a layer of fully connected layers whose input is a vector and the output is a real vector between 0 and 1. Assuming W is the weight vector of the gate and b is the bias term, the gate can be expressed as: gðxÞ ¼ σðWx þ bÞ

(4)

οt ¼ σ ðWo ⋅ ½ht 1 ; xt � þ bo Þ

(7)

The final output of the LSTM RNN is determined by the output gate and the unit state: (8)

ht ¼ οt ∘tanhðct Þ

The structure of the LSTM RNN is as shown in Fig. 1(a). The activation function of gate is sigmoid function, and the output activation function is tanh function. Their derivatives are:

σ ðzÞ ¼ y ¼

1 1þe

(9)

z

σ ’ðzÞ ¼ yð1

yÞ

tanhðzÞ ¼ y ¼

ez e ez þ e

tanhðzÞ ¼ 1

y2

(10) z z

(11) (12)

2.2. Dropout algorithm for preventing overfitting issue

(1)

Overfitting refers to the ability to fit the training data well, but does not fit the test data well. When in an over-fitting state, the neural network is unable to exert its great potential. To prevent gradient disappearance and gradient explosion, Dropout is used to train neural

LSTM RNN uses two gates to control the content of the unit state c, one is the forget gate, which determines how much the unit state ct-1 at the previous moment is retained to the current time ct; The other is the input gate, which determines how much of the input xt is saved to the 3

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

Fig. 2. (a) Supercapacitor aging test platform. (b) Capacity degradation trajectories of supercapacitors along with increased cycles under the temperature of 25 � C, 50 � C, 65 � C, 80 � C. (c) Capacity degradation trajectories of supercapacitors along with increased cycles under the voltage of 2.7 V, 2.9 V, 3.1 V, 3.2 V.

networks containing fully connected layers during network training. The training process of the neural network is to iteratively adjust the values of each parameter for each Mini-Batch using the error backpropagation algorithm. Dropout is to randomly select a part of the unit to set its output to 0 during each iteration adjustment. When calculating the error, the output value of all the cells was originally used, but since some of the cells were discarded, Dropout played a similar role to the ho­ mogenization method. However, when performing error back propa­ gation calculations on discarded units, the original output values before being discarded are still used. By constraining the parameters of the network, a reasonable network can still be trained, and the purpose of suppressing over-fitting is achieved. The Dropout neural network model is shown in Fig. 1(b) and (c).

3.2. Analysis of aging characteristics The application scenarios of supercapacitors are complex and vari­ able. Different working conditions have different effects on the aging speed of the capacitors. This section analyzes the effects of temperature and working voltage. In this experiment, the CC-CV charging protocol was used to charge the supercapacitor to a cut-off voltage 2.7 V with a constant current 3A, and the constant voltage charging was continued. Multiple sets of charge and discharge tests are performed at different temperatures and voltages, cycling hundreds of thousands of times. By analyzing the aging trend of the capacitance under different tempera­ ture conditions, it is found that the aging speed of the supercapacitor increases with the increase of temperature. The aging tendency of supercapacitors at different temperatures is shown in Fig. 2(b). The redox reaction rate of the surface functional group and the stability of the electrolyte are all closely related to the operating voltage. Therefore, the voltage is another factor that affects the aging rate of the super­ capacitor. The aging trend of supercapacitors at different voltages is shown in Fig. 2(c). Although supercapacitors may work at different voltages and tem­ peratures, the trend of capacity degradation is similar, that is, the state of health SOH follows a similar pattern of change. Therefore, the pre­ diction methods presented in this paper can be used to evaluate the health status and performance testing of supercapacitors in other work environments.

2.3. Optimization with Adam algorithm In 2014, Kingma and Lei Ba proposed the Adam optimizer. Combining the advantages of the two optimization algorithms AdaGrad and RMSProp, the first moment estimation of the gradient and the sec­ ond moment estimation are considered and calculate the update step size. The Adam algorithm differs from the traditional stochastic gradient descent. The stochastic gradient descent maintains a single learning rate to update all the weights, and the learning rate does not change during the training process. Adam calculates independent adaptive learning rates for different parameters by calculating the first-order moment estimation and second-order moment estimation of the gradient. The algorithm is easy to implement and has high computational efficiency and low memory requirements.

4. Experimental results and analysis For the RUL prediction of the supercapacitor, the epoch number of the LSTM RNN is set to 250, the initial value of the learning rate is set to 0.005, the decline factor of the learning rate is set to 0.8, the falling period is 125, the hidden layer contains 50 units, and the Dropout is set to 0.6. The SIM RNN, GRU and LSTM RNN models have the same hyperparameter settings, all containing the same number of hidden layer units and epoch times. After the experiment, the error analysis is performed on the training set and the prediction set of each super­ capacitor, and the RMSE, MAPE, MAE, and R2 indicators are used to measure the prediction effect of the model.

3. Supercapacitor aging tests 3.1. Test platform The supercapacitor aging test platform is shown in Fig. 2(a). LAND CT2001A battery test system, which can realize the discharge function of constant current, constant power, constant voltage and constant resistance of super capacitors; Digital display electric heating incubator, which is continuously adjustable to 80 � C; The computer can be used for data monitoring and storage. The experimental object is the same type of supercapacitor product of the same model of MAXWELL, the model is BCAP0010T01. The nominal capacitance value is 10F, the actual initial value of the capac­ itor is up and down 20, the rated working voltage is 2.7 V, the absolute maximum voltage is 2.85 V, the absolute maximum current is 7.2A, the operating temperature range is 40 � C–65 � C, and the maximum working temperature can reach 70 � C.

4.1. Evaluation standard The root mean square error RMSE represents the average of the er­ rors and therefore reflects the stability of the model. The mean absolute percent error MAPE not only considers the error between the predicted value and the true value, but also considers the proportional relationship between them. The mean absolute deviation MAE can avoid the problem that the errors cancel each other out, and thus can accurately reflect the actual prediction error. The smaller the above error value, the better the prediction effect. R2 is also known as the decision coefficient, reflects the 4

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

ability of the model to interpret the data. The closer the value is to 1, the better the prediction effect of the model. vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi!, ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N � u X �2 t �xðnÞ b ðN 1Þ (13) RMSE ¼ x ðnÞ � n¼1

N � X �

MAPE ¼

xðnÞ

b x ðnÞ

!, �� � b x ðnÞ � N

(14)

n¼1

MAE ¼

N � X �xðnÞ

!, � N bx ðnÞ �

(15)

n¼1

R2 ¼ 1

N X

xðnÞ 1

RMSE

MAPE(%)

MAE

R2

No Dropout

Overall error Training set Predictive set Overall error Training set Predictive set

0.0578 0.0402 0.0992 0.0261 0.0214 0.0270

0.4494 0.3049 0.9943 0.1955 0.1933 0.1946

0.0423 0.0299 0.0888 0.0188 0.0173 0.0190

0.9827 0.9891 0.9757 0.9965 0.9985 0.9951

Dropout

, N X xðnÞ

xðnÞ

�2

(16)

1

4.2. Dropout to avoid overfitting If there is no method to prevent over-fitting in the LSTM RNN, it will result in higher fitting accuracy, lower prediction accuracy, and even the RUL prediction function of the supercapacitor cannot be achieved. In order to solve the problem of over-fitting in neural networks, SC1 and SC2 (Fig. S1) are experimented based on LSTM RNN, comparing the predicted results of adding Dropout with the prediction results without using any overfitting algorithm, the effectiveness of the method is further illustrated. The SC1 was tested at 2.9 V, 3A, 65 � C. The comparison of the overfitting is shown in Fig. 3. It can be seen that the prediction effect of the LSTM RNN using the Dropout algorithm is significantly better. The specific errors are shown in Table 1. The RMSE of the training set and predictive set without any over-fitting prevention algorithm are 0.0402

Table 1 Prediction results of remaining useful life for supercapacitor No. SC1. Error

�2

Where, xðnÞ representing the actual value, b x ðnÞ representing the predicted value.

Fig. 3. Comparison of remaining useful life prediction results of supercapacitor No. SC1 (2.9 V, 3A, 65 � C) based on LSTM RNN method. (a) Prediction results with standard recurrent neural network without method to prevent overfitting. (b) Prediction results with dropout method to avoid overfitting issue.

Method

b x ðnÞ

Fig. 4. Remaining useful life prediction results and errors of supercapacitor No. SC1 with three different models at 2.9 V, 3A, 65 � C. (a) Remaining useful life prediction with LSTM RNN. (b) Prediction errors with LSTM RNN. (c) Remaining useful life prediction with GRU model. (d) Prediction errors with GRU model. (e) Remaining useful life prediction with SIM RNN. (f) Prediction errors with SIM RNN. 5

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

predictive set has a similar prediction effect with the training set and has higher prediction accuracy. In Fig. 4(c), there is a certain error between the predicted data of GRU and the actual value, and the prediction effect is worse than LSTM RNN. The RMSE of the training set is 0.0366 and the RMSE of the predictive set is 0.0463. The reason is that the GRU has only two gates and the LSTM RNN has three gates, so the LSTM RNN is more powerful and flexible. During the training phase, the RMSE of the LSTM RNN is reduced by 0.0152 compared to the GRU, and R2 is increased by 0.0054. In the prediction phase, the RMSE of the LSTM RNN is reduced by 0.0193 compared to the GRU, and R2 is increased by 0.0301. These results show that the LSTM RNN-based approach has better performance than the GRU on both the training set and the predictive set. In Fig. 4(e), the RMSE of the training set is 0.0455. The RMSE of the predictive set is 0.1369, which increase 0.0488 than the LSTM RNN and 0.0361 than the GRU. Compared with LSTM RNN and GRU, SIM RNN has a large training set and predictive set error. This is due to the long-term dependence of SIM RNN, so as the prediction interval increases, the prediction accuracy of the model decreases significantly. From the various errors, LSTM RNN provides the best performance for supercapacitor RUL prediction.

Table 2 Prediction accuracy of remaining useful life for supercapacitor No. SC1. Method

Error

RMSE

MAPE(%)

MAE

R2

LSTM RNN

Overall error Training set Predictive set Overall error Training set Predictive set Overall error Training set Predictive set

0.0261 0.0214 0.0270 0.0388 0.0366 0.0463 0.0749 0.0455 0.1369

0.1955 0.1933 0.1946 0.3506 0.3236 0.4548 0.6191 0.3918 1.4776

0.0188 0.0173 0.0190 0.0332 0.0312 0.0408 0.0575 0.0377 0.1324

0.9965 0.9985 0.9951 0.9916 0.9931 0.9651 0.9687 0.9846 0.9552

GRU SIM RNN

and 0.0992 respectively, and the predictive set error is 2.47 times of the training set error, indicating that there is obvious over-fitting phenom­ enon. From the error data with Dropout, the RMSE of the training set is 0.0214, the RMSE of the predictive set is 0.0270, and the ratio of pre­ dictive set error to training set error is 1.26. Compared with the previous prediction results, the overall RMSE error with Dropout decreased from 0.0578 to 0.0261, and the error gap between the training set and the predictive set was greatly reduced. In summary, the addition of Dropout to the LSTM RNN effectively solves the problem of over-fitting of the LSTM RNN and improves the prediction accuracy.

4.4. Model verification for offline data In this paper, two supercapacitors of the same specification have been predicted at 2.9 V, 3A, 65 � C and 2.7 V, 3A, 50 � C (Fig. S2), respectively. To further validate the applicability and generalization capabilities of the model, the model was used for prediction of the SC3 dataset that was not trained. The online update data of the RUL pre­ diction is reduced, and the calculation error is compared with the measured data, and the superiority of the model can be tested. Fig. 5 shows the prediction results of SC3 under 2.7 V, 3A, 25 � C conditions. Since the supercapacitor data under this condition is not trained, the initial prediction error is large. After a period of time, since both LSTM RNN and GRU have memory functions, the prediction effects of these two models are relatively good. Since LSTM RNN is more applicable to the processing of exponential data, the prediction error is

4.3. RUL prediction for trained data The supercapacitor data set under different temperatures and volt­ ages was trained using LSTM RNN, GRU, SIM RNN models and used for capacity prediction. The first 70 of the supercapacitor data set was used for model training, and the experimental data of the SC1 and SC2 (Fig. S2) were randomly selected as the test data for model verification. The predicted results of the SC1 are shown in Fig. 4. The specific performance of the three methods is shown in Table 2. In Fig. 4(a), the RMSE of the training set and predictive set based on the LSTM RNN prediction are 0.0214 and 0.0270, respectively. It is obvious that the

Fig. 5. Remaining useful life prediction results and errors of supercapacitor No. SC3 with three different models at 2.7 V, 3A, 25 � C. (a) Remaining useful life prediction with LSTM RNN. (b) Prediction errors with LSTM RNN. (c) Remaining useful life prediction with GRU model. (d) Prediction errors with GRU model. (e) Remaining useful life prediction with SIM RNN. (f) Prediction errors with SIM RNN. 6

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

different temperatures is predicted by LSTM RNN at 2.7 V, as shown in Fig. 6(a). The red curve is the predicted value after training, and the rest is the direct prediction value. It can be seen that the overall prediction effect is better. When the temperature is less than 60 � C, it is in the normal working temperature range of the supercapacitor, and the inclination angle of the predicted surface in the interval is basically the same, indicating that the aging speed is slow. When the temperature exceeds 60 � C, it can be seen that the inclination angle of the predicted surface increases. And when the capacity attenuation exceeds 20%, it means failure. It indicates that the aging rate increases with increasing temperature over the normal operating temperature range. The trend of SOH with different voltages is predicted by LSTM RNN at 65 � C, as shown in Fig. 6(b). The red curve is the predicted value after training. It can be seen that when the working voltage is 2.7 V–2.9 V, the predicted surface is relatively flat, indicating that the working voltage range has little effect on the aging speed. When the operating voltage exceeds 2.9 V, we can see the predicted surface inclination angle in­ creases as the voltage increases. It indicates that the working voltage has approached or even exceeded the decomposition voltage of the elec­ trolyte, causing the supercapacitor to accelerate aging.

Table 3 Prediction accuracy of remaining useful life for supercapacitor No. SC3. Method

RMSE

MAPE(%)

MAE

R2

LSTM RNN GRU SIM RNN

0.0338 0.0528 0.0646

0.2234 0.2940 0.4471

0.0230 0.0305 0.0462

0.9881 0.9711 0.9567

5. Conclusion The RUL prediction can evaluate the reliability of supercapacitor system and determine possible failure times in advance. This paper studies the influence of temperature and voltage on the aging trend of supercapacitors combined with the degradation mechanism model of supercapacitors. It’s proved that LSTM RNN performs high prediction accuracy and good robustness. Moreover, the proposed method is compared with GRU and SIM RNN for the prediction of offline data, which verifies the validity and applicability of the proposed method. However, GRU is a network created on the basis of LSTM RNN, it has attracted the attention of researchers in recent years. There are rela­ tively few studies, it still needs to be further researched and experi­ mented in the future.

Fig. 6. (a) Trend of state of health at 2.7 V under different temperatures. When the temperature exceeds 60 � C, the aging speed of the supercapacitor is accel­ erated. (b) Trend of state of health at 65 � C under different voltages. When the voltage exceeds 2.9 V, the aging speed of the supercapacitor is accelerated.

Acknowledgments

smaller. However, SIM RNN does not have a memory unit, and the prediction effect is poor. Table 3 lists the more detailed predictive per­ formance of the three models. The predicted value of LSTM RNN is close to the actual data, and the error is small. RMSE, MAPE, MAE, and R2 are 0.0338, 0.2234%, 0.0230, and 0.9881, respectively. It can be seen from the Fig. 5(a) that the prediction error is large before the cycle of 2000 times, and the prediction error gradually decreases as the number of cycles increases. It verifies the good predictive ability of LSTM RNN and can achieve higher prediction accuracy without repeated training. When the GRU predicts the lifetime of a supercapacitor, the RMSE increases by 0.0190 and the R2 decreases by 0.0139 compared to the LSTM RNN. It can be seen from the Fig. 5(c) that the prediction error is reduced after the cycle of 100,000 times, but the prediction accuracy is lower than that of the LSTM RNN. The RMSE and MAPE of SIM RNN are 0.0308 and 0.2065% higher than the error of LSTM RNN, respectively. It can be seen from the Fig. 5(e) that the late prediction error is relatively small compared with the previous period. Compared with the LSTM RNN and the GRU, the error fluctuation is large and the stability is not high. The above shows that the LSTM RNN performs better than the other two models in terms of accuracy and stability of prediction.

The work was supported by the Shandong Science and Technology Development Plan (No. GG201809230040) and (No. 2017GGX50114) and the National Natural Science Foundation Youth Fund (No. 51307012). The authors also show gratitude to the NSFC (No. 51802116) and the Natural Science Foundation of Shandong Province (No. ZR2019BEM040). We acknowledge the National Key R&D Program of China (No. 2017YFE0102700) from the Ministry of Science and Technology (MOST) of China and the Key R&D program of Shandong Province (Major Innovation Project of Science and Technology of Shandong Province) (No. 2018YFJH0503). Appendix A. Supplementary data Supplementary data to this article can be found online at https://doi. org/10.1016/j.jpowsour.2019.227149. References [1] B.L. Guo, C.X. Hou, L.H. Fan, Z. Sun, Effect of extraction temperature on hyper-coal structure and electrochemistry of coal-based activated carbon, Chin. J. Inorg. Chem. 34 (2018) 1615–1624. http://doi.org/10.11862/CJIC.2018.201. [2] Y.F. Zeng, G.X. Xin, C.K. Bulin, B.W. Zhang, One-step preparation and electrochemical performance of 3D reduced graphene oxide/NiO as supercapacitor electrodes materials, J. Inorg. Mater. 33 (2018) 1070–1076. http://doi.org/10.1 5541/jim20180032. [3] M. Zhang, Y.T. Li, Z.R. Shen, “Water-in-salt” electrolyte enhanced high voltage aqueous supercapacitor with all-pseudocapacitive metal-oxide electrodes, J. Power Sources 414 (2019) 479–485. https://doi.org/10.1016/j.jpowsour.2019.01.037. [4] J.X. Zhang, Z.Z. Zhang, Y.T. Jiao, H.X. Yang, Y.Q. Li, J. Zhang, P. Gao, The graphene/lanthanum oxide nanocomposites as electrode materials of

4.5. Predicted aging trend of supercapacitors The working conditions of supercapacitors in practical applications are very complicated. Due to the large temperature and voltage setting interval between sample experiments, LSTM RNN is used to predict other temperature and voltage conditions. The trend of SOH with 7

Y. Zhou et al.

[5]

[6]

[7] [8] [9]

[10] [11] [12]

[13]

[14]

[15] [16] [17] [18] [19]

[20]

[21]

[22] [23]

[24] [25] [26] [27]

Journal of Power Sources 440 (2019) 227149

supercapacitors, J. Power Sources 419 (2019) 99–105. https://doi.org/10.1016/j. jpowsour.2019.02.059. L.X. Shen, L.H. Du, S.Z. Tan, Z.G. Zang, C.X. Zhao, W.J. Mai, Flexible electrochromic supercapacitor hybrid electrodes based on tungsten oxide films and silver nanowires, Chem. Commun. 52 (2016) 6296–6299. https://doi.org /10.1039/c6cc01139j. A.A. Mohammed, C. Chen, Z.H. Zhu, Green and high performance all-solid-state supercapacitors based on MnO2/Faidherbia albida fruit shell derived carbon sphere electrodes, J. Power Sources 417 (2019) 1–13. https://doi.org/10.1016/j. jpowsour.2019.02.003. L. Huang, J.L. Key, P.K. Shen, Boosting the volumetric energy of supercapacitors using polytetrafluoroethylene pyrolysis gas, J. Power Sources 414 (2019) 76–85. https://doi.org/10.1016/j.jpowsour.2018.12.060. J.W. Lang, X. Zhang, L. Liu, B.J. Yang, J. Yang, X.B. Yan, Highly enhanced energy density of supercapacitors at extremely low temperatures, J. Power Sources 423 (2019) 271–279. https://doi.org/10.1016/j.jpowsour.2019.03.096. J.H. Fang, X.F. Miao, X.X. Zhang, Y.C. Liu, S.J. Chen, Y.Q. Chen, W. Wang, Y. N. Zhang, Enhancing the capacity of activated carbon electrodes by a redox mediator pair for the fabrication of flexible asymmetric solid-state supercapacitors, J. Power Sources 418 (2019) 24–32. https://doi.org/10.1016/j.jpowsour.2019.01 .076. P. Kumar, E. Di Mauro, S.M. Zhang, A. Pezzella, F. Soavi, C. Santato, F. Cicoira, Melanin-based flexible supercapacitorsþ, J. Mater. Chem. C 4 (2016) 9516–9525. https://doi.org/10.1039/C6TC03739A. G.T. Xia, C. Li, K. Wang, L.W. Li, Structural design and electrochemical performance of PANI/CNTs and MnO2/CNTs supercapacitor, Sci. Adv. Mater. 11 (2019) 1079–1086. https://doi.org/10.1166/sam.2019.3487. K. Wang, L.W. Li, W. Xue, S.Z. Zhou, Y. Lan, H.W. Zhang, Z.Q. Sui, Electrodeposition synthesis of PANI/MnO2/graphene composite materials and its electrochemical performance, Int. J. Electrochem. Sci. 12 (2017) 8306–8314. https://doi.org/10.20964/2017.09.06. A. El Mejdoubi, H. Chaoui, H. Gualous, J. Sabor, Online parameter identification for supercapacitor state-of-health diagnosis for vehicular applications, IEEE Trans. Power Electron. 32 (2017) 9355–9363. http://doi.org/10.1109/TPEL.2017. 2655578. P.J. Corral-Vega, L.M. Fern� andez-Ramírez, P. García-Trivi~ no, Hybrid powertrain, energy management system and techno-economic assessment of rubber tyre gantry crane powered by diesel-electric generator and supercapacitor energy storage system, J. Power Sources 412 (2019) 311–320. https://doi.org/10.1016/j. jpowsour.2018.11.027. L. Serban, A control strategy for microgrids: seamless transfer based on a leading inverter with supercapacitor energy storage system, Appl. Energy 221 (2018) 490–507. https://doi.org/10.1016/j.apenergy.2018.03.122. J. Sayago, U. Shafique, F. Soavi, F. Cicoira, C. Santato, TransCap: a monolithically integrated supercapacitor and electrolyte-gated transistorþ, J. Mater. Chem. C 2 (2014) 10273–10276. https://doi.org/10.1039/C4TC02119C. J. Wei, G. Dong, Z. Chen, Lyapunov-based state of charge diagnosis and health prognosis for lithium-ion batteries, J. Power Sources 397 (2018) 352–360. http:// doi.org/10.1016/j.jpowsour.2018.07.024. L. Cai, J.H. Meng, D.I. Stroe, G.Z. Luo, R. Teodorescu, An evolutionary framework for lithium-ion battery state of health estimation, J. Power Sources 412 (2019) 615–622. http://doi.org/10.1016/j.jpowsour.2018.12.001. L.R. Hou, Y.Y. Shi, C. Wu, Y.R. Zhang, Y.Z. Ma, X. Sun, J.F. Sun, X.G. Zhang, C. Z. Yuan, Monodisperse metallic NiCoSe2 hollow sub-microspheres: formation process, intrinsic charge-storage mechanism, and appealing pseudocapacitance as highly conductive electrode for electrochemical supercapacitors, Adv. Funct. Mater. 28 (2018) 1705921. https://doi.org/10.1002/adfm.201705921. J.F. Sun, C. Wu, X.F. Sun, H. Hu, C.Y. Zhi, L.R. Hou, C.Z. Yuan, Recent progresses in high-energy-density all pseudocapacitive-electrode-materials-based asymmetric supercapacitors, J. Mater. Chem. A 5 (2017) 9443–9464. https://doi.org /10.1039/c7ta00932a. L.J. Zhang, X.G. Zhang, L.F. Shen, B. Gao, L. Hao, X.J. Lu, F. Zhang, B. Ding, C. Z. Yuan, Enhanced high-current capacitive behavior of graphene/CoAl-layered double hydroxide composites as electrode material for supercapacitors, J. Power Sources 199 (2012) 395–401. https://doi.org/10.1016/j.jpowsour.2011.10.056. Y.Y. Gao, S.L. Chen, D.X. Cao, G.L. Wang, J.L. Yin, Electrochemical capacitance of Co3O4 nanowire arrays supported on nickel foam, J. Power Sources 195 (2010) 1757–1760. https://doi.org/10.1016/j.jpowsour.2011.12.001. X.J. Lu, H. Dou, C.Z. Yuan, S.D. Yang, L. Hao, F. Zhang, L.F. Shen, L.J. Zhang, X. G. Zhang, Polypyrrole/carbon nanotube nanocomposite enhanced the electrochemical capacitance of flexible graphene film for supercapacitors, J. Power Sources 197 (2012) 319–324. https://doi.org/10.1016/j.jpowsour.2011.08.112. Y. Ma, Y. Chen, X.W. Zhou, H. Chen, Remaining useful life prediction of lithiumion battery based on gauss-hermite particle filter, IEEE Trans. Control Syst. Technol. 27 (2019) 1788–1795. http://doi.org/10.1109/TCST.2018.2819965. D.T. Liu, X.H. Yin, Y.C. Song, W. Liu, Y. Peng, An on-line state of health estimation of lithium-ion battery using unscented particle filter, IEEE Access 6 (2018) 40990–41001. http://doi.org/10.1109/ACCESS.2018.2854224. A. El Mejdoubi, H. Chaoui, J. Sabor, H. Gualous, Remaining useful life prognosis of supercapacitors under temperature and voltage aging conditions, IEEE Trans. Ind. Electron. 65 (2018) 4357–4367. http://doi.org/10.1109/TIE.2017.2767550. Z.H. Chen, H. Sun, G.Z. Dong, J.W. Wei, J. Wu, Particle filter-based state-of-charge estimation and remaining-dischargeable-time prediction method for lithium-ion batteries, J. Power Sources 414 (2019) 158–166. http://doi.org/10.1016/j. jpowsour.2019.01.012.

[28] G.Z. Dong, J.W. Wei, Z.H. Chen, H. Sun, X.W. Yu, Remaining dischargeable time prediction for lithium-ion batteries using unscented Kalman filter, J. Power Sources 364 (2017) 316–327. http://doi.org/10.1016/j.jpowsour.2017.08.040. [29] L.J. Zhang, Z.Q. Mu, C.Y. Sun, Remaining useful life prediction for lithium-ion batteries based on exponential model and particle filte, IEEE Access 6 (2018) 17729–17740. http://doi.org/10.1109/ACCESS.2018.2816684. [30] M. Ahwiadi, W. Wang, An enhanced mutated particle filter technique for system state estimation and battery life prediction, IEEE Trans. Instrum. Meas. 68 (2018) 923–935. http://doi.org/10.1109/TIM.2018.2853900. [31] A. El Mejdoubi, H. Chaoui, H. Gualous, P.V.D. Bossche, N. Omar, J.V. Mierlo, Lithium-ion batteries health prognosis considering aging conditions, IEEE Trans. Power Electron. 34 (2018) 6834–6844. http://doi.org/10.1109/TPEL.2018. 2873247. [32] G.Z. Dong, Z.H. Chen, J.W. Wei, Q. Ling, Battery health prognosis using brownian motion modeling and particle filtering, IEEE Trans. Ind. Electron. 65 (2018) 8646–8655. http://doi.org/10.1109/TIE.2018.2813964. [33] K. Wang, S.Z. Zhou, Y.T. Zhou, J. Ren, L.W. Li, Y. Lan, Synthesis of porous carbon by activation method and its electrochemical performance, Int. J. Electrochem. Sci. 13 (2018) 10766–10773. https://doi.org/10.20964/2018.11.30. [34] K. Wang, J.B. Pang, L.W. Li, S.Z. Zhou, Y.H. Li, T.Z. Zhang, Synthesis of hydrophobic carbon nanotubes/reduced graphene oxide composite films by flash light irradiation, Front. Chem. Sci. Eng. 12 (2018) 376–382. https://doi.org/10. 1007/s11705-018-1705-z. [35] D.L. Yuan, C. Zhang, S.F. Tang, X. Li, J.C. Tang, Y.D. Rao, Z.B. Wang, Q.R. Zhang, Enhancing CaO2 fenton-like process by Fe(II)-oxalic acid complexation for organic wastewater treatment, Water Res. 163 (2019) 114861. https://doi.org/10.1016/j. watres.2019.114861. [36] S.F. Tang, N. Li, D.L. Yuan, J.C. Tang, X. Li, C. Zhang, Y.D. Rao, Comparative study of persulfate oxidants promoted photocatalytic fuel cell performance: simultaneous dye removal and electricity generation, Chemosphere 234 (2019) 658–667. https://doi.org/10.1016/j.chemosphere.2019.06.112. [37] Le Kang, Liang Zhao, Shuang Yao, Chongxiong Duan, A new architecture of superhydrophilic β-SiAlON/graphene oxide ceramic membrane for enhanced antifouling and separation of water/oil emulsion, Ceram. Int. 45 (2019) 16717–16721. https://doi.org/10.1016/j.ceramint.2019.05.195. [38] Y.Y. Yang, L. Kang, H. Li, Enhancement of photocatalytic hydrogen production of BiFeO3 by Gd3þ doping, Ceram. Int. 45 (2019) 8017–8022. https://doi.org/10.101 6/j.ceramint.2018.12.150. [39] J. Wu, Y.J. Wang, X. Zhang, Z.H. Chen, A novel state of health estimation method of Li-ion battery using group method of data handling, J. Power Sources 327 (2016) 457–464. http://doi.org/10.1016/j.jpowsour.2016.07.065. [40] F.J. Wang, W. Chen, L. Qiu, Hausdorff derivative Laplacian operator for image sharpening, Fractals Complex Geom. Patterns Scaling Nat. Soc. 27 (2019) 1950060. https://doi.org/10.1142/S0218348X19500609. [41] X.X. Yue, F.J. Wang, Q.S. Hua, X.Y. Qiu, A novel space-time meshless method for nonhomogeneous convection-diffusion equations with variable coefficients, Appl. Math. Lett. 92 (2019) 144–150. https://doi.org/10.1016/j.aml.2019.01.018. [42] W.Z. Qu, A high accuracy method for long-time evolution of acoustic wave equation, Appl. Math. Lett. 98 (2019) 135–141. https://doi.org/10.1016/j.aml.201 9.06.010. [43] J.L. Wang, L.H. Tang, L.Y. Zhao, Z.E. Zhang, Efficiency investigation on energy harvesting from airflows in HVAC system based on galloping of isosceles triangle sectioned bluff bodies, Energy 172 (2019) 1066–1078. https://doi.org/10.1016/j. energy.2019.02.002. [44] L. Zhao, L. Kang, S. Yao, Research and application of acoustic emission signal processing technology, IEEE Access 7 (2019) 984–993. https://doi.org/10.1109/ ACCESS.2018.2886095. [45] D. Andre, C. Appel, T. Soczka-Guth, D.U. Sauer, Advanced mathematical methods of SOC and SOH estimation for lithium-ion batteries, J. Power Sources 224 (2013) 20–27. http://doi.org/10.1016/j.jpowsour.2012.10.001. [46] H. Li, D.H. Pan, C.L.P. Chen, Intelligent prognostics for battery health monitoring using the mean entropy and relevance vector machine, IEEE Trans. Syst. Man Cybern. -Syst. 44 (2014) 851–862. http://doi.org/10.1109/TSMC.2013.2296276. [47] B. Wu, S.W. Han, K.G. Shin, W. Lu, Application of artificial neural networks in design of lithium-ion batteries, J. Power Sources 395 (2018) 128–136. http://doi. org/10.1016/j.jpowsour.2018.05.040. [48] H. Chaoui, C.C. Ibe-Ekeocha, State of charge and state of health estimation for lithium batteries using recurrent neural networks, IEEE Trans. Veh. Technol. 66 (2017) 8773–8783. http://doi.org/10.1109/TVT.2017.2715333. [49] A. Soualhi, M. Makdessi, R. German, F.R. Echeverría, H. Razik, A. Sari, P. Venet, G. Clerc, Heath monitoring of capacitors and supercapacitors using the neo-fuzzy neural approach, IEEE Trans. Ind. Inform. 14 (2018) 24–34. http://doi.org/10.11 09/TII.2017.2701823. [50] X.Y. Li, L. Zhang, Z.P. Wang, P. Dong, Remaining useful life prediction for lithiumion batteries based on a hybrid model combining the long short-term memory and Elman neural networks, J. Energy Storage 21 (2019) 510–518. http://doi. org/10.1016/j.est.2018.12.011. [51] C.S. Wang, N.Y. Lu, S.L. Wang, Y.H. Cheng, B. Jiang, Dynamic long short-term memory neural-network-based indirect remaining-useful-life prognosis for satellite lithium-ion battery, Appl. Sci.-Basel 8 (2018) 2078. http://doi.org/10.3390/app 8112078. [52] E. Chemali, P.J. Kollmeyer, M. Preindl, R. Ahmed, A. Emadi, Long short-term memory networks for accurate state-of-charge estimation of Li-ion batteries, IEEE Trans. Ind. Electron. 65 (2018) 6730–6739. http://doi.org/10.1109/TIE.201 7.2787586.

8

Y. Zhou et al.

Journal of Power Sources 440 (2019) 227149

[53] R. Ma, T. Yang, E. Breaz, Z.L. Li, P. Briois, F. Gao, Data-driven proton exchange membrane fuel cell degradation predication through deep learning method, Appl. Energy 231 (2018) 102–115. http://doi.org/10.1016/j.apenergy.2018.09.111. [54] J.W. Liu, Q. Li, W.R. Chen, Y. Yan, Y.B. Qiu, T.Q. Cao, Remaining useful life prediction of PEMFC based on long short-term memory recurrent neural networks, Int. J. Hydrogen Energy 44 (2018) 5470–5480. http://doi.org/10.1016/j.ijh ydene.2018.10.042. [55] Y.Z. Zhang, R. Xiong, H.W. He, M.G. Pecht, Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries, IEEE Trans. Veh. Technol. 67 (2018) 5695–5705. http://doi.org/10.1109/TVT.201 8.2805189. [56] R.X. Zhao, P. Kollmeyer, R.D. Lorenz, T. Jahns, A compact methodology via a recurrent neural network for accurate equivalent circuit type modeling of lithium-

ion batteries, IEEE Trans. Ind. Appl. 55 (2) (2018) 1922–1931. http://doi.org/1 0.1109/TIA.2018.2874588. [57] K. Wang, L.W. Li, Y. Lan, P. Dong, G.T. Xia, Application research of chaotic carrier frequency modulation technology in two-stage matrix converter, Math. Probl. Eng. 2019 (2019) 8, 2614327, https://doi.org/10.1155/2019/2614327. [58] S.K. Ibrahim, A. Ahmed, M.A.E. Zeidan, I. Ziedan, Machine learning methods for spacecraft telemetry mining, IEEE Trans. Aerosp. Electron. Syst. 55 (4) (Aug. 2019) 1816–1827. http://doi.org/10.1109/TAES.2018.2876586. [59] T. Ergen, S.S. Kozat, Efficient online learning algorithms based on LSTM neural networks, IEEE Trans. Neural Netw. Learn. 29 (2018) 3772–3783. http://doi.org/1 0.1109/TNNLS.2017.2741598.

9