Traffic-load prediction based on echo state network improved by Bayesian theory in 10G-EPON

The Journal of China Universities of Posts and Telecommunications April 2015, 22(2): 69–73 www.sciencedirect.com/science/journal/10058885

http://jcupt.xsw.bupt.cn

Traffic-load prediction based on echo state network improved by Bayesian theory in 10G-EPON Bai Huifeng1 (

), Zhao Xianlong1, Zhou Hongfeng1, Zhang Likun2

1. Beijing NARI SmartChip Microelectronics Company Limited, Beijing 102200, China 2. Translation Department, Beijing 100015, China

Abstract With the evolution of 10-gigabit Ethernet passive optical network (10G-EPON), the traffic-load prediction ability is necessary to support soaring services traffic with diversified characteristics and requirements. As a strong candidate to be used for the traffic-load prediction, the echo state network (ESN) may face the pseudo-regression problem and need to be improved for the better traffic-load prediction. To overcome this problem, this paper proposes an ESN based traffic-load prediction scheme using Bayesian theory in 10G-EPON for future-proof. In this proposed approach, Bayesian probability is introduced into the ESN and is used to improve the performance of ESN. According to the architecture between optical line terminal (OLT) and optical network units (ONU) in 10G-EPON, an ESN based on the Bayesian theory (B-ESN) is realized and the B-ESN based traffic load prediction scheme is also developed in OLT. Experiment results show that the proposed scheme can greatly better the accuracy of traffic-load prediction with lower complex degree. Keywords 10-gigabit passive optical network, traffic-load prediction, ESN, Bayesian theory

1 Introduction As new services emerge rapidly with soaring services traffic-load, the Ethernet passive optical network (EPON) is undergoing great evolution toward to 10G-EPON or even 40G-EPON [1]. With high capacity to provide access for multiple information services, the 10G-EPON is a strong candidate to support the communication demand of newly merging services [2]. In fact, this trend indicates the requirement of “services traffic-load prediction” ability on supporting access network architectures for future-proof [3]. Therefore, the 10G-EPON must be characterized by the ‘services traffic-load prediction’ function. The awareness of traffic-load in advance is an important issue in such as network design, network management, and network security [3]. Much effort has been made on analyzing and prediction technologies of the services traffic-load. The regression model and the time series Received date: 27-03-2014 Corresponding author: Bai Huifeng, E-mail: [email protected] DOI: 10.1016/S1005-8885(15)60641-0

algorithm are commonly used for traffic-load prediction. With research deepening, the neural network algorithm is also introduced into this field and gain widely application because of its strong ability of nonlinear mapping. Therefore, the neural network algorithm has already become a strong candidate to modeling and prediction of complex systems with nonlinearity and variability. Moreover, improvement and optimization have been done to the neural network algorithm, including the ant colony algorithm and the fuzzy reasoning algorithm, in order to increase the prediction accuracy. However, there exists the problem of large computation in the back propagation (BP) neural network, which limits the usage of the BP. The ESN is a potential candidate to the traffic-load prediction, as ESN has great advantages to enhance reliability, improve accuracy, and reduce complexity [4–5]. However, the existing ESN prediction approaches still face the problem that the classification result with nice generalization ability is hard to get. Aimed to solve the problems mentioned above, this article proposed a Bayesian theory B-ESN algorithm for the traffic-load prediction in 10G-EPON. In the B-ESN

70

The Journal of China Universities of Posts and Telecommunications

based traffic-load prediction scheme, great improvement was made, where a ‘master-agent’ structure is adopted and the services awareness function is centralized in OLT according to the passive optical network (PON) architecture. In this proposed mechanism, OLT is responsible for the main task of traffic-load prediction computation, followed by the resources allocation by dynamic bandwidth allocation (DBA). Thus, the complex traffic-load prediction function and related schedule both are centralized in OLT. The rest of this article was organized as follows. Sect. 2 discusses the principle of the ESN. Sect. 3 proposes the Bayesian theory B-ESN algorithm. Then, the B-ESN based traffic-load prediction scheme is presented in Sect. 4. The testing results and analysis are given in Sect. 5. Finally, Conclusion is given in Sect. 6.

2 Principle of the ESN As Fig. 1 shows, the architecture of the ESN is divided into three layers: the input layer, the reservoir and the output layer [4–7]. There is great number of neurons in this dynamical reservoir with quite strong learning ability. Assume that there are L input units, M output units and N units in the reservoir. Generally, the connection weight matrix is represented as W in of the input layer, while its corresponding unit in the reservoir is W. Thus, the dynamical learning ability and reliability of the ESN can be ensured. In addition, the Wout is the connection weight matrix of the output layer and the goal of data training in the ESN.

2015

regression algorithm between state-variables and output values. The states of the reservoir and the output are updated using Eqs. (1) and (2). (1) x ( n + 1) = f (Win u (n + 1) + Wx ( n ) + Wback y ( n ) )

y ( n + 1) = f out (Wout u (n + 1) + Wx ( n + 1) + Wback y ( n ) )

(2 )

where, u(n), x(n) and y(n) represents individually the input variable, state variable and output variable. And f ( ⋅ ) and f out( ⋅ ) are the activation functions of processing units of the reservoir and the output unit.

3 Bayesian theory based ESN algorithm 3.1

Description of Bayesian-probability based ESN

In the traditional ESN, all network information comes from the training set, which may lead to unstable results of training and weakening the generalization ability. Aiming at the problem, this article optimizes the output weight value by introducing the regularization method. In the traditional ESN, the sum of squared errors is adopted by Eq. (3). 2 1 N ED = ∑ ( yi − f ( xi ) ) (3) 2 i =1 where N is the total number of classification results by ESN. By introducing the regularization term in Eq. (4), the generalization ability of ESN can be improved [7]. m

EW = ∑ ωi

(4)

i =1

where the ωi is network parameter and m is the total number of these network parameters. Thus, the total error function is given in Eq. (5). (5) F (W ) = β ED + α EW where, α and β are regularization parameters of EW

Fig. 1

ESN structure

The basic idea of the ESN comes from that: the reservoir is triggered by the input signal to produce continuous state-variables within this reservoir. and the output value of ESN can be obtained by using the linear

and ED. In this article, the Bayesian theory is used to correct the weight value of the output layer in ESN. We assume that N represents the number of units in the reservoir. According to the Bayesian principle, we can draw the posterior distribution in Eq. (6) [7]. p ( D | ω, β , H ) p ( D | α , H )  p ( ω | D, α , β , H ) =  p(D |α, β, H ) (6)   p (ω | α , β , H ) = ∫ p ( D | α , β , H ) p (ω | α , H ) dω  where ω is the weight value of prior probability in Bayesian network. the H represents the architecture of network and D is the set of sample data. Assuming that the

Issue 2

Bai Huifeng, et al. / Traffic-load prediction based on echo state network improved by…

prior distribution follows the Gauss distribution, Eq. (7) can be obtained as follow: 1 2   1 p (ω | α , H ) = exp  − α ω  Z W (α )  2  (7)  ω   2π  2 Z W (α ) =    α   Similarly, Eqs. (8) and (9) can also be gained: 1 2   1 p ( D | ω, β , H ) = exp  − β ω  ZD ( β )  2  (8) N    2π  2 ZD ( β ) =     β   1  p ( ω | D, α , β , H ) = exp ( − F ( ω ) )  Z F (α , β )  (9)  1 N  − F ( ω∗ ) ∗ 2 det H ( ω ) − 1   Z F (α , β ) = ( 2π ) 2 e    And from Eqs. (7) and (9), we can get Eq. (10) Z F (α , β ) (10) p(D |α,β, H ) = Z D ( β ) Z W (α )

(

)

Combining formulas Eq. (7) to Eq. (10), the optimized values of regularization can be drawn in Eq. (11) γ  α=  2 EW ( ω∗ )   n −γ  (11) β= ∗  2 ED ( ω )  −1 γ = N − 2rank ( H )  where, γ represents the number of efficient network parameters and H is the Hessian Matrix. 3.2

Training of B-ESN

The training procedure of the Bayesian theory based ESN is depicted as follows: Step 1 Initialize each parameter of this algorithm, including the number N, Win, W and Wback of the reservoir. Step 2 Compute the state sequence x(n) of the reservoir using samples and the Eq. (1). Step 3 Set the initial value of α, β and parameters of ESN according to the Eq. (11). Step 4 Obtain the ω * using the Levenberg-Marquardt algorithm. Step 5 Calculate the γ and refresh the value of α, β. Step 6 ESN.

Repeat Eqs. (4) and (5) until convergence of

Step 7

71

Get the Wout according to x(n).

4 B-ESN based traffic-load prediction in 10G-EPON It came to say, for employment of the B-ESN based traffic-load prediction mechanism in 10G-EPON, it takes not only the time sequence in to the consideration, but also other factors including the average traffic before the target time point, the maximum traffic and the minimum traffic. These four kinds of values are input to the B-ESN as input variables, and the output result is the predicted as traffic-load. In realization of this proposed mechanism, the centralized architecture was adopted, where the OLT performs the whole procession of the traffic-load prediction. This traffic-load prediction mechanism was realized in the manner of software using C++, which includes a traffic-load prediction computation (TPC) module and a so-called traffic parameters record (TPR) module. In this proposed traffic-load prediction mechanism, the TPC module is running in OLT using the B-ESN algorithm presented above as the core, which is responsible for the complex function of prediction computation for traffic-load. And the history record of traffic-load parameters is stored and kept in the TPR module. Additionally, the TPR module abstracts and collects traffic information from each ONU. The proposed traffic-load prediction works together with the dynamic bandwidth allocation module (DBA), by giving the prediction result to the DBA to improve its resources allocation efficiency. This proposed B-ESN based traffic-load prediction scheme is able to precast the traffic of each ONU and the total traffic of the whole network system. Thus, the total view and details of traffic-load of the 10G-EPON can both be achieved. After the BP-ESN is well-trained, the procedure of traffic-load prediction in the OLT is described as follows: 1) The TPR module abstracts and collects necessary parameters information of packet traffic from report message of all ONUs in new cycle. 2) The TPR module stores these parameters collected. 3) The TPC module conducts prediction computation by using B-ESN algorithm, making full use of those traffic parameters stored in the TPR module. 4) The prediction results are given to the DBA module for further usage in OLT. Based on this centralized architecture, the B-ESN

72

The Journal of China Universities of Posts and Telecommunications

enables the OLT to form the total view of traffic-load condition of each ONU and the whole system, and to perform resources allocation as well with higher accuracy by working together with DBA module.

5 Testing results and analysis In order to evaluate the performance of the proposed mechanism, an experimental test was operated in a 10G-EPON system with 32 ONUs. The B-ESN was realized by C++ and worked in OLT. In initiation of the reservoir, Nr = 100 and input unit scales (IS) is set to be 4. In this test, comparison was made between the B-ESN and the original ESN in terms of accuracy rate. Before training, the original data must be processed via normalization process： X ( t ) − min { X ( t )} X ′ (t ) = (12) max { X ( t )} − min { X ( t )} The comparison of training performance is shown in Fig. 2, in which, the initial parameters of both the B-ESN and the original ESN are selected randomly. Two observations can be gained from Fig. 2. One is the relation between accuracy rate and training times. As the training times increase hundred by hundred, the accuracy rate of prediction soars obviously. After enough training, the accuracy rate shows the trend toward 100%. The other one is that the centralized B-ESN has higher accuracy rate than the original ESN. That is because the B-ESN is able to form the view of traffic load, which increases the average higher accuracy of the system, while the original ESN fails to do so.

2015

results are given in Fig. 3 and Fig. 4 for comparison. And it is obvious that predictions of both B-ESN and ESN are close to real traffic-load, but the B-ESN has the better overall fitting results than the ESN. That is because there are a number of neural units in the reservoir improved by the Bayesian theory, which greatly enhances the ability to compute the best value of ESN. Moreover, faster speed and more simple computation can be achieved by B-ESN when compared with the original ESN.

Fig. 3

Fig. 4

Traffic prediction of B-ESN (within 24 h)

Traffic prediction of original ESN (within 24 h)

In this article, the root mean square error (RMSE) and the mean absolute percentage error (MAPE) are used to make comparison between these two algorithms. 1  1 N 2 2 RRMSE =  ∑ ( ymax − y (n) )   (13)  N n =1     1 N y − y ( n) M MAPE = ∑ max  N n =1 y ( n)  where ymax is the maximum value of y (n). The prediction Fig. 2

Comparison of training efficiency

When taking training, both B-ESN and ESN are well-prepared, the traffic prediction of B-ESN and ESN within 24 h is both conducted immediately and their

effect from both B-ESN and ESN can be calculated by Eq. (13), and the result comparison is given in Table 1. Table 1

The prediction effect comparison

Parameter RRMSE MMAPE/(%)

Value B-ESN 2.44 1.75

ESN 2.97 2.12

Issue 2

Bai Huifeng, et al. / Traffic-load prediction based on echo state network improved by…

Combining all comparison results above, the B-ESN can obtain better performances comprehensively with higher accuracy. It can be concluded from these testing results that the centralized B-ESN based traffic-load prediction mechanism is more suitable for the special structure of 10G-EPON, and it also implies that the more tightly matching between services and 10G-EPON can be further achieved by adopting the B-ESN algorithm.

6 Conclusions As the important access network technology to support multiple services brought by emerging new services, the 10G-EPON was required to actively be aware in advance of the traffic-load of those service and to provide support with not only high capacity but also high matching degree. This article has proposed a Bayesian theory based ESN (B-ESN) traffic-load prediction mechanism for 10G-EPON. On the basis of this proposed scheme, the better prediction accuracy and overall fitting results was able to be achieved in the 10G-PON system. The experimental testing results showed the validity of the proposed B-ESN algorithm, and it also implied that the B-ESN based traffic prediction scheme could allow the 10G-EPON to better support to services with dynamic traffics.

73

Acknowledgements This work was supported by the Hi-Tech Research and Development Program of China (2012AA050804).

References 1. Tanaka K, Agata A, Horiuchi Y. et al. IEEE 802.3av 10G-EPON standardization and its research and development status. Journal of Lightwave Technology, 2010, 28(4): 651−661 2. Hajduczenia M, da Silva H J A, Monteiro P P. 10G EPON development process. Proceedings of the 9th International Conference on Transparent Optical Networks (ICTON’07): Vol 1, Jul 1−5, 2007, Rome, Italy. Piscataway, NJ, USA: IEEE, 2007: 276−282 3. Elbers J P. Optical access solutions beyond 10G-EPON/XG-PON. Proceedings of the 2010 Conference on Optical Fiber Communication (OPC’), Collocated National Fiber Optic Engineers Conference (OFC/NFOEC’10), Mar 21−25, 2010, San Diego, CA, USA. Piscataway, NJ, USA: IEEE, 2010: 3p 4. Jaeger H. The echo state approach to analyzing and training recurrent neural networks—with an erratum note. Technical Report, Vol 148. Bonn, Germany: German National Research Center for Information Technology (GMD), 2001 5. Alexandre L A, Embrechts M J, Linton J. Benchmarking reservoir computing on time-independent classification tasks. Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN’09). Jun 14−19, 2009, Atlanta, GAUSA. Piscataway, NJ, USA: IEEE, 2009: 89−93 6. Bouckaert R. Bayesian network classifiers in WEKA. Hamilton, New Zealand: Department of Computer Science, Waikato University, 2005 7. Lauret P, Fock E, Randrianarivony R N, et al. Bayesian neural network approach to short time load forecasting. Energy Conversion and Management, 2008, 49(5): 1156−1166

(Editor: Wang Xuying)