Short term load forecasting using particle swarm optimization neural network

Available online at www.sciencedirect.com

ScienceDirect

Available online at www.sciencedirect.com Procedia Computer Science 00 (2018) 000–000

ScienceDirect

www.elsevier.com/locate/procedia

Procedia Computer Science 120 (2017) 382–393

9th 9thInternational InternationalConference Conferenceon onTheory Theoryand andApplication Applicationof ofSoft SoftComputing, Computing,Computing Computingwith with Words and Perception, ICSCCW 2017, 24-25 22-23 August August 2017, 2017, Budapest, Budapest, Hungary Hungary

Short term load forecasting using particle swarm optimization neural network Ozgur Cemal Ozerdema*,Ebenezer O. Olaniyib, Oyebade K. Oyedotunb a

Electrical&Electronics Engineering, European University of Lefke, Gemikonagi-Lefke, via Mersin 10, Turkey

Abstract Energy is very important in many areas of life. Moreover, humans seem to be almost totally reliant on electrical energy in the last few decades. Although, huge efforts are invested in electronic devices which consume lesser energy or rely on alternative power sources, many emerging devices continue to rely on some sort of electrical power. Energy companies are tasked with supplying sufficient energy to consumers; hence, such companies should be able to project the amount of energy to be made available to consumers at different times. It is undesirable that lesser energy than demanded is supplied at any particular time, as this may lead to system collapse or compulsory shedding of load (some consumers experience power interruption). In this work, we model the problem of short term load forecasting using particle swarm optimized feedforward neural network. The described system is capable of predicting hourly load supplied by an energy company. Also, we investigate modeling load forecasting with conventional feedforward neural network, trained with the back propagation learning algorithm. The results obtained show that the both particle swarm and back propagation optimized feedforward networks are suitable regressors for modeling energy demand. Although, the back propagation optimized networks slightly edge on achieved mean absolute error (MAE) and mean square (MSE), the particle swarm optimized networks boasts of faster convergence. Training is roughly twice fast in particle swarm optimized networks, since error gradients computations are not required for optimization. The database used within this work is obtained from a North Cyprus based energy company. © 2018 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 9th International Conference on Theory and application of Soft Computing, Computing with Words and Perception. Keywords:Load forecasting; energy demand; regression; neural networks; particle swarm optimization.

* Corresponding author: E-mail address: [email protected] 1877-0509© 2018 The Authors. Published by Elsevier B.V.

Peer-review under responsibility of the scientific committee of the 9th International Conference on Theory and application of Soft Computing, Computing with Words and Perception. 1877-0509 © 2018 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 9th International Conference on Theory and application of Soft Computing, Computing with Words and Perception. 10.1016/j.procs.2017.11.254

2

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

383

1. Introduction Energy is vital to all living beings on earth. Modern lifestyle has further increased its importance, since population growth and the nearly inevitable haste with which we want things done meant faster and reliable transport, communication, manufacturing processes and ‘everything’. It is apparent that all these cannot be separated from sufficient and reliable energy, hence the pace of electrical technology did not disappoint the engineering world. The challenges that the field of engineeringis faced with over the years in order to introduce sufficient, economic and environmental friendly power generationschemes cannot be overemphasized. It is therefore necessary that energy delivery be optimally achieved. In many situations, energy companies have to supply some required amount of energy to consumers at different times; and of course, energy demand is subject to fluctuations from expected values. Common reasons for this include rapidweather changes, festive periods, holidays, etc. Generally, energy companies projectenergy demands at various times so that it can be adequatelyinjected into transmission lines, this energy is referred to as the based load. Since, energy demand is subject to fluctuations, it is common practice that many energy companies inject little more than is exactly required into transmission lines. In other scenarios, energy companies generate some additional energy in addition to the projected energy demand; this energy (referred to the as spinning reserve) is only made available in the advent of unforeseen huge surges in energy demand (Longlong, and Dongmei, 2013; Chen, et al. 2013) . From an economic point of view, energy companies bare the cost of unused spinning reserves, unless in situations where such energy is delivered to consumers as discussed above. Hence, ability of energy companies to forecast loads accurately great impacts on energy economics and therefore business. i.e. with more accuracy in load forecast, tighter constraints can be introduced on power injection into transmission lines and spinning reserves (Jiang, 2015). This work presents the modelling of short term load forecasting (hourly load) using learning systems. Such complex relationship between expected loads and factors described to affect load forecasting is modelled using neural networks. The factors considered within this work to affect expected hourly loads include time of the day(in hours), day of the week (Monday to Sunday), hourly load of previous week (Lpw), hourly load of two weeks back (Lp2w), and average of Lpw and Lp2w, Lav. In this work, we propose the optimization of designed feedforward neural networks using the particle swarm optimization (PSO) algorithm. PSO allows the global optimization of the feedforward networks, since, the entire weights space can be stochastically searched; in contrast to feedforward networks which rely on back propagations of local error gradients and often get stuck in local minima. We investigate both approaches to training the designed networks, and performances are presented in consideration of achieved MeanAbsoluteError (MAE) and Mean Square Error (MSE) and training time.The remaining sections present related works, brief introduction of neural networks and PSO algorithm, training and testing of networks, and conclusion.

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

384

3

2. Related Studies Feed forward neural network trained with backpropagation algorithm has been the most widely used method in forecasting the load demand using neural network. A study proposed a feedforward neural network trained with backpropagation algorithm for three types of short term electric load forecasting to forecast the daily peak (valley) load, hour and the total load(Subhes and Le, 2004). The author predicted the load for the Northern area of Vietnam by making use of a large set of data on peak data, valley load, hourly load and temperature. The results obtained indicate that applying neural network for forecasting of short term electric load is possible and will produce an accurate result. (Buhari et al. 2012) also proposed an artificial neural network for electric load forecasting of 132/133KV substation, Kano, Nigeria. The daily load profile having a lead time of 1-24hrs for the year 2005 obtained from the utility company. They made used of Levenberg-marquardt optimization techniques which is one of the best learning rate used as a backpropagation algorithm for the multilayer feed forward ANN modeling using Matlab. As a result of the stationary output of the artificial neural network, forecasted next day 24 hour peak load were collected having a performance mean square error of 5.85e-6. This was compared with the actual power utility data. These results proved that the proposed approach is more accurate and reliable in load forecasting for daily operational planning of the power system distribution in Nigeria. (Bilgic et al. 2010) developed a short term hourly load forecasting system for nine load distribution regions of Turkey based on artificial neural network (ANN) approach. The author obtained mean average percent error (MAPE) of total hourly load forecast for Turkey is found as 1.81%. (Ismet,. and Ali, 1997) also developed a new intelligent approach for short term load forecasting. This method is categorized into three basic networks. The first network employs the clustering of daily load curves using a modified kohonen algorithm. The second module determines the most appropriate supervised neural network topology and associated initial weight values for each cluster extracted from a historical database by using genetic algorithm (GA). In the third module, a genetically optimized three layered backpropagation (BP) network is trained and run to perform hourly forecasting. The author take into consideration separately, the effect of each module on the forecasting accuracy. The Turkish electrical power system load curve of the year 1993 which shows different day from different times of the year were used to test the proposed system. A promising result was obtained which shows an approximately of 1% mean error for days distributed throughout the year. Other algorithms such as swarm intelligence has also been used to determine the lowest mean absolute error system for forecasting of the load. This also have an advantage in the training time as compared with artificial neural network trained with backpropagation algorithm. (Azzam-ul-Asar et al. 2007) proposed a system for short term load forecasting based on hourly load data and adjusts its weight through the use of particle swarm optimization (PSO) algorithm. The approach gives better trained models capable of performing well over varying time window and result fairly accurate forecast. (AlRashidi et al. 2009) proposed a new approach for forecasting of the annual peak load in electrical power system. The authors adopted particle swarm optimization to minimize the error associated with the parameters of the model. The result obtained using particle swarm optimization algorithm approach was compared with other method with well known least error square estimation technique. It was confirmed that particle swarm optimization approach produces the least error square estimation as compared with other methods. 3. Data Analysis The database used in this work is obtained from a North Cyprus based energy company Cyprus Turkish Electricity Authority (Kib-Tek). The database contains the hourly load supplied to consumers for recent year. From a thorough study of the available data coupled with load forecasting knowledge, 5 inputs of factors are selected as predictors for hourly load (L/hr). The 5 factors will be used as inputs to the designed networks. These factors are briefly described below. 1. 2. 3. 4. 5.

Time: this is time of the day, t, in hours, for a level of load x delivered. Day: this is the particular day of the week, d, for which x load is delivered. Load of previous week (Lpw): this is the load delivered at time t and day d, for the previous week. Load of two weeks back (Lp2w): this is the load delivered at time t and day d, two weeks back. Average load (Lav): this is the average of Lpw and Lp2w.

4

Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000 Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393

385

Fig. 1.Average load per hour for the week

The average load per hour for Monday to Sunday is shown in Fig. 1. The coding of data attributes or inputs are described below.  Time: the range of time of the day is 00-23 hrs  Day: the days of the week range from Monday to Sunday as 1 to 7.  All loads range from 100 to 250 MW (Mega Watts). Since neural networks accept inputs in the range 0 to 1, all input attributes are normalized to the range 0 to 1 using the Equation 1.

N .V 

raw attribute value Maximum value of attribute

(1)

Where, N.V is the normalized value. The normalized data is divided into training and testing data. The test data are not part of the training data, as this allows the observation of generalization power of trained networks. 1072 training samples of 5 attributes each are used for training, and 715 samples of 5 attributes each for testing the trained networks.

386

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

5

4. Neural Network and PSO 4.1. A Neural Network Artificial neural networks are modeled after the computational principles of brain, with the specific aim of understanding and replicating human activities (Gruning and Sander, 2014) . A neural network can also be described as a system comprises of many simple processing elements operating in parallel whose function is determined by network structure, connection strength and the processing performed at computing element or nodes (Awodele, and Jegede, 2009). The artificial neural network is made up of three layers which are the input layer, hidden layer and output layer. These layers are connected by the weight called synaptic weight. The input layer is the non-processing layer of the network. This presents the data into the neural network. The hidden layer receives the product of the weighted summation of the input data and the bias. The hidden layer is made up of an activation function since it is referred to as a processing layer. The most commonly used activation function in neural network is called sigmoid function. It is commonly used because of its simplicity in its derivative and it soft switching properties. The output layer is also a processing layer that made up of activation function. The output layer determines the result of the neural network. Artificial neural networks have been used in many applications such as sales forecasting (Rene, et al. 2012). , industrial process control (Lu, and Tsai, 2006), customer research (Mark, et al. 2010), medical application (Zhenghao, and Lifeng, 2010) and risk management (Miglionico, and Parillo, 2012). In this research work, artificial neural network has been used to forecast hourly energy demand. The forecast is based on the previously data collected from the North Cyprus energy company. 4.2. Particle Swarm Optimization Particle swarm optimization belongs to a group of evolutionary computing approach referred to as swarm intelligence. Such approaches are inspired by flocks of birds, schools of fish and other similar bio-social behaviours found in nature (Reyes-Sierra, and Coello, 2006). The idea stems from the fact that when birds go randomly scouting for food, birds which are closest to found food leave signals for birds flocking behind to fly towards the found food (Abdullah, et al. 2014). In PSO, birds are translated as particles, signals as positions and velocities, and foods as the solutions. The positions and velocities are coordinates indicators of solutions and how fast particles should approach such solutions, iteratively (Rini, et al. 2011). The focus of the swarm intelligence involves particles (potential solutions) which fly through solution search space, following current particles with some pre-defined characteristics. Common characteristics include defining a fitness function for evaluating the current solutions to which the particles represents at determined intervals. Particles flock towards the particles with best fitness values each iteration, updating their positions and velocities, correspondingly. The positions of particles in search space represent solutions to modelled problems. PSO algorithms are capable of searching stochastically the whole solution space. This infers that PSO is capable of providing global optimization of modelled problems, in contrast to the back propagation algorithm which uses local gradient errors for learning, hence, often stuck in local minima. One key advantage of PSO over gradient descent based optimization algorithms is that PSO does not use local gradients; hence, problems with non-differentiable transfer functions can be optimized. In PSO, a number of particles are randomly initialized at start; initialized parameters include positions, velocities, individual best fitness values (pbest) and global best fitness value (gbest). The global best fitness value show the best fitness value achieved by any of the particles so far; individual best fitness values history for any particle are is stored (Kennedy, and Eberhart, 1995). A recent and more common version of PSO uses local best fitness values (lbest), in which particles are considered as belonging to a topological neighbourhood (sub-swarm); the size of neighbourhood is defined at initialization, and best fitness value achieved by a particle within any neighbourhood is referred to as the local best fitness value for particles belonging to that particular neighbourhood. This approach significantly reduces the chance of the PSO getting stuck in local minima.

6

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

387

The new positions, x (n), and velocities, v (n+1) of particles are updated iteratively using Equations 2 and 3

v(n  1)  w  v(n)  c1  r1( p(n)  x(n))  c2 * r 2( g (n)  x(n)) x ( n  1)  x ( n )  v ( n  1)

(2) (3)

Where, v(n) and x(n) are the current velocities and positions, respectively; w is the inertia weight, c1 is the local weight, r1 and r2 are random variables (range 0 to 1), c2 is the global weight, p(n) is the best position for each particle and g(n) is the best position achieved by any particle so far (Hajimirsadeghi, and Lucas, 2009). In this work, PSO is used to optimize the weights of feedforward networks for the short-term load forecasting problem. 5. Neural Network Training and Testing The designed networks are trained and tested on 1072 and 715 samples, respectively. The feedforward networks are all with 1 hidden layer, and 5 input neurons to accommodate the input attributes discussed in section III. The networks have 1 output neuron where expected loads for various hours of the days of the week are computed.

Fig. 2.Feedforward network topology

Fig. 2. Feedforward Topology

Fig. 2 shows the designed feedforward neural network with the 5 inputs described in section III; and 1 output, L/hr, which is the predicted load per hour for different times of the days of the week. The suitable number of hidden neurons, n, is determined heuristically in the training phase of the network; different number of hidden neurons is tried.The two approaches considered within this work, PSO and BPNN, for optimizing the weights of the feedforward network are described below. 5.1. Backpropagation Algorithm The system was trained with backpropagation neural network. Three different types of system structure were organized. These were done by choosing three best hidden neurons. These are 3, 5 and 7 hidden neuron. The learning rate and the momentum rate were varied until a value was reached which produce the optimal result for the network. These values were used on the other two networks. That is the hidden neurons were varied with learning rate and momentum rate at constant. The learning rate determines the learning power of the system and the momentum rate is the speed at which the network learns. The performance curve of the 7 hidden neurons is shown in the Fig. 3

388

Ozgur Cemal Ozerdem al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./etProcedia Computer Science 00 (2018) 000–000

7

Fig. 3.BPNN performance for the 7 hidden neurons

Table 1 shows the training parameters for the three networks with the use of BPNN for the 3 hidden neurons, 5 hidden neurons and 7 hidden neurons. Table 1. Training parameters for BPNN networks Parameter

BPNN1

BPNN2

No of training sample

1072

1072

BPNN3 1072

No of hidden neuron

3

5

7

Activation function

Sigmoid

Sigmoid

Sigmoid

Epochs

2000

2000

2000

Time(s)

27sec

35sec

48sec

Mean square error

0.01600

0.01628

0.01345

In testing the neural network to determine the network with the optimal result that has the lowest error rate. This is done by determining the minimum square error and the mean absolute square of the network. Table 2 shows the results that were obtained from the three networks considering the minimum square error (MSE) and the mean absolute error (MAE). Table 1 shows the achieved MAEs and MSEs for the three types of networks configuration. Table 2. Testing performance for BPNN networks Parameter

BPNN1

BPNN2

BPNN3

No. of test sample

715

715

715

Mean absolute error

7.22%

7.43%

5.5%

Mean squared error

1.6%

1.63%

1.35%

8

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

389

5.2. PSO Algorithm The PSO algorithm is initialized as discussed in section IV. Each initialization consists of 5 particles, using a neighbourhood size of 2. Furthermore, different number of hidden layer neurons is used as done in the back propagation algorithm. i.e. 3, 5 and 7 hidden neurons. The performance of PSO on a feedforward neural network with 5 hidden neurons is shown below.

Fig. 4.PSO performance for 5 hidden neurons

Table 3 shows the training parameters for the designed networks using 3 (PSO1), 5 (PSO2) and 7 (PSO3) neurons in the hidden layer. Table 3.Training parameters for PSO networks Parameters

PSO1

PSO2

PSO3

No. of training samples

1072

1072

1072

No. of hidden neurons

3

5

7

Activation functions

Sigmoid

Sigmoid

Sigmoid

Epochs

180

250

299

Time (secs)

13.90

20.93

21.23

Mean Square error

0.0097

0.0091

0.0106

From Table 3, it will be seen that the PSO2 (with 5 hidden neurons) achieved the lowest MSE on training in 20.93secs.

390

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

For, testing the PSO trained networks; the number of test samples used in the BPNN is also used (i.e. 715 samples). The performances of the trained networks on test data are assessed on achieved mean absolute error (MAE) and mean square error (MSE), as done with the BPNNs. Table 4 shows the achieved MAEs and MSEs for the different networks. Table 4.MAEs and MSEs for three different networks Parameter

PSO1

PSO2

PSO3

No. of test samples

715

715

715

Mean absolute error

8.88%

8.59%

9.38%

Mean squared error

16.74%

1.62%

1.83%

Runtime (secs)

0.08

0.10

0.15

9

10

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

391

Fig. 5.Plots of hourly loads for different days of the week

Fig. 5 shows the plot of actual loads, BPNN predicted loads and PSO predicted loads, using the test data. It can be seen that the prediction load curves for both BPNN and PSO closely follow that of the actual loads for all the days of the week. Since neural networks rely on interconnection weights, which serve as the long-term memory of such networks; different number of neurons in the hidden layer is experimented with as discussed earlier. Figure 6 shows the plot of achieved mean square error (MAE) against the number of neurons used in the hidden layer.

Fig. 6.MAE against number of hidden neurons

It can be seen from Figure 6 that 5 to 7 hidden neurons in the hidden layer is sufficient for learning the load forecasting task. Lowest MAEs are recorded with 7 and 5 hidden neurons for the BPNN and PSO, respectively. Also, the performances of the networks on achieved mean square error (MSE) are obtained. Figure 7 shows how the number of neurons in the hidden layer affects achieved MSE.

Fig. 7.MSE against number of hidden neurons

392

Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

11

It can be seen from Fig. 7 that lower MSEs are obtained with 5 to 7 neurons in the hidden layer for both BPNN and PSO. 6. Conclusion Load forecasting is a critical area of interest to energy companies. Such companies should be able to project loads to be supported at different times of the day for all the days of the week; this allows energy companies to inject sufficient energy into transmission grids. The failure for such preparedness in supporting required loads may lead to system collapse or compulsory power interruption to supplied power consumers. Furthermore, the ability to forecast energy demands with reasonable accuracies can be translated into tighter constrains on spinning reserves, which energy companies bare the cost unless used by consumers during huge unexpected energy demand. Different tools have been used to model the problem of load forecasting, with various degrees of successes. This work presents load forecasting based on particle swarm optimized feedforward neural network. Particle swarm optimization relies on communication between particles to allow the stochastic exploration of solution space for the problem. Also, for comparative analysis, feedforward neural networks are trained using the back propagation learning algorithm. The results obtained within this work show that both particle swarm optimized neural network and back propagation neural network are suitable for modelling load forecasting. It is observed during testing that the back propagation neural networks achieved slightly lower mean absolute errors and mean square errors, compared to the particle swarm optimized neural networks. However, it is also observed that the required times for training the back propagation networks are roughly twice of the particle swarm optimized networks. Hence, faster models can be developed with the particle swarm optimized networks with no much impact on mean absolute error and mean square error as compared to the back propagation neural network. Lastly, the authors are in support of improving the learning experiences of networks described within this work on load forecasting by incorporating other important input attributes or factors that impact on energy demand. Such factors include temperature and humidity at different times of the day. References Abdullah, A.G., Suranega, G.M., Hakim, D.L. (2014). Hybrid PSO-ANN Application for Improved Accuracy of Short Term Load Forecasting, WSEAS Transactions On Power Systems, 9(2014), 446-451 Alrashidi, M.R.El-Naggar,K.M. Al-othman,A.K. (2009).Particle Swarm Optimization Based Approach for Estimating the Fuel-Cost Function Parameters of Thermal Power Plants with Valve Loading Effect, Electric power components and System,37:1219-1230. Awodele, O. and Jegede, O. (2009). Neural Networks and Its Application in Engineering, Proceedings of Informing Science & IT Education Conference (InSITE), pp. 84-95. Azzam-ul-Asar, R. H. Syed, A. Khan, Long term electric load forecasting based on particle swarm optimization, Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, August 12-17, 2007 Bilgic, M.,Girep, C.P., Aslanoglu,S.Y.,Aydinalp-Koksal, M. (2010) Forecasting Turkey's short term hourly load with artificial neural networks, Transmission and Distribution Conference and Exposition, 2010 IEEE PES, pp. 1-7. Chen, J., Zhang, B., Wang,B., Qinglai, G. (2013) A Spinning Reserve Allocation Method for Power Generation, Dispatch Accommodating Large-Scale Wind Power Integration, Energies, 6: 5357-5381 Gruning, A and Sander, M.B. (2014). Spiking Neural Networks: Principles and Challenges, proceedings of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 23-25 Hajimirsadeghi, H. Lucas, C. (2009). A Hybrid Iwo/Pso Algorithm for Fast and Global Optimization, EUROCON, 2009. pp. 1964-1971. Ismet,E. Ali,O.(1997). Short term load forecasting using genetically optimized neural network cascaded with a modified Kohonen clustering process, Proceedings of the 12th IEEE International Symposium on Intelligent Control, 16-18 July 1997, Istanbul, Turkey. Jiang, D., (2015)Study on Short-Term Load Forecasting Method Based on the PSO and SVM model, International Journal of Control and Automation, 8: 181-188. Kennedy, J. Eberhart, R.C. 1995. Particle swarm optimization, in Proc. IEEE Int. Conference on Neural Networks, Piscataway, NJ, pp. 1942– 1948 Longlong, L., Dongmei,Z. (2013), Optimal Spinning Reserve for Power System with Wind Integrated, Energy and Power Engineering, 5: 10111015. Lu, C., Tsai, C. (2006). Predictive control using recurrent neural networks for industrial processes, Journal of process control, 22:83-92.

12



Ozgur Cemal Ozerdem et al. / Procedia Computer Science 120 (2017) 382–393 Ozgur Cemal Ozerdem et al./ Procedia Computer Science 00 (2018) 000–000

393

Mark, M.H.G., Fiona, D., Luiz, M.(2010).Determining Customer Satisfaction From Mobile Phones: A Neural Network Approach, Journal of Marketing Management, 21: 755-778. Miglionico, M.C., Parillo, F. An Application in Bank Credit Risk Management System employing A BP Neural Network based on sfloat24 custom math library using a low cost FPGA device, Advances in Computational Intelligence Communications in Computer and Information Science, Springer magazine, Vol. 300, pp. 84-93 Muhammad, B., Sanusi, S., (2012) Short-Term Load Forecasting Using Artificial Neural Network, Proceeding of International MultiConference of Engineers and Computer Scientists, Vol 1 Rene, E.R., Kumar BeheraS. and Suck Park, H. (2012). Predicting Adsorption Behavior in Engineered Floodplain Filtration System Using Backpropagation Neural Networks, Machine Learning Algorithms for Problem Solving in Computational Applications: Intelligent Techniques, (2012) 179-194. Reyes-Sierra, M., CoelloCoello, C.A. (2006). Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art, International Journal of Computational Intelligence Research, 2: 287–308. Rini, D.P., Shamsuddin, S.M., Yuhaniz, S.S. (2011). Particle Swarm Optimization: Technique, System and Challenges, International Journal of Computer Applications,14: 19-27 Subhes,C.B, Le,T.T (2004) Short-term electric load forecasting using an artificial neural network: case of Northern Vietnam, International Journal of Energy Research, 28: 463-472. Zhenghao, S., Lifeng, H., (2010). Application of Neural Networks in Medical Image Processing, Proceedings of the Second International Symposium on Networking and Network Security, 2-4, pp. 023-026, China.