- Email: [email protected]
Application of artificial neural network with extreme learning machine for economic growth estimation
Accepted Manuscript Application of artificial neural network with extreme learning machine for economic growth estimation Ljubiša Milaˇci´c, Srdan ¯ Jovi´c, Tanja Vujovi´c, Jovica Miljkovi´c PII: DOI: Reference:
S0378-4371(16)30557-X http://dx.doi.org/10.1016/j.physa.2016.08.040 PHYSA 17457
To appear in:
Physica A
Received date: 20 July 2016 Revised date: 8 August 2016 Please cite this article as: L. Milaˇci´c, S. Jovi´c, T. Vujovi´c, J. Miljkovi´c, Application of artificial neural network with extreme learning machine for economic growth estimation, Physica A (2016), http://dx.doi.org/10.1016/j.physa.2016.08.040 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Highlights (for review)
Highlights o The estimation of the gross domestic product (GDP) growth rate. o Economic growth basis on combination of different factors. o The accuracy of the extreme learning machine (ELM).
*Manuscript Click here to view linked References
Application of artificial neural network with extreme learning machine for economic growth estimation Ljubiša Milačić1, Srđan Jović2*, Tanja Vujović3, Jovica Miljković3 1.... High school of business science in Blace, Kralja Petra I 70, 18420 Blace, Serbia 2.... University of Priština, Faculty of Technical Sciences, 38220 Kosovska Mitrovica, Kneza Milosa 7, Serbia 3.... University of Priština, Faculty of Economic Science, Kosovska Mitrovica, Kolašinska 156, 38220 Kosovska Mitrovica, Serbia *
Correspondent author: Email address: [email protected], Tel: (+98) 916621955642
Abstract The purpose of this research is to develop and apply the artificial neural network (ANN) with extreme learning machine (ELM) to forecast gross domestic product (GDP) growth rate. The economic growth forecasting was analyzed based on agriculture, manufacturing, industry and services value added in GDP. The results were compared with ANN with back propagation (BP) learning approach since BP could be considered as conventional learning methodology. The reliability of the computational models were accessed based on simulation results and using several statistical indicators. Based on results, it was shown that ANN with ELM learning methodology can be applied effectively in applications of GDP forecasting. Keywords: GDP; Forecasting; Extreme Learning Machine; Economic
1. Introduction Gross Domestic Product (GDP) growth rate could be considered as essential ingredient of a healthy economy. There are many factors which have different influence on the economic growth. GDP growth results from the synthesis influence of various known or unknown and certain or uncertain factors. For the growing economy to be stable and well-behaved, it is required that the modern and traditional sectors should be substitutes and not complements [1]. Increasing of production complexity has an ambiguous effect on the level of output, but positively impacts economic growth by enhancing human capital formation [2]. A forecasting system of economic growth was proposed and software with related applications was developed [3]. Artificial neural network (ANN) achieved better results in performance and efficiency compared to conventional methods [3]. There was a positive correlation between the number of forecasters covering a given country and the forecast accuracy [4]. The GDP forecast accuracy was improved with progress in transition as well as with the expansion in the information domain [5]. The relative performance 1
of several factor models to forecast GDP growth using a large monthly dataset was analyzed [6]. The GDP growth was considered as a natural-growth process amenable to description by the logistic-growth equation [7]. There is need for more advances algorithm for GDP rate prediction. In this study the main aim was to overcome high nonlinearity of the GDP rate prediction since it is an endogenous variable by applying the ANN with extreme learning method (ELM). ELM was introduced as a learning algorithm for single layer feed forward neural network [8, 9]. It is capable to solve problems caused by gradient descent based algorithms like back propagation and to decrease required time for ANN training [10-12]. Therefore in this study was compare ANN models with ELM learning and ANN model with back propagation (BP) learning since BP learning could be considered as conventional learning approach.
2. Methodology 2.1.
Economic parameters
Many parameters affect GDP growth. In this investigation four factors were selected. The first input parameter was agriculture, which includes forestry, hunting, and fishing, as well as cultivation of crops and livestock production. The second input parameter was manufacturing which refers to value added as the net output of a sector after adding up all outputs and subtracting intermediate inputs. The third input parameter was industry which corresponds to the value added in mining, manufacturing (also reported as a separate subgroup), construction, electricity, water, and gas. The fourth input parameter was services, which correspond to value added in wholesale and retail trade, transport, and government, financial, professional, and personal services such as education, health care, and real estate services. Table 1 shows input parameters which are used in this investigation for the GDP growth rate prediction.
input 1 input 2 input 3 input 4
2.2.
Table 1: Input parameters Agriculture Added value in GDP (%) Manufacturing Added value in GDP (%) Industry Added value in GDP (%) Services Added value in GDP (%)
Extreme learning machine for neural networks
Extreme learning machine (ELM) was developed as a learning algorithm for single hidden layer feed forward networks (SLFNs) [8, 9]. This approach has several advantages as compared with conventional learning algorithms like back propagation (BP) learning algorithm. These advantages could be fast training and good generalization. 2
ELM was applied for SLFN with three layers. The hidden layer is standard single hidden layer with hidden nodes and activation function. There were weight vectors between hidden layer and output layer. Figure 1 illustrates a simple flow chart of the prediction procedure. Table 2 shows the user defined parameters ELM and BP learning algorithms for ANN.
Input parameters
Agriculture
Manufacturing
ANN training
ELM
BP
Prediction errors
Comprative study
Industry
Services
Figure 1: Flowchart of the ANN prediction procedure Table 2: Parameters of the ELM and BP learning algorithms for ANN ELM
BP
Number of layers
3
Neurons
Input: 4 Hidden: 3, 6, 10 Output: 1
3
Neurons
Input: 4 Hidden: 3, 6, 10 Output: 1
-
-
Number of iteration
1000
-
-
Activation function
Sigmoid Function
Learning rule
BP
Learning rule ELM
2.3.
Number of layers
Models Performance Evaluation
Predictive performances of the proposed ANN models were compare by the root means square error (RMSE), coefficient of determination (R2) and Pearson coefficient (r) indicators. These statistics are defined as follows: 1) Root-mean-square error (RMSE),
3
n
(P O )
RMSE
i 1
i
2
i
(1)
,
n
2) Pearson correlation coefficient (r),
n n n n Oi Pi Oi Pi i 1 i 1 i 1 r 2 2 n 2 n n 2 n n Oi Oi n Pi Pi i 1 i 1 i 1 i 1
(2)
3) Coefficient of determination (R2)
n Oi Oi Pi Pi R 2 ni 1 n Oi Oi Pi Pi
i 1
i 1
2
(7)
Oi and Pi represent the forecasted and experimental values of GDP growth rate, respectively and n denotes the sum of test data.
3. Results and discussion Figure 2 shows the prediction accuracy of the GDP growth rate with ANN models with ELM and BP learning algorithms. One can note higher coefficient of determination for the ELM learning algorithm in than BP learning algorithm. The number of underestimated values is very small for ELM learning approach. The prediction accuracy of the ANN model with BP learning algorithm has more underestimated values. Table 3 summarizes the forecasting accuracy results of the two learning approaches for the ANN based on the three statistical indicators where it was proved that ELM model outperformed the BP model according to the all statistical indicators.
4
ELM
BP 12
Forecasted values of GDP growth rate [%]
Forecasted values of GDP growth rate [%]
15 10 5 0 -5 -10
y = 0.734x + 0.5003 R² = 0.73
-15
Actual values of GDP growth rate [%]
10 8 6 4 2 -20
-10
0 -2 0
10
20
-4 -6 -8
y = 0.4598x + 1.0164 R² = 0.45
-10
Actual values of GDP growth rate [%]
Figure 2: GDP growth rate prediction accuracy with: (a) ELM and (b) BP learning algorithm Table 3: GDP growth rate prediction accuracy with ANN models with ELM and BP learning algorithm ELM BP 2 RMSE R r RMSE R2 r 1.97 0.73 0.85 2.81 0.45 0.67
4. Conclusion In present study, prediction of GDP growth rate was described by using the ANN approach with ELM learning algorithm that showcases its benefits to predict GDP growth rate in comparison with ANN model with BP learning algorithm. Agriculture, manufacturing, industry and services as added values in GDP were used as inputs for GDP growth rate forecasting. Accuracy level of predicted values was assessed in comparison to the two learning algorithms, ELM and BP. The simulation results revealed that ANN model with ELM learning algorithm was able to predict GDP favorably based on the used inputs. The ELM algorithm can be effectively utilized in GDP applications and particularly in the GDP estimations.
References [1] [2]
Joseph Zeira, Hosny Zoabi, Economic growth and sector dynamics, European Economic Review 79 (2015) 1–15. Benno Ferrarini, Pasquale Scaramozzino, Production complexity, adaptability and economic growth, Structural Change and Economic Dynamics 37 (2016) 52–61. 5
[3] [4] [5] [6] [7] [8]
[9]
[10]
[11]
[12]
Lihua Feng, Jianzhen Zhang, Application of artificial neural networks in tendency forecasting of economic growth, Economic Modelling 40 (2014) 76–80. Libor Krkoska, Utku Teksoz, How reliable are forecasts of GDP growth and inflation for countries with limited coverage?, Economic Systems 33 (2009) 376–388. Libor Krkoska, Utku Teksoz, Accuracy of GDP growth forecasts for transition countries: Ten years of forecasting assessed, International Journal of Forecasting 23 (2007) 29–45. Francisco Dias, Maximiano Pinheiro, António Rua, Forecasting Portuguese GDP with factor models: Pre- and post-crisis evidence, Economic Modelling 44 (2015) 266–272. Theodore Modis, Long-term GDP forecasts and the prospects for growth, Technological Forecasting & Social Change 80 (2013) 1557–1562. Huang, G.B., Zhu, Q.Y., Siew, C.K., 2004. Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks. International Joint Conference on Neural Networks 2, 985‒990. Wang, D., Huang, G.B., 2005. Protein Sequence Classification Using Extreme Learning Machine. Proceedings of International Joint Conference on Neural Networks 3, 1406‒1411. Huang, G.B., Zhu, Q.Y., Siew, C.K., 2003. Real-time learning capability of neural networks. Technical Report ICIS/45/2003, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. Nian, R., He, B., Zheng, B., Heeswijk, M.V., Yu, Q., Miche, Y., et al. 2014. Extreme learning machine towards dynamic model hypothesis in fish ethology research. Neurocomputing 128, 273-284. Abdullah, S.A., Malek, A.A., Abdullah, N.S., Kisi, O., Yap, K.S., 2015. Extreme Learning Machines: A new approach for prediction of reference evapotranspiration. Journal of Hydrology 527, 184-195.
6
S0378-4371(16)30557-X http://dx.doi.org/10.1016/j.physa.2016.08.040 PHYSA 17457
To appear in:
Physica A
Received date: 20 July 2016 Revised date: 8 August 2016 Please cite this article as: L. Milaˇci´c, S. Jovi´c, T. Vujovi´c, J. Miljkovi´c, Application of artificial neural network with extreme learning machine for economic growth estimation, Physica A (2016), http://dx.doi.org/10.1016/j.physa.2016.08.040 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Highlights (for review)
Highlights o The estimation of the gross domestic product (GDP) growth rate. o Economic growth basis on combination of different factors. o The accuracy of the extreme learning machine (ELM).
*Manuscript Click here to view linked References
Application of artificial neural network with extreme learning machine for economic growth estimation Ljubiša Milačić1, Srđan Jović2*, Tanja Vujović3, Jovica Miljković3 1.... High school of business science in Blace, Kralja Petra I 70, 18420 Blace, Serbia 2.... University of Priština, Faculty of Technical Sciences, 38220 Kosovska Mitrovica, Kneza Milosa 7, Serbia 3.... University of Priština, Faculty of Economic Science, Kosovska Mitrovica, Kolašinska 156, 38220 Kosovska Mitrovica, Serbia *
Correspondent author: Email address: [email protected], Tel: (+98) 916621955642
Abstract The purpose of this research is to develop and apply the artificial neural network (ANN) with extreme learning machine (ELM) to forecast gross domestic product (GDP) growth rate. The economic growth forecasting was analyzed based on agriculture, manufacturing, industry and services value added in GDP. The results were compared with ANN with back propagation (BP) learning approach since BP could be considered as conventional learning methodology. The reliability of the computational models were accessed based on simulation results and using several statistical indicators. Based on results, it was shown that ANN with ELM learning methodology can be applied effectively in applications of GDP forecasting. Keywords: GDP; Forecasting; Extreme Learning Machine; Economic
1. Introduction Gross Domestic Product (GDP) growth rate could be considered as essential ingredient of a healthy economy. There are many factors which have different influence on the economic growth. GDP growth results from the synthesis influence of various known or unknown and certain or uncertain factors. For the growing economy to be stable and well-behaved, it is required that the modern and traditional sectors should be substitutes and not complements [1]. Increasing of production complexity has an ambiguous effect on the level of output, but positively impacts economic growth by enhancing human capital formation [2]. A forecasting system of economic growth was proposed and software with related applications was developed [3]. Artificial neural network (ANN) achieved better results in performance and efficiency compared to conventional methods [3]. There was a positive correlation between the number of forecasters covering a given country and the forecast accuracy [4]. The GDP forecast accuracy was improved with progress in transition as well as with the expansion in the information domain [5]. The relative performance 1
of several factor models to forecast GDP growth using a large monthly dataset was analyzed [6]. The GDP growth was considered as a natural-growth process amenable to description by the logistic-growth equation [7]. There is need for more advances algorithm for GDP rate prediction. In this study the main aim was to overcome high nonlinearity of the GDP rate prediction since it is an endogenous variable by applying the ANN with extreme learning method (ELM). ELM was introduced as a learning algorithm for single layer feed forward neural network [8, 9]. It is capable to solve problems caused by gradient descent based algorithms like back propagation and to decrease required time for ANN training [10-12]. Therefore in this study was compare ANN models with ELM learning and ANN model with back propagation (BP) learning since BP learning could be considered as conventional learning approach.
2. Methodology 2.1.
Economic parameters
Many parameters affect GDP growth. In this investigation four factors were selected. The first input parameter was agriculture, which includes forestry, hunting, and fishing, as well as cultivation of crops and livestock production. The second input parameter was manufacturing which refers to value added as the net output of a sector after adding up all outputs and subtracting intermediate inputs. The third input parameter was industry which corresponds to the value added in mining, manufacturing (also reported as a separate subgroup), construction, electricity, water, and gas. The fourth input parameter was services, which correspond to value added in wholesale and retail trade, transport, and government, financial, professional, and personal services such as education, health care, and real estate services. Table 1 shows input parameters which are used in this investigation for the GDP growth rate prediction.
input 1 input 2 input 3 input 4
2.2.
Table 1: Input parameters Agriculture Added value in GDP (%) Manufacturing Added value in GDP (%) Industry Added value in GDP (%) Services Added value in GDP (%)
Extreme learning machine for neural networks
Extreme learning machine (ELM) was developed as a learning algorithm for single hidden layer feed forward networks (SLFNs) [8, 9]. This approach has several advantages as compared with conventional learning algorithms like back propagation (BP) learning algorithm. These advantages could be fast training and good generalization. 2
ELM was applied for SLFN with three layers. The hidden layer is standard single hidden layer with hidden nodes and activation function. There were weight vectors between hidden layer and output layer. Figure 1 illustrates a simple flow chart of the prediction procedure. Table 2 shows the user defined parameters ELM and BP learning algorithms for ANN.
Input parameters
Agriculture
Manufacturing
ANN training
ELM
BP
Prediction errors
Comprative study
Industry
Services
Figure 1: Flowchart of the ANN prediction procedure Table 2: Parameters of the ELM and BP learning algorithms for ANN ELM
BP
Number of layers
3
Neurons
Input: 4 Hidden: 3, 6, 10 Output: 1
3
Neurons
Input: 4 Hidden: 3, 6, 10 Output: 1
-
-
Number of iteration
1000
-
-
Activation function
Sigmoid Function
Learning rule
BP
Learning rule ELM
2.3.
Number of layers
Models Performance Evaluation
Predictive performances of the proposed ANN models were compare by the root means square error (RMSE), coefficient of determination (R2) and Pearson coefficient (r) indicators. These statistics are defined as follows: 1) Root-mean-square error (RMSE),
3
n
(P O )
RMSE
i 1
i
2
i
(1)
,
n
2) Pearson correlation coefficient (r),
n n n n Oi Pi Oi Pi i 1 i 1 i 1 r 2 2 n 2 n n 2 n n Oi Oi n Pi Pi i 1 i 1 i 1 i 1
(2)
3) Coefficient of determination (R2)
n Oi Oi Pi Pi R 2 ni 1 n Oi Oi Pi Pi
i 1
i 1
2
(7)
Oi and Pi represent the forecasted and experimental values of GDP growth rate, respectively and n denotes the sum of test data.
3. Results and discussion Figure 2 shows the prediction accuracy of the GDP growth rate with ANN models with ELM and BP learning algorithms. One can note higher coefficient of determination for the ELM learning algorithm in than BP learning algorithm. The number of underestimated values is very small for ELM learning approach. The prediction accuracy of the ANN model with BP learning algorithm has more underestimated values. Table 3 summarizes the forecasting accuracy results of the two learning approaches for the ANN based on the three statistical indicators where it was proved that ELM model outperformed the BP model according to the all statistical indicators.
4
ELM
BP 12
Forecasted values of GDP growth rate [%]
Forecasted values of GDP growth rate [%]
15 10 5 0 -5 -10
y = 0.734x + 0.5003 R² = 0.73
-15
Actual values of GDP growth rate [%]
10 8 6 4 2 -20
-10
0 -2 0
10
20
-4 -6 -8
y = 0.4598x + 1.0164 R² = 0.45
-10
Actual values of GDP growth rate [%]
Figure 2: GDP growth rate prediction accuracy with: (a) ELM and (b) BP learning algorithm Table 3: GDP growth rate prediction accuracy with ANN models with ELM and BP learning algorithm ELM BP 2 RMSE R r RMSE R2 r 1.97 0.73 0.85 2.81 0.45 0.67
4. Conclusion In present study, prediction of GDP growth rate was described by using the ANN approach with ELM learning algorithm that showcases its benefits to predict GDP growth rate in comparison with ANN model with BP learning algorithm. Agriculture, manufacturing, industry and services as added values in GDP were used as inputs for GDP growth rate forecasting. Accuracy level of predicted values was assessed in comparison to the two learning algorithms, ELM and BP. The simulation results revealed that ANN model with ELM learning algorithm was able to predict GDP favorably based on the used inputs. The ELM algorithm can be effectively utilized in GDP applications and particularly in the GDP estimations.
References [1] [2]
Joseph Zeira, Hosny Zoabi, Economic growth and sector dynamics, European Economic Review 79 (2015) 1–15. Benno Ferrarini, Pasquale Scaramozzino, Production complexity, adaptability and economic growth, Structural Change and Economic Dynamics 37 (2016) 52–61. 5
[3] [4] [5] [6] [7] [8]
[9]
[10]
[11]
[12]
Lihua Feng, Jianzhen Zhang, Application of artificial neural networks in tendency forecasting of economic growth, Economic Modelling 40 (2014) 76–80. Libor Krkoska, Utku Teksoz, How reliable are forecasts of GDP growth and inflation for countries with limited coverage?, Economic Systems 33 (2009) 376–388. Libor Krkoska, Utku Teksoz, Accuracy of GDP growth forecasts for transition countries: Ten years of forecasting assessed, International Journal of Forecasting 23 (2007) 29–45. Francisco Dias, Maximiano Pinheiro, António Rua, Forecasting Portuguese GDP with factor models: Pre- and post-crisis evidence, Economic Modelling 44 (2015) 266–272. Theodore Modis, Long-term GDP forecasts and the prospects for growth, Technological Forecasting & Social Change 80 (2013) 1557–1562. Huang, G.B., Zhu, Q.Y., Siew, C.K., 2004. Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks. International Joint Conference on Neural Networks 2, 985‒990. Wang, D., Huang, G.B., 2005. Protein Sequence Classification Using Extreme Learning Machine. Proceedings of International Joint Conference on Neural Networks 3, 1406‒1411. Huang, G.B., Zhu, Q.Y., Siew, C.K., 2003. Real-time learning capability of neural networks. Technical Report ICIS/45/2003, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. Nian, R., He, B., Zheng, B., Heeswijk, M.V., Yu, Q., Miche, Y., et al. 2014. Extreme learning machine towards dynamic model hypothesis in fish ethology research. Neurocomputing 128, 273-284. Abdullah, S.A., Malek, A.A., Abdullah, N.S., Kisi, O., Yap, K.S., 2015. Extreme Learning Machines: A new approach for prediction of reference evapotranspiration. Journal of Hydrology 527, 184-195.
6