Application of neural networks for the prediction of hourly mean surface temperatures in Saudi Arabia

Renewable Energy 25 (2002) 545–554 www.elsevier.nl/locate/renene

Application of neural networks for the prediction of hourly mean surface temperatures in Saudi Arabia Imran Tasadduq a, Shafiqur Rehman a

b

b,*

, Khaled Bubshait

a

Center for Economics and Management Systems, Research Institute, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia Center for Engineering Research, Research Institute, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia Received 30 November 2000; accepted 27 February 2001

Abstract This paper utilizes artificial neural networks for the prediction of hourly mean values of ambient temperature 24 h in advance. Full year hourly values of ambient temperature are used to train a neural network model for a coastal location — Jeddah, Saudi Arabia. This neural network is trained off-line using back propagation and a batch learning scheme. The trained neural network is successfully tested on temperatures for years other than the one used for training. It requires only one temperature value as input to predict the temperature for the following day for the same hour. The predicted hourly temperature values are compared with the corresponding measured values. The mean percent deviation between the predicted and measured values is found to be 3.16, 4.17 and 2.83 for three different years. These results testify that the neural network can be a valuable tool for hourly temperature prediction in particular and other meteorological predictions in general.  2001 Elsevier Science Ltd. All rights reserved. Keywords: Neural networks; Back propagation; Prediction; Temperature; Meteorology; Batch learning; Pattern learning

* Corresponding author. Address for correspondence: KFUPM Box No. 767, Dhahran 3126, Saudi Arabia. Tel.: +966-3-860-6129/3802; fax: +966-3-860-3996. E-mail address: [email protected] (S. Rehman). 0960-1481/02/$ - see front matter  2001 Elsevier Science Ltd. All rights reserved. PII: S 0 9 6 0 - 1 4 8 1 ( 0 1 ) 0 0 0 8 2 - 9

546

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

1. Introduction A thorough knowledge of the climatic parameters such as temperature, relative humidity, pressure, rainfall, visibility, etc. is of importance for the understanding of total energy available for use by a system. The knowledge of the variability of surface ambient temperature, both in temporal and spatial domains, is of importance in weather forecasting, surface budget study [1], total solar radiation estimation [2– 4], cooling and heating degree-days calculations [5,6], micrometeorological studies, initialization of planetary boundary models, calculation of thermal load on buildings [7–9], air pollution studies [10], and upper air heating rate calculations [11]. The temporal variation of temperature can be obtained by using measured values and developing physical or empirical models [12]. Njau [13] presented an electronic system for the prediction of air temperature. Boland [14] used time series analysis for the prediction of weather paramenters in general and air temperature in particular. In a recent study, Njau [15] developed a new analytical model for the prediction of temperature. More work on the prediction of surface air temperature in particular and other weather parameters can be seen in Njau [16,17]. Artificial neural networks (NNs), which are increasingly receiving attention in solving complex practical problems, are known as universal function approximators. They are capable of approximating any continuous nonlinear functions to arbitrary accuracy [18]. A feedforward multilayered NN can approximate a continuous function arbitrarily well. The other advantages inherent in NNs are their robustness, parallel architecture and fault tolerant capability. A typical NN is shown in Fig. 1. The behavior of an artificial NN depends on both the weights and the input–output function that is specified for the units. This function typically falls into one of three categories: linear, threshold or sigmoid. For linear units, the output activity is proportional to the total weighted input. For threshold units, the output is set at one of two levels, depending on whether the total input is greater than or less than some

Fig. 1.

A typical neural network structure.

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

547

threshold value. For sigmoid units, the output varies continuously but not linearly as the input changes [19]. Hunt and Sbarbaro [20] have explored the use of artificial NNs for nonlinear IMC (internal model control). In doing so, the authors have studied the invertibility of a class of nonlinear dynamical systems and derived an invertibility characterization. Sotirov and Shafai [21] have used NNs for interval identification of nonlinear systems, which is a new way of looking at such problems and may be helpful in arriving at a more tractable methodology. A hybrid econometric NN model for total monthly sales forecasting is presented in [22]. The model combines the structural characteristics of econometric models with the nonlinear pattern recognition features of NNs. The model produces an aggregate forecast by filtering forecasts from sequential stages and subsequently averaging the outputs from each stage. The performance of the model is assessed over a period of 6 months. Wind speed prediction using NNs has been studied in [23]. Through simulation studies on real data it has been shown that the NN approach outperforms the AR (autoregressive) modeling technique in predicting the wind speed. In another paper, the same authors (Mohandes et al. [24]) have successfully used NNs for estimation of global solar radiation. In a recent paper, Mohandes et al. [25] used a radial basis function for the estimation of monthly mean daily global solar radiation. They used monthly mean daily values of global solar radiation from 31 locations to train the artificial NNs. Data for 10 locations were used to test the performance of the NN model output. The results so obtained indicated the viability of the radial basis function for this kind of problem. Alexiadis et al. [26] used artificial NNs for the prediction of wind speed at six locations on the islands of the south and central Aegean Sea in Greece. They reported that the best results were obtained from artificial NNs compared to ARMA models. Other applications of this technique in renewable energy related topics can be found in Refs. [27–31]. Artificial NNs have also been used in the modeling of solar domestic water heating systems, as reported by Kalogirou et al. [32]. The authors used the artificial NN method to predict the useful energy extracted and the temperature rise in the stored water of a solar domestic water heating (SDHW) system with a minimum data input. In this study, the NNs were trained using known data from 30 case studies for collector areas varying from 1.81 to 4.38 m2. The predictions obtained from this technique were found to have an error varying from 7.1 to 9.7% when compared with the actual values. Other applications of the artificial NN can be found in Refs. [33–36]. NNs have also been used for classifying noisy patterns in magnetic storage devices. Nair and Moon [37] have presented methods for designing a feedforward NN detector employing knowledge of the magnetic channel characteristics. The network’s performance has been compared with the conventional linear equalizer in a magnetic recording channel subject to signal dependent noise and nonlinear intersymbol interference. The control of nonlinear plants has been studied in Refs. [20,38,39]. In Ref. [39], NN identification and control of nonlinear plants are discussed in detail. Dynamic backpropagation has been introduced for training the NNs. Means of utiliz-

548

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

ing both multi-layer and recurrent networks have been given. Methods have also been discussed for using NNs as explicit self-tuning controllers. In this paper, we present a NN based approach for predicting the ambient temperatures 24 h in advance. Section 2 describes this approach in detail while the results are discussed in Section 3.

2. Neural networks NNs, which are increasingly receiving attention in solving complex practical problems, are known as universal function approximators. They are capable of approximating any continuous nonlinear functions to arbitrary accuracy [18]. Consider a dynamic nonlinear relationship described by, y(t)⫽f{f(t)},

(1)

where y(t) is the output and f(t) is a nf×1 vector composed of the past output and input, nf is the number of elements in vector f(t) and f(.) is a nonlinear function that represents the dynamic relationship. A feedforward NN may be trained to assume the function f(.). Upon completion of training, the NN can express the nonlinear relationship. The feedforward NN is assumed to have multiple layers including one or more hidden layers. Units in each layer feed the units into the forward layer through a set of weights and a nonlinear differentiable activation function. Each activation function may also have a bias term. The nw×1 vector w is assumed to contain all the concatenated weights of the network (where nw is the number of elements in vector w). We further denote the output of the feedforward NN for a given weight w and input pattern f(t) as: NN{w, f(t)}.

(2)

In the present study, the NN is trained off-line using historical temperature data. If the NN were trained on-line, using recent data during forecasting, then it would be able to take care of the nonstationarity. In our case, since the deviation between the estimated and measured values was so small, such on-line training was not required.

3. Neural net training Training of a NN involves determination of weights that would minimize an error function that is based on the actual and desired outputs. In other words, training consists of adjusting the weights of the network such that the network replicates the observed phenomenon. The error is calculated using Eqs. (3) and (4). Further, at each time instant when the weights are updated, this error is computed. When the error reaches an acceptable steady-state value, training is stopped. As can be seen from Fig. 2(b), the error fails to improve after 60 sweeps. However, the value of this steady-state error will vary from problem to problem, depending upon the linearity of the function. This means, in some cases it may turn out to be very low and in some

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

549

Fig. 2. (a) Desired output and NN output after training; (b) average sum-squared error during training of the NN; (c), (e) and (g) desired and NN forecasted hourly temperatures for three different years; (d), (f) and (h) percentage error between the desired and NN forecasted temperature for three different years.

cases it may be unacceptably high, suggesting that the NN did not learn the nonlinearity well enough. In the feedforward (equation error) training the network is trained by minimizing the sum-squared error defined as:

冘 q

J⫽

E(t),

(3)

t⫽1

where E(t)⫽12储ef(t)储2⫽12储y(t)⫺yˆ (t)储2,

(4)

550

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

yˆ (t)⫽NN{w, f(t)},

(5)

and q is the total number of available data and E(t) is the sum-squared error. Training may be classified into batch learning and pattern learning [40]. In the batch learning, the weights of the NN are adjusted after a complete sweep of the entire training data. In pattern learning the weights are updated at every time step in an on-line fashion. Batch learning has a better mathematical validity in the sense that it exactly implements the gradient descent method. Pattern learning algorithms are usually derived as approximations of batch learning. However, it has the advantage of on-line implementation.

4. Batch learning The back propagation algorithm is the most widely used method in NN training. In this approach, the training is based on a simple-minded gradient scheme. Back propagation provides a convenient method to compute the gradient using a chain rule. The back propagation algorithm for the weight update is given as

冋 册

wk⫽wk−1⫺hb where

⌫b(t)⬅

冘

∂+J T ⫽wk−1⫹hb ⌫b(t)ef(t), ∂wk−1 t⫽1 q

(6)

冋 册

∂+yˆ (t) T ; (an nw⫻ny matrix), ∂wk−1

ef(t)⬅y(t)⫺NN{w, f(t)},

(7)

where hb denotes the batch learning rate, T denotes transpose and the plus sign indicates the ordered partial derivative in the network [41]. The ordered derivative can be computed conveniently by the back propagation network [42]. However, instead of backpropagating the error e(t) as normally done, we backpropagate a one through the network. This facilitates the derivative calculation. The learning rate hb can be either a matrix or a scalar. Usually a matrix-learning rate uses higher order information and converges faster. The matrix-learning rate in this case is given by hb⫽m(eI⫹H)−1; 0ⱕmⱕ2 where

冘⌫ q

H⫽

b

(t)⌫Tb(t),

t⫽1

m is a parameter that affords additional flexibility and e is a small positive quantity added to deal with the near-singular matrix H. It can be shown that in the vicinity of the optimal weight wo, the optimal value of m becomes one. However, when the weights are away from wo and/or in the presence of strong nonlinearities, the lin-

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

551

earized approximation used by the Gauss–Newton algorithm becomes poor. Based on this consideration, at times a conservative (small) value of m may yield a better result. The following variable m may be used for this purpose m(k)⫽m⬁⫺mr[m⬁⫺m(k⫺1)]; m(0)ⱕm⬁ⱕ1,

(8)

where m(0) is the initial value of m, m⬁ is the final value of m and mr, which normally ranges between 0.9 and 1, controls the rate of transition from m(0) to m⬁. 5. Pattern learning The back propagation algorithm for the pattern learning is given as:

冋

w(t)⫽w(t⫺1)⫺hp(t) where

⌫p(t)⬅

冋

∂+yˆ (t) ∂w(t−1)

册

册

∂+E(t) T ⫽w(t⫺1)⫹hp(t)⌫p(t)ef(t), ∂w(t−1)

T

and ef(t)⬅y(t)⫺NN{w(t⫺1), f(t)},

and hp(t) is the pattern learning rate. Application of the Gauss–Newton algorithm provides an expression for the matrix hp(t).

6. Results and discussion The approach consists of two steps. First, a NN is trained off-line using a backpropagation and batch learning scheme. Second, the trained NN is used to forecast hourly temperatures involving data other than those used for training. The former step can be called training while the latter is called validation. After successful validation, the NN is ready to be used for forecasting hourly temperatures for the location it is trained for. For training the NN, we have used 1 year’s hourly temperature values (365×24=8760 data points) of a coastal location, Jeddah, Saudi Arabia. The NN output is compared with temperature values that are 24 h ahead of what is fed to its input. A NN with one input, one output and one hidden layer has been trained to model these data. The number of hidden elements are obtained by trial and error. So far, no mathematically justifiable method is available for determining the hidden elements. As explained by Haykin [43], training is started with a minimum number of elements. The number of these elements is constantly increased and re-training of the NN is continued until satisfactory training is achieved. The number of hidden elements used for satisfactory training is considered as the optimal number. There is one neuron each in the input and output layers while there are four neurons in the hidden layer. Batch learning has been performed along with the matrixlearning rate given in Section 3. The parameters e, m(0), mr and m⬁ in Eq. (8) are

552

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

10, 0.01, 0.9 and 0.1, respectively. The actual and predicted outputs after the training are shown in Fig. 2(a) while the average sum-squared training error is displayed as a function of the sweep iteration in Fig. 2(b). The root mean square error (RMSE) is found to be 1.75 after reaching the steady state. The horizontal axes in Fig. 2(a– h), except 2(b), indicate the time in hours for which the predictions were made; in this case the time is 50 h, which can be extended to any number of hours. Training of the NN is validated by using data from three different years excluding the one used for training. It is found that the NN is able to forecast hourly temperatures 24 h in advance with sufficient accuracy. The predicted hourly temperatures are compared with measured ones for all the three years and the results are shown in Fig. 2(c, e, g). The corresponding percentage error between the predicted and measured values is plotted in Fig. 2(d, f, h). For the year shown in Fig. 2(c, d), the percentage error between predicted and measured temperatures is well within 5%, except for very few instances where the error crosses the 5% limit. The mean percentage error in this case is 3.16. Another year’s results are shown in Fig. 2(e, f). In this case, the percentage error deviates from the 5% interval more often than in the previous case. However, the mean percentage deviation is only 4.17. Fig. 2(g, h) show validation for a third year. Here the percentage error is well within 5% and its mean value is 2.83.

Acknowledgements The authors wish to acknowledge the support of the Research Institute, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia.

References [1] Mahrer Y, Pielke RA. A numerical study of the airflow over irregular terrain. Beitra¨ ge zur Phusik der Atmospha¨ re 1977;50:98–113. [2] Sabbagh JA, Sayigh AAM, El-Salam EMA. Estimation of the Total Solar Radiation from Meteorological Data. Solar Energy 1977;19:307–11. [3] Barra OA. Estimation of the total solar radiation from meteorological data in the Italian climate area. Solar Energy 1983;31(4):427–8. [4] Hussain M. Estimation of global and diffuse irradiation from sunshine duration and atmospheric water vapour content. Solar Energy 1984;33(2):217–20. [5] Erbs DG, Klein SA, Beckman WA. Estimation of degree-days and ambient temperature bin data from monthly-average temperatures. ASHRAE J 1983;June:60–5. [6] Erbs DG, Klein SA, Beckman WA. Sol-air heating and cooling degree-days. Solar Energy 1984;33(6):605–12. [7] Bida M, Kreider JF. Monthly-averaged cooling load calculations — residential and small commercial buildings. J Solar Energy Eng 1987;109:311–20. [8] Said SAM, Al-Zaharnah ET. An analysis of climatic variables for thermal design of building in Khamis Mushayt, Saudi Arabia. Building and Environment 1991;26(4):363–9. [9] Said S A M. Degree-day base temperature for residential building energy prediction in Saudi Arabia. ASHRAE Trans: Res 1992;98(1):346–53.

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

553

[10] Rehman S, Husain T, Halawani TO. Application of one-dimensional planetary boundary layer model to the regional transport of pollutants — a case study. Atmos Res 1990;25:521–38. [11] Rehman S, Husain T, Halawani TO. Radiative fluxes in the troposphere from upper air meteorological data in Saudi Arabia. Solar Energy 1992;49(1):35–40. [12] Alaruri SD, Amer MF. Empirical regression models for weather data measured in Kuwait during the years 1985, 1986, and 1987. Solar Energy 1993;50(3):229–33. [13] Njau EC. An electronic system for predicting air temperature and wind speed patterns. Renewable Energy 1994;4(7):793–805. [14] Boland J. Time-series analysis of climatic variables. Solar Energy 1995;55(5):377–88. [15] Njau EC. A new analytical model for temperature prediction. Renewable Energy 1997;11(1):61–8. [16] Njau EC. Prediction of meteorological parameters: I. Analytical method. Nuovo Cimento 1991;14C:473–88. [17] Njau EC. Prediction of meteorological parameters: II. Methods based on an electronic system. In: Proceedings of the International Conference on Global Warming and Climate Change, Geneva, Switzerland, 1993. [18] Cybenko G. Approximation by superposition of a sigmoidal function. Math Control Signal Syst 1989;2:303–14. [19] Hinton GE. How neural networks learn from experience. Scientific American 1992;Sept:105–9. [20] Hunt KJ, Sbarbaro D. Neural networks for nonlinear internal model control. IEE Proc D 1993;138(5):431–8. [21] Sotirov G, Shafai B. Interval identification of nonlinear systems using neural netwroks. In: Proceedings of the American Control Conference, San Francisco, CA, 1993:2529–30. [22] Luxhoj James T, Riis Jens O, Stensballe B. Hybrid econometric-neural network modeling approach for sales forecasting. Int J Prod Econ 1996;43:175–92. [23] Mohandes MA, Rehman S, Halawani TO. A neural networks approach for wind speed prediction. Renewable Energy 1998;13(3):345–54. [24] Mohandes M, Rehman S, Halawani TO. Estimation of global solar radiation using artificial neural networks. Renewable Energy 1998;14(1-4):179–84 (also presented at the Sixth Arab International Solar Energy Conference, 1998 Mar 29–Apr 1, Muscat, Sultanate of Oman). [25] Mohandes M, Balghonaim A, Kassas M, Rehman S, Halawani TO. Use of radial basis functions for estimating monthly mean daily solar radiation. Solar Energy 2000;68(2):161–8. [26] Alexiadis MC, Dokopoulos PS, Sahsamanoglou HS, Manousarids IM. Short-term forecasting of wind speed and related electrical power. Solar Energy 1998;63(1):61–8. [27] Beyer HG, Denger T, Hausmann J, Hoffmann M, Rujan P. Short term prediction of wind speed and power output of a wind turbine with neural networks. In: EWEC 94, 5th European Wind Energy Association Conference and Exhibition, Conference Proceedings, Oral Sessions, vol. 1, Thessaloniki, Greece, 1994:349–52. [28] Bossanyi E. Short term stochastic wind prediction and possible control applications. In: Proceedings of the Delphi Workshop on, “Wind Energy Applications”, Greece, 1985:66–79. [29] Chan SM, Powell DC, Yoshimura M, Curtice DH. Operations requirements of the utilities with wind power generation. IEEE Trans Power Apparatus Syst 1983;102(9):2850–60. [30] Schlueter RA, Park GL, Bouwmeester R, Shu L, Lotfalian M, Rastgoufard P, Shayanfar A. Simulation and assessment of wind array power variation based on simultaneous wind speed measurements. IEEE Trans Power Apparatus Syst 1984;103(5):1008–16. [31] Schlueter RA, Sigari G, Costi A. Wind array power prediction for improved operating economics and reliability. IEEE Trans Power Apparatus Syst 1985;104(7):137–42. [32] Kalogirou SA, Panteliou S, Dentsoras A. Modeling of solar domestic water heating systems using artificial neural networks. Solar Energy 1999;65(6):335–42. [33] Kalogirou SA. Artificial neural networks for predicting the local concentration ration of parabolic trough collectors. In: Proceedings of the International Conference EuroSun ’96, Freiburg, Germany, 1996:470–5. [34] Kalogirou SA, Neocleous CC, Schizas CN. Artificial neural networks in modeling the heat-up response of a solar steam generation plant. In: Proceedings of the International Conference EANN ’96, London, UK, 1996:1–4.

554

I. Tasadduq et al. / Renewable Energy 25 (2002) 545–554

[35] Kalogirou SA, Neocleous CC, Schizas CN. Artificial neural networks for the estimation of the performance of a parabolic trough collector steam generation system. In: Proceedings of the International Conference EANN ’97, Stockholm, Sweden, 1997:227–32. [36] Kreider JF, Wang XA. Artificial neural network demonstration for automated generation of energy use predictors for commercial buildings. In: Haberl JS, Nelson RM, Culp CC, editors. The Use of Artificial Intelligence in Building Systems. ASHRAE, 1995:193–8. [37] Nair SK, Moon J. A theoretical study of linear and nonlinear equalization in nonlinear magnetic storage channels. IEEE Trans Neural Networks 1997;8:1106–18. [38] Zhu QM, Warwick K, Douce JL. Adaptive general predictive controller for nonlinear systems. IEE Proc D 1991;138(1):33–40. [39] Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural network. IEEE Trans Neural Networks 1990;1(1):4–27. [40] Ahmed MS, Tasadduq IA. Neural-net controller for nonlinear plants: design approach through linearization. IEE Proc D 1994;141(5):323–8. [41] Webros PJ. Backpropagation through time: what it does and how to do it. Proc IEEE 1990;78:1550–60. [42] Rumelhart DE, Hinton GE, Williams RJ. Learning Internal Representation by Backpropagation. Parallel Distributed Processing: Exploration in the Microstructure of Cognition, vol. 1. Cambridge, MA: Bradford Books, 1986. [43] Haykin S. Neural Networks: A Comprehensive Foundation. IEEE Press, McMillan Publishing, 1994.