- Email: [email protected]
Quantifying uncertainties of neural network-based electricity price forecasts
Applied Energy 112 (2013) 120–129
Contents lists available at SciVerse ScienceDirect
Applied Energy journal homepage: www.elsevier.com/locate/apenergy
Quantifying uncertainties of neural network-based electricity price forecasts Abbas Khosravi ⇑, Saeid Nahavandi, Doug Creighton Centre for Intelligent Systems Research (CISR), Deakin University, Geelong, VIC 3216, Australia
h i g h l i g h t s Quantification of uncertainties associated with electricity price forecasts. The delta and bootstrap methods for prediction interval construction. Comprehensive assessment of prediction interval quality. Experiments with monthly data sets and different confidence levels. Quality prediction intervals for price forecasts.
a r t i c l e
i n f o
Article history: Received 25 May 2012 Received in revised form 5 March 2013 Accepted 27 May 2013
Keywords: Electricity price Neural networks Prediction intervals Delta Bootstrap
a b s t r a c t Neural networks (NNs) are one of the most widely used techniques in literature for forecasting electricity prices. However, nonzero forecast errors always occur, no matter what explanatory variables, NN types, or training methods are used in experiments. Persistent forecasting errors warrant the need for techniques to quantify uncertainties associated with forecasts generated by NNs. Instead of using point forecasts, this study employs the delta and bootstrap methods for construction of prediction intervals (PIs) for uncertainty quantification. The confidence level of PIs is changed between 50% and 90% to check how their quality is affected. Experiments are conducted with Australian electricity price datasets for three different months. Demonstrated results indicate that while NN forecasting errors are large, constructed prediction intervals efficiently and effectively quantify uncertainties coupled with forecasting results. It is also found that while the delta PIs have a coverage probability always greater than the nominal confidence level, the bootstrap PIs are narrower, and by that, more informative. Ó 2013 Elsevier Ltd. All rights reserved.
1. Introduction Within the last two decades, the electricity industry worldwide has been restructured and transformed from a traditional regulated and monopolistic system into a deregulated and competitive system. Accurate forecasting of electricity prices is essential for all participants of deregulated electricity markets. Electricity generators can more efficiently offer their bids to maximize their profit, and consumers can optimize their load schedules to minimize their utilization cost. Also electricity price forecasts are important for medium and long term planning such as making investment decisions and transmission expansion. Unlike other energy resources, electrical energy is economically non-storable and has an instantaneous nature. Also its transmission can be interrupted due to congestion in power networks. These features make electricity price curves volatile, non-stationary, and
⇑ Corresponding author. E-mail address: [email protected] (A. Khosravi). 0306-2619/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.apenergy.2013.05.075
outlier prone in deregulated markets. Price curves are much richer in information than load curves and have many rapid fluctuation caused by participants of the deregulated market. Therefore, price forecasting is a practically complicated task and highly prone to error. Due to its importance, a variety of methods have been proposed in literature for electricity price forecasting, including but not limited to, autoregressive models [1,2], dynamic regression [3,4], transfer function [5], GARCH models [6,7], neural networks (NNs) [8,9], neuro-fuzzy systems [10], and support vector machines [11]. No method is superior for forecasting in all cases. Contradictory conclusions and results have been reported in literature regarding performance of aforementioned methods for price forecasting. It is shown in [8] that wavelet NNs perform better than a series of other forecasting models including wavelet-ARIMA, multi-layer perceptron, radial basis function NNs, and fuzzy NNs. Time series methods, such as ARIMA and transfer functions, have been reported with a performance superior to NNs in [12]. Opposite results are reported in [13] where NNs outperform their time series
121
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
Nomenclature Abbreviation Description 3T 3-time unit ahead 24T 24-time unit ahead ARIMA autoregressive integrated moving average BS bootstrap CI confidence interval CWC coverage width-based criteria D delta GARCH generalized autoregressive conditional heteroscedasticity MAPE mean absolute percentage error NN neural network PI prediction interval PICP prediction interval coverage probability
counterparts including ARIMA models. Better results are achieved in particular for weekends and holidays. Forecasting errors always exist regardless of implemented data preprocessing mechanism, hired forecasting model type, employed explanatory variables, and applied model training method. The main focus of the research community in the last two decades has been on improving the performance of forecasting techniques, but without the discussion on quantifying uncertainties associated with forecasts. It is reasonable to look for techniques to quantify forecasting errors. As critical decisions with significant financial consequences are made based on electricity price forecasts, having access to a measure of uncertainties associated with them is highly beneficial [11,14]. A prediction interval (PI) is composed of an upper and lower bound, where the actual target lies within these bounds with a prescribed probability called the confidence level. The confidence level is often described by (1 a)%, where a varies between zero and one. PIs are more important than confidence intervals (CIs) in practice, as they carry more information about future observations. Wide PIs indicate that there is a high level of uncertainty associated with predictions (forecasts), and therefore decisions should be made more cautiously. In contrast, decisions can be made with greater confidence when constructed PIs have a narrow width. Bayesian [15,16], delta [17,18], bootstrap [19], mean–variance estimation [20], and upper lower bound estimation [23] methods have been proposed in literature for construction of PIs using NNs. Application of these techniques has been documented in the literature in the field of transportation [24,25], manufacturing [26,27], load forecasting [28,29], renewable energies [21,22], and power generation [30]. A comprehensive review and detailed discussion of these methods can be found in [31]. Recursive dynamic factor analysis and Kalman filter are also used in [32] for estimation of variance of electricity prices and construction of prediction intervals. The extreme learning machine algorithm and bootstrapping are employed in [33] for estimating the predictive uncertainties associated with points forecasts. Twelve parametric and semiparametric time series models are applied in [34] for forecasting short term electricity prices and generating intervals. It is concluded that semiparametric methods lead to more reliable intervals than parametric models. The application of CIs and PIs, compared to point prediction, is still in its infancy. A fuzzy based method was proposed in [35] for obtaining the possibility distribution of electricity prices for a given demand value. However, no discussion is made to link this to the concept of CIs and PIs. A method for calculating the probability distributions of electricity prices is proposed in [36]. This method required availability of probability distributions of load and reserve forecasts. A Quasi-Newton method is used in [37] for
PINAW SSE ti ^i y
prediction interval normalized averaged width sum of squared error the ith measured target the ith forecast i noise (error) term w⁄ NN optimal parameters g0 NN gradient F NN Jacobian matrix B Number of bootstrap models N 0; r2 normal distribution with mean zero and variance r2y^ variance due to model misspecification r2^ variance of errors
2
i
the construction of CIs for electricity prices. It is shown that the coverage probability of CIs is acceptable. A decoupled extended Kalman filter-based NN method is employed in [38] for estimation of smaller CIs than those obtained using the Bayesian method for electricity prices. Also, an ARIMA-based method is proposed in [39] for construction of CIs assuming the residual errors are in Gaussian or uniform distribution. No discussion is made regarding PIs for a future electricity price in the aforementioned references. Zhao et al. [11] proposed a hybrid method for construction of PIs using support vector machines (for estimation of price mean) and a nonlinear conditional heteroskedastic forecasting model (for estimation of price variance). It is shown PIs constructed using the proposed method outperform those constructed using GARCH method in terms of coverage probability. However, this paper ignores the important fact that constructed PIs should also be assessed based on their width, as wide PIs carry no information about variation of targets. Relying on the fact that forecasts are always wrong, Baringo and Conejo [40] recently used price intervals, instead of price predictions, to develop hourly offering curves for a price-taker producer, that participates in a pool-based electricity market. Construction of theoretically valid and informative PIs is a prerequisite for the efficient implementation of those methods in real world. Using price intervals for adjusting the hourly load level of a consumer has also been reported in [41]. This is done to optimize the consumer’s energy consumption profile, resulting in its utility maximization. In contrast with previous studies that their focus is either on point forecasts or CIs, this paper uses PIs for quantifying uncertainties surrounding electricity price forecasts generated by NNs. PIs are constructed instead of CIs, as they cover more sources of uncertainties and provide more accurate information about unseen targets. This paper exploits the delta and bootstrap methods for construction of PIs. The motivation for using these methods is their ability for construction of high quality PIs with a low computational cost [31]. Both methods are used and compared to analyze how the volatility of electricity prices affects the quality of PIs. Constructed PIs are quantitatively assessed based on their coverage probability and width. Australian energy market data sets are used for conducting experiments and examining the quality of constructed PIs. In experiments, it is shown that while NN forecasts are always prone to error, constructed PIs appropriately cover the targets and provide valuable information about associated uncertainties. The rest of this paper is organized as follows. Delta and bootstrap methods for construction of NN-based PIs are briefly introduced in Section 2. Section 3 describes performance measures for assessing the quality of PIs. Simulation results are demonstrated in Section 4. Section 5 concludes the paper with some remarks and guidelines for future work.
122
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
misspecification variance can be estimated using the variance of B model outcomes,
2. PI Construction methods 2.1. The delta method
r2y^i ¼
B b 2 1 X ^ y ^i : y B 1 b¼1 i
ð8Þ
The delta method is based on linearization of a NN model around the set of its optimal parameters, w⁄. PIs are then constructed for the linearized model. A statistical model for relating targets to the model can be assumed as follows,
For the construction of PIs, the variance of errors, r2^i , should be also estimated. r2^ can be calculated as follows,
t i ¼ f ðxi ; w Þ þ i
r2^ ’ Efðt y^Þ2 g r2y^ :
ð1Þ
where ti is the ith measured target (totally n targets), xi is the ith vector of inputs, and f (xi, w⁄) is a smooth NN model. i is the noise, also called error, with a zero expectation. In a small neighborhood of w⁄, we have,
^0 ¼ f ðx0 ; w Þ þ g T0 ðw ^ w Þ y
ð2Þ
g T0
where is the NN output gradient against the network parameters. ^ are adjusted through minimization of the sum of NN parameters, w, squared error (SSE) cost function. Under certain regularity condi^ is very close to w⁄. Accordingly, we tions, it can be shown that w have,
^0 ’ 0 þ g T0 ðw ^ w Þ t0 y
ð3Þ
The two remaining terms in right hand side of (3) are statistically independent. Assuming that the error terms are normally distributed ð Nð0; r2 ÞÞ, the second term in the right hand side of (3) can be expressed as, 1
r2y^0 ¼ r2 g T0 ðF T FÞ g 0
ð4Þ
where F is the Jacobian matrix of the NN model. The total variance can be expressed as, 1
r20 ¼ r2 ð1 þ g T0 ðF T FÞ g 0 Þ
ð5Þ
SSE An unbiased estimate of r2 is n1 . According to this, the (1 a)% PI ^i is computed as detailed in [17], for y
^0 y
1a tnp2 s
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 1 þ g T0 ðF T FÞ g 0
1a t np2
ð6Þ
a
where is the 2 quantile of a cumulative t-distribution function with n p degrees of freedom. The following formula was derived in [18] for PI construction for the case that NNs are trained using the weight decay cost function instead of the SSE cost function,
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1a ^0 tnp2 s 1 þ g T0 Wg 0 y T
1
ð7Þ T
T
1
where W = (F F + kI) (F F)(F F + kI) . Inclusion of k in (7) improves the reliability and quality of PIs, particularly for cases that FTF is nearly singular.
According to (9), a set of variance squared residuals is developed,
^i Þ2 r2y^ ; 0 r2i ¼ max ðt i y i
ð10Þ
^i and ry^2 both are available from B trained NN models and where y i (8). These residuals are linked by the set of corresponding inputs to form a new dataset,
n Dr2 ¼ ðxi ; r 2i Þ i¼1 :
ð11Þ
A new NN model, called NN r^ , can be trained using the maximum likelihood estimation method to estimate the unknown values of r2^i , so as to maximize the probability of observing the samples in Dr2 [19]. In total, B + 1 NN models are required in the bootstrap method for PI construction. Simplicity is the main advantage of using the bootstrap method for PI construction. In contrast to the delta technique, there is no need to calculate complex matrices and derivatives. 3. Performance index 3.1. Performance index for point forecasts The forecasting accuracy of NNs is measured using mean absolute percentage error (MAPE) as the performance index:
MAPE ¼
n ^ 1X yt yt n t¼1 yt
ð12Þ
^t are the actual and forecasted price at time t, respecwhere yt and y tively. n is also the number of observations used in experiments. 3.2. Performance index for prediction intervals A combinational coverage width-based criterion (CWC) is used for assessing the quality of constructed PIs,
CWC ¼ PINAWð1 þ cðPICPÞ egðPICPlÞ Þ
ð13Þ
where PINAW and PICP are the PI normalized averaged width and PI coverage probability, respectively. PICP is the spontaneous measure related to the quality of constructed PIs and measures the quantity of targets bracketed by intervals,
2.2. The bootstrap method
PICP ¼ Bootstrap is essentially a resampling technique for estimating the distribution of targets [42]. It mimics the random mechanism of the underlying target through resampling with replacement of the available data set. This leads to an empirical distribution of the underlying target that can be used for construction of CIs and PIs. B training datasets are resampled from the original dataset with replacement, fDgBb¼1 . The method estimates the variance due to model misspecification, r2y^ , by building B NN models. According to this assumption, the true regression is estimated by P ^i ¼ 1B Bb¼1 y ^bi , where y ^bi averaging the point forecasts of B models, y is the prediction of the i-th sample generated by the b-th bootstrap model. Assuming that NN models are unbiased, the model
ð9Þ
n 1X ct n t¼1
ð14Þ
where ct = 1 if yt 2 [Lt, Ut], otherwise ct = 0. Lt and Ut are the lower and upper bounds of the tth PI respectively. Theoretically, PICP P (1 a)%). Constructed PIs are not reliable and should be discarded if this condition is not satisfied. Practically, narrow PIs are more informative than wide PIs. PINAW assesses PIs from this aspect and measures how wide they are,
PINAW ¼
n 1 X ðU t Lt Þ nR t¼1
ð15Þ
123
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
(b) August 2010
(a) May 2010
(c) December 2010 Fig. 1. Australian electricity price datasets used in this study.
Table 2 Parameters used in experiments. Table 1 Inputs of NN models used for forecasting electricity prices. Forecast Horizon
Lagged prices
3-Time unit ahead 24-Time unit ahead
[yt3, yt4, yt5, yt24, yt25, yt26] [yt24, yt25, yt26, yt48, yt49, yt50]
Parameter
Numerical value
Dtrain Dtest Maximum number of neurons
80% Of each month samples 20% Of each month samples 10 0.1 50 0.90 5 10 4–2
a g l K-fold cross validation Number of bootstrap models Structure of NNr^
Split the data set into training and test sets (Dtrain and Dtest)
Perform a K-fold cross validation to determine the optimal structure for NN
Construct PIs using delta and bootstrap methods
Evaluate the quality of constructed PIs using performance index CWC Fig. 2. The experimental procedure for PI construction and evaluation.
where R is the range of the underlying target. PINAW is the average width of PIs as a percentage of the underlying target range. Inclusion of the PINAW and PICP in CWC allows one to simultaneously evaluate PIs based on their width (informativeness) and coverage probability (correctness). c(PICP) is also given by the following step function,
c¼
0
PICP P l
1 PICP < l
ð16Þ
This means that the exponential term in (13) is eliminated whenever PICP P l and CWC becomes equal to PINAW. Further information about these measures and hyperparameters g and l can be found in [23,43].
124
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
(a) MAPE 3T for May 2010
(b) MAPE3T for August 2010
(c) MAPE 3T for December 2010
(d) MAPE24T for May 2010
(e) MAPE 24T for August 2010
(f) MAPE24T for December 2010 Fig. 3. Contour plots of averaged MAPE for test samples calculated on a grid of NN structure (n1 and n2 are the number of neurons in the first and second layers of NNs, respectively).
125
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
4. Numerical experiments 4.1. Data and experimental procedure National electricity market (NEM) in Australia was deregulated and privatized more than a decade ago. The NEM spot pool market is conducted by Australian Energy Market Operator (AEMO). It receives offers by electricity generators in 5 min intervals every day. Offers are processed by AEMO and generators are instantaneously scheduled to meet the demand in the most cost effective way. Dispatch prices are advised every 5 min. Then, the spot prices are determined every 30 min by averaging the most recent six dispatch prices. These spot prices are used in the paper for examining performance of PI construction methods. Three one-month datasets of Australian electricity prices for the Victorian region during 2010 are used to examine and validate the performance of PI construction methods. May, August, and December months represent three different seasons of a year in Australia. They correspond to autumn, winter, and summer respectively. These have been selected to make the test results less subjective. Three month electricity prices with half an hour intervals are displayed in Fig. 1. Spikes are more frequent in May and August. However, previously unobserved patterns are more common in December. This means that electricity price forecasting errors in December should be greater than those for May and August. The datasets used in this study are publicly available at www.aemo.com.au. Australian electricity prices have been used in a set of studies before [44,2]. However, those studies focus on how accurate point forecasts can be generated using different types of models. In contrast, this paper deals with the problem of quantification of uncertainties associated with forecasts. Prices are far from being a stationary process when expressed in their original format [11]. In order to tackle the problem of nonstationary price series, we do a first-order differencing transformation [45]. Calculating changes from one period to the next removes the trend in mean and delivers a more stationary series. Three (3) and twenty-four (24) time unit ahead horizons are considered for forecasting electricity prices, where the time unit is 30 min. Table 1 shows lagged values of actual prices used as independent input variables to the NNs. Both sets of inputs have a similar format and include six lagged values. Selection of these lagged values is done based on analysis of autocorrelation and partial correlation of electricity prices. The Matlab package is used to carry out simulations and visualize results. The experimental procedure for construction of PIs using the delta and bootstrap methods is shown in Fig. 2. 80% of
each month samples are used for training of NN models (Dtrain), and the remaining 20% samples are applied for examining performance of developed models (Dtest). A fivefold cross validation technique is applied to determine the optimal NN structure and the number of neurons per layer. A maximum of ten neurons is allowed in each hidden layer. Once the NN model is selected, it is used for construction of PIs using the delta and bootstrap techniques. The quality of constructed PIs is then assessed using CWC as the performance index. Parameters used for implementing the delta and bootstrap methods and conducting experiments are listed in Table 2. All PIs are constructed with a 90% confidence level (a = 0.1). Ten NNy models are considered in the bootstrap method. NN r^ is considered to have two layers with four and two neurons in its hidden layers. l is set to 0.9, because the PI confidence level is 90%. Also, g is selected to be 50 in order to highly penalize PIs with a coverage probability lower than the nominal confidence level. 4.2. Results and discussion In the first stage, performance of two layer NNs is examined for forecasting electricity prices. NN forecasting performance intimately depends on its structure (number of layers) and complexity (number of neurons per layer). Preliminary results showed that two layer NNs outperform NNs with a single hidden layer. Therefore, all experiments are reported for NNs with two hidden layers. The number of neurons in the hidden layers of NNs is changed between 1 and 10. We limit the number of neurons in each hidden layer to 10 to keep NN models parsimonious. For each NN structure, experiments are repeated ten times and then the averaged performance index, MAPE, is calculated. This is done to avoid misleading results due to random initialization of NN parameters. Fig. 3 displays filled contour plots of averaged MAPEs for test samples of three one-month datasets. Horizontal and vertical axes indicate the number of neurons in the first (n1) and second (n2) hidden layers of NNs. The color map in the right hand side of each plot shows the color-coded value of MAPE. There are several peaks and valleys, which are a sign of sensitivity of MAPE to the NN structure and its initial parameters (assigned randomly). There are also regions where MAPE levels off and does not fluctuate. For the case of 3-time unit ahead forecasting, the best achieved results are 20.24%, 12.39%, and 26.69% for 3 months respectively. MAPEs for 24-time unit horizon are 20.19%, 11.46%, and 28.68% respectively. These results clearly show that forecasts obtained by NNs are far from being perfect. Large MAPEs, in particular for test samples of May and December, are unavoidable, regardless of the NN structure and the training mechanism. Also, it is important to notice
Table 3 PICP, PINAW, and CWC for PID,3T and PIBS,3T. Month
May 2010 August 2010 December 2010
Delta PIs
Bootstrap PIs
PICP (%)
PINAW (%)
CWC
PICP (%)
PINAW (%)
CWC
97.94 98.39 100
41.50 37.67 416.81
41.50 37.67 416.8
89.38 90.41 82.19
15.22 10.95 19.38
35.93 10.95 980.6
Table 4 PICP, PINAW, and CWC for PID,24T and PIBS,24T. Month
May 2010 August 2010 December 2010
Delta PIs
Bootstrap PIs
PICP (%)
PINAW (%)
CWC
PICP (%)
PINAW (%)
CWC
98.26 97.91 100
40.20 34.58 423.4
40.20 34.58 423.4
76.66 89.55 84.67
7.08 7.84 16.01
5606 17.66 246.1
126
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
that best results in all experiments are obtained by NNs with a small size. This confirms that more parsimonious NNs have a better generalization power when used for time series forecasting [46]. Hereafter for the sake of simplicity, 3T and 24T subscripts are used to indicate 3-time unit and 24-time unit ahead forecasts. Also D and BS are abbreviations for the delta and bootstrap methods for PI construction. For example, PID,3T indicates delta PIs constructed for 3-time unit ahead forecasts.
(a) PID,3T for August 2010
PICP, PINAW, and CWC for PID and PIBS constructed for 3 and 24 time unit horizon forecasts are summarized in Tables 3 and 4. PID,3T and PID,24T constructed for each month dataset are theoretically correct, as PICP P 90%. This is due to the fact that the confidence level associated with PIs is 90%. Satisfaction of this requirement by PID signifies that the delta method amply takes care of the coverage probability of PIs. PICPBS,3T and PICPBS,24T are less than 90% in five out of six case studies. The worst result is obtained for 24-time unit ahead forecasts of May, where the coverage probability of PIBS,24T is 76.66%. These results indicate that delta PIs are more reliable than bootstrap PIs (PICPD is always greater than the nominal confidence level). The greater coverage of PIs constructed using the delta method comes with the cost of wider intervals. As per the results in Tables 3 and 4, PID are significantly wider than PIBS for test samples of 3 months for both forecast horizons. The extra width of delta PIs is acceptable for some cases, e.g., August 2010, as PICP is very close to 100%. However, PID,3T and PID,24T for December 2010 are so wide that they are uninformative. PINAW is 416.8% and 423.4% for these cases. This means that the width of constructed PIs is four times greater than the target range on average. In contrast, PIBS,3T and PIBS,24T are narrow and take the same pattern of the underlying target. Tables 3 and 4 also report CWCBS/D,3T/24T. CWCBS is smaller than CWCD in four out of six cases. The small value of CWCBS is mainly due to the narrowness of PIBS, compared to excessively wide PID. CWCs indicate that for electricity prices PIBS are of a better quality than PID for both forecasting horizons. In a separate experiment, we change the level of confidence between 50% and 90% to check how width and coverage probability of PIs will vary. Simulations are done for August 2010 datasets using the delta and bootstrap methods. Fig. 4 displays four plots corresponding to PID,3T,PID,24T,PIBS,3T, and PIBS,24T. In each plot, there are five shaded bands representing PIs constructed for a 50% confidence level (the inner band with a darker color), 60%, 70%, 80%,
(b) PIBS ,3T for August 2010
(c) PID,24T for August 2010
Fig. 5. PICP for PIs constructed with confidence levels ranging between 50% and 90% for August 2010.
(d) PIBS ,24T for August 2010 Fig. 4. PID,3T, PIBS,3T, PID,24T, and PIBS,24T constructed for different values of confidence level (50%, 60%, 70%, 80%, and 90%).
Fig. 6. PINAW for PIs constructed with confidence levels ranging between 50% and 90% for August 2010.
127
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
and a 90% confidence level (the outer band with a brighter color). Targets are also shown with bright circles. With the purpose of better visualization, only PIs for the first one hundred test samples are displayed. Theoretically, it is expected that PIs with a 50% confidence level will cover at least 50% of targets and so on. It is interesting to notice that PICPD,3T and PICPD,24T are always significantly greater than the prescribed confidence level. For instance, when the confidence levels are 50% or 60%, PICPD is always more than 87%, as displayed in Fig. 5. PICPD,3T and PICPD,24T for a confidence level of 60% are 91.10% and 95.82%, respectively. These values are 67.94% and 67.94% for corresponding bootstrap PIs. Therefore, bootstrap PIs are less conservative with regard to the coverage probability when compared to delta PIs. A visual analysis makes it clear that PID,3T and PID,24T are significantly wider than their corresponding PIs constructed using the bootstrap method. The numerical values of PINAW for constructed PIs are illustrated in Fig. 6. The width of PIBS constructed with a 90% confidence level is less than the width of PID constructed with a 50% confidence level. Therefore, the former one not only is more reliable, but also is more informative. However, this conclusion is not true for data samples of the other 3 months, as shown in Tables 3 and 4. A closer look at PID,3T and PID,24T in Fig. 4 shows that widths of PIs constructed for a specific level of confidence are almost constant and do not change from one sample to another. Centers of PID fluctuate depending on input values used for forecasting electricity prices, but their widths remain almost unchanged. In contrast, PIBS,3T and PIBS,24T have variable widths. This observation
suggests that bootstrap PIs are more sensitive to the data distribution and uncertainties than the delta PIs. When there is a high level of uncertainty in training data, PIBS become wider to accommodate associated uncertainties. They remain narrow when uncertainties are low and ignorable. As per the coverage of the inner tube for PID,3T and PID,24T, one may use PIs with a confidence level of 50% or 60% for decision making. These intervals are acceptably narrow (PINAW ’ 15%) with a large PICP (approximately 90%). What the optimal value of the confidence level is and how it is linked to the process of decision-making in electricity markets needs more attention and investigation. Comparing PICP and PINAW in Figs. 5 and 6 shows that PICPD does not quickly drop as the confidence level is decreased from 90% to 50%. However, PICPBS,3T and PICPBS,24T show a strong correlation with the confidence level (0.999 and 0.998 respectively). The same decreasing pattern is also observed for PINAW as the confidence level is decreased. This signifies that there is a dominant constant term in (7) leading to wide PIs with an excessively large coverage probability. According to these results, the delta method sacrifices the informativeness of PIs in order to keep them theoretically valid. In contrast, the bootstrap method attempts to generate narrow PIs, which may occasionally lead to an unsatisfactorily low coverage probability. These results and conclusions comply with those reported in previous studies in other fields [47]. It is important to investigate why PIBS often have an unsatisfactorily low PICP. Australian electricity market prices, similar to the prices from other markets, are highly volatile, fat-tailed, skewed,
(b) August 2010
(a) May 2010
(c) December 2010 Fig. 7. Probability plots for 3 month electricity prices used in this study.
128
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
and do not follow a normal distribution. A normal probability plot is here used to aid with the clear presentation of the data distribution and characteristics. This plot provides a good indication of whether or not the data comes from a normal distribution. The normal probability plots for the 3 months are shown in Fig. 7. The color-filled circles show the empirical probability versus the data value for each point in the sample. Superimposed on the plot is a line connecting the 25th and 75th percentiles of the data, which can be seen by a dashed line in the three plots. This line is extrapolated out to the ends of the data samples for a better linearity evaluation. If all electricity prices fall near the dashed line, forming a linear plot, then it can be concluded that the data comes from a normal distribution. Electricity prices for all 3 months show a high level of curvature, which strongly denies this assumption. This means that the bootstrap method, which is based on the normality assumption, is not the best candidate for construction of PIs for electricity prices. These evidences justify the low PICPs reported in Tables 3 and 4 obtained using the bootstrap method. The ultimate goal of construction of PIs is to integrate them into decision-making algorithms and expert systems. Both delta and bootstrap PIs can be used in conjunction with point forecasts for decision-making. Depending on their roles and interests, stakeholders in the energy market can differently interpret and use PIs in their decision-making processes to maximize their benefits. A power generation company, for instance, may optimistically use the upper bound of an electricity price PI in its offer. Another company may always select the lower bound in its bids, which reflects a pessimistic attitude (worst case scenario). For an electricity buyer, wide price intervals indicate that there is a high level of uncertainty in the market; so all prices should be well negotiated. In contrast, narrow intervals are a sign of low uncertainty in the market; therefore, forecasts can be trusted. In another approach, upper and lower bounds of prices can be used as extreme values or limits in the optimization problems solved by utilities and generation companies. Design of algorithms for efficient use of PIs for decision-making in power system is a very fruitful research area. Some researchers have already started using confidence and prediction intervals instead of point forecasts to develop robust offering strategies in electricity markets [40,41]. PID and PIBS can be both integrated into those optimization techniques for generating robust solutions.
5. Conclusions The purpose of this paper is to give new momentum to the application of PI construction techniques in the field of electricity price forecasting. The key idea is to change focus from point forecasts to PIs. It is first shown that NN forecasting errors always exist and cannot be eliminated. The best MAPEs for different datasets and forecasting horizons vary between 11.46% and 28.68%. Therefore, PIs are constructed to quantify associated uncertainties with forecasts. The delta and bootstrap methods are applied in this paper for construction of neural network-based PIs for electricity price forecasts. The confidence level associated with PIs is also changed between 50% and 90%. It is observed that delta-based PIs have a more acceptable coverage probability. There is always a large gap between the coverage probability of the delta PIs and their nominal confidence level. The conservatively of the delta method to the PI coverage probability comes with the massive cost of generating too wide PIs. In contrast, bootstrap PIs are narrower, and therefore, more informative. Besides, variable width bootstrap-based PIs more appropriately respond to uncertainties in data than delta PIs. However, it is found that the fundamental normality assumption in the bootstrap method may cause some problems leading to construction of unreliable PIs.
Future research can focus on how to improve the quality of constructed PIs. Optimization methods can be applied to find a trade off between two conflicting issues of PIs: being narrow (more informative) and having a large coverage probability (correctness). Constructed PIs by the proposed methods can be used along with electricity price forecasts in the decision-making processes in energy markets. PIs signify the level of uncertainty associated with forecasts and how trustworthy they are. Acknowledgment This study is partially supported by the Australian Research Council (Discovery Grant DP120102112). References [1] Tan Z, Zhang J, Wang J, Xu J. Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models. Appl Energy 2010;87(11):3606–10. [2] Dong Y, Wang J, Jiang H, Wu J. Short-term electricity price forecast based on the improved hybrid model. Energy Convers Manage 2011;52(8–9):2987–95. [3] Pedregal DJ, Trapero JR. Electricity prices forecasting by automatic dynamic harmonic regression models. Energy Convers Manage 2007;48(5):1710–9. ISSN 0196-8904. [4] Nogales F, Contreras J, Conejo A, Espinola R. Forecasting next-day electricity prices by time series models. IEEE Trans Power Syst 2002;17(2):342–8. ISSN 0885-8950. [5] Zareipour H, Canizares C, Bhattacharya K, Thomson J. Application of publicdomain market information to forecast Ontario’s wholesale electricity prices. IEEE Trans Power Syst 2006;21(4):1707–17. ISSN 0885-8950. [6] Garcia R, Contreras J, van Akkeren M, Garcia J. A GARCH forecasting model to predict day-ahead electricity prices. IEEE Trans Power Syst 2005;20(2):867–74. ISSN 0885-8950. [7] Garcia-Martos C, Rodriguez J, Sanchez MJ. Modelling and forecasting fossil fuels, CO2 and electricity prices and their volatilities. Appl Energy 2013;101:363–75. [8] Pindoriya N, Singh S, Singh S. An adaptive wavelet neural network-based energy price forecasting in electricity markets. IEEE Trans Power Syst 2008;23(3):1423–32. ISSN 0885-8950. [9] Lin WM, Gow HJ, Tsai MT. An enhanced radial basis function network for shortterm electricity price forecasting. Appl Energy 2010;87(10):3226–34. ISSN 0306-2619. [10] Khotanzad A, Zhou E, Elragal H. A neuro-fuzzy approach to short-term load forecasting in a price-sensitive environment. IEEE Trans Power Syst 2002;17(4):1273–82. ISSN 0885-8950. [11] Zhao JH, Dong ZY, Xu Z, Wong KP. A statistical approach for interval forecasting of the electricity price. IEEE Trans Power Syst 2008;23(2):267–76. ISSN 08858950. [12] Conejo AJ, Contreras J, Espfnola R, Plazas MA. Forecasting electricity prices for a day-ahead pool-based electric energy market. Int J Forecasting 2005;21(3):435–62. ISSN 0169-2070. [13] Pino R, Parreno J, Gomez A, Priore P. Forecasting next-day price of electricity in the Spanish energy market using artificial neural networks. Eng Appl Artif Intell 2008;21(1):53–62. ISSN 0952-1976. [14] Amjady N, Hemmati M. Energy price forecasting – problems and proposals for such predictions. IEEE Power Energy Mag 2006;4(2):20–9. ISSN 1540-7977. [15] MacKay DJC. The evidence framework applied to classification networks. Neural Comput 1992;4(5):720–36. [16] Bishop CM. Neural networks for pattern recognition. Oxford: Oxford University Press; 1995. [17] Hwang JTG, Ding AA. Prediction intervals for artificial neural networks. J Am Stat Assoc 1997;92(438):748–57. ISSN 01621459. [18] Veaux RDd, Schumi J, Schweinsberg J, Ungar LH. Prediction intervals for neural networks via nonlinear regression. Technometrics 1998;40(4):273–82. ISSN 00401706. [19] Heskes T. Practical confidence and prediction intervals. In: Mozer TPM, Jordan M, editors. Neural information processing systems, vol. 9. MIT Press; 1997. p. 176–82. [20] Nix D, Weigend A. Estimating the mean and variance of the target probability distribution. In: IEEE international conference on neural networks; 1994. [21] Khosravi A, Nahavandi S. Combined non-parametric prediction intervals for wind power generation. IEEE Trans on Sust Energy, 2013. in Press. [22] Khosravi A, Nahavandi S, Creighton D. Prediction intervals for short-term wind farm power generation forecasts. IEEE Trans on Sust Energy, 2013, in press. [23] Khosravi A, Nahavandi S, Creighton D, Atiya AF. A lower upper bound estimation method for construction of neural network-based prediction intervals. IEEE Trans Neural Networks 2011;22(3):337–46. [24] Khosravi A, Mazloumi E, Nahavandi S, Creighton D, van Lint JWC. Prediction intervals to account for uncertainties in travel time prediction. IEEE Trans Intell Transport Syst 2011;12(2):537–47.
A. Khosravi et al. / Applied Energy 112 (2013) 120–129 [25] Khosravi A, Mazloumi E, Nahavandi S, Creighton D, van Lint JWC. A genetic algorithm-based method for improving quality of travel time prediction intervals. Transport Res Part C [accepted 11.04.11]. [26] Yu G, Qiu H, Djurdjanovic D, Lee J. Feature signature prediction of a boring process using neural network modeling with confidence bounds. Int J Adv Manuf Technol 2006;30(7):614–21. [27] Khosravi A, Nahavandi S, Creighton D. A prediction interval-based approach to determine optimal structures of Neural Network Metamodels. Expert Syst Appl 2010;37:2377–87. ISSN 0957-4174. [28] Khosravi A, Nahavandi S, Creighton D. Load Forecasting and Neural Networks: a prediction interval-based perspective. In: Panigrahi BK, et al., editors. Computational intelligence in power engineering. SCI 302; 2010. p. 131–50. [29] Khosravi A, Nahavandi S, Creighton D. Construction of optimal prediction intervals for load forecasting problem. IEEE Trans Power Syst 2010;25:1496–503. [30] Zio E. A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes. IEEE Trans Nucl Sci 2006;53(3):1460–78. ISSN 0018-9499. [31] Khosravi A. Construction and Optimisation of prediction intervals for neural network models, Ph.D. Thesis. Deakin University; 2010. [32] Wu HC, Chan SC, Tsui KM, A new recursive dynamic factor analysis for point and interval forecast of electricity price. IEEE Trans on Power Syst 2013. in press. http://dx.doi.org/10.1109/TPWRS.2012.2232314. [33] Chen X, Dong ZY, Meng K, Xu Y, Wong KP, Ngan H. Electricity price forecasting with extreme learning machine and bootstrapping. IEEE Trans Power Syst 2012;27(4):2055–62. ISSN 0885-8950. [34] Weron R, Misiorek A. Forecasting spot electricity prices: a comparison of parametric and semiparametric time series models. Int J Forecasting 2008;24(4):744–63. ISSN 0169-2070. [35] Niimura T, Nakashima T. Deregulated electricity market data representation by fuzzy regression models. IEEE Trans Syst Man Cyber Part C: Appl Rev 2001;31(3):320–6. ISSN 1094-6977..
129
[36] Yamin HY, Shahidehpour SM, Li Z. Adaptive short-term electricity price forecasting using artificial neural networks in the restructured power markets. Int J Electrical Power Energy Syst 2004;26(8):571–81. ISSN 01420615. [37] Zhang X, Rice JA. Short-term travel time prediction. Transport Res Part C: Emerging Technol 2003;11(3–4):187–210. ISSN 0968-090X. [38] Zhang L, Luh P. Neural network-based market clearing price prediction and confidence interval estimation with an improved extended Kalman filter method. IEEE Trans Power Syst 2005;20(1):59–66. ISSN 0885-8950. [39] Zhou M, Yan Z, Ni Y, Li G, Nie Y. Electricity price forecasting with confidenceinterval estimation through an extended ARIMA approach. IEE Proc Generation Transm Distribution 2006;153(2):187–95. ISSN 1350-2360. [40] Baringo L, Conejo AJ. Offering strategy via robust optimization. IEEE Trans Power Syst. doi:10.1109/TPWRS.2010.2092793 ISSN 0885-8950 [13.12.10]. [41] Conejo AJ, Morales JM, Baringo L. Real-time demand response model. IEEE Trans Power Syst 2010;1(3):236–42. [42] Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman and Hall; 1993. [43] Khosravi A, Nahavandi S, Creighton D. Prediction interval construction and optimization for adaptive neurofuzzy inference systems. IEEE Trans Fuzzy Syst 2011;19(5):983–8. [44] Amjady N. Day-ahead price forecasting of electricity markets by a new fuzzy neural network. IEEE Trans Power Syst 2006;21(2):887–96. ISSN 0885-8950. [45] Box G, Jenkins G, Reinsel GC. Time series analysis: forecasting and control. 4th ed. Hoboken, N.J: John Wiley; 2008. [46] Zhang GP, Qi M. Neural network forecasting for seasonal and trend time series. Eur J Operational Res 2005;160(2):501–14. ISSN 0377-2217. [47] Khosravi A, Nahavandi S, Creighton D, Atiya A. Comprehensive review of neural network-based prediction intervals and new advances. IEEE Trans Neural Networks 2011;22(9):1341–56. ISSN 1045-9227.
Contents lists available at SciVerse ScienceDirect
Applied Energy journal homepage: www.elsevier.com/locate/apenergy
Quantifying uncertainties of neural network-based electricity price forecasts Abbas Khosravi ⇑, Saeid Nahavandi, Doug Creighton Centre for Intelligent Systems Research (CISR), Deakin University, Geelong, VIC 3216, Australia
h i g h l i g h t s Quantification of uncertainties associated with electricity price forecasts. The delta and bootstrap methods for prediction interval construction. Comprehensive assessment of prediction interval quality. Experiments with monthly data sets and different confidence levels. Quality prediction intervals for price forecasts.
a r t i c l e
i n f o
Article history: Received 25 May 2012 Received in revised form 5 March 2013 Accepted 27 May 2013
Keywords: Electricity price Neural networks Prediction intervals Delta Bootstrap
a b s t r a c t Neural networks (NNs) are one of the most widely used techniques in literature for forecasting electricity prices. However, nonzero forecast errors always occur, no matter what explanatory variables, NN types, or training methods are used in experiments. Persistent forecasting errors warrant the need for techniques to quantify uncertainties associated with forecasts generated by NNs. Instead of using point forecasts, this study employs the delta and bootstrap methods for construction of prediction intervals (PIs) for uncertainty quantification. The confidence level of PIs is changed between 50% and 90% to check how their quality is affected. Experiments are conducted with Australian electricity price datasets for three different months. Demonstrated results indicate that while NN forecasting errors are large, constructed prediction intervals efficiently and effectively quantify uncertainties coupled with forecasting results. It is also found that while the delta PIs have a coverage probability always greater than the nominal confidence level, the bootstrap PIs are narrower, and by that, more informative. Ó 2013 Elsevier Ltd. All rights reserved.
1. Introduction Within the last two decades, the electricity industry worldwide has been restructured and transformed from a traditional regulated and monopolistic system into a deregulated and competitive system. Accurate forecasting of electricity prices is essential for all participants of deregulated electricity markets. Electricity generators can more efficiently offer their bids to maximize their profit, and consumers can optimize their load schedules to minimize their utilization cost. Also electricity price forecasts are important for medium and long term planning such as making investment decisions and transmission expansion. Unlike other energy resources, electrical energy is economically non-storable and has an instantaneous nature. Also its transmission can be interrupted due to congestion in power networks. These features make electricity price curves volatile, non-stationary, and
⇑ Corresponding author. E-mail address: [email protected] (A. Khosravi). 0306-2619/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.apenergy.2013.05.075
outlier prone in deregulated markets. Price curves are much richer in information than load curves and have many rapid fluctuation caused by participants of the deregulated market. Therefore, price forecasting is a practically complicated task and highly prone to error. Due to its importance, a variety of methods have been proposed in literature for electricity price forecasting, including but not limited to, autoregressive models [1,2], dynamic regression [3,4], transfer function [5], GARCH models [6,7], neural networks (NNs) [8,9], neuro-fuzzy systems [10], and support vector machines [11]. No method is superior for forecasting in all cases. Contradictory conclusions and results have been reported in literature regarding performance of aforementioned methods for price forecasting. It is shown in [8] that wavelet NNs perform better than a series of other forecasting models including wavelet-ARIMA, multi-layer perceptron, radial basis function NNs, and fuzzy NNs. Time series methods, such as ARIMA and transfer functions, have been reported with a performance superior to NNs in [12]. Opposite results are reported in [13] where NNs outperform their time series
121
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
Nomenclature Abbreviation Description 3T 3-time unit ahead 24T 24-time unit ahead ARIMA autoregressive integrated moving average BS bootstrap CI confidence interval CWC coverage width-based criteria D delta GARCH generalized autoregressive conditional heteroscedasticity MAPE mean absolute percentage error NN neural network PI prediction interval PICP prediction interval coverage probability
counterparts including ARIMA models. Better results are achieved in particular for weekends and holidays. Forecasting errors always exist regardless of implemented data preprocessing mechanism, hired forecasting model type, employed explanatory variables, and applied model training method. The main focus of the research community in the last two decades has been on improving the performance of forecasting techniques, but without the discussion on quantifying uncertainties associated with forecasts. It is reasonable to look for techniques to quantify forecasting errors. As critical decisions with significant financial consequences are made based on electricity price forecasts, having access to a measure of uncertainties associated with them is highly beneficial [11,14]. A prediction interval (PI) is composed of an upper and lower bound, where the actual target lies within these bounds with a prescribed probability called the confidence level. The confidence level is often described by (1 a)%, where a varies between zero and one. PIs are more important than confidence intervals (CIs) in practice, as they carry more information about future observations. Wide PIs indicate that there is a high level of uncertainty associated with predictions (forecasts), and therefore decisions should be made more cautiously. In contrast, decisions can be made with greater confidence when constructed PIs have a narrow width. Bayesian [15,16], delta [17,18], bootstrap [19], mean–variance estimation [20], and upper lower bound estimation [23] methods have been proposed in literature for construction of PIs using NNs. Application of these techniques has been documented in the literature in the field of transportation [24,25], manufacturing [26,27], load forecasting [28,29], renewable energies [21,22], and power generation [30]. A comprehensive review and detailed discussion of these methods can be found in [31]. Recursive dynamic factor analysis and Kalman filter are also used in [32] for estimation of variance of electricity prices and construction of prediction intervals. The extreme learning machine algorithm and bootstrapping are employed in [33] for estimating the predictive uncertainties associated with points forecasts. Twelve parametric and semiparametric time series models are applied in [34] for forecasting short term electricity prices and generating intervals. It is concluded that semiparametric methods lead to more reliable intervals than parametric models. The application of CIs and PIs, compared to point prediction, is still in its infancy. A fuzzy based method was proposed in [35] for obtaining the possibility distribution of electricity prices for a given demand value. However, no discussion is made to link this to the concept of CIs and PIs. A method for calculating the probability distributions of electricity prices is proposed in [36]. This method required availability of probability distributions of load and reserve forecasts. A Quasi-Newton method is used in [37] for
PINAW SSE ti ^i y
prediction interval normalized averaged width sum of squared error the ith measured target the ith forecast i noise (error) term w⁄ NN optimal parameters g0 NN gradient F NN Jacobian matrix B Number of bootstrap models N 0; r2 normal distribution with mean zero and variance r2y^ variance due to model misspecification r2^ variance of errors
2
i
the construction of CIs for electricity prices. It is shown that the coverage probability of CIs is acceptable. A decoupled extended Kalman filter-based NN method is employed in [38] for estimation of smaller CIs than those obtained using the Bayesian method for electricity prices. Also, an ARIMA-based method is proposed in [39] for construction of CIs assuming the residual errors are in Gaussian or uniform distribution. No discussion is made regarding PIs for a future electricity price in the aforementioned references. Zhao et al. [11] proposed a hybrid method for construction of PIs using support vector machines (for estimation of price mean) and a nonlinear conditional heteroskedastic forecasting model (for estimation of price variance). It is shown PIs constructed using the proposed method outperform those constructed using GARCH method in terms of coverage probability. However, this paper ignores the important fact that constructed PIs should also be assessed based on their width, as wide PIs carry no information about variation of targets. Relying on the fact that forecasts are always wrong, Baringo and Conejo [40] recently used price intervals, instead of price predictions, to develop hourly offering curves for a price-taker producer, that participates in a pool-based electricity market. Construction of theoretically valid and informative PIs is a prerequisite for the efficient implementation of those methods in real world. Using price intervals for adjusting the hourly load level of a consumer has also been reported in [41]. This is done to optimize the consumer’s energy consumption profile, resulting in its utility maximization. In contrast with previous studies that their focus is either on point forecasts or CIs, this paper uses PIs for quantifying uncertainties surrounding electricity price forecasts generated by NNs. PIs are constructed instead of CIs, as they cover more sources of uncertainties and provide more accurate information about unseen targets. This paper exploits the delta and bootstrap methods for construction of PIs. The motivation for using these methods is their ability for construction of high quality PIs with a low computational cost [31]. Both methods are used and compared to analyze how the volatility of electricity prices affects the quality of PIs. Constructed PIs are quantitatively assessed based on their coverage probability and width. Australian energy market data sets are used for conducting experiments and examining the quality of constructed PIs. In experiments, it is shown that while NN forecasts are always prone to error, constructed PIs appropriately cover the targets and provide valuable information about associated uncertainties. The rest of this paper is organized as follows. Delta and bootstrap methods for construction of NN-based PIs are briefly introduced in Section 2. Section 3 describes performance measures for assessing the quality of PIs. Simulation results are demonstrated in Section 4. Section 5 concludes the paper with some remarks and guidelines for future work.
122
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
misspecification variance can be estimated using the variance of B model outcomes,
2. PI Construction methods 2.1. The delta method
r2y^i ¼
B b 2 1 X ^ y ^i : y B 1 b¼1 i
ð8Þ
The delta method is based on linearization of a NN model around the set of its optimal parameters, w⁄. PIs are then constructed for the linearized model. A statistical model for relating targets to the model can be assumed as follows,
For the construction of PIs, the variance of errors, r2^i , should be also estimated. r2^ can be calculated as follows,
t i ¼ f ðxi ; w Þ þ i
r2^ ’ Efðt y^Þ2 g r2y^ :
ð1Þ
where ti is the ith measured target (totally n targets), xi is the ith vector of inputs, and f (xi, w⁄) is a smooth NN model. i is the noise, also called error, with a zero expectation. In a small neighborhood of w⁄, we have,
^0 ¼ f ðx0 ; w Þ þ g T0 ðw ^ w Þ y
ð2Þ
g T0
where is the NN output gradient against the network parameters. ^ are adjusted through minimization of the sum of NN parameters, w, squared error (SSE) cost function. Under certain regularity condi^ is very close to w⁄. Accordingly, we tions, it can be shown that w have,
^0 ’ 0 þ g T0 ðw ^ w Þ t0 y
ð3Þ
The two remaining terms in right hand side of (3) are statistically independent. Assuming that the error terms are normally distributed ð Nð0; r2 ÞÞ, the second term in the right hand side of (3) can be expressed as, 1
r2y^0 ¼ r2 g T0 ðF T FÞ g 0
ð4Þ
where F is the Jacobian matrix of the NN model. The total variance can be expressed as, 1
r20 ¼ r2 ð1 þ g T0 ðF T FÞ g 0 Þ
ð5Þ
SSE An unbiased estimate of r2 is n1 . According to this, the (1 a)% PI ^i is computed as detailed in [17], for y
^0 y
1a tnp2 s
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 1 þ g T0 ðF T FÞ g 0
1a t np2
ð6Þ
a
where is the 2 quantile of a cumulative t-distribution function with n p degrees of freedom. The following formula was derived in [18] for PI construction for the case that NNs are trained using the weight decay cost function instead of the SSE cost function,
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1a ^0 tnp2 s 1 þ g T0 Wg 0 y T
1
ð7Þ T
T
1
where W = (F F + kI) (F F)(F F + kI) . Inclusion of k in (7) improves the reliability and quality of PIs, particularly for cases that FTF is nearly singular.
According to (9), a set of variance squared residuals is developed,
^i Þ2 r2y^ ; 0 r2i ¼ max ðt i y i
ð10Þ
^i and ry^2 both are available from B trained NN models and where y i (8). These residuals are linked by the set of corresponding inputs to form a new dataset,
n Dr2 ¼ ðxi ; r 2i Þ i¼1 :
ð11Þ
A new NN model, called NN r^ , can be trained using the maximum likelihood estimation method to estimate the unknown values of r2^i , so as to maximize the probability of observing the samples in Dr2 [19]. In total, B + 1 NN models are required in the bootstrap method for PI construction. Simplicity is the main advantage of using the bootstrap method for PI construction. In contrast to the delta technique, there is no need to calculate complex matrices and derivatives. 3. Performance index 3.1. Performance index for point forecasts The forecasting accuracy of NNs is measured using mean absolute percentage error (MAPE) as the performance index:
MAPE ¼
n ^ 1X yt yt n t¼1 yt
ð12Þ
^t are the actual and forecasted price at time t, respecwhere yt and y tively. n is also the number of observations used in experiments. 3.2. Performance index for prediction intervals A combinational coverage width-based criterion (CWC) is used for assessing the quality of constructed PIs,
CWC ¼ PINAWð1 þ cðPICPÞ egðPICPlÞ Þ
ð13Þ
where PINAW and PICP are the PI normalized averaged width and PI coverage probability, respectively. PICP is the spontaneous measure related to the quality of constructed PIs and measures the quantity of targets bracketed by intervals,
2.2. The bootstrap method
PICP ¼ Bootstrap is essentially a resampling technique for estimating the distribution of targets [42]. It mimics the random mechanism of the underlying target through resampling with replacement of the available data set. This leads to an empirical distribution of the underlying target that can be used for construction of CIs and PIs. B training datasets are resampled from the original dataset with replacement, fDgBb¼1 . The method estimates the variance due to model misspecification, r2y^ , by building B NN models. According to this assumption, the true regression is estimated by P ^i ¼ 1B Bb¼1 y ^bi , where y ^bi averaging the point forecasts of B models, y is the prediction of the i-th sample generated by the b-th bootstrap model. Assuming that NN models are unbiased, the model
ð9Þ
n 1X ct n t¼1
ð14Þ
where ct = 1 if yt 2 [Lt, Ut], otherwise ct = 0. Lt and Ut are the lower and upper bounds of the tth PI respectively. Theoretically, PICP P (1 a)%). Constructed PIs are not reliable and should be discarded if this condition is not satisfied. Practically, narrow PIs are more informative than wide PIs. PINAW assesses PIs from this aspect and measures how wide they are,
PINAW ¼
n 1 X ðU t Lt Þ nR t¼1
ð15Þ
123
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
(b) August 2010
(a) May 2010
(c) December 2010 Fig. 1. Australian electricity price datasets used in this study.
Table 2 Parameters used in experiments. Table 1 Inputs of NN models used for forecasting electricity prices. Forecast Horizon
Lagged prices
3-Time unit ahead 24-Time unit ahead
[yt3, yt4, yt5, yt24, yt25, yt26] [yt24, yt25, yt26, yt48, yt49, yt50]
Parameter
Numerical value
Dtrain Dtest Maximum number of neurons
80% Of each month samples 20% Of each month samples 10 0.1 50 0.90 5 10 4–2
a g l K-fold cross validation Number of bootstrap models Structure of NNr^
Split the data set into training and test sets (Dtrain and Dtest)
Perform a K-fold cross validation to determine the optimal structure for NN
Construct PIs using delta and bootstrap methods
Evaluate the quality of constructed PIs using performance index CWC Fig. 2. The experimental procedure for PI construction and evaluation.
where R is the range of the underlying target. PINAW is the average width of PIs as a percentage of the underlying target range. Inclusion of the PINAW and PICP in CWC allows one to simultaneously evaluate PIs based on their width (informativeness) and coverage probability (correctness). c(PICP) is also given by the following step function,
c¼
0
PICP P l
1 PICP < l
ð16Þ
This means that the exponential term in (13) is eliminated whenever PICP P l and CWC becomes equal to PINAW. Further information about these measures and hyperparameters g and l can be found in [23,43].
124
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
(a) MAPE 3T for May 2010
(b) MAPE3T for August 2010
(c) MAPE 3T for December 2010
(d) MAPE24T for May 2010
(e) MAPE 24T for August 2010
(f) MAPE24T for December 2010 Fig. 3. Contour plots of averaged MAPE for test samples calculated on a grid of NN structure (n1 and n2 are the number of neurons in the first and second layers of NNs, respectively).
125
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
4. Numerical experiments 4.1. Data and experimental procedure National electricity market (NEM) in Australia was deregulated and privatized more than a decade ago. The NEM spot pool market is conducted by Australian Energy Market Operator (AEMO). It receives offers by electricity generators in 5 min intervals every day. Offers are processed by AEMO and generators are instantaneously scheduled to meet the demand in the most cost effective way. Dispatch prices are advised every 5 min. Then, the spot prices are determined every 30 min by averaging the most recent six dispatch prices. These spot prices are used in the paper for examining performance of PI construction methods. Three one-month datasets of Australian electricity prices for the Victorian region during 2010 are used to examine and validate the performance of PI construction methods. May, August, and December months represent three different seasons of a year in Australia. They correspond to autumn, winter, and summer respectively. These have been selected to make the test results less subjective. Three month electricity prices with half an hour intervals are displayed in Fig. 1. Spikes are more frequent in May and August. However, previously unobserved patterns are more common in December. This means that electricity price forecasting errors in December should be greater than those for May and August. The datasets used in this study are publicly available at www.aemo.com.au. Australian electricity prices have been used in a set of studies before [44,2]. However, those studies focus on how accurate point forecasts can be generated using different types of models. In contrast, this paper deals with the problem of quantification of uncertainties associated with forecasts. Prices are far from being a stationary process when expressed in their original format [11]. In order to tackle the problem of nonstationary price series, we do a first-order differencing transformation [45]. Calculating changes from one period to the next removes the trend in mean and delivers a more stationary series. Three (3) and twenty-four (24) time unit ahead horizons are considered for forecasting electricity prices, where the time unit is 30 min. Table 1 shows lagged values of actual prices used as independent input variables to the NNs. Both sets of inputs have a similar format and include six lagged values. Selection of these lagged values is done based on analysis of autocorrelation and partial correlation of electricity prices. The Matlab package is used to carry out simulations and visualize results. The experimental procedure for construction of PIs using the delta and bootstrap methods is shown in Fig. 2. 80% of
each month samples are used for training of NN models (Dtrain), and the remaining 20% samples are applied for examining performance of developed models (Dtest). A fivefold cross validation technique is applied to determine the optimal NN structure and the number of neurons per layer. A maximum of ten neurons is allowed in each hidden layer. Once the NN model is selected, it is used for construction of PIs using the delta and bootstrap techniques. The quality of constructed PIs is then assessed using CWC as the performance index. Parameters used for implementing the delta and bootstrap methods and conducting experiments are listed in Table 2. All PIs are constructed with a 90% confidence level (a = 0.1). Ten NNy models are considered in the bootstrap method. NN r^ is considered to have two layers with four and two neurons in its hidden layers. l is set to 0.9, because the PI confidence level is 90%. Also, g is selected to be 50 in order to highly penalize PIs with a coverage probability lower than the nominal confidence level. 4.2. Results and discussion In the first stage, performance of two layer NNs is examined for forecasting electricity prices. NN forecasting performance intimately depends on its structure (number of layers) and complexity (number of neurons per layer). Preliminary results showed that two layer NNs outperform NNs with a single hidden layer. Therefore, all experiments are reported for NNs with two hidden layers. The number of neurons in the hidden layers of NNs is changed between 1 and 10. We limit the number of neurons in each hidden layer to 10 to keep NN models parsimonious. For each NN structure, experiments are repeated ten times and then the averaged performance index, MAPE, is calculated. This is done to avoid misleading results due to random initialization of NN parameters. Fig. 3 displays filled contour plots of averaged MAPEs for test samples of three one-month datasets. Horizontal and vertical axes indicate the number of neurons in the first (n1) and second (n2) hidden layers of NNs. The color map in the right hand side of each plot shows the color-coded value of MAPE. There are several peaks and valleys, which are a sign of sensitivity of MAPE to the NN structure and its initial parameters (assigned randomly). There are also regions where MAPE levels off and does not fluctuate. For the case of 3-time unit ahead forecasting, the best achieved results are 20.24%, 12.39%, and 26.69% for 3 months respectively. MAPEs for 24-time unit horizon are 20.19%, 11.46%, and 28.68% respectively. These results clearly show that forecasts obtained by NNs are far from being perfect. Large MAPEs, in particular for test samples of May and December, are unavoidable, regardless of the NN structure and the training mechanism. Also, it is important to notice
Table 3 PICP, PINAW, and CWC for PID,3T and PIBS,3T. Month
May 2010 August 2010 December 2010
Delta PIs
Bootstrap PIs
PICP (%)
PINAW (%)
CWC
PICP (%)
PINAW (%)
CWC
97.94 98.39 100
41.50 37.67 416.81
41.50 37.67 416.8
89.38 90.41 82.19
15.22 10.95 19.38
35.93 10.95 980.6
Table 4 PICP, PINAW, and CWC for PID,24T and PIBS,24T. Month
May 2010 August 2010 December 2010
Delta PIs
Bootstrap PIs
PICP (%)
PINAW (%)
CWC
PICP (%)
PINAW (%)
CWC
98.26 97.91 100
40.20 34.58 423.4
40.20 34.58 423.4
76.66 89.55 84.67
7.08 7.84 16.01
5606 17.66 246.1
126
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
that best results in all experiments are obtained by NNs with a small size. This confirms that more parsimonious NNs have a better generalization power when used for time series forecasting [46]. Hereafter for the sake of simplicity, 3T and 24T subscripts are used to indicate 3-time unit and 24-time unit ahead forecasts. Also D and BS are abbreviations for the delta and bootstrap methods for PI construction. For example, PID,3T indicates delta PIs constructed for 3-time unit ahead forecasts.
(a) PID,3T for August 2010
PICP, PINAW, and CWC for PID and PIBS constructed for 3 and 24 time unit horizon forecasts are summarized in Tables 3 and 4. PID,3T and PID,24T constructed for each month dataset are theoretically correct, as PICP P 90%. This is due to the fact that the confidence level associated with PIs is 90%. Satisfaction of this requirement by PID signifies that the delta method amply takes care of the coverage probability of PIs. PICPBS,3T and PICPBS,24T are less than 90% in five out of six case studies. The worst result is obtained for 24-time unit ahead forecasts of May, where the coverage probability of PIBS,24T is 76.66%. These results indicate that delta PIs are more reliable than bootstrap PIs (PICPD is always greater than the nominal confidence level). The greater coverage of PIs constructed using the delta method comes with the cost of wider intervals. As per the results in Tables 3 and 4, PID are significantly wider than PIBS for test samples of 3 months for both forecast horizons. The extra width of delta PIs is acceptable for some cases, e.g., August 2010, as PICP is very close to 100%. However, PID,3T and PID,24T for December 2010 are so wide that they are uninformative. PINAW is 416.8% and 423.4% for these cases. This means that the width of constructed PIs is four times greater than the target range on average. In contrast, PIBS,3T and PIBS,24T are narrow and take the same pattern of the underlying target. Tables 3 and 4 also report CWCBS/D,3T/24T. CWCBS is smaller than CWCD in four out of six cases. The small value of CWCBS is mainly due to the narrowness of PIBS, compared to excessively wide PID. CWCs indicate that for electricity prices PIBS are of a better quality than PID for both forecasting horizons. In a separate experiment, we change the level of confidence between 50% and 90% to check how width and coverage probability of PIs will vary. Simulations are done for August 2010 datasets using the delta and bootstrap methods. Fig. 4 displays four plots corresponding to PID,3T,PID,24T,PIBS,3T, and PIBS,24T. In each plot, there are five shaded bands representing PIs constructed for a 50% confidence level (the inner band with a darker color), 60%, 70%, 80%,
(b) PIBS ,3T for August 2010
(c) PID,24T for August 2010
Fig. 5. PICP for PIs constructed with confidence levels ranging between 50% and 90% for August 2010.
(d) PIBS ,24T for August 2010 Fig. 4. PID,3T, PIBS,3T, PID,24T, and PIBS,24T constructed for different values of confidence level (50%, 60%, 70%, 80%, and 90%).
Fig. 6. PINAW for PIs constructed with confidence levels ranging between 50% and 90% for August 2010.
127
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
and a 90% confidence level (the outer band with a brighter color). Targets are also shown with bright circles. With the purpose of better visualization, only PIs for the first one hundred test samples are displayed. Theoretically, it is expected that PIs with a 50% confidence level will cover at least 50% of targets and so on. It is interesting to notice that PICPD,3T and PICPD,24T are always significantly greater than the prescribed confidence level. For instance, when the confidence levels are 50% or 60%, PICPD is always more than 87%, as displayed in Fig. 5. PICPD,3T and PICPD,24T for a confidence level of 60% are 91.10% and 95.82%, respectively. These values are 67.94% and 67.94% for corresponding bootstrap PIs. Therefore, bootstrap PIs are less conservative with regard to the coverage probability when compared to delta PIs. A visual analysis makes it clear that PID,3T and PID,24T are significantly wider than their corresponding PIs constructed using the bootstrap method. The numerical values of PINAW for constructed PIs are illustrated in Fig. 6. The width of PIBS constructed with a 90% confidence level is less than the width of PID constructed with a 50% confidence level. Therefore, the former one not only is more reliable, but also is more informative. However, this conclusion is not true for data samples of the other 3 months, as shown in Tables 3 and 4. A closer look at PID,3T and PID,24T in Fig. 4 shows that widths of PIs constructed for a specific level of confidence are almost constant and do not change from one sample to another. Centers of PID fluctuate depending on input values used for forecasting electricity prices, but their widths remain almost unchanged. In contrast, PIBS,3T and PIBS,24T have variable widths. This observation
suggests that bootstrap PIs are more sensitive to the data distribution and uncertainties than the delta PIs. When there is a high level of uncertainty in training data, PIBS become wider to accommodate associated uncertainties. They remain narrow when uncertainties are low and ignorable. As per the coverage of the inner tube for PID,3T and PID,24T, one may use PIs with a confidence level of 50% or 60% for decision making. These intervals are acceptably narrow (PINAW ’ 15%) with a large PICP (approximately 90%). What the optimal value of the confidence level is and how it is linked to the process of decision-making in electricity markets needs more attention and investigation. Comparing PICP and PINAW in Figs. 5 and 6 shows that PICPD does not quickly drop as the confidence level is decreased from 90% to 50%. However, PICPBS,3T and PICPBS,24T show a strong correlation with the confidence level (0.999 and 0.998 respectively). The same decreasing pattern is also observed for PINAW as the confidence level is decreased. This signifies that there is a dominant constant term in (7) leading to wide PIs with an excessively large coverage probability. According to these results, the delta method sacrifices the informativeness of PIs in order to keep them theoretically valid. In contrast, the bootstrap method attempts to generate narrow PIs, which may occasionally lead to an unsatisfactorily low coverage probability. These results and conclusions comply with those reported in previous studies in other fields [47]. It is important to investigate why PIBS often have an unsatisfactorily low PICP. Australian electricity market prices, similar to the prices from other markets, are highly volatile, fat-tailed, skewed,
(b) August 2010
(a) May 2010
(c) December 2010 Fig. 7. Probability plots for 3 month electricity prices used in this study.
128
A. Khosravi et al. / Applied Energy 112 (2013) 120–129
and do not follow a normal distribution. A normal probability plot is here used to aid with the clear presentation of the data distribution and characteristics. This plot provides a good indication of whether or not the data comes from a normal distribution. The normal probability plots for the 3 months are shown in Fig. 7. The color-filled circles show the empirical probability versus the data value for each point in the sample. Superimposed on the plot is a line connecting the 25th and 75th percentiles of the data, which can be seen by a dashed line in the three plots. This line is extrapolated out to the ends of the data samples for a better linearity evaluation. If all electricity prices fall near the dashed line, forming a linear plot, then it can be concluded that the data comes from a normal distribution. Electricity prices for all 3 months show a high level of curvature, which strongly denies this assumption. This means that the bootstrap method, which is based on the normality assumption, is not the best candidate for construction of PIs for electricity prices. These evidences justify the low PICPs reported in Tables 3 and 4 obtained using the bootstrap method. The ultimate goal of construction of PIs is to integrate them into decision-making algorithms and expert systems. Both delta and bootstrap PIs can be used in conjunction with point forecasts for decision-making. Depending on their roles and interests, stakeholders in the energy market can differently interpret and use PIs in their decision-making processes to maximize their benefits. A power generation company, for instance, may optimistically use the upper bound of an electricity price PI in its offer. Another company may always select the lower bound in its bids, which reflects a pessimistic attitude (worst case scenario). For an electricity buyer, wide price intervals indicate that there is a high level of uncertainty in the market; so all prices should be well negotiated. In contrast, narrow intervals are a sign of low uncertainty in the market; therefore, forecasts can be trusted. In another approach, upper and lower bounds of prices can be used as extreme values or limits in the optimization problems solved by utilities and generation companies. Design of algorithms for efficient use of PIs for decision-making in power system is a very fruitful research area. Some researchers have already started using confidence and prediction intervals instead of point forecasts to develop robust offering strategies in electricity markets [40,41]. PID and PIBS can be both integrated into those optimization techniques for generating robust solutions.
5. Conclusions The purpose of this paper is to give new momentum to the application of PI construction techniques in the field of electricity price forecasting. The key idea is to change focus from point forecasts to PIs. It is first shown that NN forecasting errors always exist and cannot be eliminated. The best MAPEs for different datasets and forecasting horizons vary between 11.46% and 28.68%. Therefore, PIs are constructed to quantify associated uncertainties with forecasts. The delta and bootstrap methods are applied in this paper for construction of neural network-based PIs for electricity price forecasts. The confidence level associated with PIs is also changed between 50% and 90%. It is observed that delta-based PIs have a more acceptable coverage probability. There is always a large gap between the coverage probability of the delta PIs and their nominal confidence level. The conservatively of the delta method to the PI coverage probability comes with the massive cost of generating too wide PIs. In contrast, bootstrap PIs are narrower, and therefore, more informative. Besides, variable width bootstrap-based PIs more appropriately respond to uncertainties in data than delta PIs. However, it is found that the fundamental normality assumption in the bootstrap method may cause some problems leading to construction of unreliable PIs.
Future research can focus on how to improve the quality of constructed PIs. Optimization methods can be applied to find a trade off between two conflicting issues of PIs: being narrow (more informative) and having a large coverage probability (correctness). Constructed PIs by the proposed methods can be used along with electricity price forecasts in the decision-making processes in energy markets. PIs signify the level of uncertainty associated with forecasts and how trustworthy they are. Acknowledgment This study is partially supported by the Australian Research Council (Discovery Grant DP120102112). References [1] Tan Z, Zhang J, Wang J, Xu J. Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models. Appl Energy 2010;87(11):3606–10. [2] Dong Y, Wang J, Jiang H, Wu J. Short-term electricity price forecast based on the improved hybrid model. Energy Convers Manage 2011;52(8–9):2987–95. [3] Pedregal DJ, Trapero JR. Electricity prices forecasting by automatic dynamic harmonic regression models. Energy Convers Manage 2007;48(5):1710–9. ISSN 0196-8904. [4] Nogales F, Contreras J, Conejo A, Espinola R. Forecasting next-day electricity prices by time series models. IEEE Trans Power Syst 2002;17(2):342–8. ISSN 0885-8950. [5] Zareipour H, Canizares C, Bhattacharya K, Thomson J. Application of publicdomain market information to forecast Ontario’s wholesale electricity prices. IEEE Trans Power Syst 2006;21(4):1707–17. ISSN 0885-8950. [6] Garcia R, Contreras J, van Akkeren M, Garcia J. A GARCH forecasting model to predict day-ahead electricity prices. IEEE Trans Power Syst 2005;20(2):867–74. ISSN 0885-8950. [7] Garcia-Martos C, Rodriguez J, Sanchez MJ. Modelling and forecasting fossil fuels, CO2 and electricity prices and their volatilities. Appl Energy 2013;101:363–75. [8] Pindoriya N, Singh S, Singh S. An adaptive wavelet neural network-based energy price forecasting in electricity markets. IEEE Trans Power Syst 2008;23(3):1423–32. ISSN 0885-8950. [9] Lin WM, Gow HJ, Tsai MT. An enhanced radial basis function network for shortterm electricity price forecasting. Appl Energy 2010;87(10):3226–34. ISSN 0306-2619. [10] Khotanzad A, Zhou E, Elragal H. A neuro-fuzzy approach to short-term load forecasting in a price-sensitive environment. IEEE Trans Power Syst 2002;17(4):1273–82. ISSN 0885-8950. [11] Zhao JH, Dong ZY, Xu Z, Wong KP. A statistical approach for interval forecasting of the electricity price. IEEE Trans Power Syst 2008;23(2):267–76. ISSN 08858950. [12] Conejo AJ, Contreras J, Espfnola R, Plazas MA. Forecasting electricity prices for a day-ahead pool-based electric energy market. Int J Forecasting 2005;21(3):435–62. ISSN 0169-2070. [13] Pino R, Parreno J, Gomez A, Priore P. Forecasting next-day price of electricity in the Spanish energy market using artificial neural networks. Eng Appl Artif Intell 2008;21(1):53–62. ISSN 0952-1976. [14] Amjady N, Hemmati M. Energy price forecasting – problems and proposals for such predictions. IEEE Power Energy Mag 2006;4(2):20–9. ISSN 1540-7977. [15] MacKay DJC. The evidence framework applied to classification networks. Neural Comput 1992;4(5):720–36. [16] Bishop CM. Neural networks for pattern recognition. Oxford: Oxford University Press; 1995. [17] Hwang JTG, Ding AA. Prediction intervals for artificial neural networks. J Am Stat Assoc 1997;92(438):748–57. ISSN 01621459. [18] Veaux RDd, Schumi J, Schweinsberg J, Ungar LH. Prediction intervals for neural networks via nonlinear regression. Technometrics 1998;40(4):273–82. ISSN 00401706. [19] Heskes T. Practical confidence and prediction intervals. In: Mozer TPM, Jordan M, editors. Neural information processing systems, vol. 9. MIT Press; 1997. p. 176–82. [20] Nix D, Weigend A. Estimating the mean and variance of the target probability distribution. In: IEEE international conference on neural networks; 1994. [21] Khosravi A, Nahavandi S. Combined non-parametric prediction intervals for wind power generation. IEEE Trans on Sust Energy, 2013. in Press. [22] Khosravi A, Nahavandi S, Creighton D. Prediction intervals for short-term wind farm power generation forecasts. IEEE Trans on Sust Energy, 2013, in press. [23] Khosravi A, Nahavandi S, Creighton D, Atiya AF. A lower upper bound estimation method for construction of neural network-based prediction intervals. IEEE Trans Neural Networks 2011;22(3):337–46. [24] Khosravi A, Mazloumi E, Nahavandi S, Creighton D, van Lint JWC. Prediction intervals to account for uncertainties in travel time prediction. IEEE Trans Intell Transport Syst 2011;12(2):537–47.
A. Khosravi et al. / Applied Energy 112 (2013) 120–129 [25] Khosravi A, Mazloumi E, Nahavandi S, Creighton D, van Lint JWC. A genetic algorithm-based method for improving quality of travel time prediction intervals. Transport Res Part C [accepted 11.04.11]. [26] Yu G, Qiu H, Djurdjanovic D, Lee J. Feature signature prediction of a boring process using neural network modeling with confidence bounds. Int J Adv Manuf Technol 2006;30(7):614–21. [27] Khosravi A, Nahavandi S, Creighton D. A prediction interval-based approach to determine optimal structures of Neural Network Metamodels. Expert Syst Appl 2010;37:2377–87. ISSN 0957-4174. [28] Khosravi A, Nahavandi S, Creighton D. Load Forecasting and Neural Networks: a prediction interval-based perspective. In: Panigrahi BK, et al., editors. Computational intelligence in power engineering. SCI 302; 2010. p. 131–50. [29] Khosravi A, Nahavandi S, Creighton D. Construction of optimal prediction intervals for load forecasting problem. IEEE Trans Power Syst 2010;25:1496–503. [30] Zio E. A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes. IEEE Trans Nucl Sci 2006;53(3):1460–78. ISSN 0018-9499. [31] Khosravi A. Construction and Optimisation of prediction intervals for neural network models, Ph.D. Thesis. Deakin University; 2010. [32] Wu HC, Chan SC, Tsui KM, A new recursive dynamic factor analysis for point and interval forecast of electricity price. IEEE Trans on Power Syst 2013. in press. http://dx.doi.org/10.1109/TPWRS.2012.2232314. [33] Chen X, Dong ZY, Meng K, Xu Y, Wong KP, Ngan H. Electricity price forecasting with extreme learning machine and bootstrapping. IEEE Trans Power Syst 2012;27(4):2055–62. ISSN 0885-8950. [34] Weron R, Misiorek A. Forecasting spot electricity prices: a comparison of parametric and semiparametric time series models. Int J Forecasting 2008;24(4):744–63. ISSN 0169-2070. [35] Niimura T, Nakashima T. Deregulated electricity market data representation by fuzzy regression models. IEEE Trans Syst Man Cyber Part C: Appl Rev 2001;31(3):320–6. ISSN 1094-6977..
129
[36] Yamin HY, Shahidehpour SM, Li Z. Adaptive short-term electricity price forecasting using artificial neural networks in the restructured power markets. Int J Electrical Power Energy Syst 2004;26(8):571–81. ISSN 01420615. [37] Zhang X, Rice JA. Short-term travel time prediction. Transport Res Part C: Emerging Technol 2003;11(3–4):187–210. ISSN 0968-090X. [38] Zhang L, Luh P. Neural network-based market clearing price prediction and confidence interval estimation with an improved extended Kalman filter method. IEEE Trans Power Syst 2005;20(1):59–66. ISSN 0885-8950. [39] Zhou M, Yan Z, Ni Y, Li G, Nie Y. Electricity price forecasting with confidenceinterval estimation through an extended ARIMA approach. IEE Proc Generation Transm Distribution 2006;153(2):187–95. ISSN 1350-2360. [40] Baringo L, Conejo AJ. Offering strategy via robust optimization. IEEE Trans Power Syst. doi:10.1109/TPWRS.2010.2092793 ISSN 0885-8950 [13.12.10]. [41] Conejo AJ, Morales JM, Baringo L. Real-time demand response model. IEEE Trans Power Syst 2010;1(3):236–42. [42] Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman and Hall; 1993. [43] Khosravi A, Nahavandi S, Creighton D. Prediction interval construction and optimization for adaptive neurofuzzy inference systems. IEEE Trans Fuzzy Syst 2011;19(5):983–8. [44] Amjady N. Day-ahead price forecasting of electricity markets by a new fuzzy neural network. IEEE Trans Power Syst 2006;21(2):887–96. ISSN 0885-8950. [45] Box G, Jenkins G, Reinsel GC. Time series analysis: forecasting and control. 4th ed. Hoboken, N.J: John Wiley; 2008. [46] Zhang GP, Qi M. Neural network forecasting for seasonal and trend time series. Eur J Operational Res 2005;160(2):501–14. ISSN 0377-2217. [47] Khosravi A, Nahavandi S, Creighton D, Atiya A. Comprehensive review of neural network-based prediction intervals and new advances. IEEE Trans Neural Networks 2011;22(9):1341–56. ISSN 1045-9227.