- Email: [email protected]
Comparison of forecast performance for homogeneous, heterogeneous and shrinkage estimators
Economics Letters 76 (2002) 375–382 www.elsevier.com / locate / econbase
Comparison of forecast performance for homogeneous, heterogeneous and shrinkage estimators Some empirical evidence from US electricity and natural-gas consumption Badi H. Baltagi a , *, Georges Bresson b , Alain Pirotte c a
Texas A& M University, Department of Economics, College Station, TX 77843 -4228, USA b ERMES( CNRS), Universite´ Paris II, 92 Rue d’ Assas, 75270 Paris, Cedex 06, France c ´ Universite Lyon II, Faculte de Sciences Economiques et de Gestion, 16 quai Claude Bernard, 69365 Lyon, Cedex 07, France Received 17 December 2001; accepted 19 February 2002
Abstract Maddala et al. [Journal of Business and Economic Statistics, 15 (1997) 90] obtained short-run and long-run elasticities of energy demand for each of 49 US states over the period 1970–1990. They showed that heterogeneous time series estimates for each state yield inaccurate signs for the coefficients, while panel data estimates are not valid because the hypothesis of homogeneity of the coefficients was rejected. Their preferred estimates are those obtained using the shrinkage estimator. This paper contrasts the out-of-sample forecast performance of heterogeneous, panel and shrinkage estimators using the Maddala et al. [Journal of Business and Economic Statistics 15 (1997) 90] electricity and natural gas data sets. Our results show that the homogeneous panel data estimates give the best out-of-sample forecasts. 2002 Elsevier Science B.V. All rights reserved. Keywords: Panel data; US electricity and natural-gas demand; Heterogeneous estimators; Shrinkage estimators JEL classification: C23
1. Introduction Maddala et al. (1997) applied classical, empirical Bayes and Bayesian procedures to the problem of estimating short-run and long-run elasticities of residential demand for electricity and natural gas in * Corresponding author. Tel.: 11-409-845-7380; fax: 11-409-847-8757. E-mail address: [email protected] (B.H. Baltagi). 0165-1765 / 02 / $ – see front matter PII: S0165-1765( 02 )00065-4
2002 Elsevier Science B.V. All rights reserved.
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
376
the US for each of 49 states. Since the elasticity estimates for each state are the ultimate goal of their study they were faced with three alternatives: The first is to use individual time series regressions for each state. These gave bad results, were hard to interpret and had several wrong signs. The second option was to pool the data and use panel data estimators. Although the pooled estimates gave the right signs and were more reasonable, Maddala et al. (1997) argued that these estimates are not valid because the hypothesis of homogeneity of the coefficients is rejected. The third option which they recommended was to allow for some (but not complete) heterogeneity or (homogeneity). This approach lead them to their preferred shrinkage estimator which gave them more reasonable parameter estimates. This approach is not without its critics. In fact, Hsiao (1986) has argued convincingly about the gains from pooling. Also, in the context of gasoline demand across 18 OECD (Organization for Economic Co-Operation and Development) countries over the period 1960–1990, Baltagi and Griffin (1997) argued for pooling the data as the best approach for obtaining reliable price and income elasticities even when the poolability assumption is rejected. They also pointed out that pure cross-section studies cannot control for unobservable country effects, whereas pure time-series studies cannot control for unobservable oil shocks or behavioral changes occurring over time. On the other extreme, Pesaran and Smith (1995) advocated abandoning the pooled approach altogether questioning the poolability of the data across heterogeneous units. Instead, they argue in favor of heterogeneous estimates that can be combined to obtain homogeneous estimates if the need arises. One such estimator being the average response from individual regressions. Depending on the extent of cross-sectional heterogeneity in parameters, researchers may prefer these heterogeneous estimators to the traditional pooled homogeneous parameter estimators or the shrinkage estimators proposed by Maddala et al. (1997). Recently, Baltagi and Griffin (1997) and Baltagi et al. (2000) showed that homogeneous panel data estimators beat the heterogeneous and shrinkage type estimators in RMSE performance for out-of-sample forecasts. These empirical results were performed for a gasoline data set across 18 OECD countries over 31 years (1960–1990) and for a cigarette demand data set across 46 states over 30 years (1963–1992). Using the Maddala et al. (1997) specification and data sets, our objective here is to compare the out-of-sample forecast performance of 10 homogeneous and nine heterogeneous estimators including the shrinkage estimators applying them to electricity and natural-gas across 49 states over the period 1970–1990. Section 2 describes the model specification of Maddala et al. (1997), the data sets and the pooled and heterogeneous estimators to be compared. Section 3 compares the forecast performance of these estimators using 1970–1985 for estimation purposes and 1986–1990 for out-of-sample forecasts. Section 4 summarizes the results and concludes.
2. Model specification and estimators
2.1. Model specification Maddala et al. (1997) considered the following standard dynamic linear regression (DLR) model for energy demand y i,t 5 bi,0 1 bi,1 y i,t 21 1 bi,2 x 1,i,t 1 bi,3 x 1,i,t 21 1 bi,4 x 2,i,t 1 bi,5 x 2,i,t21 1 bi,6 x 3,i,t 1 bi,7 x 4,i,t 1 bi,8 x 5,i,t 1 u i,t
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
377
where i 5 1, 2, . . . , 49 (states) and t 5 2, 3, . . . , 21 (years) spanning the period 1970–1990. The variables for the electricity regression are: • • • • • •
y i,t , the logarithm of residential electricity per capita consumption; x 1,i,t , the logarithm of real per capita personal income; x 2,i,t , the logarithm of real residential electricity price; x 3,i,t , the logarithm of real residential natural-gas price; x 4,i,t , heating degree days; 0 x 5,i,t , cooling degree days. For the natural-gas regression, we have:
• • • •
y i,t , the logarithm of residential natural-gas per capita consumption; x 1,i,t , the logarithm of real per capita personal income; x 2,i,t , the logarithm of real residential natural-gas price; x 3,i,t , the logarithm of real residential electricity price;
with x 4,i,t and x 5,i,t unchanged. This is a DLR (1,1) model derived by taking account of error correction and partial adjustment mechanisms (see Alogoskoufis and Smith (1991) for a recent survey of these models). For a description of the data sets, see Maddala et al. (1997). These data sets can be downloaded from the journal’s web site.
2.2. Homogeneous estimators In this case, the coefficients are homogeneous across states and we follow the usual convention of assuming that the disturbance term is specified as a one-way error component model, see Hsiao (1986): u i,t 5 ai 1 ni,t where ai denotes a state-specific effect and ni,t is IID (0,s n2 ). We first consider three standard pooled estimators, assuming the exogeneity of all the regressors. These include ordinary least squares (OLS), which ignores the state effects, the Within estimator, which allows for fixed state effects, and generalized least squares (GLS), which assumes that state effects are random. Since our model is dynamic, and even if all the explanatory variables are uncorrelated with the error components, the presence of serial correlation in the remainder error term or the presence of a random state effect renders the lagged dependent variable correlated with the error term and leads to inconsistent least squares estimates. Consequently, we also focus on pooled estimators employing two-stage least squares (2SLS) using as instruments the exogenous variables and their lagged values. In particular, we examine five alternative 2SLS pooled estimators. First, we consider a standard 2SLS estimator, making no attempt to improve efficiency by taking into account the random state effect. This estimator is consistent only if the exogenous regressors are uncorrelated with the unobservable individual effects. Second, we report the Within 2SLS estimator, which transforms the data about state means and thereby eliminates any state fixed effect. In addition, this instruments for the presence of the lagged dependent variable
378
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
by using exogenous variables and their lagged values. The third estimator is first-difference 2SLS (FD2SLS) proposed by Anderson and Hsiao (1982) in which fixed or random state effects are eliminated. However, first-differencing introduces autocorrelation in the remainder error term (ni,t 2 ni,t21 ) and thus inefficiency problems. Nevertheless, consistency is preserved by the use of predetermined variables as instruments. Keane and Runkle (1992), (hereafter denoted by KR) suggest a modification of the 2SLS estimator that allows for any arbitrary type of serial correlation in the ni,t ’s. We refer to this estimator as 2SLS-KR. Still another variant would be to allow for any arbitrary form of serial correlation in the first differenced disturbances in the manner of Keane–Runkle. This is denoted as the FD2SLS-KR estimator. Finally, following Arellano and Bond (1991), we used a generalized method of moments (GMM) estimator on the first-differences specification (FDGMM) with instruments in levels. This incorporates more orthogonality conditions than is usually used by the Anderson and Hsiao (1982) estimator as well as a general robust variance–covariance matrix specification allowed by GMM. In addition, we employ a GMM procedure on a specification in levels with instruments in first-differences (GMM). In total, we compare 10 homogeneous estimators.
2.3. Heterogeneous estimators Underlying the poolability of the data is the assumption of homogeneity of the parameters across states. However, if this assumption is invalid the dynamic pooled model could be biased because of heterogeneity in the parameters across regions. Pesaran and Smith (1995) proposed instead an average of the individual state regressions which yields a consistent estimate of the parameters as long as N and T tend to infinity. We therefore compute individual state regressions and the Pesaran and Smith average estimate to compare homogeneous as well as heterogeneous estimates of the regression coefficients. We also computed the Swamy (1970) random coefficient regression estimator which is a weighted average of the least squares estimates where the weights are inversely proportional to their variance-covariance matrices. Using a quite different approach, Maddala et al. (1994) claim that shrinkage Bayesian type estimators are superior to either the individual heterogeneous estimates or the homogeneous estimates, especially for prediction purposes. In this case, one ‘shrinks’ the individual estimates towards the pooled estimate using weights depending on their corresponding variance– covariance matrices. From the individual maximum likelihood estimators, based on the normality assumption, several shrinkage estimators have been proposed in the literature including the empirical Bayes estimator, the iterative Bayes estimator and the iterative empirical Bayes estimator, see Maddala et al. (1997) for a description of these estimators.1 Maddala et al. (1997) estimated short- and long-run elasticities of residential natural gas and electricity demand using a panel of 49 states over the period 1970–1990. They found that individual time series regressions for each state gave wrong signs and were highly unstable. They proposed the shrinkage estimators as a compromise between the unstable heterogeneous estimates and the untenable homogeneity assumption. Thus, depending on the extent of between state heterogeneity in parameters, researchers may prefer either individual state regression estimates, the Pesaran and Smith average, or the shrinkage estimates to the traditional pooled homogeneous parameters estimates. In total, we compute nine heterogeneous estimates. 1 It is important to note that these shrinkage estimators were derived based on the strict exogeneity assumption of the regressors. Nevertheless, Maddala et al. (1997) argue that these estimators are still consistent but not efficient, when a lagged dependent variable is present among the regressors.
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
379
3. Forecast comparison In this section, we use the prediction-performance criteria to help us choose among alternative estimators. Given the large data set of N 5 49 states over T 5 21 years, we estimate our model using a truncated data set (i.e. without the last 5 years of data) and then apply each estimator to an out-of-sample forecast period. Table 1 gives a comparison of various predictors using the root mean square errors criterion (RMSE) for residential electricity demand while Table 2 does the same for residential natural-gas demand. Because of the ability of an estimator to characterize long-run as well as short-run responses is at issue, the average RMSE is calculated across the 49 states at different forecast horizons. Specifically, each model was applied to each state, and out-of-sample forecasts for 5 years were calculated. The relative forecast rankings are reported in Tables 1 and 2 after 1 and 5 years. The overall average ranking for the full 5-year period is also reported. For electricity demand (Table 1), the individual OLS and individual 2SLS perform poorly vis-a-vis the homogeneous estimators ranking 15 and 19 over the 5th year horizon. Swamy’s random coefficients estimator and the Pesaran and Smith ‘Average 2SLS’ and ‘Average OLS’ estimators rank 16, 18 and 19, respectively for the 5-year average forecasts. The weak forecast performance of the Pesaran and Smith average and the Swamy estimators relative to the homogeneous estimators arise because of the parameter-instability problem of the individual state regressions. Similarly, the relatively weak performance of the shrinkage and Bayes type estimators (ranking 5 for iterative empirical Bayes, 8 for iterative Bayes and 12 for empirical Bayes for the 5-year average), can be Table 1 Comparison of forecast performance for US electricity demand Ranking
1. 2. 3. 4. 5. 6. 7. 8. 9. 10 11. 12. 13. 14. 15. 16. 17. 18. 19. a b
1st year
5th year
Estimator
RMSE
Empirical iterative Bayes Iterative Bayes GLS Individual ML Within Within-2SLS FD2SLS-KRa Empirical Bayes FDGMM FD2SLS a 2SLS-KRa OLS 2SLS Individual OLS GMM Individual 2SLS Swamy Average 2SLS Average OLS
3.032 3.230 3.252 3.352 3.355 3.441 3.603 3.622 3.624 3.842 3.846 3.879 3.905 4.886 5.286 5.521 12.427 16.408 17.208
Instruments in levels. RMSE310 22 .
b
5-year average
Estimator
RMSE
GLS Within 2SLS-KRa 2SLS OLS Within-2SLS Empirical iterative Bayes Individual ML Iterative Bayes FDGMM Empirical Bayes FD2SLS a GMM FD2SLS-KRa Individual OLS Swamy Average 2SLS Average OLS Individual 2SLS
4.733 5.092 5.552 5.609 5.676 5.829 6.544 8.420 8.774 8.917 9.407 10.826 10.899 15.157 15.594 18.933 21.829 21.830 24.288
b
Estimator
RMSE b
GLS Within Within-2SLS 2SLS-KRa Empirical iterative Bayes 2SLS OLS Iterative Bayes FDGMM Individual ML FD2SLS a Empirical Bayes GMM FD2SLS-KRa Indivividual OLS Swamy Individual 2SLS Average 2SLS Average OLS
4.207 4.482 4.817 4.892 4.981 5.019 5.020 5.998 6.236 6.465 7.045 7.275 8.084 8.598 11.913 16.239 16.397 19.665 20.009
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
380
Table 2 Comparison of forecast performance for US natural-gas demand Ranking 1st year
1. 2. 3. 4. 5. 6. 7. 8. 9. 10 11. 12. 13. 14. 15. 16. 17. 18. 19. a b
5th year
Estimator
RMSE
Within-2SLS OLS GLS Within 2SLS Iterative Bayes 2SLS-KRa Individual ML Empirical iterative Bayes Empirical Bayes FDGMM FD2SLS a Individual OLS Individual 2SLS FD2SLS-KRa GMM Swamy Average 2SLS Average OLS
6.062 6.460 6.682 7.071 7.084 7.130 7.146 7.706 7.788 8.045 8.273 9.608 9.628 12.707 14.109 23.108 44.197 44.721 52.611
b
Estimator
5-year average RMSE
b
Within-2SLS 10.049 OLS 10.517 GLS 10.788 Within 11.132 2SLS 12.564 2SLS-KRa 14.440 FDGMM 15.319 Empirical iterative Bayes 17.919 Iterative Bayes 21.493 Individual ML 21.845 Empirical Bayes 24.096 FD2SLS a 40.215 GMM 40.943 Individual OLS 41.110 Swamy 49.616 Average OLS 53.308 Average 2SLS 53.828 FD2SLS-KRa 69.431 Individual 2SLS 100.281
Estimator
RMSE b
Within-2SLS OLS GLS Within 2SLS 2SLS-KRa FDGMM Empirical iterative Bayes Iterative Bayes Individual ML Empirical Bayes FD2SLS a Indivividual OLS GMM FD2SLS-KRa Swamy Average 2SLS Average OLS Individual 2SLS
8.730 9.060 9.295 9.637 10.325 11.036 12.627 14.189 17.061 17.717 19.394 23.897 30.777 33.840 40.204 47.826 50.424 52.906 54.399
Instruments in levels. RMSE310 22 .
attributed to their reliance upon the individual state parameter estimates. This ranking depends on the type of estimator, whether empirical Bayes or iterative Bayes. Thus, what seemed as an advantage to the shrinkage estimator—that is, placing some weight on the individual state regressions—becomes a liability when parameter instability is severe. The overall RMSE forecast rankings offer a strong endorsement for the homogeneous estimators due in large part to their parameter stability. GLS ranks first, followed by Within, Within-2SLS and 2SLS-KR for the 5-year average. Endogeneity problems seem not to be severe since GLS, Within and Within-2SLS give the lowest RMSE for the overall 5-year average. See Fig. 1 which plots the RMSE forecasts for the 1 year up to 5 years ahead forecasts for 13 estimators. For natural-gas demand (Table 2), the top five ranked estimators are all homogeneous estimators whether it is for the 1 year, 5 years, or the overall 5-year average forecast performance. These are the Within-2SLS, OLS, GLS, Within and 2SLS. In contrast, the individual OLS and individual 2SLS perform poorly vis-a-vis the homogeneous estimators ranking 14 and 19 for the 5-year average. Swamy’s random coefficients estimator and the Pesaran and Smith ‘Average 2SLS’ and ‘Average OLS’ estimators rank 16, 17 and 18, respectively, for the 5-year average forecasts. The relatively weak performance of the shrinkage and Bayes type estimators (ranking 8 for iterative empirical Bayes, 9 for iterative Bayes and 11 for empirical Bayes for the 5-year average), can be attributed to their reliance upon unstable individual state parameter estimates. See Fig. 2 for a plot of the RMSE forecasts for natural gas for the 1 year up to 5 years ahead forecasts for 13 estimators. Both residential electricity and natural gas demand models offer a strong endorsement for the
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
381
Fig. 1. Comparison of RMSE of forecasts—Electricity demand in the US 1970–1985.
homogeneous estimators based on their out-of-sample forecast performance, due in large part to their parameter stability.
4. Summary and conclusions This paper confirms again the value of panel data sets and the emphasis given to pooled estimators using two US panel data sets on residential electricity and natural-gas demand across 49 states over the period 1970–1990. Our results show that when the data is used to estimate heterogeneous models across states, individual estimates offer the worst out-of-sample forecasts. Despite the fact that shrinkage estimators outperform these individual estimates, they are outperformed by simple homogeneous panel data estimates in out-of-sample forecasts. Admittedly, this is another case study using US data, but it does add to the evidence that simplicity and parsimony in model estimation offer better forecasts.
Acknowledgements The authors wish to thank Bob Trost (The George Washington University) for his assistance with obtaining the data set. Baltagi would like to thank the Bush Program of Public Policy and the Private Enterprise Research Center at Texas A&M University for their support.
382
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
Fig. 2. Comparison of RMSE of forecasts—Natural-gas demand in the US 1970–1985.
References Alogoskoufis, G., Smith, R., 1991. On error correction models: Specification, interpretation, estimation. Journal of Economic Surveys 5, 97–128. Anderson, T.W., Hsiao, C., 1982. Formulation and estimation of dynamic models using panel data. Journal of Econometrics 18, 47–82. Arellano, M., Bond, S., 1991. Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. Review of Economic Studies 58, 277–297. Baltagi, B.H., Griffin, J.M., 1997. Pooled estimators vs. their heterogeneous counterparts in the context of dynamic demand for gasoline. Journal of Econometrics 77, 303–327. Baltagi, B.H., Griffin, J.M., Xiong, W., 2000. To pool or not to pool: homogeneous versus heterogeneous estimators applied to cigarette demand. Review of Economics and Statistics 82, 117–126. Hsiao, C., 1986. Analysis of Panel Data. Cambridge University Press, Cambridge. Keane, M.P., Runkle, D.E., 1992. On the estimation of panel-data models with serial correlation when instruments are not strictly exogenous. Journal of Business and Economic Statistics 10, 1–9. Maddala, G.S., Srivastava, V.K., Li, H., 1994. Shrinkage estimators for the estimation of short-run and long-run parameters from panel data models, working paper. Ohio State University, Ohio. Maddala, G.S., Trost, R.P., Li, H., Joutz, F., 1997. Estimation of short-run and long-run elasticities of energy demand from panel data using shrinkage estimators. Journal of Business and Economic Statistics 15, 90–100. Pesaran, M.H., Smith, R., 1995. Estimating long-run relationships from dynamic heterogenous panels. Journal of Econometrics 68, 79–113. Swamy, P.A.V.B., 1970. Efficient inference in a random coefficient regression model. Econometrica 38, 311–323.
Comparison of forecast performance for homogeneous, heterogeneous and shrinkage estimators Some empirical evidence from US electricity and natural-gas consumption Badi H. Baltagi a , *, Georges Bresson b , Alain Pirotte c a
Texas A& M University, Department of Economics, College Station, TX 77843 -4228, USA b ERMES( CNRS), Universite´ Paris II, 92 Rue d’ Assas, 75270 Paris, Cedex 06, France c ´ Universite Lyon II, Faculte de Sciences Economiques et de Gestion, 16 quai Claude Bernard, 69365 Lyon, Cedex 07, France Received 17 December 2001; accepted 19 February 2002
Abstract Maddala et al. [Journal of Business and Economic Statistics, 15 (1997) 90] obtained short-run and long-run elasticities of energy demand for each of 49 US states over the period 1970–1990. They showed that heterogeneous time series estimates for each state yield inaccurate signs for the coefficients, while panel data estimates are not valid because the hypothesis of homogeneity of the coefficients was rejected. Their preferred estimates are those obtained using the shrinkage estimator. This paper contrasts the out-of-sample forecast performance of heterogeneous, panel and shrinkage estimators using the Maddala et al. [Journal of Business and Economic Statistics 15 (1997) 90] electricity and natural gas data sets. Our results show that the homogeneous panel data estimates give the best out-of-sample forecasts. 2002 Elsevier Science B.V. All rights reserved. Keywords: Panel data; US electricity and natural-gas demand; Heterogeneous estimators; Shrinkage estimators JEL classification: C23
1. Introduction Maddala et al. (1997) applied classical, empirical Bayes and Bayesian procedures to the problem of estimating short-run and long-run elasticities of residential demand for electricity and natural gas in * Corresponding author. Tel.: 11-409-845-7380; fax: 11-409-847-8757. E-mail address: [email protected] (B.H. Baltagi). 0165-1765 / 02 / $ – see front matter PII: S0165-1765( 02 )00065-4
2002 Elsevier Science B.V. All rights reserved.
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
376
the US for each of 49 states. Since the elasticity estimates for each state are the ultimate goal of their study they were faced with three alternatives: The first is to use individual time series regressions for each state. These gave bad results, were hard to interpret and had several wrong signs. The second option was to pool the data and use panel data estimators. Although the pooled estimates gave the right signs and were more reasonable, Maddala et al. (1997) argued that these estimates are not valid because the hypothesis of homogeneity of the coefficients is rejected. The third option which they recommended was to allow for some (but not complete) heterogeneity or (homogeneity). This approach lead them to their preferred shrinkage estimator which gave them more reasonable parameter estimates. This approach is not without its critics. In fact, Hsiao (1986) has argued convincingly about the gains from pooling. Also, in the context of gasoline demand across 18 OECD (Organization for Economic Co-Operation and Development) countries over the period 1960–1990, Baltagi and Griffin (1997) argued for pooling the data as the best approach for obtaining reliable price and income elasticities even when the poolability assumption is rejected. They also pointed out that pure cross-section studies cannot control for unobservable country effects, whereas pure time-series studies cannot control for unobservable oil shocks or behavioral changes occurring over time. On the other extreme, Pesaran and Smith (1995) advocated abandoning the pooled approach altogether questioning the poolability of the data across heterogeneous units. Instead, they argue in favor of heterogeneous estimates that can be combined to obtain homogeneous estimates if the need arises. One such estimator being the average response from individual regressions. Depending on the extent of cross-sectional heterogeneity in parameters, researchers may prefer these heterogeneous estimators to the traditional pooled homogeneous parameter estimators or the shrinkage estimators proposed by Maddala et al. (1997). Recently, Baltagi and Griffin (1997) and Baltagi et al. (2000) showed that homogeneous panel data estimators beat the heterogeneous and shrinkage type estimators in RMSE performance for out-of-sample forecasts. These empirical results were performed for a gasoline data set across 18 OECD countries over 31 years (1960–1990) and for a cigarette demand data set across 46 states over 30 years (1963–1992). Using the Maddala et al. (1997) specification and data sets, our objective here is to compare the out-of-sample forecast performance of 10 homogeneous and nine heterogeneous estimators including the shrinkage estimators applying them to electricity and natural-gas across 49 states over the period 1970–1990. Section 2 describes the model specification of Maddala et al. (1997), the data sets and the pooled and heterogeneous estimators to be compared. Section 3 compares the forecast performance of these estimators using 1970–1985 for estimation purposes and 1986–1990 for out-of-sample forecasts. Section 4 summarizes the results and concludes.
2. Model specification and estimators
2.1. Model specification Maddala et al. (1997) considered the following standard dynamic linear regression (DLR) model for energy demand y i,t 5 bi,0 1 bi,1 y i,t 21 1 bi,2 x 1,i,t 1 bi,3 x 1,i,t 21 1 bi,4 x 2,i,t 1 bi,5 x 2,i,t21 1 bi,6 x 3,i,t 1 bi,7 x 4,i,t 1 bi,8 x 5,i,t 1 u i,t
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
377
where i 5 1, 2, . . . , 49 (states) and t 5 2, 3, . . . , 21 (years) spanning the period 1970–1990. The variables for the electricity regression are: • • • • • •
y i,t , the logarithm of residential electricity per capita consumption; x 1,i,t , the logarithm of real per capita personal income; x 2,i,t , the logarithm of real residential electricity price; x 3,i,t , the logarithm of real residential natural-gas price; x 4,i,t , heating degree days; 0 x 5,i,t , cooling degree days. For the natural-gas regression, we have:
• • • •
y i,t , the logarithm of residential natural-gas per capita consumption; x 1,i,t , the logarithm of real per capita personal income; x 2,i,t , the logarithm of real residential natural-gas price; x 3,i,t , the logarithm of real residential electricity price;
with x 4,i,t and x 5,i,t unchanged. This is a DLR (1,1) model derived by taking account of error correction and partial adjustment mechanisms (see Alogoskoufis and Smith (1991) for a recent survey of these models). For a description of the data sets, see Maddala et al. (1997). These data sets can be downloaded from the journal’s web site.
2.2. Homogeneous estimators In this case, the coefficients are homogeneous across states and we follow the usual convention of assuming that the disturbance term is specified as a one-way error component model, see Hsiao (1986): u i,t 5 ai 1 ni,t where ai denotes a state-specific effect and ni,t is IID (0,s n2 ). We first consider three standard pooled estimators, assuming the exogeneity of all the regressors. These include ordinary least squares (OLS), which ignores the state effects, the Within estimator, which allows for fixed state effects, and generalized least squares (GLS), which assumes that state effects are random. Since our model is dynamic, and even if all the explanatory variables are uncorrelated with the error components, the presence of serial correlation in the remainder error term or the presence of a random state effect renders the lagged dependent variable correlated with the error term and leads to inconsistent least squares estimates. Consequently, we also focus on pooled estimators employing two-stage least squares (2SLS) using as instruments the exogenous variables and their lagged values. In particular, we examine five alternative 2SLS pooled estimators. First, we consider a standard 2SLS estimator, making no attempt to improve efficiency by taking into account the random state effect. This estimator is consistent only if the exogenous regressors are uncorrelated with the unobservable individual effects. Second, we report the Within 2SLS estimator, which transforms the data about state means and thereby eliminates any state fixed effect. In addition, this instruments for the presence of the lagged dependent variable
378
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
by using exogenous variables and their lagged values. The third estimator is first-difference 2SLS (FD2SLS) proposed by Anderson and Hsiao (1982) in which fixed or random state effects are eliminated. However, first-differencing introduces autocorrelation in the remainder error term (ni,t 2 ni,t21 ) and thus inefficiency problems. Nevertheless, consistency is preserved by the use of predetermined variables as instruments. Keane and Runkle (1992), (hereafter denoted by KR) suggest a modification of the 2SLS estimator that allows for any arbitrary type of serial correlation in the ni,t ’s. We refer to this estimator as 2SLS-KR. Still another variant would be to allow for any arbitrary form of serial correlation in the first differenced disturbances in the manner of Keane–Runkle. This is denoted as the FD2SLS-KR estimator. Finally, following Arellano and Bond (1991), we used a generalized method of moments (GMM) estimator on the first-differences specification (FDGMM) with instruments in levels. This incorporates more orthogonality conditions than is usually used by the Anderson and Hsiao (1982) estimator as well as a general robust variance–covariance matrix specification allowed by GMM. In addition, we employ a GMM procedure on a specification in levels with instruments in first-differences (GMM). In total, we compare 10 homogeneous estimators.
2.3. Heterogeneous estimators Underlying the poolability of the data is the assumption of homogeneity of the parameters across states. However, if this assumption is invalid the dynamic pooled model could be biased because of heterogeneity in the parameters across regions. Pesaran and Smith (1995) proposed instead an average of the individual state regressions which yields a consistent estimate of the parameters as long as N and T tend to infinity. We therefore compute individual state regressions and the Pesaran and Smith average estimate to compare homogeneous as well as heterogeneous estimates of the regression coefficients. We also computed the Swamy (1970) random coefficient regression estimator which is a weighted average of the least squares estimates where the weights are inversely proportional to their variance-covariance matrices. Using a quite different approach, Maddala et al. (1994) claim that shrinkage Bayesian type estimators are superior to either the individual heterogeneous estimates or the homogeneous estimates, especially for prediction purposes. In this case, one ‘shrinks’ the individual estimates towards the pooled estimate using weights depending on their corresponding variance– covariance matrices. From the individual maximum likelihood estimators, based on the normality assumption, several shrinkage estimators have been proposed in the literature including the empirical Bayes estimator, the iterative Bayes estimator and the iterative empirical Bayes estimator, see Maddala et al. (1997) for a description of these estimators.1 Maddala et al. (1997) estimated short- and long-run elasticities of residential natural gas and electricity demand using a panel of 49 states over the period 1970–1990. They found that individual time series regressions for each state gave wrong signs and were highly unstable. They proposed the shrinkage estimators as a compromise between the unstable heterogeneous estimates and the untenable homogeneity assumption. Thus, depending on the extent of between state heterogeneity in parameters, researchers may prefer either individual state regression estimates, the Pesaran and Smith average, or the shrinkage estimates to the traditional pooled homogeneous parameters estimates. In total, we compute nine heterogeneous estimates. 1 It is important to note that these shrinkage estimators were derived based on the strict exogeneity assumption of the regressors. Nevertheless, Maddala et al. (1997) argue that these estimators are still consistent but not efficient, when a lagged dependent variable is present among the regressors.
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
379
3. Forecast comparison In this section, we use the prediction-performance criteria to help us choose among alternative estimators. Given the large data set of N 5 49 states over T 5 21 years, we estimate our model using a truncated data set (i.e. without the last 5 years of data) and then apply each estimator to an out-of-sample forecast period. Table 1 gives a comparison of various predictors using the root mean square errors criterion (RMSE) for residential electricity demand while Table 2 does the same for residential natural-gas demand. Because of the ability of an estimator to characterize long-run as well as short-run responses is at issue, the average RMSE is calculated across the 49 states at different forecast horizons. Specifically, each model was applied to each state, and out-of-sample forecasts for 5 years were calculated. The relative forecast rankings are reported in Tables 1 and 2 after 1 and 5 years. The overall average ranking for the full 5-year period is also reported. For electricity demand (Table 1), the individual OLS and individual 2SLS perform poorly vis-a-vis the homogeneous estimators ranking 15 and 19 over the 5th year horizon. Swamy’s random coefficients estimator and the Pesaran and Smith ‘Average 2SLS’ and ‘Average OLS’ estimators rank 16, 18 and 19, respectively for the 5-year average forecasts. The weak forecast performance of the Pesaran and Smith average and the Swamy estimators relative to the homogeneous estimators arise because of the parameter-instability problem of the individual state regressions. Similarly, the relatively weak performance of the shrinkage and Bayes type estimators (ranking 5 for iterative empirical Bayes, 8 for iterative Bayes and 12 for empirical Bayes for the 5-year average), can be Table 1 Comparison of forecast performance for US electricity demand Ranking
1. 2. 3. 4. 5. 6. 7. 8. 9. 10 11. 12. 13. 14. 15. 16. 17. 18. 19. a b
1st year
5th year
Estimator
RMSE
Empirical iterative Bayes Iterative Bayes GLS Individual ML Within Within-2SLS FD2SLS-KRa Empirical Bayes FDGMM FD2SLS a 2SLS-KRa OLS 2SLS Individual OLS GMM Individual 2SLS Swamy Average 2SLS Average OLS
3.032 3.230 3.252 3.352 3.355 3.441 3.603 3.622 3.624 3.842 3.846 3.879 3.905 4.886 5.286 5.521 12.427 16.408 17.208
Instruments in levels. RMSE310 22 .
b
5-year average
Estimator
RMSE
GLS Within 2SLS-KRa 2SLS OLS Within-2SLS Empirical iterative Bayes Individual ML Iterative Bayes FDGMM Empirical Bayes FD2SLS a GMM FD2SLS-KRa Individual OLS Swamy Average 2SLS Average OLS Individual 2SLS
4.733 5.092 5.552 5.609 5.676 5.829 6.544 8.420 8.774 8.917 9.407 10.826 10.899 15.157 15.594 18.933 21.829 21.830 24.288
b
Estimator
RMSE b
GLS Within Within-2SLS 2SLS-KRa Empirical iterative Bayes 2SLS OLS Iterative Bayes FDGMM Individual ML FD2SLS a Empirical Bayes GMM FD2SLS-KRa Indivividual OLS Swamy Individual 2SLS Average 2SLS Average OLS
4.207 4.482 4.817 4.892 4.981 5.019 5.020 5.998 6.236 6.465 7.045 7.275 8.084 8.598 11.913 16.239 16.397 19.665 20.009
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
380
Table 2 Comparison of forecast performance for US natural-gas demand Ranking 1st year
1. 2. 3. 4. 5. 6. 7. 8. 9. 10 11. 12. 13. 14. 15. 16. 17. 18. 19. a b
5th year
Estimator
RMSE
Within-2SLS OLS GLS Within 2SLS Iterative Bayes 2SLS-KRa Individual ML Empirical iterative Bayes Empirical Bayes FDGMM FD2SLS a Individual OLS Individual 2SLS FD2SLS-KRa GMM Swamy Average 2SLS Average OLS
6.062 6.460 6.682 7.071 7.084 7.130 7.146 7.706 7.788 8.045 8.273 9.608 9.628 12.707 14.109 23.108 44.197 44.721 52.611
b
Estimator
5-year average RMSE
b
Within-2SLS 10.049 OLS 10.517 GLS 10.788 Within 11.132 2SLS 12.564 2SLS-KRa 14.440 FDGMM 15.319 Empirical iterative Bayes 17.919 Iterative Bayes 21.493 Individual ML 21.845 Empirical Bayes 24.096 FD2SLS a 40.215 GMM 40.943 Individual OLS 41.110 Swamy 49.616 Average OLS 53.308 Average 2SLS 53.828 FD2SLS-KRa 69.431 Individual 2SLS 100.281
Estimator
RMSE b
Within-2SLS OLS GLS Within 2SLS 2SLS-KRa FDGMM Empirical iterative Bayes Iterative Bayes Individual ML Empirical Bayes FD2SLS a Indivividual OLS GMM FD2SLS-KRa Swamy Average 2SLS Average OLS Individual 2SLS
8.730 9.060 9.295 9.637 10.325 11.036 12.627 14.189 17.061 17.717 19.394 23.897 30.777 33.840 40.204 47.826 50.424 52.906 54.399
Instruments in levels. RMSE310 22 .
attributed to their reliance upon the individual state parameter estimates. This ranking depends on the type of estimator, whether empirical Bayes or iterative Bayes. Thus, what seemed as an advantage to the shrinkage estimator—that is, placing some weight on the individual state regressions—becomes a liability when parameter instability is severe. The overall RMSE forecast rankings offer a strong endorsement for the homogeneous estimators due in large part to their parameter stability. GLS ranks first, followed by Within, Within-2SLS and 2SLS-KR for the 5-year average. Endogeneity problems seem not to be severe since GLS, Within and Within-2SLS give the lowest RMSE for the overall 5-year average. See Fig. 1 which plots the RMSE forecasts for the 1 year up to 5 years ahead forecasts for 13 estimators. For natural-gas demand (Table 2), the top five ranked estimators are all homogeneous estimators whether it is for the 1 year, 5 years, or the overall 5-year average forecast performance. These are the Within-2SLS, OLS, GLS, Within and 2SLS. In contrast, the individual OLS and individual 2SLS perform poorly vis-a-vis the homogeneous estimators ranking 14 and 19 for the 5-year average. Swamy’s random coefficients estimator and the Pesaran and Smith ‘Average 2SLS’ and ‘Average OLS’ estimators rank 16, 17 and 18, respectively, for the 5-year average forecasts. The relatively weak performance of the shrinkage and Bayes type estimators (ranking 8 for iterative empirical Bayes, 9 for iterative Bayes and 11 for empirical Bayes for the 5-year average), can be attributed to their reliance upon unstable individual state parameter estimates. See Fig. 2 for a plot of the RMSE forecasts for natural gas for the 1 year up to 5 years ahead forecasts for 13 estimators. Both residential electricity and natural gas demand models offer a strong endorsement for the
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
381
Fig. 1. Comparison of RMSE of forecasts—Electricity demand in the US 1970–1985.
homogeneous estimators based on their out-of-sample forecast performance, due in large part to their parameter stability.
4. Summary and conclusions This paper confirms again the value of panel data sets and the emphasis given to pooled estimators using two US panel data sets on residential electricity and natural-gas demand across 49 states over the period 1970–1990. Our results show that when the data is used to estimate heterogeneous models across states, individual estimates offer the worst out-of-sample forecasts. Despite the fact that shrinkage estimators outperform these individual estimates, they are outperformed by simple homogeneous panel data estimates in out-of-sample forecasts. Admittedly, this is another case study using US data, but it does add to the evidence that simplicity and parsimony in model estimation offer better forecasts.
Acknowledgements The authors wish to thank Bob Trost (The George Washington University) for his assistance with obtaining the data set. Baltagi would like to thank the Bush Program of Public Policy and the Private Enterprise Research Center at Texas A&M University for their support.
382
B.H. Baltagi et al. / Economics Letters 76 (2002) 375 – 382
Fig. 2. Comparison of RMSE of forecasts—Natural-gas demand in the US 1970–1985.
References Alogoskoufis, G., Smith, R., 1991. On error correction models: Specification, interpretation, estimation. Journal of Economic Surveys 5, 97–128. Anderson, T.W., Hsiao, C., 1982. Formulation and estimation of dynamic models using panel data. Journal of Econometrics 18, 47–82. Arellano, M., Bond, S., 1991. Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. Review of Economic Studies 58, 277–297. Baltagi, B.H., Griffin, J.M., 1997. Pooled estimators vs. their heterogeneous counterparts in the context of dynamic demand for gasoline. Journal of Econometrics 77, 303–327. Baltagi, B.H., Griffin, J.M., Xiong, W., 2000. To pool or not to pool: homogeneous versus heterogeneous estimators applied to cigarette demand. Review of Economics and Statistics 82, 117–126. Hsiao, C., 1986. Analysis of Panel Data. Cambridge University Press, Cambridge. Keane, M.P., Runkle, D.E., 1992. On the estimation of panel-data models with serial correlation when instruments are not strictly exogenous. Journal of Business and Economic Statistics 10, 1–9. Maddala, G.S., Srivastava, V.K., Li, H., 1994. Shrinkage estimators for the estimation of short-run and long-run parameters from panel data models, working paper. Ohio State University, Ohio. Maddala, G.S., Trost, R.P., Li, H., Joutz, F., 1997. Estimation of short-run and long-run elasticities of energy demand from panel data using shrinkage estimators. Journal of Business and Economic Statistics 15, 90–100. Pesaran, M.H., Smith, R., 1995. Estimating long-run relationships from dynamic heterogenous panels. Journal of Econometrics 68, 79–113. Swamy, P.A.V.B., 1970. Efficient inference in a random coefficient regression model. Econometrica 38, 311–323.