- Email: [email protected]
Evaluation of alternative leading indicators of British Columbia industrial employment
International North-Holland
Journal
of Forecasting
9 (1993)
77
77-83
Evaluation of alternative leading indicators British Columbia industrial employment Richard
A. Holmes
Faculty of Business Administration and Department of Economics, Burnaby, B.C. V5A 1 S6, Canada
Abul
of
F.M.
Department
Simon Fraser University,
Shamsuddin of Economics,
Simon Fraser University, Burnaby,
B.C.
V5A lS6,
Canada
Abstract: Leading indicators based on principal components and weighted r2 values are evaluated in this study. The tests are based on out-of-sample forecasts of British Columbia employment. Overall, the principal component approach is found to be superior in tests conducted separately by forecast horizons ranging from 1 to 12 months, and in tests conducted in the months immediately following the 1984-2 turning point. This finding is explained in the study by the methodological superiority of the leading indicator based on principal components of the input series. Keywords:
Leading
indicator,
Forecasts,
Turning
points,
1. Introduction The objective of this paper is to evaluate two alternative leading indicators used in transfer function models to forecast British Columbia industrial employment. These leading indicators are based on the same set of 21 input series but employ different weighting systems in deriving the leading indicator. One approach involves weighting the input series according to their explanatory power in forecasts, while the alternative approach uses principal components from the input series. The former approach is described in Holmes (1986) and the alternative in Terasvirta (1984). In subsequent sections of this paper, we consider the input variables employed (Section 2), the methodology underlying our leading inCorrespondence to: Richard A. Holmes, Faculty of Business Administration and Department of Economics, Simon Fraser University, Burnaby, B.C., Canada V5A 1.56. Tel: (604) 291-3560; Fax: (604) 291-4920.
0169-2070/93/$06.00
0
1993 - Elsevier
Science
Publishers
ARIMA
models,
Transfer
function
models.
dicators (Section 3), our forecasting models and results (Sections 4 and 5), and finally, the conclusions drawn (Section 6).
2. Component variables leading indicators
of the British Columbia
We intend to forecast B.C. industrial employment (BCEI) which includes employment in forestry, mining, manufacturing, construction, transportation, communication and other utilities, trade, finance, insurance and real estate, community, business and personal services, excluding education and health services. The B.C. industrial employment variable excludes only the agricultural, public administration, health and education sectors. These excluded sectors are less sensitive to general economic conditions than those included. The leading indicators have been constructed
B.V. All rights
reserved
78
R. A. Holmes, A. F. M. Shamsuddin
on the basis of 21 series, which may be divided into five broad categories as follows: (1) Three leading indicators: the Statistics Canada leading indicator (SCLI), the US leading indicator, the Japanese leading indicator. These indicators are expected to indicate future demand for B.C. exportables in both foreign and domestic markets, which in turn affect B.C. employment. (2) Three stock market indexes: the VSE index, the TSE index for paper and forest products, the TSE index for metals and minerals. These indexes are intended to predict future development in the B.C. economy with particular importance attributed to the forestry and mining sectors, since these sectors are particularly important components of the B.C. economy. (3) Four building permits series: B.C. residential permits, B.C. non-residential building permits, Canadian residential building permits, US housing permits. Building permits provide indication of future changes in construction activity, and US housing permits, in particular, is expected to forecast demand for the exports of the B.C. forest industry. (4) Four financial variables: the Canadian money supply, chartered banks prime rate, chartered banks cash reserves, US-Canadian forward foreign exchange rate (a 90-day forward rate). These variables are intended to capture the state of monetary and fiscal policies of the Canadian government. In particular, the forward exchange rate reflects people’s perception about the future state of the Canadian economy. (5) Seven other B.C. series: new motor vehiof manufactured goods, cles sales. shipments unemployment, UIC initial and renewal claims, CP Air passenger boardings in northern B.C., average weekly hours of hourly passenger boardings in northern B.C., average weekly hours of hourly rated wage earners, average weekly wage of hourly rated wage earners. These series provide variable performance as leading indicators of BCEI. In simple regressions of BCEI on the individual series lagged 12 months over the whole range of our data, the slope coefficient is insignificant only for the two TSE indexes and the two foreign leading indicators. The most effective 12 month leading indicators over the range of our data are B.C. shipments of manufactured
I British Columbia industriul employment
goods, the Canadian money supply, B.C. motor vehicle sales and CP Air passenger boardings in northern B.C. Our methodology gives greater weight to those that are better leading indicators and lesser weight to those that perform more poorly.
3. Methodology
of building
leading indicators
Traditionally, the procedure for building a leading indicator involves the use of subjective weights for the component series. The National Bureau of Economic Research has long suggested weighting the components of a leading indicator by scoring them on economic significance, statistical adequacy, historical conformity to business cycles, cyclical timing record, smoothness and promptness of publication. The average scores are employed to weight the index, with the components having the highest scores receiving the largest weights. In reality, this weighting procedure is very arbitrary where weights are subjectively rather than empirically determined.’ Here we employed two alternative ways to determine the weights empirically. 3.1.
Weights
This ( 1986). minimize specific scheme
minimizing
forecast
errors
approach has been proposed by Holmes It involves choosing the weights so as to the expected error in predicting a series. More specifically, the weighting is based on the following regression:
Y, = @ + PX,,~_,, + u, where Y, is cyclical variation in BCEI, X, is cyclical variation in the jth component of BCLI, j = 1,2,3, . . ,20 (the component series), a, p are regression parameters, and U, is a disturbance term. The r* values in these regressions are then used to derive weights for the components of the leading indicator in the following way:
Auerbach (1982) has found that voted to assigning and updating method has little value.
the extensive effort deweights in the NBER
R.A.
Holmes, A.F.M.
Shamsuddin
where Wjt is weight assigned to the jth component of BCLI in a 12 month leading indicator at time t, and r:, is a coefficient of determination in the regression of Y, on X,t_12 (i.e. I:, is obtained with a lead of 12 months on X, at time t). Thus, the leading indicator (BCLI), in this approach, is obtained by weighting each of the input variables by its explanatory power (as measured by I’) in forecasts of BCEI.2
the input variables. The object is to select vector a, to maximize the variation of P,, i.e. maximum
The principal components
approach
This approach is used to express a group of input variables by a set of orthogonal components, which in our context are the leading indicators of BCEI. The first principal component is a weighted average of the set of input variables, where the weights are chosen to ensure that the composite variable reflects the maximum possible proportion of total variation in the set. Additional principal components are orthogonal to one another and are constructed to reflect the maximum possible proportion of the remaining variation in the input variables.” The first principal component P, is defined as: P, = Xa,
(1)
where P, = m x 1 column vector, X = m X 20 matrix of observations on input variables,4 and a, = 20 x 1 column vector of weights assigned to The VSE index is available over a shorter time period than the other series and is weighted according to the procedure for other series only from 1986-1. This series has a relatively minor effect on BCLI with a weight of only 2% in our last estimation period (1987-5). The second principal component (P,) is orthogonal to the first and employs weights selected to capture the maximum possible proportion of the remaining variation in the input variables. Similarly, one can construct a third principal component. orthogonal to the first two. Since we have 20 input variables, we could construct 20 principal components, each orthogonal to the others. However, the first two or three principal components usually capture most of the variation in the original variables. For a simple exposition of the principal component analysis see Wonnacott and Wonnacott (1979). The VSE index could not be included as an input variable in the principal components approach because that series is not available over the entire range of our data. Excluding the VSE index puts the principal components approach at a slight disadvantage.
P;P, = aix’xa,
(2)
However, this variation can be made arbitrarily large by selecting a huge a,, so we impose the restriction ,a$,
3.2.
79
I British Columbia industrial employment
= 1,
(3)
to ensure that the total variation in all the principal components is equal to the total variation in all the input variables. Maximization of (2) subject to (3) yields the solution for the latent vector a,. Exhibit 1 shows that almost 80% of total variation in input variables is captured by the first three principal components (BCPCl, BCPC2, BCPC3). The leading indicator BCPC consists of all of these first three principal components. The issue we want to address is the performance of the two types of leading indicators in predicting BC industrial employment. We have conducted a comprehensive test on the predictive power of BCLI and BCPC. We run regressions of BCEI on BCLI and BCPC separately, with leads ranging from 1 to 18 months on the leading indicators. The results are reported in Exhibits 2 and 3. Clearly, one indicator is not absolutely superior to others in terms of predictive power. BCPC outperforms BCLI in terms of r2 values only with very short (<3 months) and long (>15 months) leads. This evidence is not conclusive and to obtain more information we employ each in a transfer function model and determine the accuracy of out-of-sample forecasts obtained from those transfer function models.
Exhibit 1 Eigenvalues and explained variance by principal component (based on the first estimation period: 1973-2 to 1981-12). Principal component
Eigenvalue
Cumulative explained variance (%)
PI PZ P, p, P,
6.55 4.73 3.79 1.40 0.70
0.3449 0.5941 0.7937 0.8674 0.9042
R. A. Holmes, A. F. M. Shamsuddin
80
Exhibit 2 r2 by lead BCEI.
British Columbia industrial employment
(ii)
(in months)
for BCLI
and BCPC
in forecasts
Lead in months
r*-BCLI
r2-BCPC
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1X
0.7475 0.7953 0.8353 0.8670 0.8899 0.9040 0.9102 0.9088 0.9000 0.8846 0.8625 0.8339 0.8002 0.7620 0.7199 0.6749 0.6269 0.5771
0.8391 0.8316 0.8261 0.8257 0.8220 0.8183 0.8101 0.7988 0.7908 0.7789 0.7699 0.7583 0.7432 0.7317 0.7126 0.7041 0.6915 0.6894
of
‘.OO 1 0 95
-
RSQ_BCLI
-
RS_EcPC
transfer function model using the weighted r2 leading indicator (TF-BCLI);’ (iii) transfer function model using the principal components as leading indicators (TFBCPC). The univariate ARIMA model is used as a benchmark for comparisons, while two alternative transfer function models are used to determine which method of constructing leading indicators provides better forecasts of BCEI. The estimation period includes data from 1973-2 to 1981-12. The estimation results are reported in Exhibit 4. Based on the autocorrelation function (ACF) and partial autocorrelation function (PACF) of BCEI, we have identified the UBJ-ARIMA model to be ARIMA (0,1,2) (O,l, l),,. The results on the first transfer function model have been reproduced from Holmes (1986). An alternative transfer function using the first three principal components (BCPCl, BCPC2, BCPC3) has been estimated where the noise component follows an AR(l) process. All estimated models presented in Exhibit 4 satisfy the conditions of diagnostic checking including white noise residu-
0 90
0.85
Exhibit 4 Estimation results on three alternative models of B.C. industrial employment estimation period: 1973-2 to 1981-12.
080
0 75
UBJ-ARIMA model: (1 - B)(l - E”) BCEI
0.70
0.65
= (1-
060
- 0.78B1’)e, (8.41)
Transfer function model (TF-BCLI): Weighted rz leading indicator as input variable:
0.55
0.50 1
2
3
4
5
6
7
9
9
1011121314151617181920
(1-
Lead in months Exhibit
0.25B - O.l6B*)(l (2.44) (1.58)
3. R2 by lead for BCLI
and BCPC
in forecasts of
BCEI. 1973-2 to 1987-2.
B”) BCEI = 0.75 BCLl,_ 12+ ((1 -0.89B”)/(l.O74B)}e, (31.2) (12.0) (8.4)
Transfer Principal
function model (TF-BCPC): components leading indicators
as input variables:
BCEI = 2.87 BCPCl,_,, (4.81)
4. Forecasting
models of industrial
employment
We have employed the following time series models in forecasting the monthly B.C. industrial employment index (BCEI): (i) univariate ARIMA model (UARIMA);
_
+ 0.91 BCPC2,_,, (1.49) ‘t + 1.03 BCPC3,_ 12 + ~ 1 - 0.73B (1.47) (9.71)
5 For a detailed discussion of the method of building function models see Shamsuddin et al. (1985).
transfer
R. A. Holmes, A, F. M. Shamsuddin
I British Columbia industrial employment
als, stationary AR coefficients, invertible MA coefficients and statistically significant estimates.
Forecasting error for the 12-month
Exhibit
5
using three to 1987-5).
5. Accuracy
of the out-of-sample
forecasts
The selected univariate ARIMA and transfer function models have been used to forecast the monthly industrial employment index of BC (BCEI) for the 1982-1 to 1987-5 period. Our initial estimation period runs from 1973-2 to 1981-12. These initial estimates for each model yield separate forecasts for 12 months running from 1982-1 to 1983-12. In the next step of our forecasting exercise, we include an additional month in our estimation period which then runs from 1973-2 to 1982-1. The second 12 month out-of-sample forecasts then run from 1982-2 to 1984-1. By repeating this procedure 66 times for each model, we obtain 66 out-of-sample 12 X 1 forecast vectors, the last of which runs from 1987-6 to 1988-5. Since the actual data extends to 1987-5, we have 65 l-month ahead forecasts that we can evaluate by comparing forecast to actual values. Further, we lose one observation for each month added to the forecast horizon such that we have 64 of the 2-month ahead forecasts, 63 of the 3-month ahead forecasts, down to 54 of the 12-month ahead forecasts that can be evaluated for each model. This extensive forecasting exercise enables us to assess forecasting accuracy by both the forecasting model and the forecast horizon. The evaluation of forecasting accuracy is made by employing the root mean squared percent error (RMSPE). The results on forecasting accuracy are presented in Exhibits 5 and 6. The most important of our results is that the principal component transfer function (TF-BCPC) model performs best in forecasting BCEI at all forecast horizons. The univariate ARIMA model has lower RMSPE than the weighted r2 transfer function (TF-BCLI) model at short forecast horizons (up to 3 months forecast lead), while its forecasting accuracy diminishes rapidly with the increase in forecast leads. At all leads longer than 3 months, TF-BCLI is more accurate than the UBJARIMA model. TF-BCPC is more accurate than both TF-BCLI and the UBJ-ARIMA model at all of the forecast leads considered. Forecasting accuracy of the two transfer func-
81
alternative
Length of forecasts (months)
N
1 2 3 4 5 6 7 8 9 10 11 12
65 64 63 62 61 60 59 58 57 56 55 54
H
models
out-of-sample forecasts by length of forecasts (1982-1
Root mean squared
percent
error
UBJ-ARIMA
TF-BCLI
TF-BCPC
1.77 2.53 3.19 3.78 4.29 4.79 5.20 5.56 5.69 5.83 5.79 5.88
2.22 2.81 3.37 3.67 3.91 3.94 4.04 4.13 4.08 4.24 4.24 4.25
1.56 2.39 2.89 3.22 3.39 3.45 3.45 3.37 3.28 3.22 3.14 3.12
UBJ_ARIMP
q q
TF_BCLl TF_BCPC
0 1
2
3
4
Forecast Exhibit models
5
6
7
a
9
10
11
12
lead in months
6. Forecasting accuracy by forecast lead: 1982-1
of UBJ, BCLI to 1987-5.
and
BCPC
tion models (TF-BCLI and TF-BCPC) has also been examined for 1984-2 (the bottom of a recession) and the 12 months following. For each of these 13 months, we have 12 forecasts ranging from 1 month ahead to 12 months ahead forecasts. The average error measured by the RMSPE is shown in Exhibits 7 and 8 to be relatively large at and immediately following the turning point in 1984-2. The RMSPE’s are in the 7-8% range for the first 2 months following the turning point in 1984-2. They fall to the 3-4% range in the next 2 months and to the l-2%
R. A. Holmes, A. F. M. Shamsuddin
82 Exhibit 7 Forecasting error 1984-Z to 1985-2. Months
1984
1985
6. Conclusion of TF-BCPC
and TF-BCLI
Root mean squared error (RMSPE)
February March April May June July August September October November December January February
I British Columbia industrial employment
by months:
percent
TF-BCPC
TF-BCLI
8.20 8.04 3.28 2.66 2.34 1.26 1.10 0.73 0.54 0.98 1.52 1.94 2.33
6.51 8.19 3.72 3.88 3.26 2.06 2.76 1.49 0.79 0.55 1.05 2.14 1.88
‘Oq
TF_BCLI
w
TF_BCPC
Overall our tests indicate that the leading indicator based on principal components of the input series provide more accurate forecasts than the leading indicator based on weighted r2 values. At each forecast horizon ranging from 1 to 12 months, TF-BCPC provided smaller forecast errors than either TF-BCLI or the UBJARIMA model. Since these results are based on a large number of tests ranging from 65 of the l-month ahead forecasts down to 54 of the 12month ahead forecasts, these results are quite conclusive.’ Moreover, when we examine the results separately by month following the 1984-2 turning point we find similar results. TF-BCPC is more accurate than TF-BCLI in 8 of the 9 months immediately following the turning point (1984-2 to 1984-10). We conclude that the preferred leading indicator is the one based on principal components of the input series (TFBCPC) .’ The reason for these results may be the methodological superiority of a multivariate over a univariate approach. The weighted r2 leading indicator is based on univariate regressions which omit all but one of the input variables. Since the included explanatory variable is unlikely to be orthogonal to some of the omitted variables, the resulting error variance is likely underestimated and Y’ overestimated. Consequently the weights based on relative r2 values may not reflect the true importance of an input variable in forecasting BCEI. The principal component approach, on the other hand, is a multivariate approach that generates a leading indicator that captures the maximum possible proportion of total variation in input variables, and
’ This conclusion Exhibit 8. Forecasting accuracy by month: 1984-2 to 1985-2.
of BCLI
and BCPC
models
range after that. TF-BCPC has smaller forecast errors than TF-BCLI in 8 of the 9 months immediately following the turning point (1984-2 to 1984-10). TF-BCLI is more accurate in 3 of the 4 later months (1984-11 to 1985-2).
is strengthened somewhat by the necessity of excluding the VSE index, which is not available over the entire range of our data, from the principal components approach. Had it been possible to include the VSE index in our principal components leading indicator, it would be reasonable to expect a marginally better forecasting performance from TF-BCPC. This does not necessarily mean that the principal components approach is optimal. Boschan and Banerji (1990, p. 220) argue that principal components ‘minimize the impact of indicators which do not move with the other indicators, and this may defeat the purpose of the composite index’.
R.A.
Holmes, A. F.M.
Shamsuddin
is not subject to the omitted variable problem affecting the weighted r2 leading indicator.
Acknowledgments This research was supported by SSHRC of Canada. The authors are also grateful for referees’ constructive criticisms.
References Auerbach, A.J., 1982, “The index of leading indicators: Measurement without theory, thirty-five years later”, Review of Economic Statistics, 64, 589-595. Boschan, C. and A. Banerji, 1990, ‘A reassessment of composite indexes’, in: P.A. Klein, ed., Analyzing Modern Business Cycles: Essays Honoring Geoffrey H. Moore (Sharpe, New York). Holmes, R.A., 1986, “Leading indicators of industrial employment in British Columbia”, International Journal of Forecasting, 2, 87-100.
I British Columbia industrial employment
83
Shamsuddin, A.F.M., R.A. Holmes and A.K.M.S. Alam, 1985, “Univariate ARIMA and bivariate transfer function models for inflation: Some Malaysian evidence”, The Philippine Economic Journal, XXIV, 263-287. Forecasting of Industrial Terasvirta T., 1984, “Short-term Production by Means of Quick Indicators”, Journal of Forecasting, 3, 409-416. Wonnacott, R.I. and T.H. Wonnacott, 1979, Econometrics (Wiley, Toronto). Biographies: Richard A. HOLMES is Professor in the Faculty of Business Administration and the Department of Economics at Simon Fraser University. His research interests are in business and economic forecasting. Previous publications include articles in the American Economic Review, the Journal of the American Statistical Association, the International Journal of Forecasting, the Journal of Business, and the Journal of Human Resources. Abul F.M. SHAMSUDDIN is a lecturer of Economics at the University of New England, Armidale, Australia. He is also a doctoral candidate in Economics at Simon Fraser University, Vancouver, Canada. His research interests are in international economics, economics of immigration and economic forecasting. He has contributed articles in the Journal of International Economic Integration, Journal of Economics and International Relations, and The Philippine Economic Journal.
Journal
of Forecasting
9 (1993)
77
77-83
Evaluation of alternative leading indicators British Columbia industrial employment Richard
A. Holmes
Faculty of Business Administration and Department of Economics, Burnaby, B.C. V5A 1 S6, Canada
Abul
of
F.M.
Department
Simon Fraser University,
Shamsuddin of Economics,
Simon Fraser University, Burnaby,
B.C.
V5A lS6,
Canada
Abstract: Leading indicators based on principal components and weighted r2 values are evaluated in this study. The tests are based on out-of-sample forecasts of British Columbia employment. Overall, the principal component approach is found to be superior in tests conducted separately by forecast horizons ranging from 1 to 12 months, and in tests conducted in the months immediately following the 1984-2 turning point. This finding is explained in the study by the methodological superiority of the leading indicator based on principal components of the input series. Keywords:
Leading
indicator,
Forecasts,
Turning
points,
1. Introduction The objective of this paper is to evaluate two alternative leading indicators used in transfer function models to forecast British Columbia industrial employment. These leading indicators are based on the same set of 21 input series but employ different weighting systems in deriving the leading indicator. One approach involves weighting the input series according to their explanatory power in forecasts, while the alternative approach uses principal components from the input series. The former approach is described in Holmes (1986) and the alternative in Terasvirta (1984). In subsequent sections of this paper, we consider the input variables employed (Section 2), the methodology underlying our leading inCorrespondence to: Richard A. Holmes, Faculty of Business Administration and Department of Economics, Simon Fraser University, Burnaby, B.C., Canada V5A 1.56. Tel: (604) 291-3560; Fax: (604) 291-4920.
0169-2070/93/$06.00
0
1993 - Elsevier
Science
Publishers
ARIMA
models,
Transfer
function
models.
dicators (Section 3), our forecasting models and results (Sections 4 and 5), and finally, the conclusions drawn (Section 6).
2. Component variables leading indicators
of the British Columbia
We intend to forecast B.C. industrial employment (BCEI) which includes employment in forestry, mining, manufacturing, construction, transportation, communication and other utilities, trade, finance, insurance and real estate, community, business and personal services, excluding education and health services. The B.C. industrial employment variable excludes only the agricultural, public administration, health and education sectors. These excluded sectors are less sensitive to general economic conditions than those included. The leading indicators have been constructed
B.V. All rights
reserved
78
R. A. Holmes, A. F. M. Shamsuddin
on the basis of 21 series, which may be divided into five broad categories as follows: (1) Three leading indicators: the Statistics Canada leading indicator (SCLI), the US leading indicator, the Japanese leading indicator. These indicators are expected to indicate future demand for B.C. exportables in both foreign and domestic markets, which in turn affect B.C. employment. (2) Three stock market indexes: the VSE index, the TSE index for paper and forest products, the TSE index for metals and minerals. These indexes are intended to predict future development in the B.C. economy with particular importance attributed to the forestry and mining sectors, since these sectors are particularly important components of the B.C. economy. (3) Four building permits series: B.C. residential permits, B.C. non-residential building permits, Canadian residential building permits, US housing permits. Building permits provide indication of future changes in construction activity, and US housing permits, in particular, is expected to forecast demand for the exports of the B.C. forest industry. (4) Four financial variables: the Canadian money supply, chartered banks prime rate, chartered banks cash reserves, US-Canadian forward foreign exchange rate (a 90-day forward rate). These variables are intended to capture the state of monetary and fiscal policies of the Canadian government. In particular, the forward exchange rate reflects people’s perception about the future state of the Canadian economy. (5) Seven other B.C. series: new motor vehiof manufactured goods, cles sales. shipments unemployment, UIC initial and renewal claims, CP Air passenger boardings in northern B.C., average weekly hours of hourly passenger boardings in northern B.C., average weekly hours of hourly rated wage earners, average weekly wage of hourly rated wage earners. These series provide variable performance as leading indicators of BCEI. In simple regressions of BCEI on the individual series lagged 12 months over the whole range of our data, the slope coefficient is insignificant only for the two TSE indexes and the two foreign leading indicators. The most effective 12 month leading indicators over the range of our data are B.C. shipments of manufactured
I British Columbia industriul employment
goods, the Canadian money supply, B.C. motor vehicle sales and CP Air passenger boardings in northern B.C. Our methodology gives greater weight to those that are better leading indicators and lesser weight to those that perform more poorly.
3. Methodology
of building
leading indicators
Traditionally, the procedure for building a leading indicator involves the use of subjective weights for the component series. The National Bureau of Economic Research has long suggested weighting the components of a leading indicator by scoring them on economic significance, statistical adequacy, historical conformity to business cycles, cyclical timing record, smoothness and promptness of publication. The average scores are employed to weight the index, with the components having the highest scores receiving the largest weights. In reality, this weighting procedure is very arbitrary where weights are subjectively rather than empirically determined.’ Here we employed two alternative ways to determine the weights empirically. 3.1.
Weights
This ( 1986). minimize specific scheme
minimizing
forecast
errors
approach has been proposed by Holmes It involves choosing the weights so as to the expected error in predicting a series. More specifically, the weighting is based on the following regression:
Y, = @ + PX,,~_,, + u, where Y, is cyclical variation in BCEI, X, is cyclical variation in the jth component of BCLI, j = 1,2,3, . . ,20 (the component series), a, p are regression parameters, and U, is a disturbance term. The r* values in these regressions are then used to derive weights for the components of the leading indicator in the following way:
Auerbach (1982) has found that voted to assigning and updating method has little value.
the extensive effort deweights in the NBER
R.A.
Holmes, A.F.M.
Shamsuddin
where Wjt is weight assigned to the jth component of BCLI in a 12 month leading indicator at time t, and r:, is a coefficient of determination in the regression of Y, on X,t_12 (i.e. I:, is obtained with a lead of 12 months on X, at time t). Thus, the leading indicator (BCLI), in this approach, is obtained by weighting each of the input variables by its explanatory power (as measured by I’) in forecasts of BCEI.2
the input variables. The object is to select vector a, to maximize the variation of P,, i.e. maximum
The principal components
approach
This approach is used to express a group of input variables by a set of orthogonal components, which in our context are the leading indicators of BCEI. The first principal component is a weighted average of the set of input variables, where the weights are chosen to ensure that the composite variable reflects the maximum possible proportion of total variation in the set. Additional principal components are orthogonal to one another and are constructed to reflect the maximum possible proportion of the remaining variation in the input variables.” The first principal component P, is defined as: P, = Xa,
(1)
where P, = m x 1 column vector, X = m X 20 matrix of observations on input variables,4 and a, = 20 x 1 column vector of weights assigned to The VSE index is available over a shorter time period than the other series and is weighted according to the procedure for other series only from 1986-1. This series has a relatively minor effect on BCLI with a weight of only 2% in our last estimation period (1987-5). The second principal component (P,) is orthogonal to the first and employs weights selected to capture the maximum possible proportion of the remaining variation in the input variables. Similarly, one can construct a third principal component. orthogonal to the first two. Since we have 20 input variables, we could construct 20 principal components, each orthogonal to the others. However, the first two or three principal components usually capture most of the variation in the original variables. For a simple exposition of the principal component analysis see Wonnacott and Wonnacott (1979). The VSE index could not be included as an input variable in the principal components approach because that series is not available over the entire range of our data. Excluding the VSE index puts the principal components approach at a slight disadvantage.
P;P, = aix’xa,
(2)
However, this variation can be made arbitrarily large by selecting a huge a,, so we impose the restriction ,a$,
3.2.
79
I British Columbia industrial employment
= 1,
(3)
to ensure that the total variation in all the principal components is equal to the total variation in all the input variables. Maximization of (2) subject to (3) yields the solution for the latent vector a,. Exhibit 1 shows that almost 80% of total variation in input variables is captured by the first three principal components (BCPCl, BCPC2, BCPC3). The leading indicator BCPC consists of all of these first three principal components. The issue we want to address is the performance of the two types of leading indicators in predicting BC industrial employment. We have conducted a comprehensive test on the predictive power of BCLI and BCPC. We run regressions of BCEI on BCLI and BCPC separately, with leads ranging from 1 to 18 months on the leading indicators. The results are reported in Exhibits 2 and 3. Clearly, one indicator is not absolutely superior to others in terms of predictive power. BCPC outperforms BCLI in terms of r2 values only with very short (<3 months) and long (>15 months) leads. This evidence is not conclusive and to obtain more information we employ each in a transfer function model and determine the accuracy of out-of-sample forecasts obtained from those transfer function models.
Exhibit 1 Eigenvalues and explained variance by principal component (based on the first estimation period: 1973-2 to 1981-12). Principal component
Eigenvalue
Cumulative explained variance (%)
PI PZ P, p, P,
6.55 4.73 3.79 1.40 0.70
0.3449 0.5941 0.7937 0.8674 0.9042
R. A. Holmes, A. F. M. Shamsuddin
80
Exhibit 2 r2 by lead BCEI.
British Columbia industrial employment
(ii)
(in months)
for BCLI
and BCPC
in forecasts
Lead in months
r*-BCLI
r2-BCPC
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1X
0.7475 0.7953 0.8353 0.8670 0.8899 0.9040 0.9102 0.9088 0.9000 0.8846 0.8625 0.8339 0.8002 0.7620 0.7199 0.6749 0.6269 0.5771
0.8391 0.8316 0.8261 0.8257 0.8220 0.8183 0.8101 0.7988 0.7908 0.7789 0.7699 0.7583 0.7432 0.7317 0.7126 0.7041 0.6915 0.6894
of
‘.OO 1 0 95
-
RSQ_BCLI
-
RS_EcPC
transfer function model using the weighted r2 leading indicator (TF-BCLI);’ (iii) transfer function model using the principal components as leading indicators (TFBCPC). The univariate ARIMA model is used as a benchmark for comparisons, while two alternative transfer function models are used to determine which method of constructing leading indicators provides better forecasts of BCEI. The estimation period includes data from 1973-2 to 1981-12. The estimation results are reported in Exhibit 4. Based on the autocorrelation function (ACF) and partial autocorrelation function (PACF) of BCEI, we have identified the UBJ-ARIMA model to be ARIMA (0,1,2) (O,l, l),,. The results on the first transfer function model have been reproduced from Holmes (1986). An alternative transfer function using the first three principal components (BCPCl, BCPC2, BCPC3) has been estimated where the noise component follows an AR(l) process. All estimated models presented in Exhibit 4 satisfy the conditions of diagnostic checking including white noise residu-
0 90
0.85
Exhibit 4 Estimation results on three alternative models of B.C. industrial employment estimation period: 1973-2 to 1981-12.
080
0 75
UBJ-ARIMA model: (1 - B)(l - E”) BCEI
0.70
0.65
= (1-
060
- 0.78B1’)e, (8.41)
Transfer function model (TF-BCLI): Weighted rz leading indicator as input variable:
0.55
0.50 1
2
3
4
5
6
7
9
9
1011121314151617181920
(1-
Lead in months Exhibit
0.25B - O.l6B*)(l (2.44) (1.58)
3. R2 by lead for BCLI
and BCPC
in forecasts of
BCEI. 1973-2 to 1987-2.
B”) BCEI = 0.75 BCLl,_ 12+ ((1 -0.89B”)/(l.O74B)}e, (31.2) (12.0) (8.4)
Transfer Principal
function model (TF-BCPC): components leading indicators
as input variables:
BCEI = 2.87 BCPCl,_,, (4.81)
4. Forecasting
models of industrial
employment
We have employed the following time series models in forecasting the monthly B.C. industrial employment index (BCEI): (i) univariate ARIMA model (UARIMA);
_
+ 0.91 BCPC2,_,, (1.49) ‘t + 1.03 BCPC3,_ 12 + ~ 1 - 0.73B (1.47) (9.71)
5 For a detailed discussion of the method of building function models see Shamsuddin et al. (1985).
transfer
R. A. Holmes, A, F. M. Shamsuddin
I British Columbia industrial employment
als, stationary AR coefficients, invertible MA coefficients and statistically significant estimates.
Forecasting error for the 12-month
Exhibit
5
using three to 1987-5).
5. Accuracy
of the out-of-sample
forecasts
The selected univariate ARIMA and transfer function models have been used to forecast the monthly industrial employment index of BC (BCEI) for the 1982-1 to 1987-5 period. Our initial estimation period runs from 1973-2 to 1981-12. These initial estimates for each model yield separate forecasts for 12 months running from 1982-1 to 1983-12. In the next step of our forecasting exercise, we include an additional month in our estimation period which then runs from 1973-2 to 1982-1. The second 12 month out-of-sample forecasts then run from 1982-2 to 1984-1. By repeating this procedure 66 times for each model, we obtain 66 out-of-sample 12 X 1 forecast vectors, the last of which runs from 1987-6 to 1988-5. Since the actual data extends to 1987-5, we have 65 l-month ahead forecasts that we can evaluate by comparing forecast to actual values. Further, we lose one observation for each month added to the forecast horizon such that we have 64 of the 2-month ahead forecasts, 63 of the 3-month ahead forecasts, down to 54 of the 12-month ahead forecasts that can be evaluated for each model. This extensive forecasting exercise enables us to assess forecasting accuracy by both the forecasting model and the forecast horizon. The evaluation of forecasting accuracy is made by employing the root mean squared percent error (RMSPE). The results on forecasting accuracy are presented in Exhibits 5 and 6. The most important of our results is that the principal component transfer function (TF-BCPC) model performs best in forecasting BCEI at all forecast horizons. The univariate ARIMA model has lower RMSPE than the weighted r2 transfer function (TF-BCLI) model at short forecast horizons (up to 3 months forecast lead), while its forecasting accuracy diminishes rapidly with the increase in forecast leads. At all leads longer than 3 months, TF-BCLI is more accurate than the UBJARIMA model. TF-BCPC is more accurate than both TF-BCLI and the UBJ-ARIMA model at all of the forecast leads considered. Forecasting accuracy of the two transfer func-
81
alternative
Length of forecasts (months)
N
1 2 3 4 5 6 7 8 9 10 11 12
65 64 63 62 61 60 59 58 57 56 55 54
H
models
out-of-sample forecasts by length of forecasts (1982-1
Root mean squared
percent
error
UBJ-ARIMA
TF-BCLI
TF-BCPC
1.77 2.53 3.19 3.78 4.29 4.79 5.20 5.56 5.69 5.83 5.79 5.88
2.22 2.81 3.37 3.67 3.91 3.94 4.04 4.13 4.08 4.24 4.24 4.25
1.56 2.39 2.89 3.22 3.39 3.45 3.45 3.37 3.28 3.22 3.14 3.12
UBJ_ARIMP
q q
TF_BCLl TF_BCPC
0 1
2
3
4
Forecast Exhibit models
5
6
7
a
9
10
11
12
lead in months
6. Forecasting accuracy by forecast lead: 1982-1
of UBJ, BCLI to 1987-5.
and
BCPC
tion models (TF-BCLI and TF-BCPC) has also been examined for 1984-2 (the bottom of a recession) and the 12 months following. For each of these 13 months, we have 12 forecasts ranging from 1 month ahead to 12 months ahead forecasts. The average error measured by the RMSPE is shown in Exhibits 7 and 8 to be relatively large at and immediately following the turning point in 1984-2. The RMSPE’s are in the 7-8% range for the first 2 months following the turning point in 1984-2. They fall to the 3-4% range in the next 2 months and to the l-2%
R. A. Holmes, A. F. M. Shamsuddin
82 Exhibit 7 Forecasting error 1984-Z to 1985-2. Months
1984
1985
6. Conclusion of TF-BCPC
and TF-BCLI
Root mean squared error (RMSPE)
February March April May June July August September October November December January February
I British Columbia industrial employment
by months:
percent
TF-BCPC
TF-BCLI
8.20 8.04 3.28 2.66 2.34 1.26 1.10 0.73 0.54 0.98 1.52 1.94 2.33
6.51 8.19 3.72 3.88 3.26 2.06 2.76 1.49 0.79 0.55 1.05 2.14 1.88
‘Oq
TF_BCLI
w
TF_BCPC
Overall our tests indicate that the leading indicator based on principal components of the input series provide more accurate forecasts than the leading indicator based on weighted r2 values. At each forecast horizon ranging from 1 to 12 months, TF-BCPC provided smaller forecast errors than either TF-BCLI or the UBJARIMA model. Since these results are based on a large number of tests ranging from 65 of the l-month ahead forecasts down to 54 of the 12month ahead forecasts, these results are quite conclusive.’ Moreover, when we examine the results separately by month following the 1984-2 turning point we find similar results. TF-BCPC is more accurate than TF-BCLI in 8 of the 9 months immediately following the turning point (1984-2 to 1984-10). We conclude that the preferred leading indicator is the one based on principal components of the input series (TFBCPC) .’ The reason for these results may be the methodological superiority of a multivariate over a univariate approach. The weighted r2 leading indicator is based on univariate regressions which omit all but one of the input variables. Since the included explanatory variable is unlikely to be orthogonal to some of the omitted variables, the resulting error variance is likely underestimated and Y’ overestimated. Consequently the weights based on relative r2 values may not reflect the true importance of an input variable in forecasting BCEI. The principal component approach, on the other hand, is a multivariate approach that generates a leading indicator that captures the maximum possible proportion of total variation in input variables, and
’ This conclusion Exhibit 8. Forecasting accuracy by month: 1984-2 to 1985-2.
of BCLI
and BCPC
models
range after that. TF-BCPC has smaller forecast errors than TF-BCLI in 8 of the 9 months immediately following the turning point (1984-2 to 1984-10). TF-BCLI is more accurate in 3 of the 4 later months (1984-11 to 1985-2).
is strengthened somewhat by the necessity of excluding the VSE index, which is not available over the entire range of our data, from the principal components approach. Had it been possible to include the VSE index in our principal components leading indicator, it would be reasonable to expect a marginally better forecasting performance from TF-BCPC. This does not necessarily mean that the principal components approach is optimal. Boschan and Banerji (1990, p. 220) argue that principal components ‘minimize the impact of indicators which do not move with the other indicators, and this may defeat the purpose of the composite index’.
R.A.
Holmes, A. F.M.
Shamsuddin
is not subject to the omitted variable problem affecting the weighted r2 leading indicator.
Acknowledgments This research was supported by SSHRC of Canada. The authors are also grateful for referees’ constructive criticisms.
References Auerbach, A.J., 1982, “The index of leading indicators: Measurement without theory, thirty-five years later”, Review of Economic Statistics, 64, 589-595. Boschan, C. and A. Banerji, 1990, ‘A reassessment of composite indexes’, in: P.A. Klein, ed., Analyzing Modern Business Cycles: Essays Honoring Geoffrey H. Moore (Sharpe, New York). Holmes, R.A., 1986, “Leading indicators of industrial employment in British Columbia”, International Journal of Forecasting, 2, 87-100.
I British Columbia industrial employment
83
Shamsuddin, A.F.M., R.A. Holmes and A.K.M.S. Alam, 1985, “Univariate ARIMA and bivariate transfer function models for inflation: Some Malaysian evidence”, The Philippine Economic Journal, XXIV, 263-287. Forecasting of Industrial Terasvirta T., 1984, “Short-term Production by Means of Quick Indicators”, Journal of Forecasting, 3, 409-416. Wonnacott, R.I. and T.H. Wonnacott, 1979, Econometrics (Wiley, Toronto). Biographies: Richard A. HOLMES is Professor in the Faculty of Business Administration and the Department of Economics at Simon Fraser University. His research interests are in business and economic forecasting. Previous publications include articles in the American Economic Review, the Journal of the American Statistical Association, the International Journal of Forecasting, the Journal of Business, and the Journal of Human Resources. Abul F.M. SHAMSUDDIN is a lecturer of Economics at the University of New England, Armidale, Australia. He is also a doctoral candidate in Economics at Simon Fraser University, Vancouver, Canada. His research interests are in international economics, economics of immigration and economic forecasting. He has contributed articles in the Journal of International Economic Integration, Journal of Economics and International Relations, and The Philippine Economic Journal.