- Email: [email protected]
Omega 21
International
ELSESV,IER
Journal
of Forecasting
Research
10 (1994)
647-650
on forecasting
The International Journal of Forecasting provides critiques of papers published elsewhere. The editors try to select recent papers that are likely to be of significant interest to the readers. Our review of each paper is sent to the author for comment prior to publication. Almost all authors respond with suggestions and these typically lead to improvements in the critiques. If you know of interesting papers or if you have published such a paper, please send a copy to the Editors for possible inclusion in this section of the International Journal of Forecasting. To obtain copies of the papers reviewed in this section, contact the authors of the original papers.
1975 (VCR). In addition to estimating diffusion curves, he looked at the penetration rates 15 years after introduction. Bayus also summarizes findings from seven previously published studies. In general, he concludes that there is little evidence that product life cycles are getting shorter. An extended paper on this topic “Are product life cycles really getting shorter?” will appear in the Journal of Product Innovation Management, September 1994. - J. Scott Armstrong [Barry L. Bayus, Kenan-Flagler Business School, University of North Carolina, Chapel Hill, NC 27599-34901 SSDZ 0169-2070(94)00564-H
Barry L., Bayus, (1992), Have diffusion rates been accelerating over time?, Marketing Letters, 3 215-226.
As we all know, change is accelerating. This means that product life cycles are shorter and products diffuse faster. Right? Not so according to this interesting study by Bayus. He examined diffusion rates for seven major appliances (clothes washer, refrigerator, freezer, room A/C, electric clothes dryer, microwave oven, and trash compactor), seven housewares items (vacuum cleaner, waffle iron, heating pad, blender, can opener, electric toothbrush, and electric knife), and six consumer electronics items (radio, B&W TV, color TV, turntable, cassette tape deck, and VCR). These products had been introduced from 1908 (clothes washer and vacuum cleaner) to Elsevier
Science
B.V.
G.L., Riddington, (1993), Time varying coefficient models and their forecasting performance, Omega 21, 573-583. Riddington examines an important problem. He provides a systematic evaluation of research on time-varying coefficients in forecasting. And he writes with much conviction. He states that his study “concludes conclusively that the [time varying coefficient models] approach significantly improves forecasting performance” and that “it should be automatically considered by any management scientist undertaking the modeling of causal relationships over time.” He reached these conclusions by summarizing the results
648
Research on forecasting
from 21 forecasting studies. In my opinion, he overstates his case. Riddington’s evidence is based on ex post forecast accuracy, where the forecaster uses knowledge about the predictor variables from the forecast horizon. The use of en post forecast evaluation is relevant to assessing how well models might work for predicting the effects of changes in policy variables. However, it is difficult to answer such questions as whether this procedure would lead to better decisions, so I believe that it is stating the conclusion too forcefully to say that this procedure should always be used. As to the second conclusion, about the use of time varying coefficients in assessing causality, there is some hope that this might be true. What is needed is evidence showing that (1) time varying coefficient models provide more valid estimates, and (2) the improvements in validity are likely to have value for decision makers. Riddington does not address the issue of how this procedure relates to the quality of decisions. If the time varying procedure provides substantially better parameter estimates, one would hope that this would also improve in ex ante forecasts, where one has no information from the period to be forecast. This is the situation faced by the manager. A common finding in this area is that refinements in the estimation of the parameters in econometric models do not contribute much to increased accuracy (Armstrong, 1985, pp. 225-232 reviews the empirical evidence on this issue). Riddington does not address this issue of ex ante forecast accuracy. As to the review, it would have been useful to have had more details about the studies themselves and the way in which their results were coded. It would also have been helpful to have had details on how the search was conducted. Was the search reasonably complete? For example, Wildt and Winer’s (1983) review paper on application of time-varying parameter models in marketing listed 16 studies (their table 2). Of these 16 studies, only three were reviewed by Riddington. What was the explanation for ignoring the other studies? Dziechciarz (1989) provided an extensive review on this topic yet this was ignored by Riddington’s review. In general,
it would have been helpful to know the explicit procedure by which the search was conducted, how they were screened, and what was the reliability of this search and screening process. The standards for the treatment of studies in a meta-analysis are parallel to those for the treatment of data in other studies. As a general procedure, I recommend against using time varying coefficients procedures. They are harder to understand, more expensive, and may reduce the reliability of the model. They have not been shown to improve ex ante forecasts or decision making. On the other hand, I think this area has much promise. Surely there are some conditions under which variable coefficients are useful. I believe that the search for solutions has been misdirected, and that the problem should be reframed. It is not that the parameters change because of time. They change because of shifts in causal forces. Strong causal evidence that the parameters will change, or that they have recently changed, is unlikely to be found in the time series itself. If the structural changes are recent, then it is of particular importance to capture the change. However, in such cases one has only small samples (with, perhaps, unreliable data), and this may lead to false identification of changes in parameters. It would seem useful, then, to draw upon additional data. In particular, information from domain experts might be useful. Armstrong and Collopy (1993) found that domain knowledge could be easily obtained and coded. It produced substantial improvements in accuracy for extrapolation methods, and we expect that it might also be useful for econometric methods. a company may introduce a For example, superior product that will affect the parameters for an existing product. (It may no longer be effective to advertise the old product.) The area of “causal force-varying parameters” is one that deserves further attention. Riddington’s review and analysis should help in this endeavor.
References Armstrong, J.S., 1985, Wiley, New York).
Long-Range
Forecasting.
(John
Research on forecasting Armstrong, J.S. and F. Collopy, 1993, Causal forces: Structuring knowledge for time series extrapolation, Journal oj Forecasting, 12, 103-115. Dziechciarz, J., 1989, Changing and random coefficient models: A survey, in: Peter Hackl, Statistical Analysis and Forecasting of Economic Structural Change, 217-251. Wildt, A.R. and R.S. Winer, 1983, Modelling and estimation in changing market environments, Journal of Business, 56, 365-388.
-J. Scott Armstrong [G.L. Riddington, Department of Economics, Glasgow Caledonian University, Glasgow G4 OBA, Scotland, UK] SSDI 0169-2070(94)00565-6
Rick L. Andrews, 1994, Forecasting performance of structural time series models, Journal of Business and Economic
Statistics, 12, 129-133.
Structural time series models have received increasing attention from the research community with Harvey (1989) offering a complete review. In the class of structural model, Harvey (1984) has proposed the Basic Structural Model. An extrapolative model that should have sufficient flexibility to describe most time series and, when compared to alternative time series methods, produce equally accurate forecasts. Briefly, the Basis Structural Model (BSM) is of the form: Y, = /-$ + ‘y,+ E, -the observation equation p, = pLr_1 + p, + q,-the level equation P, = P,+, + 5;
-the trend equation
and ‘ytis a seasonal factor such that CT:,’ yr_j = wr where o1 is random with zero mean. The parameters in the model are the various error variances. They are estimated through maximum likelihood. Prior to this study the Basic Structural Model had not been evaluated on a wide variety of time series. Andrews uses the 111 time series of the M-competition and compared the results from
649
using the model to the ARIMA class of models, Bayesian forecasting and the approaches due to Lewandowski and Parzen. (Makridakis et al, 1982). Andrews concluded that the Basic Structural Model “appears to perform quite well on annual, quarterly and monthly data, especially for long forecast horizons . . . and seasonal data.” He used MAPE and Median APE to evaluate the methods and noted that using MAPE benefits the BSM, i.e. it captured the characteristics of a few time series that the other methods failed substantially on, however, it typically performed somewhat worse. The study goes a little way to answering questions on the comparative performance of the BSM and supports Harvey’s contention that it has good accuracy characteristics and is readily interpretable, unlike some of the other complex approaches. However, I would have liked the author to have considered issues such as the type of series where performance characteristics differed in important respects from the other methods analysed and the effects of combining the BSM with the alternatives. -Robert Fildes [R.L. Andrews, Department of Business Administration, University of Delaware, Newark, DE 19716, USA]
References Harvey, A.C., 1984, A unified view of statistical forecasting procedures with discussion, Journal of Forecasting, 3, 245283. Harvey, A.C. 1989, Forecasting, Structural Time Series Models and the Kalman Filter, (Cambridge University Press, Cambridge). Makridakis, S., A. Andersen, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, P. Parzen and R. Winkler, 1982. The accuracy of extrapolation (time series) methods: Results of a forecasting competition, Journal of Forecasting, 1, 111-153.
SSDZ
0169-2070(94)00566-4
ELSESV,IER
Journal
of Forecasting
Research
10 (1994)
647-650
on forecasting
The International Journal of Forecasting provides critiques of papers published elsewhere. The editors try to select recent papers that are likely to be of significant interest to the readers. Our review of each paper is sent to the author for comment prior to publication. Almost all authors respond with suggestions and these typically lead to improvements in the critiques. If you know of interesting papers or if you have published such a paper, please send a copy to the Editors for possible inclusion in this section of the International Journal of Forecasting. To obtain copies of the papers reviewed in this section, contact the authors of the original papers.
1975 (VCR). In addition to estimating diffusion curves, he looked at the penetration rates 15 years after introduction. Bayus also summarizes findings from seven previously published studies. In general, he concludes that there is little evidence that product life cycles are getting shorter. An extended paper on this topic “Are product life cycles really getting shorter?” will appear in the Journal of Product Innovation Management, September 1994. - J. Scott Armstrong [Barry L. Bayus, Kenan-Flagler Business School, University of North Carolina, Chapel Hill, NC 27599-34901 SSDZ 0169-2070(94)00564-H
Barry L., Bayus, (1992), Have diffusion rates been accelerating over time?, Marketing Letters, 3 215-226.
As we all know, change is accelerating. This means that product life cycles are shorter and products diffuse faster. Right? Not so according to this interesting study by Bayus. He examined diffusion rates for seven major appliances (clothes washer, refrigerator, freezer, room A/C, electric clothes dryer, microwave oven, and trash compactor), seven housewares items (vacuum cleaner, waffle iron, heating pad, blender, can opener, electric toothbrush, and electric knife), and six consumer electronics items (radio, B&W TV, color TV, turntable, cassette tape deck, and VCR). These products had been introduced from 1908 (clothes washer and vacuum cleaner) to Elsevier
Science
B.V.
G.L., Riddington, (1993), Time varying coefficient models and their forecasting performance, Omega 21, 573-583. Riddington examines an important problem. He provides a systematic evaluation of research on time-varying coefficients in forecasting. And he writes with much conviction. He states that his study “concludes conclusively that the [time varying coefficient models] approach significantly improves forecasting performance” and that “it should be automatically considered by any management scientist undertaking the modeling of causal relationships over time.” He reached these conclusions by summarizing the results
648
Research on forecasting
from 21 forecasting studies. In my opinion, he overstates his case. Riddington’s evidence is based on ex post forecast accuracy, where the forecaster uses knowledge about the predictor variables from the forecast horizon. The use of en post forecast evaluation is relevant to assessing how well models might work for predicting the effects of changes in policy variables. However, it is difficult to answer such questions as whether this procedure would lead to better decisions, so I believe that it is stating the conclusion too forcefully to say that this procedure should always be used. As to the second conclusion, about the use of time varying coefficients in assessing causality, there is some hope that this might be true. What is needed is evidence showing that (1) time varying coefficient models provide more valid estimates, and (2) the improvements in validity are likely to have value for decision makers. Riddington does not address the issue of how this procedure relates to the quality of decisions. If the time varying procedure provides substantially better parameter estimates, one would hope that this would also improve in ex ante forecasts, where one has no information from the period to be forecast. This is the situation faced by the manager. A common finding in this area is that refinements in the estimation of the parameters in econometric models do not contribute much to increased accuracy (Armstrong, 1985, pp. 225-232 reviews the empirical evidence on this issue). Riddington does not address this issue of ex ante forecast accuracy. As to the review, it would have been useful to have had more details about the studies themselves and the way in which their results were coded. It would also have been helpful to have had details on how the search was conducted. Was the search reasonably complete? For example, Wildt and Winer’s (1983) review paper on application of time-varying parameter models in marketing listed 16 studies (their table 2). Of these 16 studies, only three were reviewed by Riddington. What was the explanation for ignoring the other studies? Dziechciarz (1989) provided an extensive review on this topic yet this was ignored by Riddington’s review. In general,
it would have been helpful to know the explicit procedure by which the search was conducted, how they were screened, and what was the reliability of this search and screening process. The standards for the treatment of studies in a meta-analysis are parallel to those for the treatment of data in other studies. As a general procedure, I recommend against using time varying coefficients procedures. They are harder to understand, more expensive, and may reduce the reliability of the model. They have not been shown to improve ex ante forecasts or decision making. On the other hand, I think this area has much promise. Surely there are some conditions under which variable coefficients are useful. I believe that the search for solutions has been misdirected, and that the problem should be reframed. It is not that the parameters change because of time. They change because of shifts in causal forces. Strong causal evidence that the parameters will change, or that they have recently changed, is unlikely to be found in the time series itself. If the structural changes are recent, then it is of particular importance to capture the change. However, in such cases one has only small samples (with, perhaps, unreliable data), and this may lead to false identification of changes in parameters. It would seem useful, then, to draw upon additional data. In particular, information from domain experts might be useful. Armstrong and Collopy (1993) found that domain knowledge could be easily obtained and coded. It produced substantial improvements in accuracy for extrapolation methods, and we expect that it might also be useful for econometric methods. a company may introduce a For example, superior product that will affect the parameters for an existing product. (It may no longer be effective to advertise the old product.) The area of “causal force-varying parameters” is one that deserves further attention. Riddington’s review and analysis should help in this endeavor.
References Armstrong, J.S., 1985, Wiley, New York).
Long-Range
Forecasting.
(John
Research on forecasting Armstrong, J.S. and F. Collopy, 1993, Causal forces: Structuring knowledge for time series extrapolation, Journal oj Forecasting, 12, 103-115. Dziechciarz, J., 1989, Changing and random coefficient models: A survey, in: Peter Hackl, Statistical Analysis and Forecasting of Economic Structural Change, 217-251. Wildt, A.R. and R.S. Winer, 1983, Modelling and estimation in changing market environments, Journal of Business, 56, 365-388.
-J. Scott Armstrong [G.L. Riddington, Department of Economics, Glasgow Caledonian University, Glasgow G4 OBA, Scotland, UK] SSDI 0169-2070(94)00565-6
Rick L. Andrews, 1994, Forecasting performance of structural time series models, Journal of Business and Economic
Statistics, 12, 129-133.
Structural time series models have received increasing attention from the research community with Harvey (1989) offering a complete review. In the class of structural model, Harvey (1984) has proposed the Basic Structural Model. An extrapolative model that should have sufficient flexibility to describe most time series and, when compared to alternative time series methods, produce equally accurate forecasts. Briefly, the Basis Structural Model (BSM) is of the form: Y, = /-$ + ‘y,+ E, -the observation equation p, = pLr_1 + p, + q,-the level equation P, = P,+, + 5;
-the trend equation
and ‘ytis a seasonal factor such that CT:,’ yr_j = wr where o1 is random with zero mean. The parameters in the model are the various error variances. They are estimated through maximum likelihood. Prior to this study the Basic Structural Model had not been evaluated on a wide variety of time series. Andrews uses the 111 time series of the M-competition and compared the results from
649
using the model to the ARIMA class of models, Bayesian forecasting and the approaches due to Lewandowski and Parzen. (Makridakis et al, 1982). Andrews concluded that the Basic Structural Model “appears to perform quite well on annual, quarterly and monthly data, especially for long forecast horizons . . . and seasonal data.” He used MAPE and Median APE to evaluate the methods and noted that using MAPE benefits the BSM, i.e. it captured the characteristics of a few time series that the other methods failed substantially on, however, it typically performed somewhat worse. The study goes a little way to answering questions on the comparative performance of the BSM and supports Harvey’s contention that it has good accuracy characteristics and is readily interpretable, unlike some of the other complex approaches. However, I would have liked the author to have considered issues such as the type of series where performance characteristics differed in important respects from the other methods analysed and the effects of combining the BSM with the alternatives. -Robert Fildes [R.L. Andrews, Department of Business Administration, University of Delaware, Newark, DE 19716, USA]
References Harvey, A.C., 1984, A unified view of statistical forecasting procedures with discussion, Journal of Forecasting, 3, 245283. Harvey, A.C. 1989, Forecasting, Structural Time Series Models and the Kalman Filter, (Cambridge University Press, Cambridge). Makridakis, S., A. Andersen, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, P. Parzen and R. Winkler, 1982. The accuracy of extrapolation (time series) methods: Results of a forecasting competition, Journal of Forecasting, 1, 111-153.
SSDZ
0169-2070(94)00566-4