- Email: [email protected]
Reply to the commentary
International North-Holland
Journal
of Forecasting
461
3 (1987) 461-462
REPLY TO THE COMMENTARY Roderick
Abstract:
J. BRODIE
and Cornelis
A. DE KLUYVER
Our conclusion that econometric market share models may not be consistently more accurate than naive models was not surprising to the commentators. The comments they made help clarify the research that is needed to identify when econometric market share models are likely to be useful for forecasting.
The commentaries have made a useful extension to our paper by providing explanations about why econometric market share models may not be consistently than naive extrapolation models for prediction under four headings.
a number of more accurate
Is naive so naive? Bass and other commentators point out that the difference between the marketing mix causal models and the ‘naive’ (i.e., the previous period market share) model may not be so great as it initially appears. This is because the reduced form of the marketing mix models usually has a lagged market share variable (i.e., the naive model) that reflects the carryover effects of marketing effort and/or consumer inertia. In most cases, the estimates for the coefficient of this variable are statistically significant, and in many cases the lagged market share variable has a dominant influence in determining market share. For example, Aaker and Jacobson refer to their own research using the PIMS data base where lagged market share made an overwhelming contribution as compared to the other commonly assumed influences of market share. This conclusion also holds for empirical studies evaluated in our study. Further support for the ‘sophistication’ of naive time series models is given by Aaker and Jacobson. They point out that many apparently naive time series models can be shown to be reduced form representations of structural models that may have sophisticated causal mechanisms. Incorrectly
specified causal models vs. naive models
A second related explanation given by Aaker and Jacobson and alluded to by other commentators is that there is a reasonably high probability that the causal market share models may be misspecified. The lack of generally accepted theory about aggregate market response and marketing mix competition means there is little a priori reason to support or reject any of a number of plausible model specifications. Tlius, there is a reasonai?le chance that one of the ‘sophisticated’ naive models may out-perform an incorrectly specified causal model (Aaker and Jacobson). 0169-2070/87/$3.50
0 1987, Elsevier Science Publishers
B.V. (North-Holland)
R.J. Brodie and C.A. de Kluyuer / Repl,v
462
Limited observations
- the trade off between bias and precision
A third related explanation mentioned by the commentators relates to the limited number of observations for model estimation and the inherent randomness in econometric models. For example, in the four studies evaluated in our paper the number of observations used for estimation ranged from 16 to 30. Erickson and Wittink point out that in such cases there is a trade off between bias and precision. The analyst may choose to construct a simpler, incompletely specified model with biased predictors but with lower variance. Alternatively, the analyst may prefer more complete model specification with predictors that are unbiased but have higher variance. Thus, the biased naive lagged market share model may sometimes produce more accurate predictions than the more complete and unbiased marketing mix models that have predictors with higher variance. The theoretical implications of this tradeoff are explored by Hagerty. He presents a model that predicts the proportion of times an econometric model is likely to outperform the naive model for different numbers of parameters and observations. Other problems
associated with data
The commentators also mentioned a number of other data-related problems that may affect the estimation and hence the predictive properties of the marketing mix models. These include problems with data aggregation, measurement errors, lack of variation among individual data series, and multi-collinearity problems that make it difficult to separate the individual effect of some of the marketing mix variables. Extending
the agenda for research
The explanations that have been summarized above give rise to a number of important questions that both clarify and extend the set of research questions given in the conclusion of our paper. These research questions all relate in some way to the general question: Under what conditions are econometric market share models likely to produce more accurate forecasts? Future empirical research needs to examine predictive validity so that generalizations can be developed to answer this question. Our survey has shown that most of the large number of published empirical studies to date have failed to do this. Considerable scope also exists for the development of Monte Carlo computer simulation studies to clarify some of the more specific issues.
Journal
of Forecasting
461
3 (1987) 461-462
REPLY TO THE COMMENTARY Roderick
Abstract:
J. BRODIE
and Cornelis
A. DE KLUYVER
Our conclusion that econometric market share models may not be consistently more accurate than naive models was not surprising to the commentators. The comments they made help clarify the research that is needed to identify when econometric market share models are likely to be useful for forecasting.
The commentaries have made a useful extension to our paper by providing explanations about why econometric market share models may not be consistently than naive extrapolation models for prediction under four headings.
a number of more accurate
Is naive so naive? Bass and other commentators point out that the difference between the marketing mix causal models and the ‘naive’ (i.e., the previous period market share) model may not be so great as it initially appears. This is because the reduced form of the marketing mix models usually has a lagged market share variable (i.e., the naive model) that reflects the carryover effects of marketing effort and/or consumer inertia. In most cases, the estimates for the coefficient of this variable are statistically significant, and in many cases the lagged market share variable has a dominant influence in determining market share. For example, Aaker and Jacobson refer to their own research using the PIMS data base where lagged market share made an overwhelming contribution as compared to the other commonly assumed influences of market share. This conclusion also holds for empirical studies evaluated in our study. Further support for the ‘sophistication’ of naive time series models is given by Aaker and Jacobson. They point out that many apparently naive time series models can be shown to be reduced form representations of structural models that may have sophisticated causal mechanisms. Incorrectly
specified causal models vs. naive models
A second related explanation given by Aaker and Jacobson and alluded to by other commentators is that there is a reasonably high probability that the causal market share models may be misspecified. The lack of generally accepted theory about aggregate market response and marketing mix competition means there is little a priori reason to support or reject any of a number of plausible model specifications. Tlius, there is a reasonai?le chance that one of the ‘sophisticated’ naive models may out-perform an incorrectly specified causal model (Aaker and Jacobson). 0169-2070/87/$3.50
0 1987, Elsevier Science Publishers
B.V. (North-Holland)
R.J. Brodie and C.A. de Kluyuer / Repl,v
462
Limited observations
- the trade off between bias and precision
A third related explanation mentioned by the commentators relates to the limited number of observations for model estimation and the inherent randomness in econometric models. For example, in the four studies evaluated in our paper the number of observations used for estimation ranged from 16 to 30. Erickson and Wittink point out that in such cases there is a trade off between bias and precision. The analyst may choose to construct a simpler, incompletely specified model with biased predictors but with lower variance. Alternatively, the analyst may prefer more complete model specification with predictors that are unbiased but have higher variance. Thus, the biased naive lagged market share model may sometimes produce more accurate predictions than the more complete and unbiased marketing mix models that have predictors with higher variance. The theoretical implications of this tradeoff are explored by Hagerty. He presents a model that predicts the proportion of times an econometric model is likely to outperform the naive model for different numbers of parameters and observations. Other problems
associated with data
The commentators also mentioned a number of other data-related problems that may affect the estimation and hence the predictive properties of the marketing mix models. These include problems with data aggregation, measurement errors, lack of variation among individual data series, and multi-collinearity problems that make it difficult to separate the individual effect of some of the marketing mix variables. Extending
the agenda for research
The explanations that have been summarized above give rise to a number of important questions that both clarify and extend the set of research questions given in the conclusion of our paper. These research questions all relate in some way to the general question: Under what conditions are econometric market share models likely to produce more accurate forecasts? Future empirical research needs to examine predictive validity so that generalizations can be developed to answer this question. Our survey has shown that most of the large number of published empirical studies to date have failed to do this. Considerable scope also exists for the development of Monte Carlo computer simulation studies to clarify some of the more specific issues.