Forecasting practices: Empirical evidence and a framework for research

ARTICLE IN PRESS

Int. J. Production Economics 108 (2007) 84–99 www.elsevier.com/locate/ijpe

Forecasting practices: Empirical evidence and a framework for research Giulio Zotteria, Matteo Kalchschmidtb, a

Dipartimento di Sistemi di Produzione ed Economia Aziendale, Politecnico di Torino, C.so Duca degli Abruzzi 24, Torino, Italy b Dipartimento di Ingegneria Gestionale, Universita` degli Studi di Bergamo, Viale Marconi 5, 24044 Dalmine (BG), Italy Available online 26 January 2007

Abstract Demand forecasting is a relevant issue both in research and practice. In the past several papers have investigated forecasting practices. This research uses data from the GMRG survey to achieve three main objectives. First of all, we aim to describe current practices in the machinery and textile sectors; in particular we investigate: (i) aims and usage of the demand forecast for decision making; (ii) structure of the forecasting process; (iii) algorithms and tools adopted; (iv) both upstream and downstream cooperative forecasting and (v) performance. On a second perspective we investigate contingent variables such as structure (company size and sector), strategy (improvement priorities) and demand characteristics (e.g. number of products) and their relationship with forecasting practices. Finally, the impact of the current forecasting process on both forecasting accuracy and overall company’s performance is investigated. In the end we highlight gaps between current research and actual companies’ practices; such gaps are discussed to identify areas where support, new tools and concepts are needed to improve companies’ practices. r 2007 Elsevier B.V. All rights reserved. Keywords: Demand forecasting; Survey; Contingent approach

1. Introduction Forecasting is a relevant issue both in research and practice. Research has devoted major attention to this topic, and practitioners are often concerned with this process. Literature on demand forecasting has contributed on many different aspects from several points of view. Contributions have come from statistics and econometrics, in particular in terms of techniques to better model future demand. Corresponding author. Tel.: +39 035 205 2360;

fax: +39 035 562 779. E-mail address: [email protected] (M. Kalchschmidt).

Similar attention has been paid to the design of more qualitative techniques. From a different perspective, literature has also studied forecasting practices. Various contributions have analysed what tools companies use, what organization structure they adopt, and so on. From one side many authors have proposed descriptive analyses of how companies forecast. Watson (1996) provides a detailed description of how companies manage forecasting in the Scottish electronics industry. Similarly, Mentzer and Kahn (1997) provide evidence on practices within 207 US companies; they describe the organizational structure adopted, the department(s) responsible for forecasting and the degree of formalization of the

0925-5273/$ - see front matter r 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.ijpe.2006.12.004

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

process. Hughes (2001) focuses attention on organizational topics by means of data collected within 81 UK companies and highlights main barriers to proper forecasting. Similarly, Chaman (2003) investigates US companies and Duran and Flores (1998) analyse forecasting practices in Mexican companies, while Klassen and Flores (2001) focus their attention on Canadian ones and Tawfik Mady (2000) on Egyptian ones. Mentzer and Cox (1984) study the degree of familiarity and satisfaction of various forecasting approaches by means of a survey analysis on 160 companies. Similar contributions are given by Winklhofer and Diamantopoulos (2002). Wisner and Stanley (1994) compare forecasting practices between just-in-time (JIT) and non-JIT purchasing departments. Many authors have taken a contingent approach, by analysing how practices change according to specific independent variables. Several authors have identified firm size as a major driver. Larger firms seem to employ forecasts for different purposes (Peterson, 1993) compared to smaller ones; larger firms produce forecasts for different aggregation levels (both in terms of time, product and market; Winklhofer and Diamantopoulos, 1997; Peterson, 1993; Peterson and Jun, 1999), tend to use a bottom-up forecasting process (Peterson, 1993), use quantitative and sophisticated forecasting techniques (Peterson, 1993; Sanders and Manrodt, 1994). However, there is no conclusive evidence regarding the impact of firm size on forecast performance (Diamantopoulos and Winklhofer, 1999). Other contributions investigate the type of industry: usually forecasters in manufacturing firms are more familiar with complex techniques than forecasters in service firms (Sanders, 1992). Conflicting evidence is found when the relationship between the type of technique used and accuracy achieved is studied (Peterson, 1990; Kahn and Mentzer, 1995). Uncertainty and environmental turbulence is also considered. Sanders and Manrodt (1994) show that environmental turbulence is correlated to the judgmental adjustment of quantitative techniques. Diamantopoulos and Winklhofer (1999) show that uncertainty is negatively related to forecast performance (Watson, 1996). For a more detailed review of empirical literature on forecasting practices we refer to Winklhofer et al. (1996). While great attention has been paid to environmental variables that influence forecasting performance and practices, other authors have also analysed what practices are more effective. Kahn

85

and Mentzer (1994) provide evidence that teambased forecasting tends to improve the accuracy of forecasts particularly for longer horizons. Companies seem also to be more satisfied with team-based forecasting. Wacker and Sprague (1995) show that performances are positively influenced by the availability of newer equipment. Similar correlation is found with the importance of sales planning. Quite interestingly authors show that the ‘‘higher’’ the organisational involvement the less accurate forecasts will be. The same authors analyse how cultural elements within the organization (e.g., individualism, power–distance, uncertainty avoidance, etc.) affect forecasting practices and thus accuracy (see Wacker and Sprague, 1998). While attention has focused on the impact of environmental variables on forecasting processes and performance, we argue that some relevant drivers have been overlooked by previous works. First of all, current literature adopts an ‘‘environment-driven’’ perspective: a company defines its forecasting practices according to its size, market uncertainty and so on. However, frequently this basic assumption seems not to hold. Companies define their processes also according to the specific objectives they have, both in terms of forecasting accuracy and operational performance. In other words, companies may differ since they have different specific business priorities; e.g., if attention is paid to cost reduction (e.g., stock and backlog reduction), forecasting accuracy will be a major issue. However, if attention is paid more to quality or safety issues, forecasting will be far less relevant. Also, the need for an accurate forecast might depend on the level of investment in flexible manufacturing systems. In other words flexibility might somehow substitute forecast accuracy. Companies do not consider only what they want (thus their objectives) but also what they do: thus companies, try to design a specific forecasting strategy that is internally consistent in terms of techniques adopted, information collected, people involved, and so on. For example, nobody will collect a great amount of information if he/she does not have any proper tool to use it or people to manage it. Also the use of formal techniques is constrained by the skills of the employees involved in the process. Compared to previous contributions, this paper adopts a broader perspective and investigates whether forecasting practices are mainly influenced by external variables (i.e., number of products,

ARTICLE IN PRESS 86

G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

company size, demand variability, etc.) or by strategic aims and internal consistency. The remainder of the paper is organized as follows: in the next section research objectives are defined and then research methodology is briefly described. Then analyses and results are discussed in detail; in the final paragraph, conclusions and guidelines for future research are provided. 2. Research aims The main scope of this work is to identify the impact of external variables compared to internal ones. Thus we do not aim at providing estimates of any specific relationship between any specific practice, objective and performance metric, but we focus on more aggregate effects. For example, we do not try to estimate whether companies that cooperate with suppliers tend to use exponential smoothing more widely than others. On the contrary, we wonder whether the choice of forecasting tools and techniques depends on the willingness to cooperate in the supply chain. We argue that external variables might not be the main drivers, but, on the contrary, companies are influenced by their forecasting objectives. Moreover, we argue that looking at forecasting accuracy and trying to link it to particular practices may be sometimes misleading. As a matter of fact, forecasting accuracy is not an objective ‘‘per se’’; it

Structural factors

Demand Characteristics

really is a mean to achieve other objectives such as service levels, inventories turnover, etc. Also, accuracy should be adjusted for the degree of demand variability and uncertainty with indexes such as Theil’s U, which however are neither applied nor well known in practice. This might make accuracy data hard to compare and read and at times misleading. Given this framework the main objective of this paper is to analyse the drivers of forecasting practices. As Fig. 1 shows, our analysis focuses on three issues: 1. Aims and structural factors: First, we will analyse contingency factors that lead to specific forecasting practices. Here we will consider structural variables (e.g., company size and sector), demand characteristics (e.g. number of products) and forecasting aims. The specific objective is to identify to what extent forecasting aims influence forecasting practices. We will analyse what elements mainly explain how companies structure their forecasting processes. So we will consider both ‘‘external’’ (e.g., elements that typically are not controlled by who is in charge of forecasting) and ‘‘internal’’ elements (e.g., specific practices, forecasting aims, and so on) and try to capture the relative strength of these drivers. As previously anticipated, we argue that companies, even if influenced at some rate by the kind of problem

Forecasting aims 1) Aims and external elements 2) Internal consistency

Forecasting Practices 1. process 2. tools 3. cooperation 4. control 5. organization

Forcasting Performance

Company Performance Fig. 1. Overall structure of the research.

3) Practices and performance

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

they have to manage, are particularly concerned with what forecasting is used for and thus try to build a process that is consistent both internally and with the forecasters’ scope. Thus in the following empirical analyses we will analyse relationships among practices using external variables as control factors. 2. Internal consistency: Then, forecasting practices will be investigated to identify the relevance of the internal consistency of the process as compared to that of external variables and forecasting aims (variables investigated in the first step). We argue that companies are more concerned with developing an internally consistent forecasting process than with the kind of problem they are dealing with. 3. Practices and performances: In the end, like other papers in the past, we will investigate the practices–performance relationships. Attention will be paid on how the process is conducted, on which organizational structure is adopted, on the degree of cooperation along the supply chain and so on. We argue (as previous authors do) that several elements are tied to forecasting performance, even if the relationship may be somehow complicated. These analyses can shed some light on how forecasting practices are influenced by ‘‘external’’ elements, how practices are influenced by the use of forecasting and how forecasting practices are tied to company performance. Our perspective is rather different from previous papers as we will not investigate exclusively the relationship between ‘‘external’’ elements and forecasting practices. To implement the above analyses, we will capture these rather broad concepts through a wide set of variables. 2.1. Structural factors A rather wide set of works shows some kind of relationship between company size and its forecasting practices. In this paper company size is measured by the number of employees and overall sales. Also we capture the industry the company belongs to. 2.2. Demand characteristics Literature shows that companies tend to change their forecasting processes according to the number

87

of products. A general understanding is that quantitative techniques become more effective than qualitative ones as the number of products increases. Here we take into account the number of products for which a forecast is developed and the level of customisation the market requires, as this might capture the number of product variants. 2.3. Forecasting aims First of all, we consider forecasting aims in terms of what companies use forecast for (i.e., production planning, budget preparation, new product development and so on). We argue that the forecasting process is designed according to what forecast is made for. For example, if a company mainly forecasts for budgeting purposes, marketing, sales and finance departments are likely to be more involved than logistics and production departments. The organization will be quite different in case the company uses forecasts mainly for production planning (probably production will be a major player). Here we will consider nine different purposes of forecasting: budget preparation, production planning, subcontracting decisions, material/ inventory planning, sales planning, human resource planning, new product development, facilities planning and equipment purchase planning. 2.4. Forecasting practices We describe forecasting practices through five classes of variables: process, tools, cooperation, control and organization.



Process: The forecasting process consists of various activities. In order to build forecasts, companies need to (i) collect information, then (ii) they can generate and modify forecasts and, in the end, (iii) they can measure forecasting accuracy. We argue that the attention paid to the various activities may depend on both internal and external variables, and if any kind of aggregation/disaggregation is conducted during this process. Literature has analysed whether aggregation or disaggregation of data can at some rate influence accuracy; however, few contributions investigate whether companies actually aggregate/disaggregate demand data (Mentzer and Kahn, 1997; Zotteri et al., 2005). Also we capture forecasting time buckets and forecasting horizons.

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

88









Tools: We will also take into account the employment of specific tools. Literature typically considers whether a quantitative or qualitative approach is adopted (Mentzer and Cox, 1984; Wisner and Stanley, 1994; Wacker and Sprague, 1995; Kahn and Mentzer, 1995). Here we will study the same dichotomy. Cooperation: We will consider the degree of cooperation along the supply chain. Literature is in fact concerned with the impact of information exchange on internal processes (e.g., Cooper et al., 1997a, b; Ellram and Cooper, 1990). Thus we will take into account the degree of information exchange with suppliers and customers. Organization: Literature has provided evidences that internal cooperation can significantly improve forecasting accuracy (Kahn and Mentzer, 1994; Mentzer and Kahn, 1997). Here attention is paid mainly to the involvement of different organizational units in the forecasting process. Control: Forecasting control is also a major issue and here we measure whether accuracy is measured formally or informally.

2.5. Methodology Analyses were conducted by means of a survey research based on the third version of the Global Manufacturing Research Group questionnaire developed in Italian non-fashion textile and machine tooling companies. A sample of 941 companies (453 machine and 488 textile) was identified, companies were invited to participate by direct contacts and 97 questionnaires were sent out. Among these, only 60 companies provided usable questionnaires thus gaining an overall response rate of 6.4% (7.9% machine and 4.9% textile). The sample consists mainly of small companies, showing both increasing and decreasing sales, thus claiming that both successful and unsuccessful companies are considered. Of the sample, 35% declared an increase in sales (on average equal to 18%), while 58% of the companies declared a decrease in sales (on average equal to 12%). The remaining companies declared stable sales over the last 2 years (see Table 1). Thus we argue that we have a sample of actual forecasting practices, and thus we can find what companies do. More normative conclusions should be rather careful as we can only suggest what companies should be doing by comparing performance among them which is often hard to do. Hence, our study has

Table 1 Sample description

Mean Median Std. dev.

Employees

Sales (Mh)

Sales increase (%)

97 38 155

18 9 28

0.6 2.7 18.5

mainly explanatory and descriptive rather than normative purposes. Information on forecasting aims and practices was collected by means of Likert scales (1–7) while demand characteristics, structural factors and performances were evaluated on absolute scales (copy of the GMRG questionnaire is available via the GMRG website: /http://www.gmrg.orgS). Since the objectives of this work are mainly explorative and given the rather wide set of variables (compared to the number of total observations), analyses were conducted by means of stepwise linear regression (the process is similar to previous contributions, see for example Wacker and Sprague (1995) and Henderson (2004)). Indeed we have no strong hypothesis on the relationship among single variables but rather assume a relationship among sets of variables such as the organizational ones and the tools adopted by the company. Replication would be important in future works to deeply analyse the relationships identified in this article. We think that in other samples each single relationship may show significant differences, however we argue that the influence of external and internal variables on practices should hold. We mainly adopted a forward regression (po0.05) but tested also backward and mixed approaches: no significant differences seemed to arise. Causal analysis, as Bayesian Networks and Structure Equation Modelling, would have been a more effective methodology, however the small sample size does not allow to adopt such methodologies efficiently. The methodology adopted however is consistent with the objective of determining main drivers of company practices and performance. The small sample size may give rise to some issues concerning the reliability of the provided analysis. According to Miles and Shavin (2001), the sample size used is fine to catch large effects if no more than six predictors are used. Thus given the explorative scope of this work, we argue that the sample size is fine to investigate this issue.

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

3. Data analysis 3.1. Aims and external variables In this section analyses will follow the overall scheme presented in Fig. 1. Thus we start by investigating the relationship between structural factors, demand characteristics, forecasting aims and forecasting practices (as measured by process, tools, organization, cooperation and control). 3.2. Process Forecasting is a rather complex process. First, companies have to gather proper information to feed a forecasting process, then they have to generate forecasts by means of some methodology/ tools, then they can modify forecast according to any information they have not provided during the first round of forecast, and in the end performance can be measured. We investigate whether the importance of these four steps is influenced by external factors rather than by the forecasting aims. Table 2 summarizes results. Efforts spent in information collection are not influenced by either structural factors or other elements. This is probably due to the overall importance companies pay to data collection despite size or scope. Interestingly, forecast generation seems to be a relevant issue when forecasting is heavily used for inventory and material planning (explaining 86% of total explained variance), while the number of product

89

lines seems to have a moderate positive effect on this activity. Forecast modification appears to be relevant when forecasts are mostly used for production planning (explaining 52% of total explained variance). Also, the number of product lines seems to drive the relevance of forecast modification, since managers have to modify forecasts at product level and thus a larger number of products require more efforts and time. An interesting, though less relevant, effect is found for what regards the size of company as measured by the number of employees. As Table 2 shows, smaller companies seem to pay more attention to this activity as compared to larger ones. In the end measuring performance appears to be quite important for companies that mainly use it for sales planning. The most interesting result is the fact that in all analyses forecasting aims seem to play a major role, thus claiming that attention to forecasting activities is mainly driven by what companies use forecasting for rather than to the environmental conditions they are facing. Though we find this result quite intriguing as it sheds light on a new area that has been so far relatively overlooked by literature, we acknowledge that this finding might depend on the sample that mostly consists of SMEs from just two industries. While previous analyses investigate the relevance of various steps of the forecasting process, we now investigate how these are performed. First, aggregation/disaggregation of forecasting is analysed. In particular we study the differences among

Table 2 Linear regression results for forecasting process Forecast activities

Mean (std. dev.)

Variable

Information retrieval Forecast generation

5.51 (1.53) 4.29 (1.61)

No significant regressor Constant No. of product lines Use forecast for inventory/ material planning

Forecast validation and modification

4.16 (1.45)

Performance measurement

3.96 (1.72)

Variable type

Demand charact. Forecasting aims

Constant No. of employees No. of product lines Use forecast for production planning

Structural elements Demand charact. Forecasting aims

Constant Use forecast for sales planning

Forecasting aims

p

R2 adj

1.512 0.079 0.544

0.0090 0.0500 o0.0001

34.20% (n ¼ 52)

1.670 0.003 0.122 0.494

0.0194 0.0245 0.0078 0.0005

21.40% (n ¼ 52)

1.39 0.594

0.0092 o0.0001

34,40% (n ¼ 55)

Estimate

Note that in the results’ tables we highlight the variables whose selection is consistent with our hypotheses.

ARTICLE IN PRESS 90

G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

companies that aggregate or disaggregate data with companies that do not. Quite interestingly, there is no statistically significant difference in external variables between the two subsamples, claiming that both small and larger companies pay attention to this issue. On the contrary, companies that do aggregate/disaggregate differ from those that do not as to forecasting aims (see Table 3). Forecasting for subcontracting decisions and facilities planning is a major concern for companies that aggregate or disaggregate. Hence, the structure of the forecasting process seems to be related to forecasting aims (see Table 4). As Table 4 shows, companies that use forecast for new product development tend to forecast over longer time horizon. Quite interestingly performance measurement seems to be very important for those companies that use forecasts for sales planning. In the end, the number of forecast revisions does not seem to be much influenced by the use of forecasts, while larger companies seem to review forecasts more frequently. 3.3. Tools No significant relationship could be found regarding tools. That means that neither the aims

of forecasting practices nor exogenous variables seem to drive the adoption of a specific forecasting approach/tool. This is actually an intriguing finding as most literature on forecasting focuses on algorithms and tools. 3.4. Cooperation Cooperation with customers and suppliers is a hot topic in these days. Here we try to identify those variables that might influence the degree of cooperation within the supply chain. In particular we considered five different modes of cooperation both with customers and suppliers. Three focus on simple information exchange whether it concerns demand data, sales forecasts or production plans; two consider the degree (if any) of joint development of forecasts and production plans. In order to simplify both analysis and interpretation of results we applied a factor analysis. Results identified two main factors as described in Table 5; Cronbach’s alpha has been used to verify the consistency. Table 6 summarizes regression results. A general overview of results shows that the degree of information sharing is influenced by forecasting aims while external elements do not seem to play a significant role. Sales planning seems

Table 3 ANOVA for aggregation of forecasting process Use of forecast

Mean

Std. error

F ratio

p

Use forecast for subcontracting decisions

No aggregation/disaggregation Aggregation/disaggregation

2.73 4.14

0.415 0.345

6.85

0.012 (n ¼ 54)

Use forecast for facilities planning

No aggregation/disaggregation Aggregation/disaggregation

3.60 4.78

0.338 0.289

7.08

0.011 (n ¼ 54)

Table 4 Linear regression results for forecasting structure

Time horizon length

Performance measurement No. of forecast revisions

R2 adj

p

Variable

Variable type

Estimate

Constant No. of product lines Use forecast for facilities planning Use forecast for new product development

9.157 0.732 2.521 1.856

0.041 0.016 0.0500 0.019

21.09% (n ¼ 54)

Demand charact. Forecasting aims Forecasting aims

Constant Use forecast for sales planning

1.39 0.594

0.0090 o0.0001

34.40% (n ¼ 54)

Forecasting aims

Constant No. of employees

5.501 0.161

0.4890 o0.0001

32.30% (n ¼ 49)

Structural factors

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

to be a key aim since it positively and significantly influences both forecast exchange and joint development. Similar results hold for cooperation: again

forecasting aims seem to influence cooperation decisively; not surprisingly, inventory and material planning seem to play a relevant role: as it is often the case, cooperation at the operational level is the easiest to achieve.

Table 5 Factor loads for cooperation along the supply chain Joint development Demand data exchange Forecast exchange Production plan exchange Forecast joint development Production plan joint development Cronbach’s alpha

3.5. Control

Information exchange

Another interesting issue concerns how forecasting errors are measured. In particular, companies, especially SMEs, tend not to quantitatively measure forecast accuracy. Here we analysed what elements (structural factors and aims) significantly differ between companies that use formal measures and those who do not. Since data on error measurement are Boolean, they were analysed by means of ANOVA. Table 7 summarizes results for those elements that showed a significant difference (po0.05).

0.8703 0.8753 0.8292 0.8496 0.8991 0.745

91

0.853

Extraction method: Principal component analysis. Rotation method: Varimax with Kaiser normalization. No. factor loads smaller than 0.5 were omitted.

Table 6 Linear regression results for cooperation along the supply chain Variable

Information exchange

Joint development

Variable type

Constant Use forecast for production planning Use forecast for sales planning Use forecast for facilities planning

Forecasting aims Forecasting aims Forecasting aims

Constant Use forecast for inventory and material planning Use forecast for sales planning

Forecasting aims Forecasting aims

Estimate

Std. error

t ratio

p

R2 adj

0.380 0.335

0.430 0.099

0.884 3.398

0.381 0.001

25.91% (n ¼ 53)

0.223

0.081

2.768

0.008

0.240

0.085

2.813

0.007

1.804

0.364

4.956

0.000

0.244

0.068

3.577

0.001

0.169

0.070

2.428

0.019

33.20% (n ¼ 53)

Table 7 ANOVA results for formal and informal measures Mean

Std. error

F ratio

p

Materials planning

Formal measure Informal measure

5.17 4.06

0.360 0.278

5.95

0.018 (n ¼ 55)

Sales planning

Formal measure Informal measure

5.38 3.91

0.338 0.261

11.75

0.001 (n ¼ 55)

Human resource planning

Formal measure Informal measure

4.71 3.89

0.301 0.233

4.73

0.034 (n ¼ 55)

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

92

Companies that measure accuracy properly use forecast more for materials, sales and human resource planning more than an average company. This makes sense as in these fields gaps between actual demand and forecast can be used for decision making purposes. In particular, materials planning needs to set safety stock, according to forecasting errors; in sales planning companies compare budget and actual sales as part of their incentive scheme, thus making error measurement a relevant topic. 3.6. Organization Here we investigate what elements influence the degree of involvement of the various departments in the forecasting process (see Table 8). Again forecasting aims seem to play a major role. Marketing departments are mostly involved when budget has to be developed and human resources

have to be planned, while its involvement is reduced when production is planned. Sales department involvement is negatively related with subcontracting and facilities decisions; quite interestingly there is no relationship between sales department involvement and sales planning (maybe some companies do not think that sales personnel should be involved when sales plans have to be defined as sales plans are often used to measure sales personnel effectiveness). As expected, finance department is mainly involved in small companies and when planning sales. Similar findings hold for administration departments. Quite interestingly, no significant relationship is found for manufacturing and logistic departments. A general overview of results shows that forecasting aims tend to significantly influence how forecasting is conducted. On the contrary, external variables do not seem to play a major role. Our data

Table 8 Linear regression results for forecasting organization Department involvement

Mean (std. dev.)

Variable

Variable type

Marketing dpt.

4.78 (1.94)

Constant Use forecast for production planning Use forecast for human resource planning

Forecasting aims Forecasting aims

Sales dpt.

Finance dpt.

6.33 (1.18)

3.12 (1.75)

Constant Use forecast for subcontracting decisions Use forecast for facilities planning Constant Use forecast for subcontracting decisions Use forecast for sales planning No. of employees

Manufacture dpt.

4.22 (1.65)

No significant regression

Logistics dpt.

3.08 (1.59)

No significant regression

Administration dpt.

3.30 (1.97)

Constant Use forecast for facilities planning No. of employees

Forecasting aims Forecasting aims Forecasting aims Forecasting aims Structural factors

Forecasting aims Structural factors

Estimate

Std. error

t ratio

p

R2 adj

4.837 0.505

0.897 0.196

4.56 2.57

o0.0001 0.0130

10.02% (n ¼ 57)

0.544

0.213

2.55

0.0130

6.549 0.216

0.451 0.082

14.53 2.64

o0.0001 0.0110

0.248

0.096

2.59

0.0120

1.429 0.287

0.665 0.118

2.15 2.43

0.0364 0.0184

0.274

0.128

2.14

0.0375

0.005

0.001

3.69

0.0005

2.053 0.421

0.665 0.147

3.09 2.87

0.0032 0.0059

0.004

0.001

3.09

0.0031

13.20% (n ¼ 56)

26.49% (n ¼ 55)

18.47% (n ¼ 56)

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

seem to suggest that the direct impact of external factors is somehow less important than the one of company’s aims. The next section investigates the relationship among practices, i.e., their internal consistency.

93

one regression and as (candidate) independent ones in all other regressions (as we use a step-wise regression and some might not qualify as statistically significant predictors). 4.2. Process

4. Data analysis 4.1. Internal consistency The previous section shows that forecasting aims significantly influence forecasting practices; however we argue that this relationship is mitigated by internal consistency among practices. In other words we think that the objectives drive forecasting practices, but companies choose among internally consistent forecasting strategies (consisting of various practices) rather than each single practice independently. Thus, we consider the relationship among practices but also control for relationships with structural factors and demand characteristics. Analyses again are conducted by means of regression. In particular we try to highlight the relationship between each single practice (e.g. collection of information) in the five groups discussed so far (process, tools, cooperation, control and organization) and try to highlight the relationships with both external variables and other practices. So all practices will be used as dependent variables in

A first interesting result (see Table 9) is that companies define how to manage their forecasting process mainly according to the particular organizational structure of the process. Almost all significant variables can be traced back to other forecasting practices, thus claiming that gaining an internally consistent structure is a major issue for companies. In particular, information retrieval is an important issue when administration department is involved, since probably this structure is capable of accessing different sets of data and information. Quite interestingly, information retrieval is less important within companies that involve deeply their financial department (maybe it means that these companies mainly try to make ends meet). Moreover, the shorter the forecasting horizon and the smaller the number of modifications of forecasts, the more information becomes important. Obviously, companies that pay attention to this issue also tend to exchange information within the supply chain. Forecast generation becomes a relevant issue when marketing and logistics are

Table 9 Linear regression results for forecasting process Process

Variable

Information retrieval

Constant Financial dpt. Administration dpt. Forecasting horizon No. of forecast revisions per year Information exchange

Forecast generation

Forecast modification

Constant Marketing dpt. Finance dpt. Logistic dpt. Quantitative techniques Market research adoption Constant No. of products Joint development

Variable type

Estimate

Std. error

t ratio

p

R2 adj

Organization Organization Process Process

6.449 0.451 0.379 0.069 0.009

0.405 0.126 0.117 0.015 0.003

15.94 3.57 3.24 4.55 2.77

o0.0001 0.0009 0.0023 o0.0001 0.0083

44.67% (n ¼ 46)

Cooperation

0.595

0.193

3.08

0.0036

Organization Organization Organization Tools

1.834 0.188 0.306 0.348 0.229

0.505 0.080 0.093 0.104 0.077

3.63 2.34 3.28 3.34 2.99

0.0007 0.0233 0.0019 0.0016 0.0044

Tools

0.213

0.091

2.34

0.0234

Demand charact. Cooperation

3.538 0.168 0.572

0.275 0.055 0.175

12.86 3.07 3.28

o0.0001 0.0037 0.0021

45.64% (n ¼ 46)

28.80% (n ¼ 49)

ARTICLE IN PRESS 94

G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

involved but finance department is somehow kept aside. As one may well have anticipated companies that pay more attention to this issue use quantitative techniques more widely. Finally, forecast modification becomes a relevant issue when many products have to be forecasted and when joint decisions are taken with customers and suppliers since, understandably, this process might require many interactions and revisions. 4.3. Tools The analysis of the drivers of the adoption of specific tools does not add much to our findings. As one may have expected both quantitative and qualitative approaches tend to be adopted by companies that focus their attention on the forecast generation phase. However, the involvement of manufacturing department tends to increase the usage of quantitative approaches. Indeed, generally speaking, operations managers tend to have a more solid quantitative background than people form other functions. Finally, companies that do not

consider their actual order backlog when forecasting tend to adopt qualitative approaches (Table 10). 4.4. Cooperation The degree of cooperation adopted seems to be an interesting element. Table 11 summarizes results. Companies that share information with customers and suppliers do not rely much on management opinion in developing their forecast but probably rely more on formal approaches. Understandably, companies that share their decision making process with customers and suppliers tend to pay more attention on forecast generation as this might be a rather time-consuming process when various organizations are involved. Also these companies tend to use quantitative approaches rather than qualitative ones. These companies also use larger time buckets but forecast for shorter time horizons. This would suggest that cooperation is used mostly for fairly simple forecasting problems, i.e., with a fairly large time bucket and for the short run.

Table 10 Linear regression results for forecasting tools Tools

Variable

Variable type

Estimate

Std. error

t ratio

p

R2 adj

Quantitative approaches

Constant Manufacturing dpt. Forecast generation

0.581 0.266 0.631

0.913 0.132 0.160

0.64 2.03 3.94

0.5271 0.0483 0.0003

33.50% (n ¼ 55)

Organization Process

Constant Forecast generation Order backlog

1.568 0.595 0.294

0.805 0.156 0.141

1.95 3.81 12.08

0.0567 0.0004 0.0419

20.26% (n ¼ 55)

Process Process

Qualitative approaches

Table 11 Linear regression results for cooperation Cooperation

Variable

Variable type

Estimate

Std. error

t ratio

p

R2 adj

Information exchange

Constant Finance dpt. Information retrieval No. of products Management opinion

0.443 0.193 0.184 0.062 0.243

0.736 0.064 0.079 0.030 0.092

0.6 3.02 2.33 2.08 2.63

0.5505 0.0042 0.0242 0.0434 0.0118

31.57% (n ¼ 48)

Organization Process Demand charact. Tools

Constant Administration dpt. Forecast generation Longer time horizon Longer time bucket Quantitative approaches Qualitative approaches

2.140 0.140 0.208 0.024 0.363 0.192 0.144

0.622 0.060 0.099 0.011 0.165 0.066 0.067

3.44 2.35 2.1 2.18 2.19 2.9 2.16

0.0013 0.0233 0.0414 0.0345 0.0337 0.0059 0.0360

35.10% (n ¼ 46)

Organization Process Process Process Tools Tools

Joint development

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

4.5. Control

95

measures of accuracy). Also, the more attention is paid to performance measurement the more it is formal, i.e., companies devote time to the definition of appropriate measures of error.

In order to analyse what elements influence the use of informal measures of performances, ANOVA was adopted, since information is boolean. Table 12 summarizes results. Quite interestingly companies that involve deeply logistics prefer to rely on formal measures: as discussed above the involvement of ‘‘operations personnel’’ that is usually mathematically more skilled tends to increase the use of quantitative methods and formal measures. On the contrary, the more attention is paid to backorders the more accuracy is measured informally (as the amount of backlog is an effect of wrong forecasts and thus might substitute the need for formal

4.6. Organization As noted previously, organizational structures are influenced by several variables (see Table 13). Sales people tend to be involved when information regarding customers is considered and when several product lines are forecasted. Its involvement, however, is lower when qualitative techniques are adopted and when several modifications of forecasts are conducted. Forecast generation seems

Table 12 ANOVA analysis for informal measure of performance Mean

Std. error

F ratio

P

Order backlogs

Formal measure Informal measure

3.55 4.53

0.42 0.24

4.803

0.033 (n ¼ 55)

Logistic dpt.

Formal measure Informal measure

3.93 2.76

1.57 1.50

7.729

0.007 (n ¼ 55)

Performance measurement

Formal measure Informal measure

4.71 3.56

1.27 1.78

6.697

0.012 (n ¼ 55)

Table 13 Linear regression results for forecasting organization Organization

Variable

Variable type

Estimate

Std. error

t ratio

p

R2 adj

Sales dpt.

Constant No. of product lines Qualitative approaches Customer information No. of forecast modifications

4.838 0.073 0.168 0.310 0.006

0.689 0.034 0.068 0.115 0.002

7.02 2.15 2.48 2.71 2.49

o0.0001 0.037 0.017 0.010 0.017

24.38% (n ¼ 46)

Demand charact. Tools Process Process

Constant Information retrieval Current economic conditions Suppliers information No. of forecast modifications Information exchange

1.916 0.302 0.288 0.467 0.014 0.629

0.906 0.132 0.122 0.109 0.004 0.210

2.11 2.28 2.36 4.26 3.66 2.99

0.041 0.028 0.023 0.000 0.001 0.005

51.56% (n ¼ 47)

Process Process Process Process Cooperation

Constant Forecast generation No. of forecast modifications

1.242 0.411 0.010

0.623 0.138 0.004

1.99 2.99 2.72

0.0525 0.0046 0.0093

26.16% (n ¼ 47)

Process Process

Constant Information retrieval Forecast generation Forecast modification Suppliers information Forecast horizon

2.513 0.576 0.525 0.592 0.505 0.071

1.277 0.173 0.196 0.202 0.119 0.024

1.97 3.34 2.68 2.93 4.25 2.93

0.055 0.002 0.010 0.005 0.000 0.005

39.24% (n ¼ 46)

Process Process Process Process Process

Finance dpt.

Logistics dpt.

Administration dpt.

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

96

Table 14 Linear regression results for forecast error Performance

Variable

Variable Type

Estimate

Std. error

t ratio

p

R2 adj

Forecast error

Constant Qualitative methods Information exchange

27.932 3.406 4.282

3.541 0.998 1.913

7.889 3.412 2.239

o0.0001 0.0001 0.031

21.20% (n ¼ 42)

Tools Cooperation

to influence the participation of logistics and administration department: the greater the relevance of this phase, the more the first department is involved and the less the second one is concerned. Finally, the more information retrieval is important the less finance is involved in the process, while administration becomes a relevant player.

5. Data analysis

Forecasting Use

Forecasting Performance

Company Performance Fig. 2. Relationship among company performance, forecasting performance and forecasting use.

5.1. Practices and performance In this section attention towards forecasting accuracy is analysed. In particular, this section aims to identify which practices lead to good performance. Forecast performance is here evaluated as average percentage error. As Table 14 shows, there seems to be a significant relationship between practices/aims and performance. The use of qualitative methods seems to reduce forecast errors, thus claiming that companies that use at least some sort of structured forecasting approach tend to perform better. However, we shall also acknowledge that qualitative methods tend to be used for less structured forecasting problems with a more variable and uncertain demand that is per se more difficult to foresee. One should not draw the conclusion that a company would improve accuracy by simply switching from qualitative to quantitative methods. Similarly, cooperation in sharing information with suppliers seems to improve forecast accuracy. However, understanding differences in accuracy is sometimes tricky. Accuracy is the main output of forecasting practices, but it is an intermediate output and companies invest in the forecasting process to improve operational performance that forecasting influences. In other words, if a company is very flexible, with short lead times, probably forecasting accuracy is not so important and thus forecasting accuracy might not look that good. For this reason we study the

relationship between forecasting performance and overall company performance (see Fig. 2). In order to evaluate performance, we collected data regarding how companies in the sample perform compared to their competitors (on 20 different business performances). We reduced these dimensions by means of factor analysis. Seven factors were identified1 (Eigen values 41; 78.5% of total variance explained) as Table 15 shows (factor names were defined according to the interpretation of the factor loads). Cronbach’s alpha is significant for all items identified, claiming for the reliability of the factors here identified. Given these factors, we considered the linear relationship between performance, forecasting objectives and forecasting error. The main purpose is to identify the main drivers of companies’ performance and, in particular, to judge to what extent forecast accuracy leads to better performance. Table 16 summarizes results. Only three of the seven factors showed significant relationships with the selected causal variables. In particular, companies that use forecasts mainly for subcontracting and not for facilities planning tend to perform poorly in terms of time (manufacturing and design) and eco-compatibility. Surprisingly, the 1 Extraction was done by means the principal components method using correlations. Varimax rotation with Kaiser Normalization was applied.

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

97

Table 15 Items and factor loads for company performances Factors

Items

Logistic

Eco-compatibility

Product

Costs

Flexibility

Time

ISO

Order fulfilment speed (0.9312) Delivery speed (0.9414)

Solid waste disposal (0.7995) Air emissions (0.9645)

Production costs (0.9180) Product total costs (0.8841)

Flexibility to change output volume (0.7309)

Manufacturing throughput time (0.851)

ISO certifications (1.0000)

Delivery as promised (0.8436)

Water emissions (0.9382)

Product features (0.8838) Product performance (0.9025) Perceived overall product quality (0.7492)

Supply chain costs (0.7651)

Flexibility to change product mix (0.8859)

Product design time (0.851)

Delivery flexibility (0.7468)

Health and safety record (0.6194)

Cronbach’s alpha

0.9138

0.8282

0.8486

0.6615

0.6154

Estimate

Std. error

t ratio

p

R2 adj

0.270 0.253

0.384 0.061

0.7 4.16

0.486 0.000

Forecasting aims

0.189

0.075

2.51

0.015

Forecasting aims

0.331

0.078

4.27

1.120

0.455

2.46

0.019

0.189

0.080

2.36

0.024

0.039

0.011

3.48

0.001

0.296

0.096

3.1

0.004

0.393 0.335

0.981 0.162

0.4 2.07

0.691 0.045

0.384

0.164

2.34

0.025

0.050

0.020

2.49

0.017

0.8982

Factor loads smaller than 0.5 were omitted.

Table 16 Linear regression results for company performances Performance

Variable

Time

Constant Use forecast for subcontracting decisions Use forecast for materials and inventory planning Use forecast for facilities planning

Ecocompatibility

Forecasting aims

Constant Use forecast for subcontracting decisions Forecast error Use forecast for facilities planning

ISO

Variable Type

Constant Use of forecast for new product development Use forecast for machinery planning Forecast error

Forecasting aims Forecasting performance Forecasting aims

Forecasting aims Forecasting aims Forecasting performance

use of forecast for materials and inventory planning seems to be negatively related to time performances. Quite interestingly R2 is quite high in all analysis (more than 25%). Interestingly forecast error is higher for companies that perform better on ecocompatibility and quality, probably due to specific competitive priorities.

34.05% (n ¼ 55)

o0.0001 25.80% (n ¼ 55)

25.02% (n ¼ 55)

6. Conclusions The analyses provide interesting hints on the drivers of forecasting practice and usage. First of all, results show that forecasting has an impact on company performance, which sounds reasonable. Quite interestingly, though, this impact depends on

ARTICLE IN PRESS 98

G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

what forecasting is used for. This suggests that companies should carefully consider how to use their forecasts according to their competitive priorities. Moreover, this claims that forecasting plays a significant role also in improving companies’ performance, thus justifying the attention paid to this issue. A second interesting result sheds light on the drivers of the forecasting process: some features of the forecasting process (e.g., organizational structures, kind of cooperation) depend primarily on what forecasts will be used for. Others (e.g., relevance of single phases) depend primarily on external variables. Thus companies design their forecasting process according to what they want to achieve and only partially to the context they are facing. We acknowledge that this might be due to the fact that companies in the sample are from only two industrial sectors and thus the variance in environmental variables is rather limited. With a more heterogeneous sample we might find that environmental variables play a more important role. However, the basic finding might hold true in broader contexts. Again, given the specific objectives of this work and the small sample considered, we do not think that each single relationship will hold in other works, but we argue that the overall influence of forecasting aims will probably endure. Furthermore, in many of the various analyses, organizational structures seem to co-vary with forecasting practices. It is difficult to claim whether forecasting practices are driven or drive organizational structures since we are not able to capture the causal relationship. This result claims that the forecasting process and the organizational structure have to be designed coherently. On the one hand, people involved in the process tend to focus on some elements rather than others, according to their needs, skills and role; on the other hand, different tools may become more or less effective according to who is using them. We claim that research is needed here since only partial evidence can be found in literature regarding organizational structures and forecasting performance. Some interesting issues and questions seem to stem from these concluding remarks. In particular, we claim that future research should investigate factors that influence forecasting with a broader perspective. In particular, the complexity and uncertainty of the forecasting problem deserve further attention. Complexity is tied to the number of variables that have to be considered and the

number of relationships among them. This work confirms recent findings (see literature review) and shows that variables such as the number of products, the time bucket and the forecasting horizon have some kind of influence on how forecasting is conducted. However, this paper fails to provide concluding evidence on all the variables that should be considered when designing a forecasting process and how they shall drive specific practices. Literature has provided some hints regarding the impact of elements such as demand variability and information quality on the accuracy of forecasting models, but there is no clear understanding of their role in the design of a forecasting strategy. Also, we argue that, given a specific forecasting problem (in terms of complexity and uncertainty), there is no one best way since, as this work shows, company’s aims and organization play a major role in designing how forecasting is conducted. However, more normative research suggesting what companies should be doing is needed. This work shows that relevant attention is paid to (and we argue it should be paid to) information exchange along the supply chain in order to better foresee future demand. However, the way information should be used depends on the organizational structure adopted and thus should be consistent with the forecasting process the company adopts. In the end, this work shows that organizational structures influence how forecasting is conducted. However, it is not clear whether there are some specific organizational configurations among which companies have to choose when designing their forecasting approach. It would be interesting to analyse a wider sample, both in terms of number of companies and industrial sectors, to evaluate whether and under which circumstances organizational configurations may exist. This may help companies to identify which is the best structure given a specific forecasting problem. Acknowledgments Authors have contributed jointly to the present work; however Giulio Zotteri has edited the following paragraphs: Research aims, Methodology and Conclusions. Matteo Kalchschmidt has written the remaining paragraphs Introduction and Data analysis. Also we wish to thank two anonymous referees for their helpful guidance and insightful comments and suggestions.

ARTICLE IN PRESS G. Zotteri, M. Kalchschmidt / Int. J. Production Economics 108 (2007) 84–99

References Chaman, L.J., 2003. Benchmarking the forecasting process. Journal of Business Forecasting Fall, 11–14. Cooper, M., Ellram, L.M., Gardner, J.T., Hanks, A.M., 1997a. Meshing multiple alliances. Journal of Business Logistics 18 (1), 67–89. Cooper, M.C., Lambert, D.M., Pagh, J.D., 1997b. Supply chain management: more than a new name for logistics. The International Journal of Logistics Management 8 (1), 1–14. Diamantopoulos, A., Winklhofer, H., 1999. The impact of firm and export characteristics on the accuracy of export sales forecasts: evidence from UK exporters. International Journal of Forecasting 15, 67–81. Duran, J.A., Flores, B.E., 1998. Forecasting practices in Mexican companies. Interfaces 26 (6), 56–62. Ellram, L.M., Cooper, M.C., 1990. Supply chain management, partnerships, and the shipper-third-party relationship. The International Journal of Logistics Management 1 (2), 1–10. Henderson, L.S., 2004. Encoding and decoding communication competencies in project management—an exploratory study. International Journal of Project Management 22, 469–476. Hughes, M.C., 2001. Forecasting practice: organisational issues. Journal of the Operational Research Society 52, 143–149. Kahn, K.B., Mentzer, J.T., 1994. The impact of team-based forecasting. Journal of Business Forecasting Summer, 18–21. Kahn, K.B., Mentzer, J.T., 1995. Forecasting in consumer and industrial markets. Journal of Business Forecasting 14 (2), 21–28. Klassen, R.D., Flores, B.E., 2001. Forecasting practices of Canadian firms: survey results and comparisons. International Journal of Production Economics 70, 163–174. Mentzer, J.T., Cox, J.E., 1984. Familiarity, application and performance of sales forecasting techniques. Journal of Forecasting 3, 27–36. Mentzer, J.T., Kahn, K.B., 1997. State of sales forecasting systems in corporate America. Journal of Business Forecasting Spring, 21–27. Miles, J., Shavin, M., 2001. Applying Regression and Correlation: A Guide for Students and Researchers. Sage, London.

99

Peterson, R.T., Jun, M., 1999. Forecasting sales in wholesale industry. Journal of Business Forecasting 18 (2), 15–18. Peterson, R.T., 1993. Forecasting practices in retail industry. Journal of Business Forecasting 12 (1), 11–14. Peterson, R.T., 1990. The role of experts’ judgment in sales forecasting. Journal of Business Forecasting 9 (2), 16–21. Sanders, N.R., 1992. Corporate forecasting practices in the manufacturing industry. Production and Inventory Management 33, 54–57. Sanders, N.R., Manrodt, K.B., 1994. Forecasting practices in US corporations: survey results. Interfaces 24 (2), 92–100. Tawfik Mady, M., 2000. Sales forecasting practices of Egyptian public enterprises: survey evidence. International Journal of Forecasting 16 (3), 359–368. Wacker, J.G., Sprague, L.G., 1998. Forecasting accuracy: comparing the relative effectiveness of practices between seven developed countries. Journal of Operations Management 16, 271–290. Wacker, J.G., Sprague, L.G., 1995. The impact of institutional factors on forecast accuracy: manufacturing executives perspective. International Journal of Production Research 33 (11), 2945–2958. Watson, M.C., 1996. Forecasting in the Scottish electronics industry. International Journal of Forecasting 12, 361–371. Winklhofer, H., Diamantopoulos, A., Witt, S.F., 1996. Forecasting practice: a review of the empirical literature and an agenda for future research. International Journal of Forecasting 12, 193–221. Winklhofer, H., Diamantopoulos, A., 1997. Organisational aspects of export sales forecasting: an empirical investigation. Journal of Strategic Marketing 5 (3), 167–185. Winklhofer, H., Diamantopoulos, A., 2002. Managerial evaluation of sales forecasting effectiveness: a MIMIC modeling approach. International Journal of Research in Marketing 19 (2), 151–166. Wisner, J.D., Stanley, L.L., 1994. Forecasting practices of JIT and non-JIT purchasers. European Journal of Purchasing and Supply Management 1 (4), 219–225. Zotteri, G., Kalchschmidt, M., Caniato, F., 2005. The impact of aggregation level on forecasting performances. International Journal of Production Economics 93–94, 471–491.