Performance evaluation of purchasing and supply management using value chain DEA approach

European Journal of Operational Research 207 (2010) 197–205

Contents lists available at ScienceDirect

European Journal of Operational Research journal homepage: www.elsevier.com/locate/ejor

Production, Manufacturing and Logistics

Performance evaluation of purchasing and supply management using value chain DEA approach Haritha Saranga a,*, Roger Moser b a b

Production and Operations Management Area, Indian Institute of Management Bangalore, Bannerghatta Road, Bangalore 560076, Karnataka, India European Business School, Wiesbaden 65201, Germany

a r t i c l e

i n f o

Article history: Received 10 June 2009 Accepted 23 April 2010 Available online 21 May 2010 Keywords: Data envelopment analysis Purchasing and supply management Performance drivers Performance outcomes Performance evaluation

a b s t r a c t Purchasing and Supply Management (PSM) today is increasingly becoming more important to senior management due to its potential to strategically influence both operational performance as well as financial performance outcomes. However the cross-functional nature of many PSM activities has led to inadequate data collection and performance measurement resulting in weak performance evaluation methodologies and mixed results. We address this gap in the current study, firstly by using an external assessment survey methodology that complements the internal perceptional measures of PSM performance, to collect data for a sample of over 120 firms across the globe with more than 3 billion US dollar turnover, representing seven industry sectors. Next, we develop a comprehensive performance measurement framework using the classical and two-stage Value Chain Data Envelopment Analysis models, which make use of multiple PSM measures at various stages and provide a single efficiency measure that estimates the all-round performance of a PSM function and its contribution to the long term corporate performance in each of these seven industry sectors. The relevance of this measurement methodology is demonstrated through an in-depth analysis of the distribution of efficiencies within and across industry sectors and through the estimation of target PSM performance levels. Ó 2010 Elsevier B.V. All rights reserved.

1. Introduction The ever increasing competitive pressures across the globe are forcing corporations to look internally and cut costs to survive the downturns through operational excellence. Today, one of the major components of cost is the purchasing spend, which on an average accounts for 40–70% of a firm’s sales volume (depending upon the degree of vertical integration in the industry), and hence offers large scope for the creation of competitive advantages (CAPS, 2009). Global corporations like Wal-Mart, Dell, HP, Nokia and Zara have demonstrated that it is possible to achieve industry leadership through the efficient and effective management of purchasing and supply practices, irrespective of the nature of the industry. Consequently, many companies are sourcing raw materials and other supplies in recent times from across the globe in pursuit of lower costs. As a result, the role of the Purchasing and Supply Management (PSM) function has been widened significantly and its impact on corporate performance began to receive considerable attention from senior management as well as academia over the last few decades (Das and Narasimhan, 2000; Ellram et al., 2002; van Weele, 1984).

* Corresponding author. Tel.: +91 80 26993130; fax: +91 80 26584050. E-mail address: [email protected] (H. Saranga). 0377-2217/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2010.04.023

The role of today’s PSM function has therefore transformed from a mere clerical activity to a competence with the capability to structure, develop and manage the supply base in alignment with corporate objectives (Das and Narasimhan, 2000). In order to develop these capabilities, organizations are following best practices in recruiting and training employees in the PSM function, are establishing processes that enable cross-functional collaboration and are developing systems for supplier collaboration. These PSM activities drive the performance of the PSM function in terms of cost savings, better quality of products, or co-innovations with suppliers (Das and Narasimhan, 2000). However, the ultimate goal of a PSM function from the senior management perspective is the role it plays in improving the financial performance at the corporate level. Therefore, the need for an alignment between purchasing strategies and corporate strategies cannot be overemphasized in the current economic scenario where firms are plagued by price pressures and margins are driven primarily through cost savings. However, as the PSM function plays more of a supporting role rather than directly adding value to products and services offered by the firm, measurement of the direct value addition made by this function to corporate performance has become a major challenge (Nollet et al., 2008). There is also the need to create a measurement system and an appropriate incentive structure that can motivate PSM employees towards corporate goals considering the indirect nature of their activities.

198

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

Although PSM performance evaluation seems to be still in its infancy (Easton et al., 2002), there is a general agreement on categorizing performance measures into performance drivers and outcome measures (Kaplan and Norton, 1992). PSM performance drivers are measures concerning PSM activities such as global sourcing, demand and specification management, supplier development, procure-to-pay management etc. that facilitate outcomes, while PSM performance outcome measures refer to the PSM results such as cost savings, quality improvements, supplier integration etc. achieved through deployment of PSM drivers. One can address the PSM staff motivation part by linking the incentive system with efficient utilization of performance drivers to achieve PSM performance outcomes (Wagner and Kaufmann, 2004). However, this is too simplistic and may not necessarily result in better financial performance either because the PSM objectives are not aligned with corporate objectives or due to the inefficient conversion of PSM outcomes such as cost savings into corporate financial outcomes (e.g. profit margins). On the other hand, the conversion efficiency of PSM drivers into corporate financial performance without taking into consideration the intermediate outputs (PSM performance outcomes) seems also not the right approach because these PSM outcomes have a significant impact on financial performance and one may end up with spurious results that do not adequately represent the contribution of PSM (Ellram et al., 2002). Finally, the changing role and broadening scope of PSM activities have also magnified the difficulties of measuring the overall impact of PSM on corporate success. Especially the increased cross-functional nature of PSM activities, the lack of aggregate performance data and the intangible nature of some of the performance outcomes, have all contributed to PSM performance evaluation problems in academia (Ellram et al., 2002) and industry (van Weele, 1984). In order to address these issues, we propose a performance evaluation methodology based on Data Envelopment Analysis (DEA), which can incorporate multiple inputs and outputs in multiple stages and results in a single relative efficiency measure. Since the conventional DEA models are found to be ineffective in measuring the performance of various supply chain related functions, many multi-stage DEA models have been developed to accommodate various indirect processes and their contribution to corporate performance (Chen and Zhu, 2004; Liang et al., 2006; Golany et al., 2006; Kao and Hwang, 2007; Kao, 2009). In the current paper, we use the two-stage value chain DEA model developed by Chen and Zhu (2004) to incorporate the effect of mediating and moderating variables, such as IT investments, on firm performance. This two-stage value chain DEA model is capable of accommodating intermediary PSM outputs such as cost savings or level of cross-functional collaboration along with initial PSM drivers and ultimate corporate financial performance, and hence is comprehensive enough to meet the needs of both PSM staff and the Chief Purchasing Officer (CPO) as well as senior management executives. Therefore, we strongly feel, this twostage DEA model can address the existing gaps in the performance evaluation of PSM functions and can also make a significant contribution in terms of highlighting the criticality and contribution of the PSM function in an organization. Besides the need for a multi-stage performance evaluation model, the typical data collection approaches in PSM literature, which are based on perceptional views of purchasing managers, pose another set of challenges for its performance measurement. It has been found that traditional questionnaire-based surveys alone may not provide valid and reliable measurement of the spirit of the overall system with which PSM best practices are implemented. Pagell (2004) attributes these measurement challenges to (1) issues of social desirability and to (2) overly negative impressions of respondents’ own practices.

In this paper we make two contributions to the PSM performance evaluation literature. Firstly, we apply a diversified data collection approach including the external assessment survey methodology, following Pagell (2004) and Bloom and van Reenen (2007) to measure PSM performance drivers and performance outcomes. We integrate information from a variety of sources such as public databases, questionnaire-based survey data (evaluating facts, not perceptions) and interviewer-rated data to supplement the deficiencies of different survey based methodologies as well as to avoid problems with common method bias (Podsakoff et al., 2003). Secondly, we present a PSM performance evaluation methodology that incorporates intermediary outcomes and evaluates the PSM performance of an organization by measuring the relative efficiency of its PSM function vis-à-vis PSM functions of other organizations in the same industry. This approach also enables one to benchmark a PSM function against industry best practices and learn from the frontier firms. We thus analyze the efficient transformation of PSM performance drivers into PSM and corporate financial performance for seven industries, each industry consisting of a set of sample firms, using both conventional as well as multi-stage DEA methodologies, which enable the integration of precise as well as imprecise data and quantify the performance gaps of inefficient PSM functions. The remainder of the paper is organized as follows. In Section 2, we briefly discuss the related PSM literature and develop the proposed evaluation framework. The data collection methodology is presented in Section 3. Section 4 narrates the various DEA models used for PSM performance evaluation in the current study. The results from the application of the DEA models for seven industry sectors are presented in Section 5. Finally, we conclude with a discussion on the managerial relevance of our study in Section 6.

2. Purchasing and supply management performance evaluation Early conceptual developments of performance evaluation in PSM focused on cost issues only, which resulted in the emergence of two major concepts (Easton et al., 2002). The first is concerned with the proper utilization of purchasing personnel, and considers lower overhead cost being equivalent to better PSM performance. The second deals with end product cost as the ultimate measure of PSM performance. These traditional measures, which mainly consisted of costs and profits, continued until the 1980’s. Due to this simplistic perspective, PSM performance evaluation failed to attract adequate attention from academia and resulted in lack of appropriate definitions and differentiations in the scope of PSM practices (van Weele, 1984). As the PSM function evolved over time to meet the needs of manufacturing and services organizations in a globalized world, a strong need is arising for broader PSM performance concepts that do not just focus on costs but also include quality or supplier performance measures (van Weele, 1984). Easton et al. (2002) attribute drawbacks of current PSM measurement approaches also to the failure to incorporate the efficiency aspect of PSM activities into PSM metrics. For example, PSM activities such as supplier development focus only on performance outcomes in the form of supplier quality but ignore the amount of inputs required. The categorization of PSM measures into performance drivers and outcome measures lends itself nicely as input–output measures in the DEA application, since the association between these two sets of measures, although obvious, has also been empirically established by earlier studies (Easton et al., 2002). In order to incorporate all appropriate categories of measures that capture the relationship between the primary performance drivers (as inputs), their direct output measures (as intermediary outputs) and the corporate financial performance (as the final output), we have

199

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

Analysis 4 (Value Chain DEA-based PSM Performance Measurement Approach)

PSM Performance Drivers:

VC DEA Stage 1

PSM Performance Outcomes

- No. Strategic Buyers

-Cost Savings

- No. Transactional Buyers

- Cross-Functional Collaboration

- No. Suppliers

- Supplier Performance

Analysis 2 (PSM Performance Measurement Approach, e.g. Das & Narasimhan, 2000)

VC DEA Stage 2

Corporate Performance Outcome -EBITDA

Analysis 3 (PSM Performance Measurement Approach, e.g. similar to Narasimhan et al., 2001)

Analysis 1 (PSM Performance Measurement Approach, e.g. Ellram et al, 2002) Fig. 1. The PSM performance evaluation framework using DEA. (See above-mentioned references for further information.)

developed the performance evaluation framework depicted in Fig. 1. This framework consists of four different PSM performance measurement approaches, which make use of three PSM performance drivers, three PSM performance outcomes and one corporate performance outcome, to investigate the efficiency of transformation of PSM drivers into corporate financial performance. This framework not only captures relevant information related to PSM but also incorporates the indirect and intangible contributions by the PSM function towards corporate performance. We then carry out a comparative analysis of all the results from different approaches to recommend the most appropriate approach for PSM performance evaluation. We briefly explain each measurement approach and corresponding input and output measures in the following sections. 2.1. Conversion of PSM performance drivers into corporate financial performance We first analyze the relative efficiencies of PSM functions in transforming performance drivers into corporate performance outcome (Analysis 1 in Fig. 1) in various industries. Based on our literature review on purchasing resources (e.g. Easton et al., 2002) and discussions with PSM managers we have identified three PSM performance drivers as inputs for this analysis: (1) number of strategic buyers and managers, (2) number of transactional buyers and (3) number of suppliers representing 80% of the managed purchasing volume. The strategic PSM managers and buyers are involved in the strategic management of the supplier base, selection of suitable supply markets and in intensive interactions with other functional departments to understand corporate requirements and optimize the demand and specification needs of other functions. This PSM driver focuses not just on cost savings but also on improving bundling initiatives, product and service innovations as well as outsourcing initiatives which finally are expected to lead to better corporate financial performance. The number of transactional buyers on the other hand represents the human resources a PSM function applies in the operational management of the supply base and third party logistics service providers. Their activities allow the PSM function to leverage the resources and capabilities of the supplier base and to provide the required inputs to production and other organizational functions on a day-to-day basis. Their operational efficiency is essential to create further value at the corporate level. The third performance driver is chosen to be the number of suppliers representing 80% of the purchasing volume, in order to incorporate the majority of suppliers of each of the sample companies

and to ensure comparability of the results. Today, many organizations across various industries are trying to reduce the number of direct suppliers in order to reduce their supply base complexity and improve quality control over the supplied materials and services. Therefore, PSM functions that are able to manage with lower number of suppliers are expected to be more efficient in transforming PSM performance drivers into corporate financial performance. Finally, a three year average of EBITDA (Earnings Before Interest, Tax, Depreciation and Amortization) as percentage of sales is used as a measure of corporate financial performance, because it provides the means to estimate how effectively the purchasing function is able to utilize their inputs to improve corporate performance in the long run. Thus, analysis 1 essentially focuses on the transformation efficiency of PSM drivers (represented by the three inputs discussed above) into corporate financial performance (represented by the output EBITDA). 2.2. Conversion of PSM performance drivers into PSM performance outcomes Based on existing literature (e.g. Nollet et al., 2008), we have identified three PSM performance outcomes to evaluate the transformation efficiency of PSM drivers into PSM performance at a functional level (Analysis 2 in Fig. 1). In line with prior empirical findings (Smith-David et al., 1999) and as per the trends in practice we lay more emphasis on the effectiveness of PSM strategies on multiple fronts rather than on functional cost efficiency alone because the average cost of the PSM function is typically only about one percent of the purchasing spend (CAPS, 2009). Consequently, we include three PSM performance outcomes on a functional level into our analysis including a hard measure (1) cost savings (Nollet et al., 2008), and two soft measures (2) cross-functional collaboration (Trent and Monczka, 1994) and (3) supplier performance management (Tan et al., 1998). Cost savings traditionally have been considered as the primary objective of PSM functions. We measure cost savings as ‘average achieved cost savings across the firm over three years’ (based on detailed information provided by each firm), after controlling for inflation. Cross-functional collaboration integrates the contribution of the PSM function towards improvements in other functions such as production or R&D. Such a performance measure is important to capture all relevant contributions of PSM that may not directly translate into cost savings but may contribute towards performance improvement of other functions. Supplier performance management serves as an indicator for the optimal supplier base

200

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

structure in terms of reduced complexity and improved synergies within a global supply base. Note that both these measures, cross-functional collaboration and supplier management performance, are intangible in nature and hence can only be obtained through perceptual measurement methods. Strategic PSM managers and buyers can significantly improve all three PSM performance outcome measures. For example, skilled strategic buyers achieve a higher acceptance from internal customers, which can result either in cost savings through improved demand and specification management or increased cross-functional collaboration and early purchasing involvement. Similarly, transactional buyers can improve the supplier performance management capabilities and the cost reduction efforts through frequent interactions and information exchange with each supplier. Finally, the reduction in number of suppliers can also have a positive influence as less complexity will free up more resources for improvement projects and innovative initiatives. Thus, in accordance with this premise, we evaluate the relative efficiencies of our sample firms in transforming PSM performance drivers into the three PSM performance outcomes as discussed above. 2.3. Conversion of PSM performance outcomes into corporate financial performance If purchasing and supply management is indeed a potential source of sustained competitive advantage, advances in PSM should become visible in corporate financial performance (Tan et al., 1998). In fact, there seem to be plenty of ways for PSM to contribute to a firm’s financial performance (Analysis 3 in Fig. 1). On the one hand, PSM may be able to contribute to savings in non-traditional areas such as the efficient management of contract labour, maintenance, or travel expenses. On the other hand, crossfunctional PSM activities may support at least a sustainable price premium through product or service innovations. Reduction of capital employed could be achieved by PSM involvement in strategic outsourcing, inventory management and supply chain optimization decisions. Moreover, corporate performance can be positively influenced either by the successful integration of innovations from the supplier base, or by superior customer service in terms of quality and speed through improved supply chain performance (Tan et al., 1998). Since PSM can influence corporate performance through many different routes, EBITDA is used as the financial performance outcome in this study, as it can best integrate the numerous contributions of an improved PSM performance on the bottom and top line of a firm. The Chief Executive Officer (CEO) or Chief Financial Officer (CFO) of an organization would primarily be interested in knowing how efficiently the PSM performance outcomes are being transformed into corporate financial performance relative to its industry peers, to justify the investments and evaluate the purchasing function’s contribution to the overall organizational performance. 2.4. Conversion of PSM performance drivers into corporate financial performance through PSM performance outcomes Prior research results suggest that the direct impact of PSM performance drivers on corporate financial performance could be difficult to identify due to the high number of additional influencing factors on corporate performance (Ellram et al., 2002). From a conceptual perspective, the primary objective of the PSM function is to manage the sourcing process of a firm leading to improved PSM and subsequently to better corporate financial performance (Analysis 4 in Fig. 1). Analyzing each single PSM activity contributing to corporate success is empirically impossible not only for researchers but even for PSM controlling departments within companies (Nollet et al., 2008), as PSM activities are diverse in range and

integrate a strong cross-functional character. Therefore, the overall PSM performance depends upon how efficiently PSM drivers are converted into PSM outcomes and in turn how efficiently PSM outcomes are converted into corporate financial outcome. Any efficiency loss in either of these two conversions will result in overall inefficiency. For example, cost savings or improved material and service quality from suppliers may not reach the bottom or top line if the savings are diverted or reinvested in other functions such as marketing (e.g. more expensive ad campaigns) or production (e.g. new machinery) or simply go unaccounted for due to poor accounting management. In such situations, neither the CPOs nor the CFOs will be able to fully capture the contribution of PSM to corporate financial performance. In order to capture and measure the efficiency of the PSM function in transforming its purchasing practices into corporate financial success in a comprehensive manner we propose the use of PSM performance outcomes as intermediary measures to link the PSM performance drivers with corporate financial performance through a two-stage value chain DEA methodology developed by Chen and Zhu (2004). Since two of the intermediary outcomes are measured using Likert scale, we slightly modify Chen and Zhu’s value chain DEA model to accommodate Likert scale data using the data transformation procedure proposed by Zhu (2004), as described in detail in Section 4.

3. Data collection For this study, we chose all firms with global revenues above US $ 3 billion as the target population across industry sectors around the world. This revenue threshold had been chosen to ensure sufficient complexity of the organization for questions regarding cross-functional collaboration or supplier performance management, in addition to ensuring that companies have similar sizes. Out of the initial sampling frame of 2251 firms from the OneSource data base, a stratified random sample of 1000 firms was selected. Because neither a Chief Purchasing Officer (CPO) nor any equivalent senior PSM professional could be identified in 232 cases, the final sample comprised of 768 firms. Of these firms, 129 fully participated in the survey conducted in autumn 2005, equaling an effective response rate of 16.8%. The final data sample contained firms from seven industries, namely, automobile, pharmaceutical and chemical, packaged goods, financial institutions, high-tech, energy and construction. While 63% of all firms were Europe-based, 29% had their headquarters in North-America, and the remaining 8% were from Asia, Africa, and South-America. The average annual revenue of a firm in our sample was around USD 21 billion. We have chosen a different data collection process for each of the construct groups to cope with the indicated challenges and avoid common method bias (Podsakoff et al., 2003). We chose an interview-based approach with external assessments for two of the three PSM performance outcomes, namely (1) Cross-Functional Collaboration and (2) Supplier Performance Management, because it overcomes the subjectivity of CPOs towards social desirability similar to the approaches from Pagell (2004) and Bloom and van Reenen (2007) (please see Appendix for a detailed description of the external assessment approach we have used for this data collection). Based on a comprehensive literature review on the one hand and the collective experience of industry experts who had been involved in more than 600 PSM projects on the other hand, we developed detailed assumptions on typical PSM performance levels. These questions were complemented by a scoring grid to guide the interviewers to a rating on a scale of 1 (i.e., minimum performance level) to 5 (i.e., best performance level). In line with the approaches of Pagell (2004) as well as Bloom and van Reenen

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

(2007), the assessment of PSM performance outcomes was therefore done by the interviewers, not by the respondents. The PSM performance drivers, namely (1) Number of strategic PSM managers and buyers, (2) Number of transactional buyers and (3) Number of suppliers covering 80% of the managed sourcing volume as well as the PSM performance outcome (1) Cost savings have been measured using a traditional paper-based questionnaire. In order to collect this basic data from each firm the time-intensive interview approach with the CPO was not required. We simply made sure that the CPO had filled in the questionnaire with the support of his staff by the time the interview was scheduled so that open questions could be clarified and both data sets (interview and questionnaire) were complete after the interview was conducted. As argued in Section 2, corporate financial performance is supposed to be the ultimate outcome of PSM’s impact on corporate success. As an indicator of ‘corporate financial performance’ we use the profit measure ‘EBITDA’ and collected this data from publicly available financial databases such as Bloomberg, Research Insight and Amadeus; and the annual reports of the companies for three consecutive years, 2002–2004. Only data that was found to be consistent in at least two of these sources was used to determine the EBITDA as a percentage of sales over three years of the sample companies.

4. Application of DEA to measure PSM efficiency The most popularly known PSM evaluation methodologies typically use single input–output ratios, which can only partially measure the performance, based on the specific set of inputs and outputs used. To address such issues, the non parametric DEA methodology is widely used in many different fields as a comprehensive measure of performance evaluation, especially in the presence of multiple input and output measures. DEA models are capable of incorporating maximum information about the system under consideration through multiple dimensions (irrespective of the units of measures); ultimately resulting in a single measure of relative efficiency that reflects the overall performance. The additional information obtained during the application of DEA models, such as the reference sets, the benchmark targets and the returns to scale information etc. have also been found to be very useful in gaining further insights into a system’s lack of performance. The existence of multiple performance measures in a PSM function and an attempt to aggregate these individual performance measures into a single index of overall performance compelled a few researchers to adopt the DEA methodology for the performance evaluation of PSM functions. Easton et al. (2002) compute a single PSM measure of efficiency score through the successful application of the Variable Returns to Scale (VRS) DEA model to a sample of 18 petroleum firms, using six input and output values and compare them with seven different standard benchmarks used in the petroleum industry. Through this study Easton et al. (2002) demonstrate the general suitability of the DEA methodology in performance evaluation of a PSM function vis-à-vis other benchmarking techniques that use single input and single output measures. They also showcase the desirability of additional management information that becomes available from DEA evaluation through interviews of 54 purchasing executives. Some of the important reasons for adopting a DEA methodology to the current study are: (i) DEA’s ability to evaluate each Decision Making Unit (DMU) relative to its industry peers, (ii) the availability of advanced DEA models that can incorporate intermediary inputs such as PSM performance outcomes into the performance evaluation, (iii) DEA’s applicability to financial, operational and perceptional data as long as they can be quantified (Dyson et al., 2001) and finally, (iv) the avail-

201

ability of DEA methodologies which enable the integration of precise and imprecise data in the performance evaluation. We outline the input and output oriented VRS DEA models (models (1) and (2)) in the Appendix, for the sake of PSM readers who are new to DEA literature. Please note that these basic DEA models are used to carry out the analyses 1–3 in the empirical investigation described in Section 2 through the framework presented in Fig. 1. Models (1) and (2) listed in the Methodology section of Appendix are appropriate in a context where the objective is to determine the efficiency of a production process that uses a set of inputs and generates a final set of outputs, such as a manufacturing operation which uses raw material and manpower as inputs and converts them into finished goods. However, one cannot accurately evaluate the efficiency of a supporting function, such as IT or PSM, which only create intermediary outputs, which further help create final outcomes, using these single stage models, as they do not have the ability to incorporate the intermediary measures in the efficiency calculations. Many researchers have therefore developed multi-stage DEA models and network based DEA models to accommodate a variety of production processes that cannot be modeled using a single stage model (Seiford and Zhu, 1999; Chen and Zhu, 2004; Liang et al., 2006 and Golany et al., 2006; Kao and Hwang, 2007; Kao, 2009). The multi-stage production system framework proposed by Golany et al. (2006) for example, computes the efficiency of multiple sub systems that are connected in series and the efficiency of the aggregate system, simultaneously and considers the optimum allocation of resources across the sub systems. The two-stage DEA model developed by Kao and Hwang (2007) also takes into account the series relationship between the two sub processes and decomposes efficiency of the aggregate system into the efficiencies of the two sub systems. Kao (2009) extended this further to a network based multi-stage model, based on a parallel and series structure of sub processes and suggests a method to decompose the system efficiency into product of efficiencies of each individual stage. Seiford and Zhu (1999), on the other hand developed a two-stage DEA that is capable of incorporating a new set of inputs, along with an existing set of intermediary inputs in the second stage in the context of evaluating the profitability and marketability of 55 US commercial banks. In the current context, since our primary objective is to incorporate the indirect nature of the PSM function in generating financial profits as a support function, the most appropriate DEA models are those that are capable of measuring the conversion efficiency of PSM drivers into PSM performance outcomes as well as the conversion efficiency of PSM outcomes into ultimate corporate financial performance. The two-stage value chain DEA model, which was developed by Chen and Zhu (2004) to model the indirect impact of IT systems on firm performance by incorporating intermediary inputs into efficiency calculations is found to be most appropriate in the current context as it accommodates the indirect nature of the PSM function in influencing corporate financial performance. We also recommend Seiford and Zhu’s (1999) two-stage model for PSM function’s performance evaluation, in cases where the primary objective is to evaluate the PSM’s contribution to corporate performance in conjunction with other functions such as production, finance and marketing etc., as this two-stage model can bring in new set of intermediary measures during the stage 2 evaluations. However, in the current paper, since our primary focus is the PSM function, we use Chen and Zhu’s (2004) Two stage Value Chain DEA (VCDEA) model which is described below:

Min x1 h  x2 /

h;/;kj ;lj ;~z

subject to the constraints

202

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

Stage-1: n X

kj xij 6 hxiO ;

i ¼ 1; 2; . . . ; m;

j¼1 n X

kj zlj P ~zlO ;

l ¼ 1; 2; . . . ; L;

j¼1

X

kj ¼ 1; and kj P 0;

j ¼ 1; 2; . . . ; n:

j

Stage-2: n X

lj zlj 6 ~zlO ; l ¼ 1; 2; . . . ; L;

j¼1 n X

lj ykj P /ykO ; k ¼ 1; 2; . . . ; s;

j¼1

X

lj ¼ 1; and lj P 0; j ¼ 1; 2; . . . ; n;

j

where xij is the ith input and ykj kth output of DMU j and i = 1, . . . , m, k = 1, . . . , s, j = 1, . . . , n and ‘O’ is the DMU under evaluation. h and / are the efficiency scores corresponding to stage-1 and stage-2 respectively; x1 and x2 are the weights assigned to the efficiency scores in stage-1 and stage-2 respectively; and zlj are the intermediary inputs, which are outputs of stage-1 and become inputs for stage-2. ~zlO are the unknown decision variables. If both stages are of equal importance, one normally assigns x1 = x2 = 1. However, the expert judgment and/or the incentive structure of the Chief Purchasing Officer (CPO) in the organization under consideration play a critical role in the choice of values for x1 and x2 in practice. For example, if the performance of CPO is measured primarily based on PSM outcomes alone, he/she is likely to give more weightage to x1, rather than x2. To be more objective one can also make use of tools such as Analytical Hierarchy Process (AHP) to come up with appropriate weights for x1 and x2, using the specific priorities and long term objectives of the organization under consideration. The two-stage VCDEA model essentially looks to minimize the inputs xij and simultaneously maximize the final output ykj given the level of intermediary inputs zlj. Note that only those firms that achieve h = / = 1 are considered to be efficient by the VCDEA model. Since we are looking for a single efficiency score to compare the results of analysis 4 with the results of analyses 1 to 3, we take the average of h and / as the efficiency score corresponding to analysis 4.1 One important point to note here is that all the classical DEA models were originally developed for cardinal data and hence cannot be applied to the ordinal data directly (rank based data collected using Likert scale as in the context of our study). Many eminent researchers have constructed various DEA models and frameworks that can accommodate the ordinal data in the past twenty years or so (Cook et al., 1993; Cooper et al., 1999; Cook and Zhu, 2006). These models are popularly known as Imprecise Data Envelopment Analysis (IDEA) and involve nonlinear programming, typically resulting in some modifications of the original DEA models with scale transformations and variable alterations. In the more recent times, Zhu (2004) has proposed a pure data transformation IDEA methodology that converts the imprecise data into exact data and subsequently makes use of standard DEA models to calculate efficiency scores and to carry out corresponding sensitivity analysis. Cooper et al.’s (1999) IDEA model relies on the process of scale transformation and variable alteration. This converts a 1

There are more recent network DEA models (e.g. Kao, 2009) that can be used to endogenously estimate system efficiency by taking into account interrelationships between individual process efficiencies in each stage.

non linear problem involving imprecise data into a linear programming problem. The presence of rank data, Likert scale data as well as ratio bounded data, can be dealt with using Cooper’s model. However, the model is computationally difficult to implement as each DMU provides a different kind of objective function and constraints and hence Zhu’s model is preferred for IDEA calculations. In the current study, we convert the inputs and outputs that are ordinal in nature into exact data using Zhu’s (2004) data transformation methodology and then apply the DEA models, whose results are presented in the following section. 5. Empirical results Table 1 below reports the descriptive statistics for all the input/ output parameters corresponding to the seven industry sectors in total. As discussed in Section 2, the individual analyses as shown in Fig. 1 refer to primary concerns of different senior executives. While the transformation efficiency of PSM activities into corporate financial performance (analysis 1) and into PSM performance outcomes (analysis 2) is the major focus of the CPO, the transformation efficiency of PSM performance outcomes into corporate financial performance (analysis 3) is not only in the interest of the CPO but also the CFO and the CEO. The results of the DEA analysis for all four steps for the seven industry sectors are reported in Table 2. Here we analyze the transformation efficiency of companies compared to their industry peers as well as try to identify commonalities in those between different industries. As one may note, analysis 3, which tests the efficiency of the conversion of PSM performance outcomes into corporate performance (using the output oriented VRS model (2) described in the Appendix), reveals the maximum number of efficient firms across most industry sectors. Although anecdotal evidence indicates that cost savings and other PSM performance results are often not directly translated into improved bottom line performance, our study results show that many companies in our sample set of seven industries have been quite efficient in transforming PSM performance outcomes into corporate financial performance. We believe that our data collection approach2 based on Pagell (2004) and Bloom and van Reenen (2007) as well as the choice of realized cost savings data in contrast to perceptual cost savings data has contributed to this result. Analysis 1, which measures the transformation efficiency of PSM drivers directly into corporate performance (using the input oriented VRS model (1) described in the Appendix), reveals a relatively higher efficiency of conversion for the automotive, high-tech and energy sector while they are somewhat lower for the chemicals/pharmaceuticals, packaged goods, financial services and construction industry. Based on these results, it seems that industries where the efficient use of strategic and transactional buyers as well as a lower number of suppliers is highly relevant show a higher efficiency rate in this transformation step. Automotive and high-tech are primarily driven by a structured supplier base with a clear objective to reduce the number of direct suppliers (for relatively higher sourcing volumes compared to other industries) significantly. The energy sector on the other hand enables companies to buy large volumes of similar goods and hence seem to be able to manage with lesser numbers of strategic and transactional buyers. In contrast, industries such as chemicals/pharmaceuticals, packaged goods, financial services or the construction sector are characterized by a relatively high number of input categories or a fairly unstructured supplier base, resulting in lower effi2 We use an interview based approach to collect data for two of the three performance outcomes (cross-functional collaboration and supplier performance management) as discussed in Section 3 and measure the third performance outcome using actual cost savings.

203

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205 Table 1 Descriptive statistics of all input and output variables. Variablea b

Number of strategic buyers Number of transactional buyersb Number of suppliers covering 80% of the managed spendb Cost savings (in % p.a.)b Cross-functional collaboration (Scale 1–5)c Supplier performance management (Scale 1–5)b Average % of EBITDA margin over 3 yearsd

Mean

Std Dev

Minimum

Maximum

Lower quartile

Median

Upper quartile

65.44 105.2 1277.83 2.74 3.24 2.89 15.93

82.47 122.51 2041.31 1.33 0.93 1.055 9.43

7 9 150 0.5 1 1 0.055

456 630 15,000 8 5 5 39.77

23 36 400 2 3 2 8.71

36 67 550 2.5 3 3 13.9

67 99 1200 3.4 4 4 23.16

a

The total sample size, N = 129, which includes all seven industry sectors. These variables are measured using a traditional paper-based questionnaire evaluating the precise data for each firm. c These variables are measured using interviews complemented by the scoring grid to guide the interviewers to a rating on a scale of 1 (i.e., worst performance level) to 5 (i.e., best performance level). d EBITDA margin is measured as a % of sales. b

Table 2 Number of efficient firms in each industry sector in the respective analyses 1–4. Industry sector

Number of firms

Analysis 1

Analysis 2

Analysis 3

Analysis 4

Automotive Chemicals and pharmaceuticals Energy Financial institutions High tech Construction Packaged goods

15 17

6 2

2 2

5 11

2 2

15 21

6 3

6 4

12 3

4 1

22 22 17

8 5 3

6 8 3

9 7 9

2 3 1

ciency levels in converting PSM drivers into corporate performance. Analysis 2, which measures the efficiency of the conversion of PSM drivers into PSM performance outcomes (using the input oriented VRS model (1) described in the Appendix), shows similar results as analysis 1. However, industries showing high or low transformation rates are slightly different. While energy and high-tech still show high conversion efficiencies automotive has decreased and the construction sector has increased in this respect. These changes might be due to the fact that, key PSM performance measures such as cross-functional collaboration and supplier performance management in the automotive industry require higher number of transactional and strategic buyers than in other sectors due to the high product complexity and highly integrated global supply chains. In the construction industry however, given the commodity type characteristics of their major supplies, many firms are able to achieve efficient transformation of their PSM drivers into PSM performance. A comparison of the results from analysis 1 and 2 shows that even though firms are inefficient in the short run in converting the PSM drivers into PSM performance outcomes, the intangible benefits of this process of conversion in the long run contributes towards better financial performance.3 The results from analysis 3, which measures the direct conversion efficiency of PSM performance outcomes into corporate performance, also substantiate this conjecture. Although the analysis evaluating similarities between the seven industries reveals some common characteristics and cross-industry trends, each sector shows an individual conversion efficiency pattern. For example, in the financial sector, there are only three firms out of 21 firms that fall on the frontier in analysis 1, four firms in analysis 2 and three firms in analysis 3, respectively. This shows that, perhaps, the PSM function has not evolved to the same degree in the financial sector as compared to other sectors, which 3 The financial performance is measured as average EBITDA as a percentage of sales over a 3 year period.

is justified considering the fact that the industry is characterized by a strong focus on indirect category sourcing and therefore PSM has still a limited role to play. This could also be contributing to the lack of interest in establishing strong cross functional collaborative processes and in pursuing PSM cost savings. The automotive industry shows a different picture where cross-functional collaboration and supplier base complexity are relevant challenges for all PSM functions in this sector and have a high impact on performance. Moreover, substantial cost savings are difficult to achieve and require significant investments due to the maturity of this industry in this context. These characteristics may be then contributing towards the low transformation efficiencies in analysis 2. A final illustrative example is the packaged goods industry where we could only identify a relatively high efficiency of conversion from PSM performance outcomes into corporate performance. This might be due to the circumstances that companies in this industry require a high degree of cross-functional collaboration and face in general a diversified supply base. Companies with PSM functions that master all those challenges combined with significant cost savings, might be able to achieve a superior corporate performance in this sector, due to the relatively low degree of vertical integration and the consequent impact of PSM on corporate financial performance. The two-stage Value Chain DEA (VCDEA) results corresponding to analysis 4 in Fig. 1, throw up the least number of efficient firms for all industries. Understandably, there are fewer efficient firms in analysis 4 than in the other analyses, since it requires an efficient conversion of both PSM drivers into PSM performance and PSM performance outcomes into corporate financial performance. One can also further investigate the value chain model by looking at the distribution of efficiency scores as described in Table 3. For example, one may note from Table 3 that the industry with the relatively highest number of efficient firms in analysis 4 in Table 2 also shows the highest mean (0.70) of conversion efficiency. Furthermore, the analysis of the quartiles can provide additional insights into the efficiency structures of a single industry. For example, the construction industry, which constitutes the second highest number of frontier companies in analysis 4 (Table 2), does not show the second highest average efficiency as one might expect. The reason becomes evident from various statistics given in Table 3, as the construction industry seems to be characterized by a few highly efficient firms (0.68 for upper quartile) while the lower quartile shows a very low threshold of 0.34. Such analyses might be taken into account when benchmarking a PSM function with peers of its industry. For a true and fair benchmarking of a PSM function, the minor differentiating characteristics between companies such as size, product portfolio, international coverage in the upper and the lower quartile could be analyzed. These insights might then be used to create a reasonable level of expectations

204

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

Table 3 Distribution of efficiency scores across each industry sector for analysis 4. Industry sector

Number of firms

Analysis 4: Efficient number of firms

Mean

Std Dev

Lower quartile

Median

Upper quartile

Automobile Chemicals and pharmaceuticals Energy Financial institutions High tech Construction Packaged goods

15 17 15 21 22 22 17

2 2 4 1 2 3 1

0.64 0.49 0.70 0.53 0.57 0.53 0.56

0.21 0.26 0.24 0.18 0.24 0.27 0.18

0.43 0.30 0.50 0.41 0.38 0.34 0.44

0.62 0.41 0.72 0.53 0.54 0.50 0.53

0.84 0.54 1.00 0.62 0.71 0.68 0.59

Table 4 Original intermediary outputs and the corresponding efficient targets for the automobile sector. Firm

Value chain efficiency score

Original cost savings

Target cost savings

Original crossfunctional collaboration

Target cross-functional collaboration

Original supplier performance management

Target supplier performance management

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0.36 1 0.62 0.42 0.62 0.84 0.54 0.51 0.85 0.43 0.55 0.66 0.43 1 0.75

3.6 1.4 2.3 4.2 3 4.4 2.2 1.7 2.7 2 4.1 2.5 2.7 2.6 2.1

2.3 1.4 2.3 2.3 3 4.4 2.9 2.1 2.3 2.4 2.3 2.5 2.3 2.6 2.1

4 4 3 3 4 3 3 3 3 2 5 3 4 4 3

3 4 3 4 4 3 4 3 3 3 4 3 3 4 3

4 5 2 3 4 4 4 3 3 3 4 4 2 1 4

4 5 4 4 4 4 4 4 4 4 5 4 4 1 4

towards a specific PSM function where the firm shows more similar characteristics with companies in the lower quartile than the upper quartile. Thus, senior executives such as CPOs, CFOs or CEOs that are aiming for an all-round PSM performance that contributes to long term corporate financial performance are well advised to use this two-stage value chain analysis for benchmarking themselves with their industry peers. Another advantage of the proposed model is the provision of ‘output targets’ which enable the inefficient firms to benchmark and improve their performance to catch up with the frontier firms. To demonstrate this, we present in Table 4 the original intermediary outputs and the corresponding targets that are computed using the results from the two-stage VCDEA model for the automotive sector. Note that, one of the intermediary outputs (cost savings) is cardinal in nature and the remaining two intermediary outputs, cross-functional collaboration and supplier performance management are ordinal in nature (Likert scale data) and hence are transformed into exact data during the efficiency calculations of VCDEA using Zhu’s (2004) data transformation procedure. The detailed procedure used for computation of intermediary output targets for the inefficient DMUs which are presented in Table 4, is described in the Appendix. As one may note from Table 4, all the inefficient firms have slightly different targets from the original values for at least one of the PSM performance outcomes. Also, some of the target values are in fact lower than the original values, which basically mean, for the input/output mix of these inefficient companies, these are the right mix of intermediary targets. Please also note that the intermediary targets are different from the efficiency values obtained from analysis 4; and that the intermediary output targets basically show the direction for improvement for the inefficient firms. It essentially means that, if the inefficient firm manages to reduce the proportion of initial inputs by ‘h’ and increase the proportion of final outputs by ‘/’, and reach the targets set for the intermediary outputs, it will become efficient in both stages.

6. Conclusions and discussion We have tried to address some of the challenges in the PSM performance evaluation literature by providing an alternative rigorous data collection methodology and an evaluation framework that takes into account multiple inputs and outputs at various levels of PSM activities and performance outcomes and provides a single efficiency measure that accurately captures the comprehensive performance of a PSM function and its ultimate contribution to corporate performance. This paper contributes to the PSM performance evaluation literature in two major aspects. In order to address the problems associated with the traditional, questionnaire-based surveys with respect to the measurement of implemented PSM activities and actual performance outcomes, a new approach to data collection is proposed, which incorporates data from a variety of sources via different evaluation techniques including external performance reviews and hence avoids common method bias and problems associated with perceptual measures. Second, the two-stage value chain DEA method is applied to the PSM performance evaluation incorporating precise as well as imprecise data for the first time in the PSM literature. The results of this paper have some substantial managerial implications. Firstly, it is a well established fact that PSM performance evaluation in practice is quite difficult due to the broad and often directly immeasurable influence of the PSM function on overall corporate performance. The existing practice of looking at several single input/output comparisons will not provide an accurate picture of the true strategic value of PSM to the senior executives. The DEA framework proposed in the current paper provides a single measure of performance evaluation, which can be used to benchmark with peer companies within the industry at a given point in time as well as with own performance at different points in time. Secondly, the results reveal that each industry has its very own characteristics and PSM performance patterns, which

H. Saranga, R. Moser / European Journal of Operational Research 207 (2010) 197–205

in other words imply benchmarking efforts need to be focused on companies with similar characteristics. Senior executives using an industry benchmarking dataset have to therefore ensure that they also understand the characteristics of each participating firm to determine whether their own PSM function belongs to the upper or lower quartile of the efficiency distribution, given the characteristic’s relevance, and identify the benchmark peer group accordingly. Finally, our framework addresses the challenges faced by the senior executives, who are finding the need to incorporate not just precise data such as cost savings but also imprecise data such as the level of cross-functional collaboration in performance evaluations, in order to accurately evaluate PSM function’s contribution to purchasing and corporate performance. However, this paper also has some limitations. Firstly, there are other PSM drivers and outcomes, depending on the industry structure, which could be playing an important role in fashioning PSM performance and hence corporate performance. Due to the small size of each industry sample, we could not incorporate too many performance drivers and outcomes in the current study. However, there is plenty of scope for future studies, especially if they are based on a single industry and have access to data from a larger sample, to incorporate other drivers and performance outcomes such as PSM knowledge management systems or supply risk management into the efficiency evaluation. Secondly, this paper focuses only on the impact of a PSM function on corporate performance via the PSM performance outcomes. Senior managers may be interested in the impact of PSM performance outcomes combined with other functions such as production, marketing and new product development etc. In such instances, application of other multi-stage DEA models (Seiford and Zhu, 1999; Chen and Zhu, 2004; Liang et al., 2006 and Golany et al., 2006; Kao and Hwang, 2007; Kao, 2009) which have the ability to incorporate the inputs from these other functional areas at appropriate stages of evaluation would be more useful. Acknowledgements This research project has been supported by the EADS-SMI Endowed Chair for Sourcing and Supply Management, IIM Bangalore. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.ejor.2010.04.023. References Bloom, N., van Reenen, J., 2007. Measuring and explaining management practices across firms and countries. Quarterly Journal of Economics 122, 1351–1408.

205

Center for Advanced Purchasing Studies (CAPS), 2009. Cross-Industry Metric Report. Tempe, AZ. Chen, Y., Zhu, J., 2004. Measuring information technology’s indirect impact on firm performance. Information Technology and Management 5, 9–22. Cook, W.D., Zhu, J., 2006. Rank order data in DEA: A general framework. European Journal of Operational Research 174, 1021–1038. Cook, W.D., Kress, M., Seiford, L.M., 1993. On the use of ordinal data in data envelopment analysis. Journal of Operational Research Society 44, 133–140. Cooper, W.W., Park, K.S., Yu, G., 1999. IDEA and AR-IDEA: Models for dealing with imprecise data in DEA. Management Science 45 (4), 597–607. Das, A., Narasimhan, R., 2000. Purchasing competence and its relationship with manufacturing performance. Journal of Supply Chain Management 36, 17–28. Dyson, R.G., Allen, R., Camanho, A.S., Podinovski, V.V., Sarrico, S.S., Shale, E.A., 2001. Pitfalls and protocol in DEA. European Journal of Operational Research 132, 245–259. Easton, L., Murphy, D.J., Pearson, J.N., 2002. Purchasing performance evaluation with data envelopment analysis. European Journal of Purchasing and Supply Management 8, 123–134. Ellram, L.M., Zsidisin, G.A., Siferd, S., Stanly, M., 2002. The impact of purchasing and supply management activities on corporate success. The Journal of Supply Chain Management: A Global Review of Purchasing and Supply 38 (1), 4–17. Golany, B., Hackman, S.T., Passy, U., 2006. An efficiency measurement framework for multi-stage production systems. Annals of Operations Research 145, 51–68. Kao, C., 2009. Efficiency decomposition in network data envelopment analysis: A relational model. European Journal of Operational Research 192, 949–962. Kao, C., Hwang, S.N., 2007. Efficiency decomposition in two-stage data envelopment analysis: An application to non-life insurance companies in Taiwan. European Journal of Operational Research 185, 418–429. Kaplan, R.S., Norton, D.P., 1992. The balanced scorecard – Measures that drive performance. Harvard Business Review 70 (1), 71–79. Liang, L., Yang, F., Cook, W.D., Zhu, J., 2006. DEA models for supply chain efficiency evaluation. Annals of Operations Research 145, 35–49. Narasimhan, R.J., Jayaram, J., Carter, J.R., 2001. An empirical examination of the underlying dimensions of purchasing competence. Production and Operations Management 10 (1), 1–15. Nollet, J., Calvi, R., Audet, E., Cote, M., 2008. When excessive cost savings measurement drows the objectives. Journal of Purchasing and Supply Management 14, 125–135. Pagell, M., 2004. Understanding the factors that enable and inhibit the integration of operations, purchasing and logistics. Journal of Operations Management 22 (5), 459–487. Podsakoff, P.M., MacKenzie, S.M., Lee, J., Podsakoff, N.P., 2003. Common method variance in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology 88, 879–903. Seiford, L.M., Zhu, J., 1999. Profitability and marketability of the top 55 US commercial banks. Management Science 45 (9), 1270–1288. Smith-David, J., Hwang, Y., Pei, B.K.W., Reneau, J.H., 1999. The Impact of Purchasing on Financial Performance. Tempe, AZ. Tan, K.C., Kannan, V.R., Handfield, R.B., 1998. Supply chain management: Supplier performance and firm performance. International Journal of Purchasing and Materials Management 34 (3), 2–9. Trent, R.J., Monczka, R.M., 1994. Effective cross-functional sourcing teams: Critical success factors. International Journal of Purchasing and Materials Management 30 (4), 3–12. van Weele, A.J., 1984. Purchasing performance measurement and evolution. Journal of Purchasing and Material Management 20 (3), 16–22. Wagner, S.M., Kaufmann, L., 2004. Overcoming the main barriers in initiating and using purchasing – BSCs. Journal of Purchasing and Supply Management 10 (6), 269–281. Zhu, J., 2004. Imprecise DEA via standard linear DEA models with a revisit to a Korean mobile telecommunications firm. Operations Research 52, 323–329.