Commercial branch performance evaluation and results communication in a Canadian bank––a DEA application

European Journal of Operational Research 156 (2004) 719–735 www.elsevier.com/locate/dsw

O.R. Applications

Commercial branch performance evaluation and results communication in a Canadian bank––a DEA application Joseph C. Paradi *, Claire Schaffnit Centre for Management of Technology and Entrepreneurship, Faculty of Applied Science and Engineering, University of Toronto, 200 College Street, Toronto, Ont., Canada M5S 3E5 Received 16 January 2002; accepted 21 January 2003

Abstract In this paper, we focus on evaluating the performance of the commercial branches of a large Canadian bank using data envelopment analysis. Two production models are considered in this country-wide evaluation. One model, looking directly at resource usage, is most useful to the branch manager. The other model, incorporating financial results, is more geared towards senior management. We introduce non-discretionary factors to reflect specific aspects of the environment a branch is operating in, such as risk and economic growth rate of the region. Both input and output multipliers are constrained by incorporating market prices as well as managerial preferences, in order to get effectiveness measures. The cost-minimisation study led to valuable results pertaining to the performance of individual branches. Notable is the methodology introduced here that shows how to present graphical and numeric outcomes to managers. Gap maps, pie charts and target tables are produced for each branch to provide performance goals for the managers. Useful information has also been obtained at the district level. Output oriented models were analysed to reflect the BankÕs recent emphasis towards growth in some areas.
2003 Elsevier B.V. All rights reserved. Keywords: Data envelopment analysis; Management; Banking; Performance improvement; Results presentation

1. Introduction In a rapidly changing world with an increasingly global business environment, continuous improvement is vital for any successful organisa-

*

Corresponding author. Tel.: +1-416-978-6924; fax: +1-416978-3877. E-mail address: [email protected] (J.C. Paradi).

tion. Historically, banks have been in the forefront of trying to improve their operating efficiency, but typically have not had at their disposal a sound performance analysis system to evaluate their branch networks. Capturing the essential aspects of the process, such a system should yield a relevant and trustworthy measure, suitable to establish benchmarks. Indicative of the unitÕs ability to use its resources to generate outcomes, this measure should lead to a better understanding of the process in terms of what is achieved and how it is

0377-2217/$ - see front matter
2003 Elsevier B.V. All rights reserved. doi:10.1016/S0377-2217(03)00108-5

720

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

achieved. It should allow a meaningful investigation of hypotheses concerning the sources of inefficiency, in order to separate the effects of the environment, as well as to isolate the impact of differences in production technology. Finally, it should provide management with a control mechanism with which to monitor performance. This paper presents a performance analysis of the country-wide commercial branch network of a large Canadian bank using data envelopment analysis (DEA). It addresses directly managementÕs need for consistent benchmarking, targetsetting, and designing focused on-site audits, in an attempt to identify and transfer best-practices. We emphasise the presentation of the results in an effort to reach management at different levels. Indeed, their understanding of the findings will be crucial in promoting change within the organisation. Two production models have been devised in collaboration with the Bank; they are tailored to the needs of two levels of management. A measure of risk is incorporated as well as an environmental factor capturing the level of economic growth in each geographical area. In addition to the input/ output data from all the branches, we introduce information on managerial preferences and market prices to construct models with multiplier constraints and move from technical to overall efficiency. The models were compared in order to gain further insight into branch performance. Given the diversity of the branches and of their managerial objectives, we investigated alternate paths towards the frontier: either through input reduction or output-enhancement. In order to disseminate the results as effectively as possible within the organisation, we designed individual reports and gap maps incorporating the findings, which are most relevant to management. The models and results presented in this paper illustrate the usefulness of DEA as a methodology for providing important insights beyond mere efficiency measures. Both senior and branch managers acquired a much better understanding of the performance drivers in their districts or branches respectively because the results from our analyses offered truly useful and practical goals which they could actually implement.

The remainder of the paper is organised as follows. Section 2 briefly introduces the literature relevant to this study. Section 3 describes the sample, the production models and the methodology. Section 4 presents the results of the analysis and discusses some of the implications. Section 5 investigates different paths towards the frontier. Section 6 describes a format for the presentation of the results to the BankÕs management and, finally, some concluding remarks are given in Section 7.

2. Efficiency and DEA in bank branch performance measurements This study focuses on the commercial banking network of a large Canadian bank. This major financial institution has found it important to carry out a thorough and meaningful evaluation of the various aspects of their commercial branch networkÕs performance expecting to obtain a factbased rationale for future actions. As opposed to traditional methods, such as performance ratios and regression analyses, which are usually not very effective in assessing complex processes over a fairly large network, DEA has the ability to handle multiple non-commensurate factors and provide specific and consequential information for performance improvement. Hence, we adopted this method in this study. DEA was first proposed by Charnes et al. (1978), they based their work on the seminal paper by Farrell (1957), and is a non-parametric method of efficiency analysis: it does not require assumptions regarding the shape of the production frontier and it makes simultaneous use of multiple inputs and outputs. The production units are often referred to as decision making units (DMU); a fortuitous choice of terminology as the issue at hand is the measurement of people performance, which is dependent upon their own decision making. DEA defines the relative efficiency for each DMU (bank branches, engineering teams, hospitals, schools, etc.) by comparing its input and output data to all other DMUs in the same ‘‘cultural’’ environment. In addition to relative efficiency measures, the outcomes of a DEA study are:

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

1. A piecewise linear empirical envelopment surface to represent the best practice frontier, consisting of units which exhibit the highest attainable outputs in relation to all other DMUs in the population, for their given level of inputs. 2. An efficiency metric to represent the maximal performance measure for each DMU measured by its distance to the frontier. 3. Specific Targets or efficient projections onto the frontier are given for each inefficient DMU. 4. An efficient reference set or peer group for each DMU made up of the efficient units closest to it. DEA models are classified with respect to the type of envelopment surface, the efficiency measurement and the orientation (input or output). There are two basic types of envelopment surfaces in DEA: Charnes et al. (1978) introduced the constant returns-to-scale (CRS) and Banker et al. (1984) introduced the variable returns-to-scale (VRS) model. DEA models are also classified as radial input oriented, radial output oriented or additive (both inputs and outputs are optimised) based on the direction of the projection of the inefficient unit onto the frontier. Although we utilise both the radial input and output oriented VRS models in our study of commercial bank branches, we will not present the mathematical formulation here, instead direct the reader to the complete mathematical presentation of the applicable DEA models in Cooper et al. (2000). DEA is a framework well suited for performance analysis and it offers many advantages over traditional methods such as performance ratios and regression analysis. Largely the result of multi-disciplinary research during the last two decades in economics, engineering and management, DEA is best described as an effective way of visualising and analysing performance data. Technically, it represents the set of non-parametric, linear programming techniques used to construct empirical production frontiers and to evaluate the relative efficiency of production units. DEA is particularly effective in handling complex processes, where these DMUs use multiple inputs to produce multiple outputs. There has been a significant interest in evaluating bank branch activities, both by practitioners

721

and academics. Traditionally based on profitability measures, the banksÕ assessment of their branch networks has started to change towards more comprehensive benchmarking programs. Academics have used frontier analysis as a sophisticated way to evaluate the relative performance of production units, assessing how close the financial units are to a best-practice frontier. The first of these applications using DEA was by Sherman and Gold (1985); they defined the broad approach to DEA applications when used in bank branch productivity measurements. Schaffnit et al. (1997) contains a review of the DEA studies of bank branches published prior to 1995. Then, a comprehensive paper by Berger and Humphrey (1997) reviewed the literature concerning the efficiency of financial institutions, including bank branches, using non-parametric (DEA and variations) and parametric frontier analysis. Lovell and Pastor (1997) looked at setting targets for bank branches; Camanho and Dyson (1999) evaluated Portuguese bank branches; Kantor and Maital (1999) examined activity based accounting in bank branches; Soteriou and Zenios (1999) focused on operations, quality and profitability in banking services; and Golany and Storbeck (1999) examined operational efficiencies in bank branches. There are other studies too many to cite here, but there are a few that resulted in an adaptation of DEA by the bank on an on-going basis. Oral and Yolalan (1990) examined 20 branches of a Turkish Commercial Bank where DEA was used to reallocate resources between branches. Building on the previous work by Sherman and Gold (1985), Sherman and Ladino (1995) reported on the implementation of DEA results in the restructuring process of 36 US branches of a bank that led to actual annual savings of over $6 million. Zenios et al. (1999) studied the Bank of Cyprus where the bank adopted their model and findings to establish policy guidelines and provide operational support for productivity improvements. Then, Athanassopoulos and Giokas (2000) examined 47 branches of the Commercial Bank of Greece and the DEA results were used to implement the proposed changes in the bankÕs performance measurement system. Closer to home is the study by Cook et al. (2000) when they applied DEA to a large Canadian

722

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

BankÕs branches and the bank accepted their new performance rating system based on DEA. This work builds on the previous studies and emphasises the importance of obtaining results of direct relevance to the BankÕs management (see Sherman and Ladino, 1995; Golany and Storbeck, 1999). Two models are analysed here, with the choice of inputs and outputs aimed at addressing particular managerial needs. Also, from a technical perspective, we apply advanced DEA models enabling us to move from technical to overall efficiency. Banker and Morey (1986) applied DEA in a fast food environment where exogenously fixed variables were introduced. Ray (1991) examined resource use efficiency in a public school environment where they regressed DEA results against socio-economic factors. Ruggiero (1996) also worked in the public-sector and suggested that environmental variables need to be used in DEA analyses because otherwise technical efficiencies will be over estimated. The papers all dealt with non-discretionary variables in real situations. We incorporate a risk factor and a growth factor as non-discretionary variables in an attempt to reflect the diversity of environments in which the branches are operating. Unrestricted DEA can yield quite unrealistic results from a managerial point of view and there are situations where additional information is available that allows the analyst to impose conditions on the components of the multiplier vectors. Thompson et al. (1986) introduced the technology they referred to as the ‘‘assurance region’’ and then, Charnes et al. (1990) published their ‘‘cone ratio’’ approach. Here, we use DEA models with output multiplier constraints to place bounds on the trade-offs between the different outputs, based on managementÕs value system and constraints, incorporated on the input side. For model comparisons, we use a simple technique introduced in Schaffnit et al. (1997) to assess differences between radial models. One of the fundamental assertions in DEA analyses is that it is fair and equitable. The typical managerial push-back, when one attempts to measure their performance, is that scale size is meaningful and comparisons between large and

small DMUs are inherently unfair. The VRS models used here do address this particular concern as the variable returns-to-scale frontier ensures that scale is taken into proper account. However, establishing scale size in DEA is not a simple effort. A number of papers exist in the literature on this aspect of DEA starting with Banker et al. (1984), then Banker and Maindiratta (1988), Banker and Thrall (1992), F€are and Grosskopf (1994) and Sueyoshi (1999) just to cite the main ones. The reader is advised to study these references if work is to be done where scale size is of central importance. Finally, we investigate alternative paths towards the frontier, and stress the importance of communicating the key findings to management through the design of specific individual reports. In particular, this last point is a major issue in DEA research due to the fact that for the most part, practical applications in DEA fail to impress management because the outcomes are presented in a manner that is simply too complex and hard to understand from a lay personÕs point of view. The Figures presented later are examples of the approach that had turned out to be an easy way to explain how an individual branch manager might apply the results from the DEA analysis as the managers at this bank had done so.

3. The analysis: Models and methodology Canadians enjoy services from 66 banks, 14 domestic and 42 foreign bank subsidiaries, which manage over $1.75 trillion ($CAD) in assets. They serve Canadians with 235,600 employees in Canada and 32,600 abroad, 8329 branches and 17,174 ABMs as reported by Canadian Bankers Association (2002). The five largest of these banks are very large by any standard and have assets in the 300 billion-CAD range. Banks are very heavily regulated in Canada. The Bank Act, first enacted in 1871, defines what the banks can and cannot do. Moreover, the Office of the Superintendent of Financial Institutions (OSFI), CanadaÕs federal financial supervisor, monitors the banksÕ operations. The Bank studied here has a commercial network that consists of 90 branches spread across

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

723

3.1. Choice of inputs and outputs

who needs to know how best to use the branchÕs resources to produce the required outputs. But to meet Senior managementÕs needs, a strategic model was designed that accepts as inputs various factors which they are interested in minimising; the outputs include financial measures that the bank desires to maximise. How to choose inputs and outputs for the actual DEA models is the subject of a never ending debate. However, in this case, we had the close cooperation and involvement of the Senior VicePresident in charge of Commercial Banking and one of his direct reports, the Vice-President responsible for performance measurement. They obtained all the details from their staff (four individuals) with whom we also worked very closely. There were numerous meetings between them and us where in-depth discussions took place with the bank staff suggesting input and output measures while we judged the appropriateness to the models. After much work, we had both agreed to the modelsÕ construction. Prices (multiplier constraints) were decided upon in a similar manner. In addition to their specific discretionary inputs and outputs, both models incorporate additional factors, environmental variables, which are not fully under managerial control, but are likely to affect the branchesÕ performance. Two such factors were included in this study: a measure of risk for the branchÕs investment portfolio, and a growth factor, specific to each district.

The BankÕs senior management needs a sound methodology in order to ensure that the branch networkÕs performance is assessed appropriately and can be improved in order to be in line with the organisationÕs goals. The branch manager has the responsibility to optimise his/her branchÕs operations and align their goals with senior managementÕs bottom line expectations. Therefore, to comply with these requirements, two models are considered. A production model is directly based on the process, following the ‘‘physical’’ flow of resources within a branch, and has a set of inputs corresponding to the various types of resources utilised. Its outputs include the different types of services provided in the branch. This processbased model is most useful to the branch manager,

• A weighted average borrower risk rating (BRR) is calculated using historical data together with branch specific information to arrive at this factor. Each account is assigned a risk value, which is used to compute a weighted average at the branch level: the branch rating is a portfolio quality measure representing the distribution of assets in each risk category. The BRR reflects the credit worthiness of the business environment of the branch and is incorporated to avoid having a branch ‘‘unfairly’’ deemed inefficient when it handles more risky portfolios. • As elsewhere, Canada has more and less advantaged districts where the success of enterprises, and banks, is a reflection of their economic environment. The factor reflecting economic

Canada. Although this group is but a small percentage of their entire branch network consisting of over 1000 branches, their business is sufficiently different from the rest that they are separated and measured differently. These branches have little to do with personal banking activities and concentrate on dealing with large corporations, Institutions and Governments. Moreover, there are some specialised branches such as oil and gas and real estate. Clearly, the Canadian banking scene is very different from that of the US and Europe but is most often compared to Australia which has a similar banking environment and country: large, sparsely populated, few very large banks. For administrative purposes, these commercial branches are subdivided into eight districts, the geographical limits of which are based more or less on Provincial borders. For confidentiality purposes, they are called here D1 to D8. Some districts can be quite different from the others, from an economic and/or cultural point of view. If the reader is at all familiar with Canadian geography and economic life, it is easy to see that there is a substantial difference between Vancouver, British Columbia and St. JohnÕs, Newfoundland. This was taken into account in the development of the models as described below. For solving the DEA models, we utilised the commercial software package, Ideas by Software Consulting.

724

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

activity used here is the average rate of change of the real provincial gross domestic product between the years 1993 and 1996. It is included in the model to reduce the likelihood that a branch is assigned a target that was operating in a more conducive economic environment. For example, it would be totally unfair to compare an inefficient branch in Atlantic Canada to an efficient one in Central Canada, but it may well be appropriate to go the other way. 3.1.1. Production model The production model was designed to provide information on the process from the branch managerÕs point of view. It includes as inputs four types of resources: staff, information technology, premises, and other non-interest expenses. • Staff is classified into five types, each measured as the number of people working in the branch:
Vice-Presidents/Manager BBC/Deputies,
Account Managers/Team Leaders,
Assistant Manager Customer Service/Account Officers,
Secretaries/Administrative Assistants/Other Clerical Staff (receptionist, clerks), and Managers/Officers Cash Management Services. • Equipment is measured by the amount of IT expenses. • To quantify premises usage, rent was used as a proxy. 1 • Other non-interest expenses include business development, advertising, telecommunications, travel, stationery, etc. Hence, all resources but staff are measured in dollars. These resources are devoted to providing four kinds of services: deposits, loans, operating services, and account maintenance. The first three 1 The bank owns many historic and new properties as well as rents premises at widely different rental rates. Hence, rent per square foot is a standard charge by the BankÕs head office, independent of the real rent being paid. This is logical since the bank manager has absolutely no say in where the branch is located, or what rents are actually paid.

constitute the three aspects of the business, the main services a branch provides to its customers (all three are measured in dollars): • Deposits • Loans, and • Fee income, used as a proxy for the amount of non-interest earning activities, such as cash management. • Maintenance activities can be subdivided into five types, as appropriate, for each of the five tiers of connections: A, B, C, D and E. A connection is essentially an individual customer, regardless of the number of accounts involved. For example, if the Bank deals with an entrepreneur who owns several companies, and thus has various accounts with the Bank, this represents one connection. A connection belongs to one of five tiers according to its average annual revenue. This measure was chosen as a proxy for maintenance activities, because the maintenance work required in the branch is approximately proportional to the number of connections handled. This production model includes, as an environmental factor, the growth factor introduced above to reflect the economic environment in which the branch is operating. 3.1.2. Strategic model The strategic model was designed to meet the specific needs of the BankÕs senior management. The inputs gather the four types of resources mentioned above (staff, equipment, rent and noninterest expenses), plus an additional factor which the Bank is also interested in minimising: • Non-accrual loans (nal), measured in dollars. Typically, non-accrual loans are those with principal and interest unpaid for at least 90 days. The outputs considered here include the three direct services offered to customers (deposits, loans and operating services) as well as two proxies for the incomes generated, respectively, by deposits and loans:

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

• Deposit spread • Loan spread. The two environmental variables introduced before (growth factor and risk rating) are both incorporated into this model. The strategic model is the model most directly relevant to senior management, and unless stated otherwise, it is used in the following discussion. The data used in this study are for the year 1995. The statistics characterising the data set are given in Table 1. The breakdown by districts and the corresponding economic growth factor can be seen in Table 3 and will be discussed later. 3.2. DEA models Our group of DMUs includes all the BankÕs commercial branches of various sizes, hence, the VRS model, accounting for possible scale effects, was a natural choice. We used both the constant and the variable returns to scale model for the initial models in order to investigate scale efficiency. It should be mentioned, however, that while there was a significant variability of branch sizes (see Table 1), the business of the branches was quite comparable, but for some notable exceptions. We did find that certain branches that specialise in a narrow market segment (real estate, oil and gas) had to be removed from the analysis because they could not reasonably be used as peers to an inefficient branch. Since the branches typically have little or no direct control over the amount of services their customers require, input-orientation was chosen for the first model presented in this study. 3.2.1. AR output multiplier constraints Typically, sharper efficiency estimates, based on more realistic frontiers, can be obtained by introducing additional information on the process, such as knowledge about trade-offs between different factors. For example, for the production model, it would be desirable to place bounds on the tradeoffs between the different types of outputs based on resource usage, if such were available. Since this is not the case, the output weights were left unconstrained for the production model.

725

Values (prices), derived from market prices or managerial information, may also be introduced to move from technical to overall efficiency. Even if the pricing information is not precisely known, bounds on the prices, or relationships between them, can be incorporated into the DEA model as multiplier restrictions. Prices were introduced for each output of the strategic model, according to managementÕs judgement of their relative importance. They are reported with the data statistics in Table 1, Panel B. The range was taken to be the price 20%. All the multiplier constraints in this study have been introduced through bounds set on ratios of pairs of multipliers, thus defining the assurance regions (Thompson et al., 1986). It can be shown that, in the limit when prices are exactly known, the DEA model incorporating such constraints reduces to an allocative model (see Schaffnit et al., 1997). 3.2.2. AR input multiplier constraints Given that one of managementÕs objectives is to reduce costs, we investigated the cost-minimising behaviour of the branches by introducing input weight constraints based on salary ranges offered by management. The prices used for the inputs are also reported in Table 1. The price for each type of staff is based on their corresponding salary range. The inputs measured in dollars have a weight of 1. The price assigned to the factor nal has to reflect the cost of $1 of nal, relative to the cost of the other inputs. We considered two issues for estimating this value. Clearly, not all nal dollars will be lost, some amounts will be recovered. Also, when $1 of loan is lost, its cost may be estimated at $1 plus the interest due on this dollar (the extra work required in handling the issue will be reflected through the other inputs). An estimate of the price range for the input nal was thus derived, based on the ratio of the amount of loan loss experienced over nal, and an average interest rate of 15%. Although this is a fairly wide range, it can be easily refined as and when management desires it. 3.2.3. Orientation As mentioned above, the choice of input-oriented models reflects the usual lack of control the

726

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

Table 1 Data statistics and applicable prices for the production and strategic models Mean

Standard deviation

Minimum

Maximum

Costs

1.6 14.3 2.8 6 1.4 58,584 321,590 140,756

0.9 8.6 2.9 3.9 2 50,415 400,039 105,158

1 2 0 0 0 2057 0 7280

4 41 13 17 11 304,000 2,640,000 536,264

57,800–120,000 44,200–78,515 20,127–45,333 22,900–34,930 25,000–56,975 1 1 1

Environmental BRR Growth factor

3.15 2.96

0.97 0.45

0 2

5.09 3.3

Outputs Deposits (1000) Loans (1000) Fee income (1000) Tier A connections Tier B connections Tier C connections Tier D connections Tier E connections

143,332 286,985 2673 24 52 72 667 3067

145,410 295,817 2602 32 49 57 442 2478

6220 32,493 192 0 0 0 32 77

795,896 1,293,645 17,728 179 229 298 2290 10,907

1.6 14.3 2.8 6 1.4 58,584 321,590 140,756 4,612,654

0.9 8.6 2.9 3.9 2 50,415 400,039 105,158 8,911,167

1 2 0 0 0 2,057 0 7,280 26,966

4 41 13 17 11 304,000 2,640,000 536,264 54,800,000

Environmental BRR Growth factor

3.15 2.96

0.97 0.45

0 2

5.09 3.3

Outputs Deposits (1000) Loans (1000) Fee income (1000) Deposit spread (1000) Loan spread (1000)

143,332 286,985 2673 4802 5202

145,410 295,817 2602 4509 4287

6,220 32,493 192 358 925

795,896 1,293,645 17,728 25,225 21,932

Panel A: Production model Inputs Managers Account managers Assistants Secretaries Cash managers IT expense Rent Other NIE

Panel B: Strategic model Inputs Managers Account managers Assistants Secretaries Cash managers IT expense Rent Other NIE Non-accrual loans

branches have over customer demand for services. They also yield scores and targets consistent with managementÕs objective of improving staff efficiency at the current levels of service. At the same time, as the Canadian economic environment

57,800–120,000 44,200–78,515 20,127–45,333 22,900–34,930 25,000–56,975 1 1 1 0.3–1.2

20% 15 15 20 20 18

continued to improve during the last half of the 1990Õs, managementÕs priorities started to shift from a strict focus on cost saving towards an interest in maximising revenue. Output-oriented models, with benchmarks based on potential out-

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

put enhancements, are used to address this need. In fact, we decided to look at both the input-oriented and output-oriented models. Thus, an inefficient branch whose location offers good potential to attract more business may attempt to improve its performance by reaching the output-enhancement target. Conversely, if its location prohibits any further output-enhancement, inputs should be reduced according to the cost-minimising target. We will expand on this topic in Section 5. 3.2.4. Model comparison In order to compare the results of different radial efficiency models, we use the spread ratios introduced in Schaffnit et al. (1997). In order to evaluate a given DMU0 having respective scores hA and hB under two input radial models A and B to be compared, we define the spread ratio of model A to model B for this DMU as rAB ðDMU0 Þ ¼

hA ðDMU0 Þ : hB ðDMU0 Þ

For example, if models A and B are, respectively, the CRS and VRS models, rAB measures scale efficiency. Similarly, if the models compute overall (cost-minimising) efficiency and technical efficiency (such as the allocative DEA and the VRS model respectively), rAB captures the amount of pure allocative efficiency. The spread ratio can also be used to assess models with different numbers of variables, multiplier constraints, technologies, data points, etc. In general, if model A is more ‘‘restrictive’’ than model B (fewer variables, more constraints,. . .), rAB will be less than 1. In this study, we use the spread ratios to assess scale efficiency and the effects of introducing knowledge about the trade-offs. We also use it in a modified fashion to compare the two sets of factors analysed (strategic and production models).

4. Results 4.1. Strategic model In order to explore the effect of introducing values or prices, the additional weight constraints mentioned above have been incorporated gradu-

727

ally. The basic (unconstrained) DEA model yields a technical efficiency measure. Incorporating output multiplier constraints based on managementÕs value system yields an overall effectiveness measure. The model with both input and output multiplier constraints yields an overall cost-effectiveness measure. Table 2 gives a summary of the results for each of these three models. For the basic strategic model, 62% of the branches are technically efficient, with an average 6% technical inefficiency for the 90 branches. Please note that a detailed efficiency study, under different circumstances, would necessitate an analysis of the slacks that are not accounted for in this radial score. This is not done here, since our final model includes multiplier constraints on both the input and output sides, which result in all slacks being zero because of the complementary slackness conditions. We also looked at the CRS formulation for this basic model: about 71% of the efficient branches appear to operate under constant returns to scale. In order to account for the scale effects, we used the VRS model for the rest of the analysis.

Table 2 Efficiency, effectiveness and cost-effectiveness DEA results for the strategic model Efficiency

Effectiveness

Costeffectiveness

% Technically efficient branches Average VRS score

62

37

19

0.94

0.86

0.51

% Technically and scale efficient branches CRS % among the technically efficient branches Average CRS score

44

19

6

71

52

29

0.81

0.54

0.20

0.91a

0.59a

0.50a

0.08a

Spread ratio technical efficiency: average Minimum a

With respect to the previous column.

728

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

D8 also appear to be overall better performers. This analysis proved to be of great interest to senior management, showing a combination of anticipated results and surprising ones, which they proceeded to investigate further.

The model with output multiplier constraints identifies 37% of the effective branches, and an average 14% of radial inefficiency. Only about half of these effective branches (19% of the total sample) are found cost-effective (with all multipliers constrained), and the average inefficiency is 49%. As expected, the branches which are most affected by the additional constraints on the input multipliers are those using unusually high levels of at least one input. With more flexibility, these inputs had been assigned very low multipliers (relative to the other inputs) that are no longer allowed with the additional constraints. The DEA scores and the spread ratios provide a convenient way of pinpointing ÔoutliersÕ in the data. We also examined the distribution by districts. Indeed, at the top level, the whole commercial branch network is managed from the head office. At the other end, the branch manager is responsible for day-to-day operations, but there is an additional management level in each district. Hence, management was interested in looking at performance at the district level. Table 3 presents the corresponding results. District D1 has the highest number of effective branches (70%), and a high average cost-effectiveness (0.91). But as we anticipated, and confirmed by an additional analysis not incorporating the environmental growth factor, these extremely favourable results are largely due to the fact that this district has the lowest growth factor. Given the restriction on the choice of peers imposed by this environmental factor, D1 branches are compared only to themselves. This illustrates well the problem one can encounter when both environmental factors and a relatively low number of DMUs are involved in the work–– results need to be carefully scrutinised for data related problems such as this one. Districts D5 and

4.2. Cluster analysis While it is an important step of the benchmarking process to identify the relatively efficient branches, the Bank is sometimes interested in going further in pinpointing the most valuable bestpractice branches. They may want to schedule focused on-site audits to identify which practices allow this specific branch to be among the best and, if they are transferable, attempt to reproduce these good practices in similar branches. DEA can help identify the most valuable branches in this respect through a cluster analysis of the ks and the best-practice branches can be ranked according to the frequency of their occurrence as peers of the less efficient branches. Out of the 17 branches deemed cost-effective, six are most widely used as peers (at least 10 times) by less cost-effective branches, with average peer coefficient (ks) ranging from 0.2 to 0.5. These will be first candidates for audit. The cluster analysis has an additional benefit: it provides a way of checking whether the top branches in terms of their frequency of use as peers are ÔlegitimateÕ. As it happened in a preliminary analysis for this project, such a branch may happen to be extremely specific, because of the very focused nature of its market for instance (large real estate lending or oil and gas exploration), or due to other factors not captured in the model. If such specific circumstances are anticipated at the outset of the study, the branch should not be included in

Table 3 Overall results by district for the strategic model District

D1

D2

D3

D4

D5

D6

D7

D8

Growth factor # Branches # Effective % Effective Average score

2 10 7 70 0.91

2.5 12 1 8 0.4

3.3 27 1 4 0.38

3.3 13 1 8 0.36

2.7 6 4 67 0.83

2.5 2 0 0 0.36

2.9 8 1 12 0.62

3.3 12 2 17 0.6

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

the initial reference set. Sometimes, a preliminary analysis is needed to pinpoint special branches, which ought to be subsequently removed. If the frontier is too heavily ‘‘tilted’’ towards such branch(es), the DEA results may be distorted. 4.3. Production model, and model comparison With the production model, based on physical work flow, we determined the technical efficiency and the cost-effectiveness of each branch in their use of resources to deliver the required services. A summary of the results is presented in Table 4. Approximately 57% of the branches are technically efficient, and the basic model identifies about 4% of radial inefficiency. With additional input multiplier constraints, about a quarter of the branches are found cost-effective, and the average cost-effectiveness is 0.79. The strategic and production models used here differ by the set of factors they incorporate: the strategic model does not include the five types of connections but has the spreads as outputs, and has nal as an input. (Also the DEA models used to assess cost-effectiveness do not include any information on trade-offs between the outputs for the production model.) Given these differences, the scores obtained in each case have different meaning, and are not comparable per se. However, one

Table 4 Efficiency and cost effectiveness DEA results for the production model % Technically efficient branches Average VRS score % Technically and scale efficient branches CRS % among the technically efficient branches Average CRS score

Efficiency

Cost-effectiveness

57

23

0.96

0.79

54

16

96

67

0.89

0.79

Spread ratio technical efficiency: average Minimum a

With respect to the previous column.

0.82a )0.47a

729

can gain some insight into a given branch operations by comparing its performance achievements for each model. First, we looked at the two basic models (unrestricted multipliers): 86% (48) of the 56 branches found technically efficient under the strategic model are also efficient under the production model, and eight other branches appear as efficient under the production model. This high level of consistency is not surprising given the similarities between the two models. But, the discrepancies are interesting in the sense that they can guide management in their exploration of the data. For instance, in order to gain further insight into the cost-effectiveness behaviour of the branches, we calculated the ratio between the Ôstrategic cost-effectivenessÕ and the Ôproduction cost-effectivenessÕ. It is not strictly speaking a spread ratio as defined above, because this ratio can be less or greater than unity. However, due to the additional output multiplier constraints in the strategic model, 68 branches have a lower score under this model (13 of them have a higher score and nine the same); for these branches, the ratio varies between 0.08 and 0.99, averaging at 0.49. A branch with a significantly higher score under the production model than under the strategic model appears, relatively to the other branches, more effective at using its resources to provide services than at satisfying managementÕs requirements for the strategic model (spreads and trade-offs between outputs). It is thus likely to exhibit at least one of the following features: • it has a relatively high amount of non-accrual loans at its current output level; • it yields relatively low spread values, or • its outputs are associated with unreasonable weights and if trade-offs on outputs based on resource usage were available, its score would significantly drop. A look at the corresponding data will clarify the difference in most cases: the branch with the lowest ratio of 0.08 has the second highest level of nal and is significantly smaller than the one having the highest level. A note of caution for analysts is that right at the beginning of the work, data

730

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

verification must be performed. It is well understood that outliers, for example, may be outliers because of data errors. Many remove outliers from the dataset without examining the possibility of data errors and correcting them before the final runs are made. We double-checked all data at the outset to eliminate such problems.

5. Alternative paths towards the frontier 5.1. Output-oriented models As mentioned above, input-oriented models are a natural choice to assess the branches since, in general, they assume no direct control over the amount of services their customers require. Nevertheless, it is quite likely that branch staff can effect their business levels (outputs) by actively soliciting new business and keeping existing business in place. Hence, when the BankÕs strategy includes growth as an objective, it makes sense to evaluate the branches in terms of their ability to generate outputs. Thus, if the current output levels do not justify current relatively high input values, one may consider increasing output production while staying at the current input level. In order to evaluate a realistic output-enhancement target, other factors have to be taken into account to capture market constraints. Socio-economic measures of the branchÕs neighbourhood, e.g. the number of firms, could be used to describe the potential for business. A competition index can also be factored in to represent activities taken by competitors that may have adverse effects on market share and success. Hence, managementÕs judgement based on their experience will be needed to assess whether the targets obtained from DEA are achievable or not. This analysis with output-oriented models was completed in order to get a first estimate of the branchesÕ ability to generate outputs. The targets obtained here indicate an upper bound of potential output enhancement based on the performance achieved by the best-practice branches identified through the strategic model used. Consistent with the input-oriented model, we introduced output prices based on managementÕs current value sys-

tem. In doing so, we evaluated the branchesÕ ability to fulfil managerial output-enhancement objectives. (Their revenue-generation ability could be studied by using prices reflecting the dollar value of the various outputs). The inputs are valued according to their market prices. Using the same set of multiplier constraints as in the inputoriented model also enables us to deal with the same best-practice frontier and consider alternate paths towards the frontier. This model thus identifies the same number of effective branches, 19% of the total sample. The output-maximisation effectiveness scores range from 1 to about 7, with an average of 2.5. Clearly, we cannot conclude that on average, the branches should more than double their output values; however, it directs management attention towards those branches which ought to substantially grow their business (outputs), given the choice of peers identified through this model. 5.2. Choice of path to move towards the frontier For each branch, we looked at the scores obtained with both the input-oriented and the output-oriented models. The corresponding targets (based on the peers) identify two alternate ways of moving towards the frontier. It is up to management to decide which one, or which combination of the two, may be appropriate. • Ineffective branches whose location allows good potential to attract more business should focus on output-augmentation to increase their market share. This can be accomplished through a better focus on marketing their services as well as improving customer satisfaction. If these ineffective branches manage to increase their outputs towards the target identified in this study, their input levels may be justified. • On the other hand, if a branch location prevents any further output enhancement, inputs should be reduced according to the cost-minimisation target. We suggest, based on past experience, that in practice a middle ground between these two paths may yield the maximum results.

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

6. Communication of the results Obtaining meaningful performance evaluation results is certainly an important step. The ability to explain them across the organisation is arguably even more important, otherwise the results–– however beneficial, will not find their way to proper implementation. With this objective in mind, we designed, in collaboration with the Bank, individual reports for each inefficient branch gathering the most relevant DEA results. 6.1. Individual reports Individual reports, such as the one shown in Fig. 1, were designed to provide the information pertaining to the performance of an ineffective

731

branch. First, the scores are given for the two orientations. The corresponding level of inefficiency is illustrated through the bar chart at the top of the figure showing both the amounts of potential savings to be gained, and the potential output enhancement expected if the branch was to become effective. The pie chart represents the relative importance of the peers. For example, P1 has a major influence on Branch B in the output orientation model with a 0.81 share of the target definition. More information is provided in a table listing the input and output values of each of the significant peers; it also shows the current and target values of the branch under study. The table is divided into two sections (left and right of the current data column), one for each path towards the frontier.

Fig. 1. Individual report for Branch B.

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

6.2. Comment on target interpretation As discussed above, we first obtained the technical efficiency scores with a basic DEA model. Then, overall efficiency was measured with the incorporation of economic values or particular managerial goals, through additional constraints involving trade-offs between the different variables. We can thus distinguish between two components of efficiency: • a technical efficiency as defined above; • an effectiveness component describing how well particular goals are being achieved; by adjusting the current input or output mix of a branch in light of preferred trade-offs, it can become more effective. The use of DEA models with additional multiplier constraints brings up the issue of target definition. Indeed, the peer-based target, and the one obtained through the radial score and the possible slacks, do not coincide for such models, due to the existence of the residuals (dual variables of the additional constraints). We decided to use the peer-based target, which we feel to be more relevant to, and easier to understand by, management. Fig. 2 was designed in an attempt to help a few selected managers at the Bank get a feel for this issue, in a simple 2 inputs/1 output case (at a fixed output level). Branch B can become technically efficient after a proportional reduction in inputs, moving it to the best-practice frontier at point E. In order to become effective, however, it has to satisfy the cost-minimisation objective and be located on the minimum cost dashed line. 2 The minimum cost dashed line is a line parallel to the pre-defined iso-cost line representing managementÕs preferred trade-offs/slopes. In order to realistically reach the minimum cost line, B must move along the best-practice frontier and adjust its input mix to the level of its effective peer A. In-

2 Note that in order to simplify the graph, this discussion assumes fixed prices; it also applies for price ranges, as used in this study, by replacing the cost-line by a set of lines, with slopes satisfying the corresponding ranges.

Best practice frontier

Input 2

732

Effective (cost minimising) branches

M ini

m um

Iso -c o

co s

Peer-based target t li

ne

Branch B E

A st

lin

e

F

C

Efficient target

Proportional reduction in inputs in order to be effective O

Input 1

Fig. 2. Target definition.

deed, based on the example used here, F is unachievable as it lies outside the production possibility set defined by the empirical frontier. Nevertheless, the distance EF represents the improvement in effectiveness. • The target considered will thus be the empirical, feasible, effective combination of the peers (A on the graph). It should be noted that this implies that the target value for some of the inputs may actually be higher than the actual data (take C as an example); the focus here is to generate a target that is overall more cost-effective. • Two scores will be considered. The efficiency score (OE/OB) represents the distance to the efficient frontier. The overall effectiveness score (OF/OB on the graph) reflects the distance to the iso-cost line intersecting with the best-practice frontier. (The pure effectiveness score would be the ratio of these two scores, represented by OF/OE. Here, ‘‘effectiveness’’ refers to overall effectiveness.) 6.3. Gap maps Novel gap maps were designed with the specific goal of illustrating the potential input reduction for the input-oriented model, and the potential output enhancement for the output-oriented model. They are presented as bar charts shown in Fig. 3. The bar charts display the ratio of current data for an inefficient DMU (in this case Branch B) over its target values based on its peers. For the

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

733

Fig. 3. Gap map for Branch B.

top chart, the focus is on potential input reduction, reflected by the portion of the bars located above the line data/target ¼ 1.0. Due to the presence of additional multiplier restrictions incorporated into the LP (and the corresponding residuals), some input targets may be higher than current data (bars below the line), and some output targets may be lower. Compared to its peers, Branch B exhibits high rental expenses and other non-interest expenses, and has an unusually high nal value. The bottom chart focuses on potential output enhancement, represented by the gap between the output bars and the line data/target ¼ 1.0. Again, in a standard radial DEA model all input targets would be equal to or, in the presence of slacks, less than current values. Based on its peersÕ performance, this branch is expected to improve all outputs, but particularly increase its level of loans.

7. Concluding remarks In this paper, we present an interesting study of the commercial network of a large Canadian bank with an emphasis on the strategic relevance of the models and results. Bank management accepted our findings and based on them made several changes in the way they evaluate their branches. They even involved us in a series of targeted onsite audits in branches we selected based on the analysis, in order to start identifying performance drivers and to plan an implementation phase based on our results. Furthermore, there were some attempts to actually establish a group of analysts within the Bank to carry on with the work we began. After some consideration and even hiring one of our graduates, the plan was not carried out due to other priorities (and some budget cuts at the

734

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

time). Instead, we continued with the work and for some time carried out specific DEA studies for them. Following the reassignment of the two key bank people, the program was halted temporarily but since then a new approach was instituted to serve this need. In fact, their program has recently (2002) began to re-incorporate DEA into the bankÕs performance measurement methodology. We illustrated again the usefulness of DEA in providing much more than efficiency measures in its ability to capture multiple dimensions. It yields fact-based information on best practices, and helps set specific, achievable targets for inefficient units. It can be used to identify potential savings and most profitable ways to improve performance. Different scenarios can be considered depending on whether the organisationÕs current objective is to minimise costs, or to implement a growth phase. We have shown that DEA results can be presented as visually appealing reports of direct relevance to management. This is a major achievement because the main reason DEA has not been accepted by the practitioner community is precisely because in the typical situation neither line nor senior management really understands the outcomes from the analysis. However, we must caution the reader that although this paper may give the impression that DEA can be applied easily or that it is an ‘‘ideal’’ system for this purpose, there are many pitfalls and difficulties to be dealt with in the process of implementing DEA in a real situation. Specifically, DEA requires a consistent ‘‘culture’’ for all DMUs and this can be a problem even when all of the DMUs are part of the same firm. We had to adjust for risk and provincial economic conditions before credible results could be presented. Moreover, no matter what one does, there are specific local differences between DMUs (bank branches) which can not be factored into the models. One has to rely on senior management to understand and allow for these in their implementation programs. Appropriate multiplier constraints can be a real challenge because these require consensus between managers with potentially conflicting goals (branch vs. senior level managers), hence in addition to well designed models one must also be an accomplished diplomat to successfully apply DEA in the real world.

At a higher level, an integrated performance measurement system can be built, encompassing several dimensions such as customer satisfaction, internal business, financial, and innovation and learning, as included in the balanced scorecard approach of Kaplan and Norton (1996). Rouse et al. (1997) discuss how DEA can be incorporated in a general managerial framework for performance measurement and we have shown here that this is, indeed, quite achievable. A positive side-benefit of using a multi-dimensional tool like DEA is the modelling of the process in collaboration with management, when they have to focus on the issues at hand to define the model. The important aspects of the process have to be identified in order to select the relevant factors; the choice of orientation will question managementÕs views on the strategic objectives of the organisation. The most challenging and productive discussions encountered in this study were perhaps those pertaining to the choice of output multiplier restrictions for the strategic model. These represent managementÕs judgement of their relative importance, and their incorporation provides the basis for implementing a sense of direction, with performance measures supporting current strategic goals. The difficulty of quantifying some of these priorities has been recognised in past literature, and reiterated in Allen et al. (1997). This area could be explored further, as well as the issue of target definition in the presence of additional multiplier restrictions. The BankÕs way of assessing performance at the time of this study did not have the rigour and flexibility of DEA, and finding the correct balance between corporate objectives and the individual unitÕs considerations led to interesting discussions.

Acknowledgements The authors would especially like to thank Tara M., as well as Archie M., Andre M., Paul M. and Peter M. for their invaluable participation in the project, and Bruce J. for his support of DEA applications within the Bank. We also wish to thank David Reese and Gloria Yan for fruitful discussions and research assistance. This research was

J.C. Paradi, C. Schaffnit / European Journal of Operational Research 156 (2004) 719–735

made possible by the generous financial support of SSHRC and NSERC of Canada, Toronto Dominion Bank, Royal Bank of Canada and Bell Canada. References Allen, R., Athanassopoulos, A., Dyson, R.G., Thanassoulis, E., 1997. Weight restrictions and value judgments in DEA: Evolution, development and future directions. Annals of Operations Research 73, 13–34. Athanassopoulos, A.D., Giokas, D., 2000. The use of data envelopment analysis in banking institutions: Evidence from the Commercial Bank of Greece. Interfaces 30 (2), 81–95. Banker, R.D., Maindiratta, A., 1988. Models for the estimation of technical and scale inefficiencies in data envelopment analysis. Econometrica, 1315–1332. Banker, R.D., Morey, R.C., 1986. Efficiency analysis for exogenously fixed inputs and outputs. Operations Research 34 (4), 513–521. Banker, R.D., Thrall, R.M., 1992. Estimation of returns to scale using data envelopment analysis. European Journal of Operational Research 62, 74–84. Banker, R.D., Charnes, A., Cooper, W.W., 1984. Models for estimating technical and scale efficiencies in DEA. Management Science 30 (9), 1078–1092. Berger, A.N., Humphrey, D.B., 1997. Efficiency of financial institutions: International survey and directions for future research. European Journal of Operational Research 98, 175–212. Camanho, A.S., Dyson, R.G., 1999. Efficiency, size, benchmarks and targets for bank branches: An application of data envelopment analysis. Journal of the Operational Research Society 50, 903–915. Canadian Bankers Association. Canadian Bank Facts––Fast Stats, 2002 ed. Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the efficiency of decision making units. European Journal of Operational Research 2 (6), 429–444. Charnes, A., Cooper, W.W., Huang, Z.M., Sun, D.B., 1990. Polyhedral cone-ratio DEA models with an illustrative application to large commercial banks. Journal of Econometrics 40, 73–91. Cook, W.D., Hababou, M., Tuenter, H.J., 2000. Multicomponent efficiency measurement and shared inputs in data envelopment analysis: An application to sales and service performance in bank branches. Journal of Productivity Analysis 14, 209–224. Cooper, W.W., Seiford, L.M., Tone, K., 2000. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software. Kluwer Academic Publishers, Dordrecht.

735

F€are, R., Grosskopf, S., 1994. Estimation of returns to scale using data envelopment analysis. European Journal of Operational Research 79, 379–382. Farrell, M.J., 1957. The measurement of productive efficiency. Journal of the Royal Statistical Society A 120, 253–281. Golany, B., Storbeck, J.E., 1999. A data envelopment analysis of the operation efficiency of bank branches. Interfaces 29 (3), 14–26. Kantor, J., Maital, S., 1999. Measuring efficiency by product group: Integrating DEA with activity-based accounting in a large Mideast bank. Interfaces 29 (3), 27–36. Kaplan, R.S., Norton, D.P., 1996. Balanced Scorecard: Translating Strategy into Action. Harvard Business School Press. Lovell, C.A.K., Pastor, J.T., 1997. Target setting: An application to a bank branch network. European Journal of Operational Research 98, 290–299. Oral, M., Yolalan, R., 1990. An empirical study on measuring operating efficiency and profitability of bank branches. European Journal of Operational Research 46, 282–294. Ray, S.C., 1991. Resource-use efficiency in public schools: A study of connecticut data. Management Science 37 (12), 1620–1628. Rouse, P., Putterill, M., Ryan, D., 1997. Towards a general managerial framework for performance measurement: A comprehensive highway maintenance application. Journal of Productivity Analysis 8, 127–149. Ruggiero, J., 1996. On the measurement of technical efficiency in the public-sector. European Journal of Operational Research 90 (3), 553–565. Schaffnit, C., Rosen, D., Paradi, J.C., 1997. Best practice analysis of bank branches: An application of DEA in a large Canadian Bank. European Journal of Operational Research 98, 269–289. Sherman, H.D., Gold, F., 1985. Bank branch operating efficiency: Evaluation with data envelopment analysis. Journal of Banking and Finance 9, 297–315. Sherman, H.D., Ladino, G., 1995. Managing bank productivity using data envelopment analysis. Interfaces 25 (2), 60– 73. Soteriou, A., Zenios, S.A., 1999. Operations, quality, and profitability in the provision of banking services. Management Science 45 (9), 1221–1238. Sueyoshi, T., 1999. DEA duality on returns to scale (RTS) in production and cost analyses: An occurrence of multiple solutions and differences between production-based and cost-based RTS estimates. Management Science 45, 1593– 1608. Thompson, R.G., Singleton Jr., F.D., Thrall R.M, Smith, B.A., 1986. Comparative site evaluations for locating a high energy physics lab in Texas. Interfaces 16, 35–49. Zenios, C.V., Zenios, S.A., Agathocleous, K., Soteriou, A.C., 1999. Benchmarks of the efficiency of bank branches. Interfaces 29, 37–51.