- Email: [email protected]
Benchmarking for Distribution Utilities: A Problematic Approach to Defining Efficiency
Benchmarking for Distribution Utilities: A Problematic Approach to Defining Efficiency
Veronica Irastorza is a Senior Analyst at National Economic Research Associates, Inc. (NERA) in San Francisco. She specializes in matters involving electric industry restructuring, regulation, and pricing. She received an M.P.P. at the University of California at Berkeley and a B.S. in Economics from the Universidad Iberoamericana in Mexico. She can be contacted at [email protected]. This article expresses solely the views of its author, and she accepts full responsibility for the opinions and any errors contained herein. The author thanks Hethie Parmesano and Sarah P. Voll for their helpful comments.
30
Benchmarking for ratemaking purposes may not be feasible. A proper benchmarking analysis would be extremely burdensome since it would be necessary to carefully review all the costs for all the companies included in the benchmarking study. Regulators who attempt to simplify the methodology to render it more manageable risk making arbitrary judgments that confuse inefficiency with heterogeneity and that are potentially harmful to companies who cannot ‘‘improve’’ their way out of fundamental differences in circumstances. Veronica Irastorza
I. Introduction Benchmarking in its simpler forms has been around in the U.S. for decades as a way to measure performance. Around the world, it is receiving new attention, especially in Europe where it is being used as a regulatory tool to set the X-factor in price cap regimes. European
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
regulators are adopting new techniques that are apparently more sophisticated than those that have been employed in the U.S. However, the application of benchmarking as a way to set the X-factor has many important problems, no matter how ‘‘sophisticated’’ the techniques are: It is subjective, lacks transparency, foments disputes, The Electricity Journal
and puts utilities at financial risk. For these reasons, I argue that benchmarking has very limited value for ratemaking. his article reviews several benchmarking methods in the context of price regulation for electricity distribution companies to point out the problems with these methods.1 I do not discuss quality of service, although quality standards are usually a component of incentive regulation schemes. Quality standards often accompany such schemes in the guise of protecting consumers from cost reductions that might come at the expense of service quality.
T
II. Roles for Benchmarking The goal of traditional regulation, especially of distribution utilities, is to balance the interests of consumers and those of investors by ensuring that utilities provide adequate service at reasonable prices, while remaining financially viable and receiving clear investment incentives.2 Independent regulators of investor-owned utilities have emphasized the importance of transparent, robust, and objective methods for setting the revenue requirement: Only if investors understand how the method works and trust its objectivity will they be willing to invest in the utility and thereby provide sufficient funds for maintenance and expansion of the system. December 2003
Traditionally, regulators have based electric rates on the cost of providing service. Under cost-of-service methodologies, rates are based on the utility’s actual expenses or projected known and measurable expenses (sometimes with disallowances for expenditures the regulator finds unreasonable or not related to utility service) plus a return on investment (sometimes with exclusions of investment
Independent regulators have emphasized the importance of transparent, robust, and objective methods for setting the revenue requirement.
considered imprudent) that reflects the cost of capital. Because it is difficult for regulators to gauge the reasonableness of investment and expense levels, cost-of-service regulation has been criticized for not providing incentives for cost savings and efficiency improvements and, in the rate-of-return regulation (ROR) of investorowned utilities, for rewarding over-investment. n an attempt to increase the incentive to improve efficiency and reduce the incentive to over-invest, regulators around the world have used various forms of benchmarking, which set revenue
I
requirements (at least in some years) based on some measure of ‘‘efficient’’ costs rather than on the utility’s actual costs.3 They have sought ways to compare a specific distribution utility’s efficiency level with some reference level in order to estimate the reasonableness of the utility’s costs and to set rates. Benchmarking, which is the comparison of one utility’s costs or other characteristics to those of other utilities, is an attempt to develop this reference and regulators have tried or proposed to use benchmarking of various sorts in a variety of rate setting applications:4 In the U.S., benchmarking in its simpler forms has been used to measure performance and/or provide incentives in ROR ratemaking both for overall rate levels and for specific elements of costs or outputs (e.g., fuel procurement or customer satisfaction). Mississippi Power, for example, uses a comparison between its average retail price per kWh and the weighted average price of the utilities in the Southeastern Electric Exchange to adjust the company’s performance-based return on investment.5 Regulators in the U.K., the Netherlands, and Australia have used benchmarking as a means of setting the X-factor in their price cap methodologies and this application is also getting new attention elsewhere in Europe. The usual incentive regulation price (or revenue) cap formula takes the form of CPI-X and
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
31
allows rates to rise by the inflation index less ‘‘X,’’ which is an assumed rate of improvement in the utility’s productivity. If the utility improves efficiency faster than X it can keep the difference in profits. owever, regulators have found that a like-with-like comparison of performance in the same market is not possible for regional monopolies, since costs and quality are usually different for legitimate reasons. Benchmarking can be misleading if it consists simply of comparing costs for companies without accounting for differences in factors that affect their costs, such as variations in customer base (number and customer classes), population density, terrain (hills or flat, vegetation, etc.), consumption patterns (peaks), etc. Overly simplistic benchmarking may result in inefficient utilities making profits and efficient utilities not being able to recover their prudently incurred costs. In an attempt to account for these differences, several more sophisticated approaches have been created.6 Nevertheless, these ‘‘sophisticated’’ approaches have problems in and of themselves as I discuss in the next section.
H
III. Approaches Benchmarking methods can be broadly divided in two groups: average and frontier methods. Average methods are those that compare the target utility to some measure of average 32
performance, while frontier methods compare the target utility to the most ‘‘efficient’’ comparable utilities. Some of the most common methods are described below. A. Average methods 1. Simple comparisons. The simple comparisons approach is a partial performance measurement method. It relies on simple
Regulators have found that a like-with-like comparison of performance in the same market is not possible for regional monopolies.
relative performance indicators, usually ratios of a single input and single output (i.e., MWh distributed per employee). Comparisons are generally against the average, but can also be against the best result. Simple comparisons are relatively inexpensive and easy to implement, but are also very limited in scope. They do not account for relationships among different inputs and outputs; therefore, a utility that does relatively well on some indicators may do relatively badly on others, while another utility may have the opposite results. It then is necessary to assign weights to each
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
indicator; however this can be complicated and subjective. Because of their simplicity, simple comparisons will be misleading if the choice of benchmark companies includes utilities that have characteristics very different from the subject utility. 2. Ordinary least square (OLS). Regulators sometimes use regression analysis to try to estimate differences in productivity among a group of utilities. A regression is used to estimate an average production or cost function of sample firms and the costs of an individual utility are compared to the estimated average costs of a group of utilities. The regression line represents the supposed average efficiency level. All companies above the average are considered inefficient while all those below are efficient. The main criticisms of the regression method are that (1) it requires comparable data for each company in the group, (2) the results can be sensitive to the functional specification (choice of variables, functional form, etc.) and, most importantly, (3) residuals measure not only inefficiencies but also unexplained factors and data errors—which is to say that any residual cannot be assumed to have anything to do with relative efficiency levels. B. Frontier methods (DEA, COLS, SFA) The Netherlands, U.K., Norway, and other countries have experimented with frontier methods to The Electricity Journal
set their X-factors. Frontier methods purport to estimate the best practice or most efficient performance level by creating an efficiency frontier from a sample of utilities. This frontier is used as the benchmark against which performance of the subject utility is measured. Frontier methods are based on the concept that all utilities in a selected sample should be able to operate at an efficiency level determined by the most efficient utilities in the sample. Frontier methods compare ratios of inputs and outputs, and the utilities on the frontier are those that use the minimum inputs to produce a given level of output. egulators have used frontier methods to set X-factors for individual companies based on their relative inefficiency. The purported efficiency is generally measured between zero and one, with the most efficient firm’s score equal to one. A utility with an efficiency score of 0.8 could presumably reduce costs (or increase output) by 20 percent. The main problem with these methods is that they cannot distinguish between efficiency, omitted variables, and data errors in their error terms. Most frontier methods assume implicitly—but unreliably—that high costs are due to inefficiency. In fact, utilities’ costs may lie above a subjective frontier due to any number of factors not captured in the analyses. A utility wishing to defend itself against
R
December 2003
the accusation that it is inefficient must identify the special factors that account for its deviation from the frontier. This requires detailed information about the other utilities in the sample. And while testing such factors is not difficult, finding the necessary information may be impossible. 1. Data envelopment analysis (DEA). In DEA, the purported efficiency of a utility is computed using linear programming. DEA can use either production or cost functions, but for electric distribution utilities, cost functions are usually the preferred choice since these utilities cannot choose the level of output and can only minimize costs for a given output. he method creates a frontier using information on the assumed most efficient firms and measures the relative efficiency of the rest of the utilities.7 DEA attempts to approximate the efficient frontier by a ‘‘piece-wise’’ linear approximation based on the sample. Efficiency scores are constructed by measuring how far a utility is from the frontier. In general, best-practice efficiency is normalized to a score of one and then the efficiency of a particular utility is measured by how far it is from one. For example, a score of 0.75 is assumed to mean that the utility is 25 percent below the best practice. A score of 1 presumably means that the utility is efficient. However, it does not mean that the utility cannot improve its performance, but only that no
T
Figure 1: Simplified Illustration of DEA, Showing Cost and Output Relationships for Six Utilities
other utility in the sample is performing better. Figure 1, a simplified illustration of DEA, shows the cost and output relationships for six utilities. L, K, and M would be interpreted as efficient by DEA users, while the rest would not. J’s ‘‘inefficiency’’ is the distance between J and the frontier. DEA’s appeal is that it does not require that the underlying shape of the cost function be determined and therefore it does not impose any functional form on the relationships between inputs and outputs. However, the method has many more problems than advantages: a. Unexplained portions are regarded as inefficiencies DEA scores are prone to incorrect interpretation since the gap between a company’s score and a perfect score of 1 might be due to factors that were not captured in the model. A DEA score of 75 percent does not necessarily imply that 25 percent of the costs are due to inefficiency, but rather that the model fails to explain some of the actual costs. DEA scores measure deficiencies in the model as well as inefficiency in the utility. b. DEA is sensitive to the number of variables
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
33
DEA is sensitive to the number of inputs, outputs, and control variables. Adding variables never decreases DEA scores and therefore gives utilities a chance to better explain their own particular level of costs. If the model includes many variables, but only a small sample size, every utility may be able to call upon at least one of the output variables to explain its costs. Such a model would place many utilities at or near the frontier. To preserve variation in scores, the number of explanatory variables has to be restricted and such a modeling procedure does not accommodate all the specific factors that affect each utility’s costs. c. Outliers can be identified as efficient DEA identifies observations on a ‘‘frontier’’ and creates a comparator for each utility by identifying a point on the frontier between the nearest observations. However, some utilities are bound to be outliers. Their data place them on the edge of the group but this can be the result of exceptionally positive conditions (i.e., a distribution utility whose territory is mostly flat while the rest of the utilities in the group are hilly). Being on the edge of the group, such utilities are highly likely to be identified as lying on the frontier, whether they are efficient or not. d. DEA is biased against aggregated data DEA suffers an intrinsic bias against observations produced by aggregating data. For example, 34
combining two outliers with different characteristics will produce a merged utility that has mixed (more typical) characteristics. Since it is no longer so likely to be an outlier, it is also less likely to appear on (or even close to) the frontier. However, simply combining the data for two utilities does not change any of their real costs or efficiency levels. The model’s implication–that the two utilities have moved from being ‘‘efficient’’ to being ‘‘inefficient’’– does not reflect any real change in efficiency, and is merely a quirk in the methodology. e. DEA’s results cannot be tested statistically The results of DEA models cannot be tested statistically to see whether or not they are robust. This means that the reliability of the results is unknown. 2. Corrected ordinary least squares (COLS). In COLS the purported relative efficiency of a utility requires the specification of the cost function, since it starts with a regression. A regression is estimated using OLS and then shifted to create an efficient frontier. With COLS, only one firm of the sample is considered efficient. As in DEA, the frontier then is used to measure the relative efficiency of the rest of the utilities.8 Figure 2 shows the difference between OLS and COLS. In COLS the origin is decreased by the distance between the OLS line and the most efficient utility (A). In other words, the frontier is shifted until it envelops all data.
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
Figure 2: Simplified Illustration of COLS
The efficiency score for utility Z is the ratio WX/ZX. Similar to DEA, COLS assumes that all deviations are due to inefficiencies. A very important problem with COLS is that the results are highly sensitive to the single most efficient firm (A). 3. Stochastic frontier analysis (SFA). This method is similar to COLS, but SFA allows an estimation of random effects on efficiency and reduces the reliance on a single firm.9 SFA involves an assumed split of the error term (unexplained cost differences) between random errors and inefficiencies. Under the simplest form of SFA, a regression is estimated with two error components independent from each other. As in DEA and COLS, purported efficiencies are computed as the distance between the data and the efficiency frontier. This distance is usually shorter when calculated using SFA as opposed to COLS. owever, SFA does not solve the problem of deciding if unexplained costs are inefficiencies or not. The method just produces a split of the unexplained cost differences. SFA is prone to bias due to correlations between
H
The Electricity Journal
explanatory variables and the error terms. In fact, Ofwat (the U.K. water regulator) has criticized SFA for its fundamental reliance on regression analysis, noting that it requires a number of strong assumptions about the form of the relationship between expenditure and the explanatory factors.10
the objective (100 percent ‘‘efficient’’): If the company has five years to decrease its costs the annual X-factor would be about 7 percent as opposed to 35 percent if it only had one year. his example assumes a constant annual rate factor; however, other options include a high X-factor for the first year and regular factors afterwards,
T
IV. From Benchmarking to X-factors There is no standard method of converting benchmarking results of ‘‘efficient’’ cost levels into the annual rate of change defined by an X-factor. Regulators using benchmarking must not only determine a company’s inefficiency level (perhaps erroneously) but must also determine how long it should take for that company to reach the presumed efficient level. Thus, it is necessary to specify an end-date, and hence the number of years allowed for catching up, in order to convert a percentage cut in total costs into a percentage rate of allowed annual changes in prices or revenues. he X-factor takes different values depending upon the expected date to reach the target level (or assumed ‘‘efficient costs’’ level). For example a DEA score of 65 percent means that the utility should decrease costs by 35 percent to be at the ‘‘efficient cost’’ level. The 65 percent score could be converted to the following annual X-factors depending on when should the company reach
T
December 2003
of 0 to 3 percent per annum. In the U.K., the Office of Gas and Electricity Markets (Ofgem) assumed that the least efficient distribution utilities should move three-fourths of its distance to the frontier in two years and then retain that position until the next price review (three years later).12 This implied a drastic operating cost reduction for some utilities. Norweb, for example, was expected to lower its costs by almost 30 percent.
V. Applications
decreasing X-factors, increasing X-factors, etc. Practical experience shows that regulators converting efficiency scores into X-factors tend to adopt arbitrary standards, based on separate (and subjective) estimates of the feasible rate of change. The Norwegian Electricity Regulator (NVE) obtained scores (60–100 percent) very similar to those of the Dutch Regulator (DTe), but DTe converted them into much higher X-factors by assuming a much shorter ‘‘catch-up’’ period (three years instead of six).11 NVE established a common X-factor of 1.5 percent per annum, and augmented it by a ‘‘stretch factor’’ related to DEA scores
Benchmarking is a fairly new approach, so it is still early to determine the financial consequences it may have on the companies to which it has been applied. However, the three cases below illustrate the difficulties in implementing benchmarking and the potential problems it can create. A. Great Britain The application of benchmarking by Ofgem demonstrates that the technique will be ultimately subjective whenever the estimate of costs and manipulation of the frontier are subjective. Ofgem regulates electric distribution utilities under a PBR program. Every five years, Ofgem reanalyzes the relative efficiency of the whole sector and the individual companies to re-set the price cap for each company. The price reduction in the first year of the price control period (P0) and the
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
35
rate of decline for the subsequent years (X-factors) are determined in tandem. Ofgem forecasts the costs for the next period (using benchmarking as part of the equation) and determines the net present value of the costs. Then, it sets P0 and X so that the net present value of revenues equals net present value of cost. Since there are many possible combinations of P0 and X Ofgem decided to use a 3 percent X for all companies resulting in extremely high P0, as shown in Table 1. Ofgem attempted to establish the efficient level of operating expenditures (opex) for each utility using COLS. Capital expenditures (capex) were analyzed separately. The main rationale for separating operating and capital costs appears to be that it is easier to get more accurate cost models in this way, because the cost drivers for different types of cost are Table 1: Ofgem’s Extremely High Price Reductions 2000–2005 P0 (%)
X-factor (%)
East East Midlands
28 23
3 3
London Manweb
27 21
3 3
Midlands
23
3
Northern Norweb
24 27
3 3
Seeboard Southern
33 19
3 3
Swalec
26
3
South Western Yorkshire
20 23
3 3
Scottish Hydro
13 4
3 3
36
very different and the operating models are easier to generate than capital expansion and replacement models. These are good reasons for analyzing the cost categories separately, but this approach adds the problem of how to combine separate efficiency estimates. The tradeoffs between opex and capex mean that any one utility is highly unlikely to be able to achieve ‘‘best practice’’ efficiencies in every cost category separately. fgem has affirmed that there is a tradeoff between opex and capex costs, and agrees that this is an important issue that needs to be dealt with. Ofgem makes a very simplistic assessment of capital efficiency and then rewards more efficient utilities through the revenue allowance. They suggest that this reward for capital efficiency will match the extra opex that arises from lower capex and hence take into account any tradeoffs. This misses the point that the total cost allowed on the basis of both capital and operating efficiency targets may be unachievable. In other words, the company with the lowest operating costs is likely to be different from the one with the lowest capital costs and to have both could be almost impossible. A composite explanatory variable is used for operating costs with the weights of the component explanatory variables fixed by Ofgem. The basis of the weights appears to be an Ofgem ‘‘engineering judgment,’’ taking into account some of the utilities’ points of view, though Ofgem
O
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
also accepts that there are other explanatory factors that are not being captured. The weights appear not to have any statistical basis. The majority of network monopoly companies have even argued that the present arrangements give rise to distorted incentives and that this could lead to their adopting an inefficient mix of opex and capex.13 Even Ofgem now considers that the existing arrangements can provide companies with distorted incentives between opex and capex.14 B. Holland The problems with DEA specification and the impact on efficiency scores can be illustrated by an example from Holland. In 2000 the Dutch Electricity Regulatory Service (DTe) undertook a benchmarking exercise based on the DEA method to set the X-factors of the Dutch electricity distribution businesses, and therefore the revenues that the utilities were allowed to earn. DTe did not base revenues on direct measures of actual costs. Instead, DTe calculated the distributor’s estimated revenues in 2000 excluding non-controllable elements (e.g., rates charged by the transmission company) and then adjusted the estimated controllable costs to normalize the rates of depreciation and set a common rate of return. DTe then multiplied the utility’s controllable costs by their DEA scores to obtain target costs for 2003. The Electricity Journal
F
inally, DTe took the controllable costs calculated for 2003 and adjusted them to allow for a further rate of productivity growth, equal to an expected rate of decline in ‘‘efficient’’ costs of 2 percent per year.15 The final figure of total costs was compared with the 2000 figure to establish the annual rate of change necessary to achieve the new level by 2003. The X-factor was defined as this annual rate of change. Many electricity distribution companies contested the calculation of the X-factors and took DTe to court (and won). One of the distributors that appealed asked DTe to revise the dataset used. DTe agreed, adopted a new database, and re-standardized book values and depreciation terms. As a result, both the rate of return and depreciation components of the total controllable costs changed and the distributor’s efficiency score rose from 65 to 95 percent. The instability of the results demonstrates a fundamental problem in applying the DEA approach for ratemaking purposes. Presumably other companies with low scores would be able to make an equally persuasive case that the dataset should be revised, and obtain similarly dramatic improvements in their scores. C. New South Wales, Australia The experience of the Independent Pricing and Regulatory Tribunal of New South Wales December 2003
(IPART) illustrates the difficulties of modeling efficient costs using DEA and the risk that modeled results will lack credibility. As part of its first Electricity Distribution Price Review, IPART commissioned a DEA efficiency benchmarking study as an input to its inquiry into the prices of the monopoly electricity distribution businesses in New South Wales
for the five-year period from 2000.16 The database for the study included 219 distributors from Australia, New Zealand, England and Wales, and the U.S. The model included operating and management expenditures, Km of network route and MVa of transformer capacity as inputs. Energy delivered, customer numbers, and peak demand were specified as outputs. The model does not recognize such factors as the location and quality dimensions of the electricity distribution process. The DEA results suggested that the six NSW distributors could proportionally reduce their inputs by between 56 percent and 34 percent, and still be able to pro-
duce the same level of output if they were ‘‘efficient.’’ These results were adjusted to take into account factors outside the control of the company (customer density, proportion of residential customers, etc.). Even when adjustments for environmental factors were made, ‘‘inefficiencies’’ of between 41 and 13 percent are identified, resulting in substantial controversy and debate over the accuracy of the results. IPART acknowledged that a ‘‘major limitation of DEA is that it suggests all inputs can be reduced proportionally.’’17 To reduce inputs proportionally is not always a viable option. For example, it assumes that distributors could reduce their networks route by miles and still offer a secure supply. n the end, IPART recognized the limitations of the benchmarking studies and gave it only limited weight in its final decision.18 Northpower & Integral Energy, for example, scored 0.65 and 0.60, respectively in the DEA study (implying a 35 and 40 percent price reduction to achieve ‘‘efficiency’’) and IPART required only a 15 percent price reduction over the five-year period.19
I
VI. Conclusions Benchmarking, even in its more sophisticated formulations, has been an interesting, but thus far unsuccessful, regulatory effort. While it is usually presented as an objective technique, its application in regulation has been
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
37
inherently subjective. The lack of robustness and transparency can result in contentious debates that make it difficult to implement. enchmarking for ratemaking purposes, that is, for actually setting tariffs, may not be feasible. A proper benchmarking analysis would be extremely burdensome since it would be necessary to carefully review all the costs for all the companies included in the benchmarking study. Regulators who attempt to simplify the methodology to render it more manageable risk making arbitrary judgments that confuse inefficiency with heterogeneity and that are potentially harmful to companies who cannot ‘‘improve’’ their way out of fundamental differences in circumstances. Even if implementation problems are solved, benchmarking can result in lower rates in the short run, but with increased financial risk for the companies and underinvestment that create problems in the long run. This is not to argue that benchmarking has no role in regulation. Comparisons can indeed provide useful information to both the utilities and their regulators: They can be warning signs of problem areas, which the regulator can use to request follow-up studies or data. They can be useful for the utility to analyze its own performance and find areas for improvement. They can signal situations beyond the control of the utility,
B
38
which if changed, could reduce costs to all customers.& Endnotes: 1. Other methods not discussed in this article include the ‘‘engineering economic model’’ used in Spain and Chile. This method uses a theoretically efficient company as a benchmark. 2. This article discusses distribution utilities in particular, although much
7. DEA was introduced by Charnes, Cooper and Rhodes in Measuring the Efficiency of Decision-Making Units, EUROPEAN J. OPERATIONAL RES., 1978, 2 (6). They extended Farell’s (1957) idea. See M.J. Farell, Measurement of Productive Efficiency, J. ROYAL STATISTICAL SOC., 1957, Series A, 120 part 3. 8. See J. Richmond, Estimating the Efficiency of Production, INT’L ECON. REV., 1974, 15 (2). 9. SFA was first developed by Aigner, Lovell, and Schmidt in 1977. Aigner, Lovell, and Schmidt, Formulation and Estimation of Stochastic Frontier Production Function Models, J. ECONOMETRICS, 1977 (6) 1. 10. Cabinet Office (U.K. Government), Improving Analysis and Modeling in Central Government, Case Studies, 2000. 11. For details see Norwegian Water Resources and Energy Administration (NVE), Regulation of Electricity Monopolies, Efficiency Analysis of Transmission and Distribution, slide presentation by Thor Martin Neurauter, Sept. 1999. 12. Ofgem, Review of Public Electricity Suppliers 1998 to 2000. Distribution Price Control Review. Final Proposal, Dec. 1999, at 21.
of the material also applies to vertically integrated utilities. 3. Most incentive regulation schemes operate between full tariff reviews, in which the traditional ROR approach is used. 4. Setting revenues according to other companies’ costs is also known as yardstick competition. See A. Schleifer, A Theory of Yardstick Competition, RAND J. ECON., Autumn 1985, 16 (3). 5. Measurements of customer satisfaction and customer service reliability are also part of the adjustment formula. For more details see Mississippi Power Performance, Evaluation Plan, Rate Schedule PEP-3, May 23, 2002, in Docket 2001-AD-0826. 6. In econometric approaches, heterogeneity actually helps to produce more reliable estimates of the impact of different conditions on costs. However, the source of error terms is still unknown.
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
13. Ofgem, Developing Network Monopoly Price Controls, Update document, Feb. 2003. 14. Id., at 29. 15. It also adjusted the controllable costs to allow for the imposition of tax from 2002. 16. For more information about the problems with the DEA analysis see NERA, A Review of DEA Benchmarking of the NSW Distribution Businesses, Report for Energy Australia, March 1999. 17. Independent Pricing and Regulatory Tribunal of New South Wales, Pricing for Electricity Networks and Retail Supply, Vol. I, June 1999, at 128. 18. Independent Pricing and Regulatory Tribunal of New South Wales, Regulation of NSW Electric Distribution Networks, Determination and Rules Under the National Electricity Code, Dec. 1999, at 67. 19. Id., at 65.
The Electricity Journal
Veronica Irastorza is a Senior Analyst at National Economic Research Associates, Inc. (NERA) in San Francisco. She specializes in matters involving electric industry restructuring, regulation, and pricing. She received an M.P.P. at the University of California at Berkeley and a B.S. in Economics from the Universidad Iberoamericana in Mexico. She can be contacted at [email protected]. This article expresses solely the views of its author, and she accepts full responsibility for the opinions and any errors contained herein. The author thanks Hethie Parmesano and Sarah P. Voll for their helpful comments.
30
Benchmarking for ratemaking purposes may not be feasible. A proper benchmarking analysis would be extremely burdensome since it would be necessary to carefully review all the costs for all the companies included in the benchmarking study. Regulators who attempt to simplify the methodology to render it more manageable risk making arbitrary judgments that confuse inefficiency with heterogeneity and that are potentially harmful to companies who cannot ‘‘improve’’ their way out of fundamental differences in circumstances. Veronica Irastorza
I. Introduction Benchmarking in its simpler forms has been around in the U.S. for decades as a way to measure performance. Around the world, it is receiving new attention, especially in Europe where it is being used as a regulatory tool to set the X-factor in price cap regimes. European
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
regulators are adopting new techniques that are apparently more sophisticated than those that have been employed in the U.S. However, the application of benchmarking as a way to set the X-factor has many important problems, no matter how ‘‘sophisticated’’ the techniques are: It is subjective, lacks transparency, foments disputes, The Electricity Journal
and puts utilities at financial risk. For these reasons, I argue that benchmarking has very limited value for ratemaking. his article reviews several benchmarking methods in the context of price regulation for electricity distribution companies to point out the problems with these methods.1 I do not discuss quality of service, although quality standards are usually a component of incentive regulation schemes. Quality standards often accompany such schemes in the guise of protecting consumers from cost reductions that might come at the expense of service quality.
T
II. Roles for Benchmarking The goal of traditional regulation, especially of distribution utilities, is to balance the interests of consumers and those of investors by ensuring that utilities provide adequate service at reasonable prices, while remaining financially viable and receiving clear investment incentives.2 Independent regulators of investor-owned utilities have emphasized the importance of transparent, robust, and objective methods for setting the revenue requirement: Only if investors understand how the method works and trust its objectivity will they be willing to invest in the utility and thereby provide sufficient funds for maintenance and expansion of the system. December 2003
Traditionally, regulators have based electric rates on the cost of providing service. Under cost-of-service methodologies, rates are based on the utility’s actual expenses or projected known and measurable expenses (sometimes with disallowances for expenditures the regulator finds unreasonable or not related to utility service) plus a return on investment (sometimes with exclusions of investment
Independent regulators have emphasized the importance of transparent, robust, and objective methods for setting the revenue requirement.
considered imprudent) that reflects the cost of capital. Because it is difficult for regulators to gauge the reasonableness of investment and expense levels, cost-of-service regulation has been criticized for not providing incentives for cost savings and efficiency improvements and, in the rate-of-return regulation (ROR) of investorowned utilities, for rewarding over-investment. n an attempt to increase the incentive to improve efficiency and reduce the incentive to over-invest, regulators around the world have used various forms of benchmarking, which set revenue
I
requirements (at least in some years) based on some measure of ‘‘efficient’’ costs rather than on the utility’s actual costs.3 They have sought ways to compare a specific distribution utility’s efficiency level with some reference level in order to estimate the reasonableness of the utility’s costs and to set rates. Benchmarking, which is the comparison of one utility’s costs or other characteristics to those of other utilities, is an attempt to develop this reference and regulators have tried or proposed to use benchmarking of various sorts in a variety of rate setting applications:4 In the U.S., benchmarking in its simpler forms has been used to measure performance and/or provide incentives in ROR ratemaking both for overall rate levels and for specific elements of costs or outputs (e.g., fuel procurement or customer satisfaction). Mississippi Power, for example, uses a comparison between its average retail price per kWh and the weighted average price of the utilities in the Southeastern Electric Exchange to adjust the company’s performance-based return on investment.5 Regulators in the U.K., the Netherlands, and Australia have used benchmarking as a means of setting the X-factor in their price cap methodologies and this application is also getting new attention elsewhere in Europe. The usual incentive regulation price (or revenue) cap formula takes the form of CPI-X and
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
31
allows rates to rise by the inflation index less ‘‘X,’’ which is an assumed rate of improvement in the utility’s productivity. If the utility improves efficiency faster than X it can keep the difference in profits. owever, regulators have found that a like-with-like comparison of performance in the same market is not possible for regional monopolies, since costs and quality are usually different for legitimate reasons. Benchmarking can be misleading if it consists simply of comparing costs for companies without accounting for differences in factors that affect their costs, such as variations in customer base (number and customer classes), population density, terrain (hills or flat, vegetation, etc.), consumption patterns (peaks), etc. Overly simplistic benchmarking may result in inefficient utilities making profits and efficient utilities not being able to recover their prudently incurred costs. In an attempt to account for these differences, several more sophisticated approaches have been created.6 Nevertheless, these ‘‘sophisticated’’ approaches have problems in and of themselves as I discuss in the next section.
H
III. Approaches Benchmarking methods can be broadly divided in two groups: average and frontier methods. Average methods are those that compare the target utility to some measure of average 32
performance, while frontier methods compare the target utility to the most ‘‘efficient’’ comparable utilities. Some of the most common methods are described below. A. Average methods 1. Simple comparisons. The simple comparisons approach is a partial performance measurement method. It relies on simple
Regulators have found that a like-with-like comparison of performance in the same market is not possible for regional monopolies.
relative performance indicators, usually ratios of a single input and single output (i.e., MWh distributed per employee). Comparisons are generally against the average, but can also be against the best result. Simple comparisons are relatively inexpensive and easy to implement, but are also very limited in scope. They do not account for relationships among different inputs and outputs; therefore, a utility that does relatively well on some indicators may do relatively badly on others, while another utility may have the opposite results. It then is necessary to assign weights to each
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
indicator; however this can be complicated and subjective. Because of their simplicity, simple comparisons will be misleading if the choice of benchmark companies includes utilities that have characteristics very different from the subject utility. 2. Ordinary least square (OLS). Regulators sometimes use regression analysis to try to estimate differences in productivity among a group of utilities. A regression is used to estimate an average production or cost function of sample firms and the costs of an individual utility are compared to the estimated average costs of a group of utilities. The regression line represents the supposed average efficiency level. All companies above the average are considered inefficient while all those below are efficient. The main criticisms of the regression method are that (1) it requires comparable data for each company in the group, (2) the results can be sensitive to the functional specification (choice of variables, functional form, etc.) and, most importantly, (3) residuals measure not only inefficiencies but also unexplained factors and data errors—which is to say that any residual cannot be assumed to have anything to do with relative efficiency levels. B. Frontier methods (DEA, COLS, SFA) The Netherlands, U.K., Norway, and other countries have experimented with frontier methods to The Electricity Journal
set their X-factors. Frontier methods purport to estimate the best practice or most efficient performance level by creating an efficiency frontier from a sample of utilities. This frontier is used as the benchmark against which performance of the subject utility is measured. Frontier methods are based on the concept that all utilities in a selected sample should be able to operate at an efficiency level determined by the most efficient utilities in the sample. Frontier methods compare ratios of inputs and outputs, and the utilities on the frontier are those that use the minimum inputs to produce a given level of output. egulators have used frontier methods to set X-factors for individual companies based on their relative inefficiency. The purported efficiency is generally measured between zero and one, with the most efficient firm’s score equal to one. A utility with an efficiency score of 0.8 could presumably reduce costs (or increase output) by 20 percent. The main problem with these methods is that they cannot distinguish between efficiency, omitted variables, and data errors in their error terms. Most frontier methods assume implicitly—but unreliably—that high costs are due to inefficiency. In fact, utilities’ costs may lie above a subjective frontier due to any number of factors not captured in the analyses. A utility wishing to defend itself against
R
December 2003
the accusation that it is inefficient must identify the special factors that account for its deviation from the frontier. This requires detailed information about the other utilities in the sample. And while testing such factors is not difficult, finding the necessary information may be impossible. 1. Data envelopment analysis (DEA). In DEA, the purported efficiency of a utility is computed using linear programming. DEA can use either production or cost functions, but for electric distribution utilities, cost functions are usually the preferred choice since these utilities cannot choose the level of output and can only minimize costs for a given output. he method creates a frontier using information on the assumed most efficient firms and measures the relative efficiency of the rest of the utilities.7 DEA attempts to approximate the efficient frontier by a ‘‘piece-wise’’ linear approximation based on the sample. Efficiency scores are constructed by measuring how far a utility is from the frontier. In general, best-practice efficiency is normalized to a score of one and then the efficiency of a particular utility is measured by how far it is from one. For example, a score of 0.75 is assumed to mean that the utility is 25 percent below the best practice. A score of 1 presumably means that the utility is efficient. However, it does not mean that the utility cannot improve its performance, but only that no
T
Figure 1: Simplified Illustration of DEA, Showing Cost and Output Relationships for Six Utilities
other utility in the sample is performing better. Figure 1, a simplified illustration of DEA, shows the cost and output relationships for six utilities. L, K, and M would be interpreted as efficient by DEA users, while the rest would not. J’s ‘‘inefficiency’’ is the distance between J and the frontier. DEA’s appeal is that it does not require that the underlying shape of the cost function be determined and therefore it does not impose any functional form on the relationships between inputs and outputs. However, the method has many more problems than advantages: a. Unexplained portions are regarded as inefficiencies DEA scores are prone to incorrect interpretation since the gap between a company’s score and a perfect score of 1 might be due to factors that were not captured in the model. A DEA score of 75 percent does not necessarily imply that 25 percent of the costs are due to inefficiency, but rather that the model fails to explain some of the actual costs. DEA scores measure deficiencies in the model as well as inefficiency in the utility. b. DEA is sensitive to the number of variables
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
33
DEA is sensitive to the number of inputs, outputs, and control variables. Adding variables never decreases DEA scores and therefore gives utilities a chance to better explain their own particular level of costs. If the model includes many variables, but only a small sample size, every utility may be able to call upon at least one of the output variables to explain its costs. Such a model would place many utilities at or near the frontier. To preserve variation in scores, the number of explanatory variables has to be restricted and such a modeling procedure does not accommodate all the specific factors that affect each utility’s costs. c. Outliers can be identified as efficient DEA identifies observations on a ‘‘frontier’’ and creates a comparator for each utility by identifying a point on the frontier between the nearest observations. However, some utilities are bound to be outliers. Their data place them on the edge of the group but this can be the result of exceptionally positive conditions (i.e., a distribution utility whose territory is mostly flat while the rest of the utilities in the group are hilly). Being on the edge of the group, such utilities are highly likely to be identified as lying on the frontier, whether they are efficient or not. d. DEA is biased against aggregated data DEA suffers an intrinsic bias against observations produced by aggregating data. For example, 34
combining two outliers with different characteristics will produce a merged utility that has mixed (more typical) characteristics. Since it is no longer so likely to be an outlier, it is also less likely to appear on (or even close to) the frontier. However, simply combining the data for two utilities does not change any of their real costs or efficiency levels. The model’s implication–that the two utilities have moved from being ‘‘efficient’’ to being ‘‘inefficient’’– does not reflect any real change in efficiency, and is merely a quirk in the methodology. e. DEA’s results cannot be tested statistically The results of DEA models cannot be tested statistically to see whether or not they are robust. This means that the reliability of the results is unknown. 2. Corrected ordinary least squares (COLS). In COLS the purported relative efficiency of a utility requires the specification of the cost function, since it starts with a regression. A regression is estimated using OLS and then shifted to create an efficient frontier. With COLS, only one firm of the sample is considered efficient. As in DEA, the frontier then is used to measure the relative efficiency of the rest of the utilities.8 Figure 2 shows the difference between OLS and COLS. In COLS the origin is decreased by the distance between the OLS line and the most efficient utility (A). In other words, the frontier is shifted until it envelops all data.
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
Figure 2: Simplified Illustration of COLS
The efficiency score for utility Z is the ratio WX/ZX. Similar to DEA, COLS assumes that all deviations are due to inefficiencies. A very important problem with COLS is that the results are highly sensitive to the single most efficient firm (A). 3. Stochastic frontier analysis (SFA). This method is similar to COLS, but SFA allows an estimation of random effects on efficiency and reduces the reliance on a single firm.9 SFA involves an assumed split of the error term (unexplained cost differences) between random errors and inefficiencies. Under the simplest form of SFA, a regression is estimated with two error components independent from each other. As in DEA and COLS, purported efficiencies are computed as the distance between the data and the efficiency frontier. This distance is usually shorter when calculated using SFA as opposed to COLS. owever, SFA does not solve the problem of deciding if unexplained costs are inefficiencies or not. The method just produces a split of the unexplained cost differences. SFA is prone to bias due to correlations between
H
The Electricity Journal
explanatory variables and the error terms. In fact, Ofwat (the U.K. water regulator) has criticized SFA for its fundamental reliance on regression analysis, noting that it requires a number of strong assumptions about the form of the relationship between expenditure and the explanatory factors.10
the objective (100 percent ‘‘efficient’’): If the company has five years to decrease its costs the annual X-factor would be about 7 percent as opposed to 35 percent if it only had one year. his example assumes a constant annual rate factor; however, other options include a high X-factor for the first year and regular factors afterwards,
T
IV. From Benchmarking to X-factors There is no standard method of converting benchmarking results of ‘‘efficient’’ cost levels into the annual rate of change defined by an X-factor. Regulators using benchmarking must not only determine a company’s inefficiency level (perhaps erroneously) but must also determine how long it should take for that company to reach the presumed efficient level. Thus, it is necessary to specify an end-date, and hence the number of years allowed for catching up, in order to convert a percentage cut in total costs into a percentage rate of allowed annual changes in prices or revenues. he X-factor takes different values depending upon the expected date to reach the target level (or assumed ‘‘efficient costs’’ level). For example a DEA score of 65 percent means that the utility should decrease costs by 35 percent to be at the ‘‘efficient cost’’ level. The 65 percent score could be converted to the following annual X-factors depending on when should the company reach
T
December 2003
of 0 to 3 percent per annum. In the U.K., the Office of Gas and Electricity Markets (Ofgem) assumed that the least efficient distribution utilities should move three-fourths of its distance to the frontier in two years and then retain that position until the next price review (three years later).12 This implied a drastic operating cost reduction for some utilities. Norweb, for example, was expected to lower its costs by almost 30 percent.
V. Applications
decreasing X-factors, increasing X-factors, etc. Practical experience shows that regulators converting efficiency scores into X-factors tend to adopt arbitrary standards, based on separate (and subjective) estimates of the feasible rate of change. The Norwegian Electricity Regulator (NVE) obtained scores (60–100 percent) very similar to those of the Dutch Regulator (DTe), but DTe converted them into much higher X-factors by assuming a much shorter ‘‘catch-up’’ period (three years instead of six).11 NVE established a common X-factor of 1.5 percent per annum, and augmented it by a ‘‘stretch factor’’ related to DEA scores
Benchmarking is a fairly new approach, so it is still early to determine the financial consequences it may have on the companies to which it has been applied. However, the three cases below illustrate the difficulties in implementing benchmarking and the potential problems it can create. A. Great Britain The application of benchmarking by Ofgem demonstrates that the technique will be ultimately subjective whenever the estimate of costs and manipulation of the frontier are subjective. Ofgem regulates electric distribution utilities under a PBR program. Every five years, Ofgem reanalyzes the relative efficiency of the whole sector and the individual companies to re-set the price cap for each company. The price reduction in the first year of the price control period (P0) and the
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
35
rate of decline for the subsequent years (X-factors) are determined in tandem. Ofgem forecasts the costs for the next period (using benchmarking as part of the equation) and determines the net present value of the costs. Then, it sets P0 and X so that the net present value of revenues equals net present value of cost. Since there are many possible combinations of P0 and X Ofgem decided to use a 3 percent X for all companies resulting in extremely high P0, as shown in Table 1. Ofgem attempted to establish the efficient level of operating expenditures (opex) for each utility using COLS. Capital expenditures (capex) were analyzed separately. The main rationale for separating operating and capital costs appears to be that it is easier to get more accurate cost models in this way, because the cost drivers for different types of cost are Table 1: Ofgem’s Extremely High Price Reductions 2000–2005 P0 (%)
X-factor (%)
East East Midlands
28 23
3 3
London Manweb
27 21
3 3
Midlands
23
3
Northern Norweb
24 27
3 3
Seeboard Southern
33 19
3 3
Swalec
26
3
South Western Yorkshire
20 23
3 3
Scottish Hydro
13 4
3 3
36
very different and the operating models are easier to generate than capital expansion and replacement models. These are good reasons for analyzing the cost categories separately, but this approach adds the problem of how to combine separate efficiency estimates. The tradeoffs between opex and capex mean that any one utility is highly unlikely to be able to achieve ‘‘best practice’’ efficiencies in every cost category separately. fgem has affirmed that there is a tradeoff between opex and capex costs, and agrees that this is an important issue that needs to be dealt with. Ofgem makes a very simplistic assessment of capital efficiency and then rewards more efficient utilities through the revenue allowance. They suggest that this reward for capital efficiency will match the extra opex that arises from lower capex and hence take into account any tradeoffs. This misses the point that the total cost allowed on the basis of both capital and operating efficiency targets may be unachievable. In other words, the company with the lowest operating costs is likely to be different from the one with the lowest capital costs and to have both could be almost impossible. A composite explanatory variable is used for operating costs with the weights of the component explanatory variables fixed by Ofgem. The basis of the weights appears to be an Ofgem ‘‘engineering judgment,’’ taking into account some of the utilities’ points of view, though Ofgem
O
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
also accepts that there are other explanatory factors that are not being captured. The weights appear not to have any statistical basis. The majority of network monopoly companies have even argued that the present arrangements give rise to distorted incentives and that this could lead to their adopting an inefficient mix of opex and capex.13 Even Ofgem now considers that the existing arrangements can provide companies with distorted incentives between opex and capex.14 B. Holland The problems with DEA specification and the impact on efficiency scores can be illustrated by an example from Holland. In 2000 the Dutch Electricity Regulatory Service (DTe) undertook a benchmarking exercise based on the DEA method to set the X-factors of the Dutch electricity distribution businesses, and therefore the revenues that the utilities were allowed to earn. DTe did not base revenues on direct measures of actual costs. Instead, DTe calculated the distributor’s estimated revenues in 2000 excluding non-controllable elements (e.g., rates charged by the transmission company) and then adjusted the estimated controllable costs to normalize the rates of depreciation and set a common rate of return. DTe then multiplied the utility’s controllable costs by their DEA scores to obtain target costs for 2003. The Electricity Journal
F
inally, DTe took the controllable costs calculated for 2003 and adjusted them to allow for a further rate of productivity growth, equal to an expected rate of decline in ‘‘efficient’’ costs of 2 percent per year.15 The final figure of total costs was compared with the 2000 figure to establish the annual rate of change necessary to achieve the new level by 2003. The X-factor was defined as this annual rate of change. Many electricity distribution companies contested the calculation of the X-factors and took DTe to court (and won). One of the distributors that appealed asked DTe to revise the dataset used. DTe agreed, adopted a new database, and re-standardized book values and depreciation terms. As a result, both the rate of return and depreciation components of the total controllable costs changed and the distributor’s efficiency score rose from 65 to 95 percent. The instability of the results demonstrates a fundamental problem in applying the DEA approach for ratemaking purposes. Presumably other companies with low scores would be able to make an equally persuasive case that the dataset should be revised, and obtain similarly dramatic improvements in their scores. C. New South Wales, Australia The experience of the Independent Pricing and Regulatory Tribunal of New South Wales December 2003
(IPART) illustrates the difficulties of modeling efficient costs using DEA and the risk that modeled results will lack credibility. As part of its first Electricity Distribution Price Review, IPART commissioned a DEA efficiency benchmarking study as an input to its inquiry into the prices of the monopoly electricity distribution businesses in New South Wales
for the five-year period from 2000.16 The database for the study included 219 distributors from Australia, New Zealand, England and Wales, and the U.S. The model included operating and management expenditures, Km of network route and MVa of transformer capacity as inputs. Energy delivered, customer numbers, and peak demand were specified as outputs. The model does not recognize such factors as the location and quality dimensions of the electricity distribution process. The DEA results suggested that the six NSW distributors could proportionally reduce their inputs by between 56 percent and 34 percent, and still be able to pro-
duce the same level of output if they were ‘‘efficient.’’ These results were adjusted to take into account factors outside the control of the company (customer density, proportion of residential customers, etc.). Even when adjustments for environmental factors were made, ‘‘inefficiencies’’ of between 41 and 13 percent are identified, resulting in substantial controversy and debate over the accuracy of the results. IPART acknowledged that a ‘‘major limitation of DEA is that it suggests all inputs can be reduced proportionally.’’17 To reduce inputs proportionally is not always a viable option. For example, it assumes that distributors could reduce their networks route by miles and still offer a secure supply. n the end, IPART recognized the limitations of the benchmarking studies and gave it only limited weight in its final decision.18 Northpower & Integral Energy, for example, scored 0.65 and 0.60, respectively in the DEA study (implying a 35 and 40 percent price reduction to achieve ‘‘efficiency’’) and IPART required only a 15 percent price reduction over the five-year period.19
I
VI. Conclusions Benchmarking, even in its more sophisticated formulations, has been an interesting, but thus far unsuccessful, regulatory effort. While it is usually presented as an objective technique, its application in regulation has been
# 2003, Elsevier Inc., 1040-6190/$–see front matter doi:/10.1016/j.tej.2003.09.010
37
inherently subjective. The lack of robustness and transparency can result in contentious debates that make it difficult to implement. enchmarking for ratemaking purposes, that is, for actually setting tariffs, may not be feasible. A proper benchmarking analysis would be extremely burdensome since it would be necessary to carefully review all the costs for all the companies included in the benchmarking study. Regulators who attempt to simplify the methodology to render it more manageable risk making arbitrary judgments that confuse inefficiency with heterogeneity and that are potentially harmful to companies who cannot ‘‘improve’’ their way out of fundamental differences in circumstances. Even if implementation problems are solved, benchmarking can result in lower rates in the short run, but with increased financial risk for the companies and underinvestment that create problems in the long run. This is not to argue that benchmarking has no role in regulation. Comparisons can indeed provide useful information to both the utilities and their regulators: They can be warning signs of problem areas, which the regulator can use to request follow-up studies or data. They can be useful for the utility to analyze its own performance and find areas for improvement. They can signal situations beyond the control of the utility,
B
38
which if changed, could reduce costs to all customers.& Endnotes: 1. Other methods not discussed in this article include the ‘‘engineering economic model’’ used in Spain and Chile. This method uses a theoretically efficient company as a benchmark. 2. This article discusses distribution utilities in particular, although much
7. DEA was introduced by Charnes, Cooper and Rhodes in Measuring the Efficiency of Decision-Making Units, EUROPEAN J. OPERATIONAL RES., 1978, 2 (6). They extended Farell’s (1957) idea. See M.J. Farell, Measurement of Productive Efficiency, J. ROYAL STATISTICAL SOC., 1957, Series A, 120 part 3. 8. See J. Richmond, Estimating the Efficiency of Production, INT’L ECON. REV., 1974, 15 (2). 9. SFA was first developed by Aigner, Lovell, and Schmidt in 1977. Aigner, Lovell, and Schmidt, Formulation and Estimation of Stochastic Frontier Production Function Models, J. ECONOMETRICS, 1977 (6) 1. 10. Cabinet Office (U.K. Government), Improving Analysis and Modeling in Central Government, Case Studies, 2000. 11. For details see Norwegian Water Resources and Energy Administration (NVE), Regulation of Electricity Monopolies, Efficiency Analysis of Transmission and Distribution, slide presentation by Thor Martin Neurauter, Sept. 1999. 12. Ofgem, Review of Public Electricity Suppliers 1998 to 2000. Distribution Price Control Review. Final Proposal, Dec. 1999, at 21.
of the material also applies to vertically integrated utilities. 3. Most incentive regulation schemes operate between full tariff reviews, in which the traditional ROR approach is used. 4. Setting revenues according to other companies’ costs is also known as yardstick competition. See A. Schleifer, A Theory of Yardstick Competition, RAND J. ECON., Autumn 1985, 16 (3). 5. Measurements of customer satisfaction and customer service reliability are also part of the adjustment formula. For more details see Mississippi Power Performance, Evaluation Plan, Rate Schedule PEP-3, May 23, 2002, in Docket 2001-AD-0826. 6. In econometric approaches, heterogeneity actually helps to produce more reliable estimates of the impact of different conditions on costs. However, the source of error terms is still unknown.
# 2003, Elsevier Inc., 1040-6190/$ – see front matter doi:/10.1016/j.tej.2003.09.010
13. Ofgem, Developing Network Monopoly Price Controls, Update document, Feb. 2003. 14. Id., at 29. 15. It also adjusted the controllable costs to allow for the imposition of tax from 2002. 16. For more information about the problems with the DEA analysis see NERA, A Review of DEA Benchmarking of the NSW Distribution Businesses, Report for Energy Australia, March 1999. 17. Independent Pricing and Regulatory Tribunal of New South Wales, Pricing for Electricity Networks and Retail Supply, Vol. I, June 1999, at 128. 18. Independent Pricing and Regulatory Tribunal of New South Wales, Regulation of NSW Electric Distribution Networks, Determination and Rules Under the National Electricity Code, Dec. 1999, at 67. 19. Id., at 65.
The Electricity Journal