- Email: [email protected]
Measuring equity of educational outcomes in the presence of inefficiency
European Journal of Operational Research 142 (2002) 642–652 www.elsevier.com/locate/dsw
Interfaces with Other Disciplines
Measuring equity of educational outcomes in the presence of inefficiency John Ruggiero a
a,*
, Jerry Miner b, Lloyd Blanchard
c
Department of Economics and Finance, The University of Dayton, 300 College Park, Dayton, OH 45469-2251, USA b Center for Policy Research, Syracuse University, Syracuse, NY 13210, USA c Daniel J. Evans School of Public Affairs, University of Washington, Seattle, WA 98195-3055, USA Received 29 March 2000; accepted 31 August 2001
Abstract Disparities in school expenditures have been a major concern in school finance. Equalization of spending presumably fulfills the ideal of equal educational opportunity. Differences in spending, however, result not only from differences in service provision, but also from variations in resource prices, exogenous cost environments and efficiency. As a result, equity measures that use expenditures per pupil as the object will capture these other variations and, therefore, may not be indicative of unequal educational outcomes. The purpose of this paper is to provide a systematic non-parametric framework for measuring outcome equity of school districts. Data Envelopment Analysis is used to adjust expenditures for differences in cost environments and to control for inefficiency in New York State school districts. The results suggest that nearly one-half of the measured inequity can be attributed to sources other than disparities in outcomes. Ó 2002 Elsevier Science B.V. All rights reserved. Keywords: Data Envelopment Analysis; Efficiency; Equity
1. Introduction School finance equity remains at the forefront of educational reform more than 25 years after the Serrano V. Priest (1971) decision by the California Supreme Court sparked intense debate. The court challenges which commenced with this case continue to the present day. A new round of court
*
Corresponding author. Tel.: +1-937-229-2550; fax: +1-937229-2477. E-mail address: [email protected] (J. Ruggiero).
cases has broadened traditional equity standards in calling for remedies that go beyond equalization of per-pupil spending (e.g., Ohio, Kentucky and Texas) to standards for equity based on adequacy of resources and outcomes see Reschovsky (1994). This paper addresses two central issues regarding school equity that, despite the newly expanded views, have received inadequate attention by the courts and in legislative approaches to reform. One concerns the effects that exogenous environmental factors have on the cost of producing outcomes. The second is the efficiency with which school resources are used and services are provided.
0377-2217/02/$ - see front matter Ó 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 0 1 ) 0 0 3 1 1 - 3
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
Most of the school finance debate has focused on differences in fiscal capacities of school districts and the efficacy of aid formulas intended to compensate (i.e. ‘‘equalize’’) for low-capacity. Equity across districts and hence, the effectiveness of compensatory aid generally is measured in relation to expenditure per pupil. This standard, however, is severely flawed because it omits differences in district costs and efficiency, both of which have major impacts on the levels of per pupil educational services provided across districts and on the educational outcomes that result. District ‘‘cost’’ refers, here, not to outlays, but rather, to the minimum expenditure necessary to provide a given level of output. Defined in this way, differences in cost among school districts can be attributed to: (1) variations in the prices of school inputs, (2) variations in the amounts of inputs needed to produce a quality-adjusted unit of school services or activities, and (3) variations in the quantity of such services that lead to given outcomes. The first factor is straightforward and is illustrated by the higher salaries that some districts must pay to obtain teachers of a given quality (i.e. training and experience). The second and third are due to what are commonly called environmental factors; that is, the different circumstances and characteristics of districts and pupils that affect the transformation of school inputs into student outcomes. For example, the second is shown by the additional inputs per pupil required to provide limited English proficiency students with a given quality of classroom instruction. In contrast, the third is illustrated by more hours of quality adjusted instruction in literature or science required to attain a given test score result by those with limited English proficiency. While the latter two may be difficult to disentangle in practice, they are conceptually distinct. These considerations imply that for comparisons of equity, observed measures of per pupil expenditure need to be adjusted to reflect variations in the cost environment. Unfortunately, however, courts have focused almost exclusively on equalization of unadjusted expenditure per pupil; see Odden and Picus (1992). If inequalities in actual expenditure are inappropriate indicators of equity, what adjustments of expenditure would
643
provide suitable indication of inequitable differentials, i.e. would measure differences in outputs per pupil? There is, however, a prior question which must be addressed; why is it necessary at all to examine expenditure if the goal is to measure disparities in outcomes? That is, why not measure disparities directly by showing differences in outcomes? The answer is somewhat paradoxical. Historically, the emphasis on expenditure derives from the absence of systematically collected data regarding outcomes. With the collection of various outcome measures, the obstacle to direct measurement is the variety of outcome measures in the absence of a natural scale for weighting them. A direct measure of inequities in outcomes among districts would require an index of district outcomes and any such index would be arbitrary in its procedures for aggregating the mix of various outcomes into a single metric. As will be explained below, a measure that constitutes a proxy for outcomes can be derived by adjusting expenditure for differences in factor prices among districts (e.g. costs of equivalent teachers), for the spatial and structural characteristics of school districts that affect cost (e.g. sparsity and proportion of secondary and disabled pupils) and for ‘‘environmental’’ characteristics of pupils (e.g. limited English proficiency and disadvantaged family circumstances). Such an adjusted expenditure measure reflects the ‘‘real’’ cost or expenditure of a district and hence, its outcomes. Even if fully adjusted for cost differences, however, expenditure will not be indicative of output and equity so long as there is variation in the efficiency of service provision. Studies of educational production and costs have found little systematic relationship between inputs or expenditures and outcomes, suggesting that school districts vary in the efficiency of service provision; see Hanushek (1993). With such variation two districts will not provide the same level of student outcomes even if they spend equivalent amounts per pupil and face identical cost environments. The conclusion is clear: cost-adjusted spending remains a distorted measure of output and hence, of equity unless, ultimately, it is adjusted for districts’ relative efficiency.
644
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
In response to the awareness that analysis of inequities in public service provision requires measures of outcomes not expenditures, the last decade has witnessed development of ingenious empirical methods for achieving this purpose. In one approach, researchers estimate a cost equation where expenditures are regressed on input costs and structural and environmental factors, along with either direct measures of outcomes or variables reflecting demand for the services. Applied to education such an equation controls for differences in the costs of educational inputs, the harshness of the production environment, and in levels of various school outcomes; see Duncombe et al. (1996). The cost equation is used to estimate the cost of each district’s outcomes assuming it faces average cost conditions. Alternatively, researchers can use Data Envelopment Analysis (DEA), a linear programming approach that projects observed expenditures to a ‘‘best-practice’’ cost frontier and effectively controls for cost inefficiency and environmental cost factors including resource prices. By projecting to the best-practice frontier, cost and efficiency is controlled and therefore, the adjusted expenditure reflects variations in outcomes. Standard equity measures are then applied to the adjusted expenditure and to observed expenditure to analyze the effects of adjustment for outcomes on the measures of equity of service provision. Findings for New York State school districts reveal that more than half of the measured inequity based on observed expenditure differences is attributable to differential costs and inefficiency and not to variations is outcomes. The next section of the paper briefly discusses standards of equity. It then develops a model of public sector costs that incorporates variations in cost, outcomes, and efficiency across districts and also presents the DEA model used to adjust expenditures for cost and efficiency differences. The model is applied to data from New York State to measure cost minimizing expenditures per pupil given outputs, assuming that all districts are equally efficient and face the most favorable cost environment. Horizontal equity is analyzed through the standard indicators of overall dispersion of district-by-district spending and categorical
equity through association of district outcomes with measures of their wealth. The concluding section describes policy implications and directions for future research.
2. Equity standards While examination of the equity of local public services has concerned both the level of services provided and the burden on local taxpayers of financing the service, this paper focuses on variations in the provision of school services. Although equivalence of service level across jurisdictions is a reasonable, prima facie, standard of equity for publicly provided goods, it leads, inevitably, to the difficult questions of defining and ultimately measuring public service levels; see Wood (1995). One approach defines service levels produced in terms of quantities of activities performed. As regards schools, activities include classes taught, pupilmiles of transportation provided, and square feet of buildings maintained and heated. An obvious difficulty with applying this approach is the need to control for quality as well as quantity of activities. If the standard of equity, however, is not activities but outcomes, even an index of activities which takes account of quality does not reflect outcomes because it ignores the likelihood that environmental variations across districts require differential activities per pupil. Recent discussions of standards for evaluating school performance use the term ‘‘inadequacy’’ as indicative of insufficient inputs (activities) to produce a specified ‘‘adequate’’ level of educational outcomes. To develop an index reflective of equity (adequacy) of outcomes requires adjustments to expenditure not only for input prices and extra inputs per activity, but also for the differential quantity of activities needed per pupil needed to attain equal (specified) outcomes, and, finally, for the efficiency of transforming activities into outcomes. This is precisely what is accomplished by the estimated minimum required spending derived from the DEA analysis. Horizontal equity is based on view that fairness requires equal services or outcomes across
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
all provider units. Full, or complete, horizontal equity in schooling is attained if all students receive the same level of an appropriate measure of services. The most appropriate such measure for schools is outcomes, although court challenges have focused on spending. Horizontal equity measures gauge how far the actual distribution across districts departs from complete equity. In addition to such conventional statistical measures of variation and of the extent of departure from complete equivalence as the range and the variance, the coefficient of variation and the Gini coefficient, students of school finance have utilized other measures such as the restricted range, federal range ratio, and Brazer’s coefficient of variation, that control in a special way for the extremes of the distribution. See Berne and Steifel (1984) and Fenner (1991) for useful discussions of each measure. An alternative principle, categorical equity, is the view that equity obtains when services are provided without regard to ability to pay (Feldstein, 1975). This view is the basis for the concept of wealth neutrality, a situation where no statistical relationship exists between the level of services provided and community wealth. Categorical equity does not imply absence of variations across districts in service provision or in outcomes; rather, it obtains as long as variations in service provision are attributable to differences in preferences but are not systematically associated with wealth. Thus, categorical equity does not necessarily imply horizontal equity, but horizontal equity does imply categorical equity. There are three standard measures of the extent of wealth neutrality: correlation, slope and elasticity. Each measures the relationship between the level of services or outcomes provided and the ability of a school district to finance these services (local wealth). Perfect categorical equity would be indicated by a correlation coefficient of zero. The slope parameter from a linear regression of services on wealth provides a measure of the impact of increasing wealth on services. Wealth neutrality ensues if the slope parameter is equal to zero. Alternatively, a log-linear regression assumes a constant elasticity; an elasticity of zero indicates wealth neutrality.
645
3. Educational production, costs and efficiency As explained above, most analyses of school equity treat observed expenditure per pupil as indicative of services and/or outcomes. (Berne and Steifel (1984) recognize the importance of adjusting expenditures for resource price differentials. Fenner (1991) develops a model which allows not only for resource price differentials but also for variations in other exogenous cost factors. Neither paper, however, recognizes variations due to inefficiency.) The rationale for this presumption is that higher expenditures per pupil, ceteris paribus, indicate increased inputs, services and eventually outcomes. Consequently, the conclusion is that variations in spending are inequitable. However, this assumes that school districts face the same cost environment and are equally efficient in providing services. Without controlling for these factors, expenditure measures of equity are misleading. We model educational costs with the following standard model: TC ¼ CðS; W ; ZÞ:
ð1Þ
This cost function relates the minimum level of expenditure, i.e. the minimum cost, necessary to provide a given level of services given the production function, exogenous resource prices, and environmental factors. Further, this function provides the basis to analyze differences in cost that arise when services are efficiently provided. The minimum cost of providing a given level of services varies with differential factor prices and/or environmental conditions. That is, there are multiple cost frontiers. Expenditure and outcome under conditions of multiple cost frontiers is shown in Fig. 1, where it is assumed that school districts (indicated as A, B, C, G, F ) face different cost environments (CH ; CM ; CF ) and produce a single outcome. All districts other than G are assumed to be cost minimizers and are therefore on their cost frontier. As shown, district A faces the harshest environment and, therefore, must spend more to produce the given outcome. Districts B, F , and G, on the other hand, face more favorable environments relative to A and,
646
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
Fig. 1. Cost frontiers in the public sector.
therefore, if efficient can provide higher levels of the service with less spending than A. The cost function explicitly assumes that the decision making units are cost minimizers. Based on the bureaucratic model of supply, public choice literature poses an alternative expenditure determination model that allows inefficiency; see Niskanen (1971, 1975). The bureaucratic model postulates that decision makers have an incentive to increase expenditure beyond the minimum cost level in order to obtain pecuniary benefits, leading to budgetary slack. (For a further discussion, see Wyckoff, 1990; Duncombe et al., 1997.) The empirical literature provides evidence that costminimization is not a reasonable assumption, suggest that expenditure variations are largely due to differences in efficiency (see Hanushek, 1986, 1993). The difference between minimum costs and actual per pupil expenditures ðEÞ defines cost inefficiency, where E P CðS; W ; ZÞ:
ð2Þ
In this case, if a school district is cost efficient, observed expenditure represents minimum costs. On the other hand, if a school district is cost inefficient, actual expenditure will be greater than the minimum cost of providing the observed services. Referring to Fig. 1, district G (which by assumption faces the most favorable cost envi-
ronment) is cost inefficient. Since district G has the most favorable cost environment, if efficient it could have provided its observed level of services with the observed expenditure of district F . The errors of analyzing equity using unadjusted observed expenditures are revealed in Fig. 1. From the standpoint of observed expenditure, inequity is perceived since there are three levels of spending. Based on observed expenditures, an equity analysis would inappropriately indicate that district G provides the most services and district F the least. In fact, however, district D (with average observed spending) produces the highest outcome, and F , G, and B all produce the same lower level of outcome. These stylized facts highlight the problem of using unadjusted expenditures to measure the equity of outcome production. Outcome based measures of inequity, however, can be derived from expenditures adjusted by DEA analysis for differences in the cost environment and inefficiency. An equity analysis based on these adjusted expenditures reveals the true magnitude of disparities in outcomes. The operations research technique known as DEA, pioneered in Charnes et al. (1978), provides a measure of Farrell (1957) efficiency. The original DEA linear programming model was introduced to identify technical inefficiency, measured by the equiproportional reduction of actual inputs that is possible, holding output at the observed level. Based on minimal assumptions, the technique identifies the best-practice frontier and evaluates each producer relative to the frontier. Grosskopf and Yaisawarng (1990) extended the approach to allow measurement of cost efficiency given expenditure and output data. Assume that each of N school districts produces L outcomes (i.e. DEA outputs). Let sjl denote the production of output l by district j and Ej denote the observed spending by district j. Now consider the following linear program used in the analysis of school district i (for i ¼ 1; . . . ; N ): ci ¼ Minimize subject to
gi
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652 N X
8l ¼ 1; . . . ; L;
hj sjl P sil
j¼1 N X
h j Ej 6 g i Ei ;
ð3Þ
j¼1 N X
hj ¼ 1;
hj P 0 8j ¼ 1; . . . ; N :
j¼1
The solution of (3) for each school district provides an index consisting not only of relative exogenous costs but also cost inefficiency. School district expenditures are contracted to a bestpractice cost frontier where the minimum cost of outcome production is achieved assuming that each district faces the most-favorable environment and is equally efficient. The resulting DEA measure is useful for deflating expenditures for cost and efficiency differences. Consequently, the adjusted expenditures are a more useful equity measure. The construction and application of the bestpractice frontier is illustrated in Fig. 2, which is similar to Fig. 1. However, rather than showing all cost curves, only the DEA best-practice frontier is illustrated. Contracting each district’s observed expenditure to the best-practice frontier reveals the minimum expenditure required to produce the observed level of outcome assuming that each district has the same favorable environment and is equally efficient. The ratio of the minimum to the observed expenditure is a combined measure of environmental harshness and inefficiency.
647
As shown in the diagram, districts G, B, and F all produce an equivalent outcome of 80. Lowest cost district F produces the outcome by spending $1000. Because F is on the best-practice frontier, F ’s adjusted expenditures would be $1000. Consider next district G, which produces the same outcome as district F . Recalling the earlier assumption that G faces the most favorable environment and is inefficient, we know that G could have produced the same outcome by spending only $1000 (and is thus wasting $1000). The adjusted expenditures for G is therefore $1000. Basing the equity analysis on adjusted expenditures reveals that G and F are providing the same outcome level. The adjusted expenditures of B is also $1000. However, unlike G, B is cost efficient because the $500 difference in adjusted and observed expenditures is attributed to differential cost environments and not inefficiency. Regardless, an equity analysis based on adjusted expenditures reveals that no outcome inequities exist between districts G, B and F . The adjusted expenditures of district A are also $1000 resulting from output slack that remains after projection. District D is on the best-practice frontier and its adjusted and observed expenditure is $1500, reflecting higher outcome production than all other districts. It is essential to recognize that the explanation and illustration of the workings of DEA in Fig. 2 rely on the simplification of a single outcome. In fact, DEA is designed for multiple outcomes, and no simple diagram can fully illustrate its operation. Intuitively, in actual DEA analysis, the linear minimum cost function in Fig. 2 has as its counterpart a minimum cost for all observed combinations of outcomes, and to get the minimum cost for each district, its expenditure is projected to the lowest spending district with an equivalent mix of outcomes.
4. Empirical analysis of New York school districts
Fig. 2. DEA best-practice cost frontier.
We estimate best-practice expenditures per pupil and analyze equity for 631 of the 695 school districts in New York in 1991. Despite limitations
648
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
due to missing observations, the sample appears representative of the major regions in New York State. This section describes measures, data sources, and empirical analysis of education costs, and compares measures of equity under different assumptions regarding environmental cost differentials and school district efficiency. The measure of expenditure is a district’s Approved Operating Expenses (AOE) per pupil. AOE includes salaries and fringe benefits of teachers and other school staff, other instructional expenditure, and all other expenditure related to operation and maintenance of schools. The DEA model (3) presented above contracts expenditures per pupil holding outcomes at the level produced by the districts. We chose three outcomes to use as DEA outputs. The first derives from the state’s Pupil Evaluation Program, given to all third- and sixth-grade students in reading and math. The specific measure is the average percentage of students performing above the standard reference point on these four exams. The second is the percentage of students receiving a Regents diploma upon graduation from high school. Regents diplomas are given only to high
school students who pass standardized state tests. To balance this measure of achievement, the third outcome is the percentage of students who graduate. For a discussion of variables reflecting school outcomes; see Duncombe et al. (1996). Descriptive statistics are reported in Table 1. We also include cost factors to allow us to illustrate the problems of standard equity analyses that use expenditures per pupil. These factors were not used in the DEA model because we are not interested in measuring efficiency per se. Instead, we are interested in deflating expenditures for inefficiency and cost differences that exist in school districts. By employing DEA model (3) we are essentially controlling for efficiency and cost environmental factors. The state average level of observed expenditure is $6054 per pupil. This expenditure, however, captures not only variations in outcomes, but also variations in efficiency and cost environments. After controlling for the environment and the level of inefficiency by multiplying the DEA measure and the expenditures per pupil, we are able to derive adjusted expenditures per pupil; the average adjusted expenditure is $3801.
Table 1 Descriptive statistics (New York school districts in 1991, n ¼ 631) Variable
Mean
Standard deviation
Minimum
Maximum
6054
2143
3166
25,376
Outcomes PEP scores Percent receiving Regents diploma Percent non-dropouts
94.24 40.44 97.59
3.79 13.07 1.84
64.50 0.00 88.10
100.00 75.38 100.00
Cost factors Entry teacher salaries Enrollment Female-headed households (%) Handicapped students (%) Limited English proficiency (%) High school students (%)
24,727 2383 8.79 10.64 0.99 28.97
2957 3025 2.71 3.37 1.27 3.70
12,790 68 2.46 1.63 0 20.09
34,982 46,195 34.68 30.68 1.96 63.10
Efficiency results DEA cost and efficiency index DEA adjusted expenditures
0.67 3801
0.16 1076
0.16 3166
1.00 18,760
Expenditures ($/pupil) a
Sources: New York State Department of Education, Comprehensive Assessment Report, Basic Education Data System and Fiscal Profile, and National Center for Education Statistics, School District Data Book. a The outcomes are the outputs used in the DEA analysis. Projection to the best-practice frontier is achieved with reductions in expenditures holding outcomes constant.
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
The shortcomings of using unadjusted expenditures to analyze equity analysis is illustrated in Table 2, which presents data on observed and adjusted per pupil expenditure and on selected factors indicative of environmental costs for three of the school districts in the study. Two of the districts, F and V , spent approximately the state average of $6050 per pupil; the third, S, spent nearly 50% more. The $3000 difference per pupil appears to indicate a considerable amount of inequity. Closer inspection, however, reveals little inequity in outcomes. While district F ’s spending is essentially equivalent to the state average, it faces the harshest cost environment of the three selected districts. This can be seen by examining the selected cost factors. First, F pays entry level teachers $5000 more than the state average and hence, faces a higher labor price. In addition, relative to the state average, this district has 3600 more pupils, nearly double the percentage of single female-headed household, and eight times the percent of limited English students. Adjusted for these higher costs (and for inefficiency) F ’s expenditure per pupil (of and indicator of outcomes) $3515 is 92% of the state average.
649
Relative to district F , district V pays entry level teachers 33% less. In addition, district V has a more favorable environment due to its relatively low percentages of limited English students, handicapped students and single female-headed households. Consequently, it faces lower costs. While not reflected by the unadjusted expenditure, these lower costs are reflected by the adjusted expenditure of $4337 which is 14% above the adjusted state average, and, implies that district F ’s outcomes are 14% above the state average. This is consistent with the measures of outcomes in Table 2, as district V achieves a 10% higher PEP score than F and graduates 50% more students with a Regents Diploma, and has an above average overall graduation rate. Finally, consider district S, whose observed spending is 33% more than districts F , V and the state average. Despite facing an average cost environment, district S produces outcomes similar to district V . The DEA analysis yields adjusted expenditure of $4109 indicating outcomes slightly less than those produced by V , and 8% above the state average. This is confirmed by examination of the selected output measures. In this case, then, the low DEA cost/efficiency index of 0.444 for district
Table 2 Selected school district results Variable
School district F
V
S
Expenditure ($/pupil) Unadjusted Relative to state average DEA cost and efficiency adjusted Relative to state average DEA cost and efficiency index
6042 0.99 3515 0.92 0.582
6057 1.00 4337 1.14 0.716
9255 1.50 4109 1.08 0.444
Outcomes Pep scores Percent receiving Regents diploma Percent non-dropouts
87.50 30.38 98.50
96.25 60.20 99.10
96.25 58.39 98.90
Cost factors Entry teacher salaries ($) Enrollment Female-headed households (%) Handicapped students (%) Limited English proficiency (%) High school students (%)
29,822 5916 16.35 10.06 8.14 26.96
20,473 1177 5.43 6.94 0.64 30.11
27,451 7362 7.09 10.14 0.88 35.39
650
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
ter adjusting expenditures per pupil using DEA, a significant amount of the perceived inequity disappears. Most equity measures are substantially reduced, ranging from a decline in the range of 30% to a decrease in the restricted range of 74%. The federal range ratio, the variance and Brazer’s coefficient of variation all fell by two-thirds. The most widely used measure of inequity, the Gini coefficient, declined by 44%. (We note, however, that the coefficient of variation only decreases by 3.3%.) These reductions suggest that a considerable amount of what has traditionally been perceived as inequity can be attributed to inefficiency and exogenous costs. These findings constitute strong evidence supporting further development of education cost and efficiency indices and their use in education production and equity research. Past analyses of educational inequities based on measures of unadjusted expenditure per pupil are subject to the biases described above that lead to gross overstatements of inequities. Obviously, this raises serious questions about the often used approach in equity research of measuring differences in resources without accounting for these important factors.
S reflects the presence of inefficiency rather than high cost factors. A proper equity analysis should determine that little inequity exists between S and V , but that F provides significantly fewer services/outcomes than the other two districts. The improper conclusions derived from an analysis of observed expenditure would indicate inequity attributable to the higher spending of S relative to F and V . The proper conclusion is reached only after adjusting expenditures per pupil for inefficiency and cost factors.
5. Equity results Tables 3 and 4 show the results of an examination of expenditure equity in New York State. Variations in AOE per pupil are evaluated using two approaches: (1) with unadjusted AOE, and (2) with AOE adjusted for cost differences and inefficiency using the non-parametric DEA approach. A variety of equity measures are used to compare the outcomes. 5.1. Measures of horizontal equity As presented in Table 3, analysis of unadjusted AOE provides ‘‘evidence’’ of considerable inequality. For example, the federal range ratio indicates that the district at the ninety-fifth percentile of expenditures spends 1.43 times the district at the fifth percentile. Other measures reveal similar evidence of variation. However, af-
5.2. Measures of categorical equity Table 4 presents results analyzing the relationship between district spending and wealth, i.e. categorical equity. Two measures of district wealth are employed: (1) the Combined Wealth Ratio
Table 3 Selected measures of horizontal equity of New York State school districts in 1991 ðn ¼ 631Þ Measure
Unadjusted expenditures
DEA cost and efficiency adjusted expenditures
Percent reduction
Range ($/pupil) Restricted range ($/pupil) Federal range ratio Variance ($/pupil) Coefficient of variation Brazer’s coefficient of variation Gini coefficient Theil’s measure
22,210 5878 1.43 3,521,462 0.30 0.18
15,594 1532 0.47 1,178,004 0.29 0.06
29.8 73.4 67.1 66.6 3.3 66.7
0.16 0.04
0.09 0.03
43.8 25.0
DEA cost and efficiency adjusted expenditures were obtained by multiplying approved operating expenditures by the DEA cost and efficiency index obtained from (3).
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652 Table 4 Selected measures of categorical equity of New York State school districts in 1991 ðn ¼ 631Þ Unadjusted expenditures Correlation CWR FV/pupil Slope CWR FV/pupil Elasticity CWR FV/pupil
0.78 0.72 2242 0.009 0.40 0.35
DEA cost and efficiency adjusted expenditures
Percent reduction
0.38 0.32
51.3 55.6
547 0.002
75.6 77.8
0.13 0.10
67.5 71.4
DEA cost and Efficiency adjusted expenditures were obtained by multiplying approved operating expenditures per pupil by the DEA index. CWR, the combined wealth ratio in 1990–91, is a measure of a district’s relative wealth weighted equally by income and property value per pupil. FV/pupil is the full valuation of property value per pupil in a district.
(CWR) measure used by the state to set its state aid ratio for operating aid, which weights adjusted gross income and assessed property value equally, and (2) the full valuation of property per pupil. The relation between spending and each of the two measures of wealth is estimated by correlation, slope, and elasticity. Separate estimates of these coefficients are made for unadjusted spending (AOE/pupil) and for cost and efficiency adjusted expenditures using DEA. The unadjusted expenditure approach shows a correlation of 0.78 between expenditures and CWR. This correlation measure decreased by 51% after controlling for costs and efficiency using DEA. The correlation between expenditures and property value decreased by a slightly larger amount. Notably, all other measures of categorical equity are reduced by at least two-thirds. Thus, the correlation between district wealth and observed spending is stronger than the correlation between district wealth and service provision, as measured by adjusted expenditures. This finding is consistent with the conclusion of Duncombe et al. (1997) that wealthier districts are more inefficient, although without analysis of the relation of costs and wealth it is not possible to distinguish the effects of inefficiency and high cost in accounting for the lower
651
correlations of outcomes to wealth. The other categorical equity measures further support the hypothesis that the perceived inequity based on district wealth is not so strong when costs and efficiency are properly controlled for. Still, categorical inequity remains even after adjusting expenditure. Not surprisingly, wealthier districts provide a higher level of services. These are extremely important and disturbing findings for education equity research, particularly considering the actions taken by state courts based in part on equity study findings. Though this study does not claim that expenditure inequities or the relationship between expenditures and wealth in New York school districts are non-existent, it does find that costs and inefficiency account for a substantial portion of the observed inequities.
6. Conclusions This paper has focused on central issues that have not addressed in the school finance equity debate. Equity analyses based on unadjusted expenditure per pupil fail to recognize that school districts generally face different cost environments and that educational services may not be efficiently provided. As a result, to a considerable extent variations in spending reflect differences in cost environments and relative inefficiencies and not differences in educational outcomes. Distorted measures of inequity result if these other variations are not properly controlled. The results of this analysis indicate that most perceived inequity actually is due to these factors and not to variations in school outcomes. A major implication of these findings is that there should be considerably more effort devoted to studies of school efficiency including those features of school organization and governance associated with lesser or greater levels of outcomes per unit of input. Whatever the limitations of the specific methods used to control for costs and efficiency, this study clearly shows that less reliance should be given to unadjusted measures of spending in determining whether arrangements for school finance are in conflict with state constitutional provisions. It raises fundamental questions
652
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
regarding state policies intended to reduce inequities among local schools. Clearly, aid needs to be based on cost considerations as well as fiscal capacity, and policies need to be examined as regards their effects on efficient operation of schools. Finally, the analysis has illustrated an important use for DEA to analyze a key area of public policy.
References Berne, R., Steifel, L., 1984. The Measurement of Equity in School Finance: Conceptual, Methodological, and Empirical Dimensions. Johns Hopkins University Press, Baltimore, MD. Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the efficiency of decision making units. European Journal of Operational Research 2, 429–444. Duncombe, W., Miner, J., Ruggiero, J., 1997. Empirical evaluation of bureaucratic models of inefficiency. Public Choice 93, 1–18. Duncombe, W., Ruggiero, J., Yinger, J., 1996. Alternative approaches to measuring the cost of education. In: Helen, F.L. (Ed.), Holding Schools Accountable: Performance Based Reform in Education. Brookings Institution, Washington, DC, pp. 327–356. Farrell, M.J., 1957. The measurement of productive efficiency. Journal of the Royal Statistical Society Series A, General 120, 253–281.
Feldstein, M.S., 1975. Wealth neutrality and local choice in public education. American Economic Review 65, 75– 89. Fenner, R., 1991. The effect of equity of New York state’s system of aid for education. Ph.D. Dissertation, Department of Economics, Syracuse University, Syracuse, NY. Grosskopf, S., Yaisawarng, S., 1990. Economies of scope in the provision of local public services. National Tax Journal 43, 61–74. Hanushek, E., 1986. The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature 24, 1141–1177. Hanushek, E., 1993. Can equity be separated from efficiency in school finance debates. In: Emily, H. (Ed.), Essays on the Economics of Education. Upjohn Institute, pp. 35– 74. Niskanen, W.A., 1971. Bureaucracy and Representative Government. Aldine-Atherton, Chicago. Niskanen, W.A., 1975. Bureaucrats and politicians. Journal of Law and Economics 18, 617–643. Odden, A., Picus, L., 1992. School Finance: A Policy Perspective. McGraw-Hill, New York. Reschovsky, A., 1994. Fiscal equalization and school finance. National Tax Journal 47, 211–224. Serrano V. Priest, 1971. 487 P.2d 1241. Wood, R.C., 1995. Adequacy issues in recent educational finance litigation. Developments in School Finance, National Center for Educational Statistics, US Department of Education, pp. 29–37. Wyckoff, P., 1990. The simple analytics of slack-maximizing bureaucracy. Public Choice 67, 35–67.
Interfaces with Other Disciplines
Measuring equity of educational outcomes in the presence of inefficiency John Ruggiero a
a,*
, Jerry Miner b, Lloyd Blanchard
c
Department of Economics and Finance, The University of Dayton, 300 College Park, Dayton, OH 45469-2251, USA b Center for Policy Research, Syracuse University, Syracuse, NY 13210, USA c Daniel J. Evans School of Public Affairs, University of Washington, Seattle, WA 98195-3055, USA Received 29 March 2000; accepted 31 August 2001
Abstract Disparities in school expenditures have been a major concern in school finance. Equalization of spending presumably fulfills the ideal of equal educational opportunity. Differences in spending, however, result not only from differences in service provision, but also from variations in resource prices, exogenous cost environments and efficiency. As a result, equity measures that use expenditures per pupil as the object will capture these other variations and, therefore, may not be indicative of unequal educational outcomes. The purpose of this paper is to provide a systematic non-parametric framework for measuring outcome equity of school districts. Data Envelopment Analysis is used to adjust expenditures for differences in cost environments and to control for inefficiency in New York State school districts. The results suggest that nearly one-half of the measured inequity can be attributed to sources other than disparities in outcomes. Ó 2002 Elsevier Science B.V. All rights reserved. Keywords: Data Envelopment Analysis; Efficiency; Equity
1. Introduction School finance equity remains at the forefront of educational reform more than 25 years after the Serrano V. Priest (1971) decision by the California Supreme Court sparked intense debate. The court challenges which commenced with this case continue to the present day. A new round of court
*
Corresponding author. Tel.: +1-937-229-2550; fax: +1-937229-2477. E-mail address: [email protected] (J. Ruggiero).
cases has broadened traditional equity standards in calling for remedies that go beyond equalization of per-pupil spending (e.g., Ohio, Kentucky and Texas) to standards for equity based on adequacy of resources and outcomes see Reschovsky (1994). This paper addresses two central issues regarding school equity that, despite the newly expanded views, have received inadequate attention by the courts and in legislative approaches to reform. One concerns the effects that exogenous environmental factors have on the cost of producing outcomes. The second is the efficiency with which school resources are used and services are provided.
0377-2217/02/$ - see front matter Ó 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 0 1 ) 0 0 3 1 1 - 3
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
Most of the school finance debate has focused on differences in fiscal capacities of school districts and the efficacy of aid formulas intended to compensate (i.e. ‘‘equalize’’) for low-capacity. Equity across districts and hence, the effectiveness of compensatory aid generally is measured in relation to expenditure per pupil. This standard, however, is severely flawed because it omits differences in district costs and efficiency, both of which have major impacts on the levels of per pupil educational services provided across districts and on the educational outcomes that result. District ‘‘cost’’ refers, here, not to outlays, but rather, to the minimum expenditure necessary to provide a given level of output. Defined in this way, differences in cost among school districts can be attributed to: (1) variations in the prices of school inputs, (2) variations in the amounts of inputs needed to produce a quality-adjusted unit of school services or activities, and (3) variations in the quantity of such services that lead to given outcomes. The first factor is straightforward and is illustrated by the higher salaries that some districts must pay to obtain teachers of a given quality (i.e. training and experience). The second and third are due to what are commonly called environmental factors; that is, the different circumstances and characteristics of districts and pupils that affect the transformation of school inputs into student outcomes. For example, the second is shown by the additional inputs per pupil required to provide limited English proficiency students with a given quality of classroom instruction. In contrast, the third is illustrated by more hours of quality adjusted instruction in literature or science required to attain a given test score result by those with limited English proficiency. While the latter two may be difficult to disentangle in practice, they are conceptually distinct. These considerations imply that for comparisons of equity, observed measures of per pupil expenditure need to be adjusted to reflect variations in the cost environment. Unfortunately, however, courts have focused almost exclusively on equalization of unadjusted expenditure per pupil; see Odden and Picus (1992). If inequalities in actual expenditure are inappropriate indicators of equity, what adjustments of expenditure would
643
provide suitable indication of inequitable differentials, i.e. would measure differences in outputs per pupil? There is, however, a prior question which must be addressed; why is it necessary at all to examine expenditure if the goal is to measure disparities in outcomes? That is, why not measure disparities directly by showing differences in outcomes? The answer is somewhat paradoxical. Historically, the emphasis on expenditure derives from the absence of systematically collected data regarding outcomes. With the collection of various outcome measures, the obstacle to direct measurement is the variety of outcome measures in the absence of a natural scale for weighting them. A direct measure of inequities in outcomes among districts would require an index of district outcomes and any such index would be arbitrary in its procedures for aggregating the mix of various outcomes into a single metric. As will be explained below, a measure that constitutes a proxy for outcomes can be derived by adjusting expenditure for differences in factor prices among districts (e.g. costs of equivalent teachers), for the spatial and structural characteristics of school districts that affect cost (e.g. sparsity and proportion of secondary and disabled pupils) and for ‘‘environmental’’ characteristics of pupils (e.g. limited English proficiency and disadvantaged family circumstances). Such an adjusted expenditure measure reflects the ‘‘real’’ cost or expenditure of a district and hence, its outcomes. Even if fully adjusted for cost differences, however, expenditure will not be indicative of output and equity so long as there is variation in the efficiency of service provision. Studies of educational production and costs have found little systematic relationship between inputs or expenditures and outcomes, suggesting that school districts vary in the efficiency of service provision; see Hanushek (1993). With such variation two districts will not provide the same level of student outcomes even if they spend equivalent amounts per pupil and face identical cost environments. The conclusion is clear: cost-adjusted spending remains a distorted measure of output and hence, of equity unless, ultimately, it is adjusted for districts’ relative efficiency.
644
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
In response to the awareness that analysis of inequities in public service provision requires measures of outcomes not expenditures, the last decade has witnessed development of ingenious empirical methods for achieving this purpose. In one approach, researchers estimate a cost equation where expenditures are regressed on input costs and structural and environmental factors, along with either direct measures of outcomes or variables reflecting demand for the services. Applied to education such an equation controls for differences in the costs of educational inputs, the harshness of the production environment, and in levels of various school outcomes; see Duncombe et al. (1996). The cost equation is used to estimate the cost of each district’s outcomes assuming it faces average cost conditions. Alternatively, researchers can use Data Envelopment Analysis (DEA), a linear programming approach that projects observed expenditures to a ‘‘best-practice’’ cost frontier and effectively controls for cost inefficiency and environmental cost factors including resource prices. By projecting to the best-practice frontier, cost and efficiency is controlled and therefore, the adjusted expenditure reflects variations in outcomes. Standard equity measures are then applied to the adjusted expenditure and to observed expenditure to analyze the effects of adjustment for outcomes on the measures of equity of service provision. Findings for New York State school districts reveal that more than half of the measured inequity based on observed expenditure differences is attributable to differential costs and inefficiency and not to variations is outcomes. The next section of the paper briefly discusses standards of equity. It then develops a model of public sector costs that incorporates variations in cost, outcomes, and efficiency across districts and also presents the DEA model used to adjust expenditures for cost and efficiency differences. The model is applied to data from New York State to measure cost minimizing expenditures per pupil given outputs, assuming that all districts are equally efficient and face the most favorable cost environment. Horizontal equity is analyzed through the standard indicators of overall dispersion of district-by-district spending and categorical
equity through association of district outcomes with measures of their wealth. The concluding section describes policy implications and directions for future research.
2. Equity standards While examination of the equity of local public services has concerned both the level of services provided and the burden on local taxpayers of financing the service, this paper focuses on variations in the provision of school services. Although equivalence of service level across jurisdictions is a reasonable, prima facie, standard of equity for publicly provided goods, it leads, inevitably, to the difficult questions of defining and ultimately measuring public service levels; see Wood (1995). One approach defines service levels produced in terms of quantities of activities performed. As regards schools, activities include classes taught, pupilmiles of transportation provided, and square feet of buildings maintained and heated. An obvious difficulty with applying this approach is the need to control for quality as well as quantity of activities. If the standard of equity, however, is not activities but outcomes, even an index of activities which takes account of quality does not reflect outcomes because it ignores the likelihood that environmental variations across districts require differential activities per pupil. Recent discussions of standards for evaluating school performance use the term ‘‘inadequacy’’ as indicative of insufficient inputs (activities) to produce a specified ‘‘adequate’’ level of educational outcomes. To develop an index reflective of equity (adequacy) of outcomes requires adjustments to expenditure not only for input prices and extra inputs per activity, but also for the differential quantity of activities needed per pupil needed to attain equal (specified) outcomes, and, finally, for the efficiency of transforming activities into outcomes. This is precisely what is accomplished by the estimated minimum required spending derived from the DEA analysis. Horizontal equity is based on view that fairness requires equal services or outcomes across
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
all provider units. Full, or complete, horizontal equity in schooling is attained if all students receive the same level of an appropriate measure of services. The most appropriate such measure for schools is outcomes, although court challenges have focused on spending. Horizontal equity measures gauge how far the actual distribution across districts departs from complete equity. In addition to such conventional statistical measures of variation and of the extent of departure from complete equivalence as the range and the variance, the coefficient of variation and the Gini coefficient, students of school finance have utilized other measures such as the restricted range, federal range ratio, and Brazer’s coefficient of variation, that control in a special way for the extremes of the distribution. See Berne and Steifel (1984) and Fenner (1991) for useful discussions of each measure. An alternative principle, categorical equity, is the view that equity obtains when services are provided without regard to ability to pay (Feldstein, 1975). This view is the basis for the concept of wealth neutrality, a situation where no statistical relationship exists between the level of services provided and community wealth. Categorical equity does not imply absence of variations across districts in service provision or in outcomes; rather, it obtains as long as variations in service provision are attributable to differences in preferences but are not systematically associated with wealth. Thus, categorical equity does not necessarily imply horizontal equity, but horizontal equity does imply categorical equity. There are three standard measures of the extent of wealth neutrality: correlation, slope and elasticity. Each measures the relationship between the level of services or outcomes provided and the ability of a school district to finance these services (local wealth). Perfect categorical equity would be indicated by a correlation coefficient of zero. The slope parameter from a linear regression of services on wealth provides a measure of the impact of increasing wealth on services. Wealth neutrality ensues if the slope parameter is equal to zero. Alternatively, a log-linear regression assumes a constant elasticity; an elasticity of zero indicates wealth neutrality.
645
3. Educational production, costs and efficiency As explained above, most analyses of school equity treat observed expenditure per pupil as indicative of services and/or outcomes. (Berne and Steifel (1984) recognize the importance of adjusting expenditures for resource price differentials. Fenner (1991) develops a model which allows not only for resource price differentials but also for variations in other exogenous cost factors. Neither paper, however, recognizes variations due to inefficiency.) The rationale for this presumption is that higher expenditures per pupil, ceteris paribus, indicate increased inputs, services and eventually outcomes. Consequently, the conclusion is that variations in spending are inequitable. However, this assumes that school districts face the same cost environment and are equally efficient in providing services. Without controlling for these factors, expenditure measures of equity are misleading. We model educational costs with the following standard model: TC ¼ CðS; W ; ZÞ:
ð1Þ
This cost function relates the minimum level of expenditure, i.e. the minimum cost, necessary to provide a given level of services given the production function, exogenous resource prices, and environmental factors. Further, this function provides the basis to analyze differences in cost that arise when services are efficiently provided. The minimum cost of providing a given level of services varies with differential factor prices and/or environmental conditions. That is, there are multiple cost frontiers. Expenditure and outcome under conditions of multiple cost frontiers is shown in Fig. 1, where it is assumed that school districts (indicated as A, B, C, G, F ) face different cost environments (CH ; CM ; CF ) and produce a single outcome. All districts other than G are assumed to be cost minimizers and are therefore on their cost frontier. As shown, district A faces the harshest environment and, therefore, must spend more to produce the given outcome. Districts B, F , and G, on the other hand, face more favorable environments relative to A and,
646
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
Fig. 1. Cost frontiers in the public sector.
therefore, if efficient can provide higher levels of the service with less spending than A. The cost function explicitly assumes that the decision making units are cost minimizers. Based on the bureaucratic model of supply, public choice literature poses an alternative expenditure determination model that allows inefficiency; see Niskanen (1971, 1975). The bureaucratic model postulates that decision makers have an incentive to increase expenditure beyond the minimum cost level in order to obtain pecuniary benefits, leading to budgetary slack. (For a further discussion, see Wyckoff, 1990; Duncombe et al., 1997.) The empirical literature provides evidence that costminimization is not a reasonable assumption, suggest that expenditure variations are largely due to differences in efficiency (see Hanushek, 1986, 1993). The difference between minimum costs and actual per pupil expenditures ðEÞ defines cost inefficiency, where E P CðS; W ; ZÞ:
ð2Þ
In this case, if a school district is cost efficient, observed expenditure represents minimum costs. On the other hand, if a school district is cost inefficient, actual expenditure will be greater than the minimum cost of providing the observed services. Referring to Fig. 1, district G (which by assumption faces the most favorable cost envi-
ronment) is cost inefficient. Since district G has the most favorable cost environment, if efficient it could have provided its observed level of services with the observed expenditure of district F . The errors of analyzing equity using unadjusted observed expenditures are revealed in Fig. 1. From the standpoint of observed expenditure, inequity is perceived since there are three levels of spending. Based on observed expenditures, an equity analysis would inappropriately indicate that district G provides the most services and district F the least. In fact, however, district D (with average observed spending) produces the highest outcome, and F , G, and B all produce the same lower level of outcome. These stylized facts highlight the problem of using unadjusted expenditures to measure the equity of outcome production. Outcome based measures of inequity, however, can be derived from expenditures adjusted by DEA analysis for differences in the cost environment and inefficiency. An equity analysis based on these adjusted expenditures reveals the true magnitude of disparities in outcomes. The operations research technique known as DEA, pioneered in Charnes et al. (1978), provides a measure of Farrell (1957) efficiency. The original DEA linear programming model was introduced to identify technical inefficiency, measured by the equiproportional reduction of actual inputs that is possible, holding output at the observed level. Based on minimal assumptions, the technique identifies the best-practice frontier and evaluates each producer relative to the frontier. Grosskopf and Yaisawarng (1990) extended the approach to allow measurement of cost efficiency given expenditure and output data. Assume that each of N school districts produces L outcomes (i.e. DEA outputs). Let sjl denote the production of output l by district j and Ej denote the observed spending by district j. Now consider the following linear program used in the analysis of school district i (for i ¼ 1; . . . ; N ): ci ¼ Minimize subject to
gi
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652 N X
8l ¼ 1; . . . ; L;
hj sjl P sil
j¼1 N X
h j Ej 6 g i Ei ;
ð3Þ
j¼1 N X
hj ¼ 1;
hj P 0 8j ¼ 1; . . . ; N :
j¼1
The solution of (3) for each school district provides an index consisting not only of relative exogenous costs but also cost inefficiency. School district expenditures are contracted to a bestpractice cost frontier where the minimum cost of outcome production is achieved assuming that each district faces the most-favorable environment and is equally efficient. The resulting DEA measure is useful for deflating expenditures for cost and efficiency differences. Consequently, the adjusted expenditures are a more useful equity measure. The construction and application of the bestpractice frontier is illustrated in Fig. 2, which is similar to Fig. 1. However, rather than showing all cost curves, only the DEA best-practice frontier is illustrated. Contracting each district’s observed expenditure to the best-practice frontier reveals the minimum expenditure required to produce the observed level of outcome assuming that each district has the same favorable environment and is equally efficient. The ratio of the minimum to the observed expenditure is a combined measure of environmental harshness and inefficiency.
647
As shown in the diagram, districts G, B, and F all produce an equivalent outcome of 80. Lowest cost district F produces the outcome by spending $1000. Because F is on the best-practice frontier, F ’s adjusted expenditures would be $1000. Consider next district G, which produces the same outcome as district F . Recalling the earlier assumption that G faces the most favorable environment and is inefficient, we know that G could have produced the same outcome by spending only $1000 (and is thus wasting $1000). The adjusted expenditures for G is therefore $1000. Basing the equity analysis on adjusted expenditures reveals that G and F are providing the same outcome level. The adjusted expenditures of B is also $1000. However, unlike G, B is cost efficient because the $500 difference in adjusted and observed expenditures is attributed to differential cost environments and not inefficiency. Regardless, an equity analysis based on adjusted expenditures reveals that no outcome inequities exist between districts G, B and F . The adjusted expenditures of district A are also $1000 resulting from output slack that remains after projection. District D is on the best-practice frontier and its adjusted and observed expenditure is $1500, reflecting higher outcome production than all other districts. It is essential to recognize that the explanation and illustration of the workings of DEA in Fig. 2 rely on the simplification of a single outcome. In fact, DEA is designed for multiple outcomes, and no simple diagram can fully illustrate its operation. Intuitively, in actual DEA analysis, the linear minimum cost function in Fig. 2 has as its counterpart a minimum cost for all observed combinations of outcomes, and to get the minimum cost for each district, its expenditure is projected to the lowest spending district with an equivalent mix of outcomes.
4. Empirical analysis of New York school districts
Fig. 2. DEA best-practice cost frontier.
We estimate best-practice expenditures per pupil and analyze equity for 631 of the 695 school districts in New York in 1991. Despite limitations
648
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
due to missing observations, the sample appears representative of the major regions in New York State. This section describes measures, data sources, and empirical analysis of education costs, and compares measures of equity under different assumptions regarding environmental cost differentials and school district efficiency. The measure of expenditure is a district’s Approved Operating Expenses (AOE) per pupil. AOE includes salaries and fringe benefits of teachers and other school staff, other instructional expenditure, and all other expenditure related to operation and maintenance of schools. The DEA model (3) presented above contracts expenditures per pupil holding outcomes at the level produced by the districts. We chose three outcomes to use as DEA outputs. The first derives from the state’s Pupil Evaluation Program, given to all third- and sixth-grade students in reading and math. The specific measure is the average percentage of students performing above the standard reference point on these four exams. The second is the percentage of students receiving a Regents diploma upon graduation from high school. Regents diplomas are given only to high
school students who pass standardized state tests. To balance this measure of achievement, the third outcome is the percentage of students who graduate. For a discussion of variables reflecting school outcomes; see Duncombe et al. (1996). Descriptive statistics are reported in Table 1. We also include cost factors to allow us to illustrate the problems of standard equity analyses that use expenditures per pupil. These factors were not used in the DEA model because we are not interested in measuring efficiency per se. Instead, we are interested in deflating expenditures for inefficiency and cost differences that exist in school districts. By employing DEA model (3) we are essentially controlling for efficiency and cost environmental factors. The state average level of observed expenditure is $6054 per pupil. This expenditure, however, captures not only variations in outcomes, but also variations in efficiency and cost environments. After controlling for the environment and the level of inefficiency by multiplying the DEA measure and the expenditures per pupil, we are able to derive adjusted expenditures per pupil; the average adjusted expenditure is $3801.
Table 1 Descriptive statistics (New York school districts in 1991, n ¼ 631) Variable
Mean
Standard deviation
Minimum
Maximum
6054
2143
3166
25,376
Outcomes PEP scores Percent receiving Regents diploma Percent non-dropouts
94.24 40.44 97.59
3.79 13.07 1.84
64.50 0.00 88.10
100.00 75.38 100.00
Cost factors Entry teacher salaries Enrollment Female-headed households (%) Handicapped students (%) Limited English proficiency (%) High school students (%)
24,727 2383 8.79 10.64 0.99 28.97
2957 3025 2.71 3.37 1.27 3.70
12,790 68 2.46 1.63 0 20.09
34,982 46,195 34.68 30.68 1.96 63.10
Efficiency results DEA cost and efficiency index DEA adjusted expenditures
0.67 3801
0.16 1076
0.16 3166
1.00 18,760
Expenditures ($/pupil) a
Sources: New York State Department of Education, Comprehensive Assessment Report, Basic Education Data System and Fiscal Profile, and National Center for Education Statistics, School District Data Book. a The outcomes are the outputs used in the DEA analysis. Projection to the best-practice frontier is achieved with reductions in expenditures holding outcomes constant.
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
The shortcomings of using unadjusted expenditures to analyze equity analysis is illustrated in Table 2, which presents data on observed and adjusted per pupil expenditure and on selected factors indicative of environmental costs for three of the school districts in the study. Two of the districts, F and V , spent approximately the state average of $6050 per pupil; the third, S, spent nearly 50% more. The $3000 difference per pupil appears to indicate a considerable amount of inequity. Closer inspection, however, reveals little inequity in outcomes. While district F ’s spending is essentially equivalent to the state average, it faces the harshest cost environment of the three selected districts. This can be seen by examining the selected cost factors. First, F pays entry level teachers $5000 more than the state average and hence, faces a higher labor price. In addition, relative to the state average, this district has 3600 more pupils, nearly double the percentage of single female-headed household, and eight times the percent of limited English students. Adjusted for these higher costs (and for inefficiency) F ’s expenditure per pupil (of and indicator of outcomes) $3515 is 92% of the state average.
649
Relative to district F , district V pays entry level teachers 33% less. In addition, district V has a more favorable environment due to its relatively low percentages of limited English students, handicapped students and single female-headed households. Consequently, it faces lower costs. While not reflected by the unadjusted expenditure, these lower costs are reflected by the adjusted expenditure of $4337 which is 14% above the adjusted state average, and, implies that district F ’s outcomes are 14% above the state average. This is consistent with the measures of outcomes in Table 2, as district V achieves a 10% higher PEP score than F and graduates 50% more students with a Regents Diploma, and has an above average overall graduation rate. Finally, consider district S, whose observed spending is 33% more than districts F , V and the state average. Despite facing an average cost environment, district S produces outcomes similar to district V . The DEA analysis yields adjusted expenditure of $4109 indicating outcomes slightly less than those produced by V , and 8% above the state average. This is confirmed by examination of the selected output measures. In this case, then, the low DEA cost/efficiency index of 0.444 for district
Table 2 Selected school district results Variable
School district F
V
S
Expenditure ($/pupil) Unadjusted Relative to state average DEA cost and efficiency adjusted Relative to state average DEA cost and efficiency index
6042 0.99 3515 0.92 0.582
6057 1.00 4337 1.14 0.716
9255 1.50 4109 1.08 0.444
Outcomes Pep scores Percent receiving Regents diploma Percent non-dropouts
87.50 30.38 98.50
96.25 60.20 99.10
96.25 58.39 98.90
Cost factors Entry teacher salaries ($) Enrollment Female-headed households (%) Handicapped students (%) Limited English proficiency (%) High school students (%)
29,822 5916 16.35 10.06 8.14 26.96
20,473 1177 5.43 6.94 0.64 30.11
27,451 7362 7.09 10.14 0.88 35.39
650
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
ter adjusting expenditures per pupil using DEA, a significant amount of the perceived inequity disappears. Most equity measures are substantially reduced, ranging from a decline in the range of 30% to a decrease in the restricted range of 74%. The federal range ratio, the variance and Brazer’s coefficient of variation all fell by two-thirds. The most widely used measure of inequity, the Gini coefficient, declined by 44%. (We note, however, that the coefficient of variation only decreases by 3.3%.) These reductions suggest that a considerable amount of what has traditionally been perceived as inequity can be attributed to inefficiency and exogenous costs. These findings constitute strong evidence supporting further development of education cost and efficiency indices and their use in education production and equity research. Past analyses of educational inequities based on measures of unadjusted expenditure per pupil are subject to the biases described above that lead to gross overstatements of inequities. Obviously, this raises serious questions about the often used approach in equity research of measuring differences in resources without accounting for these important factors.
S reflects the presence of inefficiency rather than high cost factors. A proper equity analysis should determine that little inequity exists between S and V , but that F provides significantly fewer services/outcomes than the other two districts. The improper conclusions derived from an analysis of observed expenditure would indicate inequity attributable to the higher spending of S relative to F and V . The proper conclusion is reached only after adjusting expenditures per pupil for inefficiency and cost factors.
5. Equity results Tables 3 and 4 show the results of an examination of expenditure equity in New York State. Variations in AOE per pupil are evaluated using two approaches: (1) with unadjusted AOE, and (2) with AOE adjusted for cost differences and inefficiency using the non-parametric DEA approach. A variety of equity measures are used to compare the outcomes. 5.1. Measures of horizontal equity As presented in Table 3, analysis of unadjusted AOE provides ‘‘evidence’’ of considerable inequality. For example, the federal range ratio indicates that the district at the ninety-fifth percentile of expenditures spends 1.43 times the district at the fifth percentile. Other measures reveal similar evidence of variation. However, af-
5.2. Measures of categorical equity Table 4 presents results analyzing the relationship between district spending and wealth, i.e. categorical equity. Two measures of district wealth are employed: (1) the Combined Wealth Ratio
Table 3 Selected measures of horizontal equity of New York State school districts in 1991 ðn ¼ 631Þ Measure
Unadjusted expenditures
DEA cost and efficiency adjusted expenditures
Percent reduction
Range ($/pupil) Restricted range ($/pupil) Federal range ratio Variance ($/pupil) Coefficient of variation Brazer’s coefficient of variation Gini coefficient Theil’s measure
22,210 5878 1.43 3,521,462 0.30 0.18
15,594 1532 0.47 1,178,004 0.29 0.06
29.8 73.4 67.1 66.6 3.3 66.7
0.16 0.04
0.09 0.03
43.8 25.0
DEA cost and efficiency adjusted expenditures were obtained by multiplying approved operating expenditures by the DEA cost and efficiency index obtained from (3).
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652 Table 4 Selected measures of categorical equity of New York State school districts in 1991 ðn ¼ 631Þ Unadjusted expenditures Correlation CWR FV/pupil Slope CWR FV/pupil Elasticity CWR FV/pupil
0.78 0.72 2242 0.009 0.40 0.35
DEA cost and efficiency adjusted expenditures
Percent reduction
0.38 0.32
51.3 55.6
547 0.002
75.6 77.8
0.13 0.10
67.5 71.4
DEA cost and Efficiency adjusted expenditures were obtained by multiplying approved operating expenditures per pupil by the DEA index. CWR, the combined wealth ratio in 1990–91, is a measure of a district’s relative wealth weighted equally by income and property value per pupil. FV/pupil is the full valuation of property value per pupil in a district.
(CWR) measure used by the state to set its state aid ratio for operating aid, which weights adjusted gross income and assessed property value equally, and (2) the full valuation of property per pupil. The relation between spending and each of the two measures of wealth is estimated by correlation, slope, and elasticity. Separate estimates of these coefficients are made for unadjusted spending (AOE/pupil) and for cost and efficiency adjusted expenditures using DEA. The unadjusted expenditure approach shows a correlation of 0.78 between expenditures and CWR. This correlation measure decreased by 51% after controlling for costs and efficiency using DEA. The correlation between expenditures and property value decreased by a slightly larger amount. Notably, all other measures of categorical equity are reduced by at least two-thirds. Thus, the correlation between district wealth and observed spending is stronger than the correlation between district wealth and service provision, as measured by adjusted expenditures. This finding is consistent with the conclusion of Duncombe et al. (1997) that wealthier districts are more inefficient, although without analysis of the relation of costs and wealth it is not possible to distinguish the effects of inefficiency and high cost in accounting for the lower
651
correlations of outcomes to wealth. The other categorical equity measures further support the hypothesis that the perceived inequity based on district wealth is not so strong when costs and efficiency are properly controlled for. Still, categorical inequity remains even after adjusting expenditure. Not surprisingly, wealthier districts provide a higher level of services. These are extremely important and disturbing findings for education equity research, particularly considering the actions taken by state courts based in part on equity study findings. Though this study does not claim that expenditure inequities or the relationship between expenditures and wealth in New York school districts are non-existent, it does find that costs and inefficiency account for a substantial portion of the observed inequities.
6. Conclusions This paper has focused on central issues that have not addressed in the school finance equity debate. Equity analyses based on unadjusted expenditure per pupil fail to recognize that school districts generally face different cost environments and that educational services may not be efficiently provided. As a result, to a considerable extent variations in spending reflect differences in cost environments and relative inefficiencies and not differences in educational outcomes. Distorted measures of inequity result if these other variations are not properly controlled. The results of this analysis indicate that most perceived inequity actually is due to these factors and not to variations in school outcomes. A major implication of these findings is that there should be considerably more effort devoted to studies of school efficiency including those features of school organization and governance associated with lesser or greater levels of outcomes per unit of input. Whatever the limitations of the specific methods used to control for costs and efficiency, this study clearly shows that less reliance should be given to unadjusted measures of spending in determining whether arrangements for school finance are in conflict with state constitutional provisions. It raises fundamental questions
652
J. Ruggiero et al. / European Journal of Operational Research 142 (2002) 642–652
regarding state policies intended to reduce inequities among local schools. Clearly, aid needs to be based on cost considerations as well as fiscal capacity, and policies need to be examined as regards their effects on efficient operation of schools. Finally, the analysis has illustrated an important use for DEA to analyze a key area of public policy.
References Berne, R., Steifel, L., 1984. The Measurement of Equity in School Finance: Conceptual, Methodological, and Empirical Dimensions. Johns Hopkins University Press, Baltimore, MD. Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the efficiency of decision making units. European Journal of Operational Research 2, 429–444. Duncombe, W., Miner, J., Ruggiero, J., 1997. Empirical evaluation of bureaucratic models of inefficiency. Public Choice 93, 1–18. Duncombe, W., Ruggiero, J., Yinger, J., 1996. Alternative approaches to measuring the cost of education. In: Helen, F.L. (Ed.), Holding Schools Accountable: Performance Based Reform in Education. Brookings Institution, Washington, DC, pp. 327–356. Farrell, M.J., 1957. The measurement of productive efficiency. Journal of the Royal Statistical Society Series A, General 120, 253–281.
Feldstein, M.S., 1975. Wealth neutrality and local choice in public education. American Economic Review 65, 75– 89. Fenner, R., 1991. The effect of equity of New York state’s system of aid for education. Ph.D. Dissertation, Department of Economics, Syracuse University, Syracuse, NY. Grosskopf, S., Yaisawarng, S., 1990. Economies of scope in the provision of local public services. National Tax Journal 43, 61–74. Hanushek, E., 1986. The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature 24, 1141–1177. Hanushek, E., 1993. Can equity be separated from efficiency in school finance debates. In: Emily, H. (Ed.), Essays on the Economics of Education. Upjohn Institute, pp. 35– 74. Niskanen, W.A., 1971. Bureaucracy and Representative Government. Aldine-Atherton, Chicago. Niskanen, W.A., 1975. Bureaucrats and politicians. Journal of Law and Economics 18, 617–643. Odden, A., Picus, L., 1992. School Finance: A Policy Perspective. McGraw-Hill, New York. Reschovsky, A., 1994. Fiscal equalization and school finance. National Tax Journal 47, 211–224. Serrano V. Priest, 1971. 487 P.2d 1241. Wood, R.C., 1995. Adequacy issues in recent educational finance litigation. Developments in School Finance, National Center for Educational Statistics, US Department of Education, pp. 29–37. Wyckoff, P., 1990. The simple analytics of slack-maximizing bureaucracy. Public Choice 67, 35–67.