- Email: [email protected]
A note on robustness of the efficient DMUs in data envelopment analysis
European Journal of Operational Research 112 (1999) 240±244
Short Communication
A note on robustness of the ecient DMUs in data envelopment analysis Valter Boljuncic
1
University of Rijeka, Faculty of Economics and Tourism ``Dr. Mijo Mirkovi c'', Preradovi ceva 1/1, 52100 Pula, Croatia Received 13 October 1997; accepted 11 May 1998
Abstract Sensitivity analysis of extreme ecient DMUs was studied by Zhu (J. Zhu, EOR 90 (3) (1996) 451±460) using the modi®ed CCR (A. Charnes, W.W. Cooper, E. Rhodes, EOR 2 (1978) 429±444) model. Upward proportional variations of inputs and downward proportional variations of outputs were considered. The procedure suggested by Zhu is a new approach to sensitivity analysis and can result in sucient and necessary conditions on the changes which do not alter the eciency of DMUs. However, in certain instances the results presented by Zhu do not hold. The aim of this note is to point out shortcomings which can be met following procedure proposed by Zhu. This includes ineciency of socalled projected points, domination of obtained hyperplane, case of simultaneos changes of inputs and outputs, and case of infeasibility even if all data are greater than 0. Ó 1999 Elsevier Science B.V. All rights reserved. Keywords: DEA; CCR model; Sensitivity analysis
1. Introduction Data envelopment analysis (DEA) is a set of methods and models, based on mathematical programming, and used for assessment of relative eciency of observed decision making units (DMUs). Charnes et al. [1] developed a classical model, to be referred to as the CCR model. After performing analysis, DMU is classi®ed as ecient or inecient. Following the notation from [2], we assume that there are n DMUs, each using m inputs and producing s outputs. For DMUj we use
1
E-mail: [email protected].
the notation xij ; i 1; . . . ; m for inputs and yrj ; r 1; . . . ; s for outputs. One of the topics of interest in DEA is sensitivity analysis of a speci®c DMU0 , unit under evaluation. Sensitivity analysis is used to assess how much inputs and outputs of DMU0 can change, without changing its eciency ranking. We are interested in sensitivity analysis of an extreme ecient DMU, following the classi®cation given in the CCT model, [3], where we consider proportional increase of inputs and decrease of outputs. Hence, for DMU0 we have the following perturbed data: x^i0 bi xi0 ;
0377-2217/99/$ ± see front matter Ó 1999 Elsevier Science B.V. All rights reserved. PII S 0 3 7 7 - 2 2 1 7 ( 9 8 ) 0 0 1 8 5 - 4
bi P 1; i 1; . . . ; m;
1
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
y^r0 ar yr0 ;
0 6 ar 6 1; r 1; . . . ; s:
2
Dierent approaches to sensitivity analysis exist, and one of the most important goals in each of these approaches is to ®nd out sucient and possibly necessary conditions for changes (1) and 2 under which the DMU0 will remain ecient. Zhu [2], proposes a new approach to sensitivity analysis, resulting in regions of stability of DMU0 . This is done using so-called projected points of DMU0 on the production possibility set generated with remaining n ) 1 DMUs. We can have m + s projected points, one along each input and each output. The projected point along the ith input is obtained increasing, minimally, this input, as in Eq. (1), with all other data unchanged, until such a point can be represented as a linear combination of other DMUs. This is done using the modi®ed CCR model in which the observed DMU0 is excluded from the reference set. For the case of inputs we have m linear programs of the form, min s:t:
bk n X
kj xkj 6 bk xko ;
j1 j60
n X
kj xij 6 xio ;
i 1; . . . ; m; i 6 k;
j1 j60
n X
kj yrj P yro ;
3
r 1; . . . ; s;
j1 j60
kj ; b k P 0 and similarly for the case of outputs. Based on obtained projected points, a hyperplane containing them is de®ned via the system of linear equations. This hyperplane serves as a border to possible changes of type (1) and (2) which do not alter the eciency of the perturbed DMU0 , resulting in sucient and necessary conditions that possible changes, (1) and (2), must ful®ll in order to preserve the eciency of the perturbed DMU0 . Zhu in [2] considers situations where: only inputs are changed (Theorem 2); only outputs are changed (Theorem 4); and the general case where inputs and outputs are changed (Theorem 5). A situation
241
where infeasibility of program (3) is present is also considered (Theorems A.1 and A.2). However, in certain instances these results do not hold. This note presents some examples to show that. In Section 2 we show that the projected point can be inecient. In Section 3 we show that the obtained hyperplane can be dominated, and is no more border to possible changes of type (1) and (2). In Section 4 we show that in the general case where inputs and outputs are changed, the obtained hyperplane passes through the origin if it is not dominated, and in Section 5 we show that infeasibility can occur even when all data are greater than 0. 2. Ineciency of projected points In order to obtain projected point along the kth input we solve program (3). This program can be infeasible. If the program is feasible, then bk ,the optimal objective function value of Eq. (3), is greater than 1. Theorem 1 in [2] states that DMU0 stays ecient if and only if we change kth input with bk , as Eq. (1), where 1 6 bk 6 bk . Let us solve the following example with three DMUs, each one using two inputs to produce two outputs (see Table 1). All three DMUs are ecient. If we use the procedure suggested in [2] on DMU3 we obtain b1 72, as optimal objective function value for the ®rst input, and the projected point is P13 7; 2; 1; 3T . This point is not ecient, it is dominated with the point obtained as the linear combination of DMU1 and DMU2 . In fact, it is possible that there exists an optimal solution to
Table 1 Data for the three DMUs with two outputs and two inputs ± case of ineciency of projected point DMU1
DMU2
DMU3
Inputs x1j x2j
1 5
5 1
2 2
Outputs y1j y2j
2 2
2 2
1 3
242
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
Eq. (3) with slack variables greater than 0, in this particular case we have slack variable in the third constraint, s 1 2. However, the obtained projected point is part of the frontier of the production possibility set, but it is in the set F [3,4]. It is scale ecient, but not technically ecient. To avoid this, slack variables should be considered in the way CCT model does [3], or the two-step approach used by Ali and Seiford [5]. Further, if we proceed we obtain b2 72 as optimal objective function value for the second input. Following the procedure from [2], we obtain input hyperplane
Table 2 Data for the four DMUs with one output and two inputs ± case of dominated hyperplane
2 2 b b 1: 4 9 1 9 2 Theorem 2 from [1] states that the observed DMU stays ecient after changes (1) if and only if 7 b1 ; b2 2 C b1 ; b2 j 1 6 b1 6 ; 2 7 2 2 5 1 6 b 2 6 ; b1 b2 6 1 : 2 9 9
7
If we use any linear combination of b1 and b2 which results with equality in Eq. (5), i.e. 2=9b1 2=9b2 1; we obtain inecient point, which is scale ecient, element of the frontier, but technically inecient. 3. Dominated hyperplane Next, it can be shown that even if we obtain ef®cient projected points with Eq. (3), in some situations Theorem 2 from [2] does not hold. This is the case when the obtained input hyperplane is dominated by some point from the production possibility set. The example of four DMUs with one output and two inputs (see Table 2) will show it. DMU1 , DMU2 and DMU3 are ecient but DMU4 is not. We consider DMU3 . Using Eq. (3) we obtain b1 2 and b2 2. The obtained input hyperplane is of the form 1 1 6 b b 1; 3 1 3 2 and Theorem 2 states that after changes (1) the perturbed DMU3 is ecient if and only if
DMU1
DMU2
DMU3
DMU4
Inputs x1j x2j
1 10
10 1
3 3
4 4
Output y1j
1
1
1
1
b1 ; b2 2 C
b1 ; b2 j 1 6 b1 6 2; 1 1 1 6 b2 6 2; b1 b2 6 1 : 3 3
In the above example this does not hold because if we take b1 b2 32 ; Eq. (7) is satis®ed, but the perturbed DMU3 has the following data: y^13 y13 1; 9 x^13 ; 2 9 x^23 ; 2
8
and it is not ecient because DMU4 dominates it, so Eq. (7) does not represent possible changes to inputs of DMU3 which will not alter its eciency. The same is for the case of output hyperplane, and for the general case where inputs and outputs are changed simultaneously.
4. Case of simultaneous changes of inputs and outputs Zhu in [1] gives also procedure in the case of simultaneous changes of inputs and outputs, resulting in Theorem 5 where sucient conditions were given. Problems of ineciency of projected points, or dominance of obtained hyperplane are also present. But, there are also some dierences in a way hyperplane is obtained. If we look at the example from Table 4 in [2], we have values b1 32 ; b2 32 ; a 45 : The system of equation is
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
3 C1 C2 C3 1; 2 3 9 C1 C2 C3 1; 2 4 C1 C2 C3 1: 5 The solution given in [2] is C1 C2 2 and C5 ÿ5, and the hyperplane is of the form 2b1 2b2 ÿ 5a 1:
10
However, this solution does not satisfy the equations in (9). Substituting the values results with zero right hand side. Next, the determinant of the system (9) is 0. This fact is not the property of the speci®c example. Recall that when we exclude the DMU0 from the reference set, the remaining DMUs form the production possibility set. The hyperplane we want to obtain should be one of the generating hyperplanes of this set, if there is no dominance. Because we have constant returns to scale, these hyperplanes pass through the origin, resulting that we have 0 as a right hand side in Eq. (10). Determinant of the system should also be considered. If the determinant is not 0, what can also be the case, then we should have 1 as right hand side, other ways we will obtain trivial solution. In that case the obtained hyperplane is dominated. So, if Theorem 5 in [2] holds, we should have 0 as right hand side in the equation of the hyperplane. Obtaining hyperplane which passes through the origin does not assure that it is not dominated. Also, the results in [2] should be compared with [6], where the CCR model was used; hence this work is relevant for the analysis of [2], and not with [7], where the additive model was considered. 5. Infeasibility Zhu in [2] states that infeasibility of the modi®ed CCR model is due to certain pattern of zeroes in data. He uses the modi®ed CCR model with all inputs changed and also compares it with the work of Andersen and Petersen [8]. However, the situation with all inputs changed cannot be used as a representative since there is a loss of generality. If certain pattern of zeroes is present (or not present),
243
Table 3 Data for the three DMUs with one output and two inputs ± case of infeasibility DMU1
DMU2
DMU3
Inputs x1j x2j
3 4
4 1
2 2
Output y1j
1
1
1
it can be the reason for the infeasibility of the modi®ed CCR model. This can be seen in [3] where the situation with zeroes in data was considered. Hence, the program (A.1) in [2] cannot be regarded as a representative. But, if some inputs are not changed, then infeasibility can occur even with all data greater than 0. We will use the example where we have 3 DMUs with 1 output and 2 inputs (see Table 3). DMU3 is extreme ecient, all data are greater than 0, but program (3) is infeasible for the second input. Next, if we want to perform the procedure described in [2] for inputs, then no input of DMU0 can be zero. Let us suppose that x1o equals 0. From program (3) it is obvious that b1 x1o 0 x1o for any value of b1 . So, if the program is feasible, this will result that DMU0 can be represented a as linear combination of other DMUs, which contradicts with the assumption that it is extreme ecient. 6. Conclusion The procedure proposed by Zhu in [2] is a new approach to sensitivity analysis in DEA. It can result in sucient and necessary conditions on changes to data that extreme ecient DMU0 can have, Eqs. (1) and (2). However, as shown with previous examples, in certain situations theorems from [2] do not hold. This paper points out some of the shortcomings in order to bypass them in future works. References [1] A. Charnes, W.W. Cooper, E. Rhodes, Measuring the eciency of decision making units, European Journal of Operational Research 2 (1978) 429±444.
244
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
[2] J. Zhu, Robustness of the ecient DMUs in data envelopment analysis, European Journal of Operational Research 90 (3) (1996) 451±460. [3] A. Charnes, W.W. Cooper, R.M. Thrall, A structure for classifying and characterizing eciency and ineciency in data envelopment analysis, The Journal of Productivity Analysis 2 (1991) 197±237. [4] R.D. Banker, A. Charnes, W.W. Cooper, Some models for estimating technical and scale ineciencies in data envelopment analysis, Management Science 30 (9) (1984) 1078± 1092. [5] A.I. Ali, L.M. Seiford, The mathematical programming approach to eciency measurement, in: H. Fried, C.A.
Knox Lovell, S. Schmid (Eds.), The Measurement of Productivity Eciency: Techniques and Applications, Oxford University Press, London, 1993. [6] A. Charnes, L. Neralic, Sensitivity analysis in data envelopment analysis 3, Glasnik Matematicki 27 (47) (1992) 191± 201. [7] A. Charnes, L. Neralic, Sensitivity analysis of the additive model in data envelopment analysis, European Journal of Operational Research 48 (1990) 332±341. [8] P. Andersen, N.C. Petersen, A procedure for ranking ecient units in data envelopment analysis, Management Science 39 (1993) 1261±1264.
Short Communication
A note on robustness of the ecient DMUs in data envelopment analysis Valter Boljuncic
1
University of Rijeka, Faculty of Economics and Tourism ``Dr. Mijo Mirkovi c'', Preradovi ceva 1/1, 52100 Pula, Croatia Received 13 October 1997; accepted 11 May 1998
Abstract Sensitivity analysis of extreme ecient DMUs was studied by Zhu (J. Zhu, EOR 90 (3) (1996) 451±460) using the modi®ed CCR (A. Charnes, W.W. Cooper, E. Rhodes, EOR 2 (1978) 429±444) model. Upward proportional variations of inputs and downward proportional variations of outputs were considered. The procedure suggested by Zhu is a new approach to sensitivity analysis and can result in sucient and necessary conditions on the changes which do not alter the eciency of DMUs. However, in certain instances the results presented by Zhu do not hold. The aim of this note is to point out shortcomings which can be met following procedure proposed by Zhu. This includes ineciency of socalled projected points, domination of obtained hyperplane, case of simultaneos changes of inputs and outputs, and case of infeasibility even if all data are greater than 0. Ó 1999 Elsevier Science B.V. All rights reserved. Keywords: DEA; CCR model; Sensitivity analysis
1. Introduction Data envelopment analysis (DEA) is a set of methods and models, based on mathematical programming, and used for assessment of relative eciency of observed decision making units (DMUs). Charnes et al. [1] developed a classical model, to be referred to as the CCR model. After performing analysis, DMU is classi®ed as ecient or inecient. Following the notation from [2], we assume that there are n DMUs, each using m inputs and producing s outputs. For DMUj we use
1
E-mail: [email protected].
the notation xij ; i 1; . . . ; m for inputs and yrj ; r 1; . . . ; s for outputs. One of the topics of interest in DEA is sensitivity analysis of a speci®c DMU0 , unit under evaluation. Sensitivity analysis is used to assess how much inputs and outputs of DMU0 can change, without changing its eciency ranking. We are interested in sensitivity analysis of an extreme ecient DMU, following the classi®cation given in the CCT model, [3], where we consider proportional increase of inputs and decrease of outputs. Hence, for DMU0 we have the following perturbed data: x^i0 bi xi0 ;
0377-2217/99/$ ± see front matter Ó 1999 Elsevier Science B.V. All rights reserved. PII S 0 3 7 7 - 2 2 1 7 ( 9 8 ) 0 0 1 8 5 - 4
bi P 1; i 1; . . . ; m;
1
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
y^r0 ar yr0 ;
0 6 ar 6 1; r 1; . . . ; s:
2
Dierent approaches to sensitivity analysis exist, and one of the most important goals in each of these approaches is to ®nd out sucient and possibly necessary conditions for changes (1) and 2 under which the DMU0 will remain ecient. Zhu [2], proposes a new approach to sensitivity analysis, resulting in regions of stability of DMU0 . This is done using so-called projected points of DMU0 on the production possibility set generated with remaining n ) 1 DMUs. We can have m + s projected points, one along each input and each output. The projected point along the ith input is obtained increasing, minimally, this input, as in Eq. (1), with all other data unchanged, until such a point can be represented as a linear combination of other DMUs. This is done using the modi®ed CCR model in which the observed DMU0 is excluded from the reference set. For the case of inputs we have m linear programs of the form, min s:t:
bk n X
kj xkj 6 bk xko ;
j1 j60
n X
kj xij 6 xio ;
i 1; . . . ; m; i 6 k;
j1 j60
n X
kj yrj P yro ;
3
r 1; . . . ; s;
j1 j60
kj ; b k P 0 and similarly for the case of outputs. Based on obtained projected points, a hyperplane containing them is de®ned via the system of linear equations. This hyperplane serves as a border to possible changes of type (1) and (2) which do not alter the eciency of the perturbed DMU0 , resulting in sucient and necessary conditions that possible changes, (1) and (2), must ful®ll in order to preserve the eciency of the perturbed DMU0 . Zhu in [2] considers situations where: only inputs are changed (Theorem 2); only outputs are changed (Theorem 4); and the general case where inputs and outputs are changed (Theorem 5). A situation
241
where infeasibility of program (3) is present is also considered (Theorems A.1 and A.2). However, in certain instances these results do not hold. This note presents some examples to show that. In Section 2 we show that the projected point can be inecient. In Section 3 we show that the obtained hyperplane can be dominated, and is no more border to possible changes of type (1) and (2). In Section 4 we show that in the general case where inputs and outputs are changed, the obtained hyperplane passes through the origin if it is not dominated, and in Section 5 we show that infeasibility can occur even when all data are greater than 0. 2. Ineciency of projected points In order to obtain projected point along the kth input we solve program (3). This program can be infeasible. If the program is feasible, then bk ,the optimal objective function value of Eq. (3), is greater than 1. Theorem 1 in [2] states that DMU0 stays ecient if and only if we change kth input with bk , as Eq. (1), where 1 6 bk 6 bk . Let us solve the following example with three DMUs, each one using two inputs to produce two outputs (see Table 1). All three DMUs are ecient. If we use the procedure suggested in [2] on DMU3 we obtain b1 72, as optimal objective function value for the ®rst input, and the projected point is P13 7; 2; 1; 3T . This point is not ecient, it is dominated with the point obtained as the linear combination of DMU1 and DMU2 . In fact, it is possible that there exists an optimal solution to
Table 1 Data for the three DMUs with two outputs and two inputs ± case of ineciency of projected point DMU1
DMU2
DMU3
Inputs x1j x2j
1 5
5 1
2 2
Outputs y1j y2j
2 2
2 2
1 3
242
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
Eq. (3) with slack variables greater than 0, in this particular case we have slack variable in the third constraint, s 1 2. However, the obtained projected point is part of the frontier of the production possibility set, but it is in the set F [3,4]. It is scale ecient, but not technically ecient. To avoid this, slack variables should be considered in the way CCT model does [3], or the two-step approach used by Ali and Seiford [5]. Further, if we proceed we obtain b2 72 as optimal objective function value for the second input. Following the procedure from [2], we obtain input hyperplane
Table 2 Data for the four DMUs with one output and two inputs ± case of dominated hyperplane
2 2 b b 1: 4 9 1 9 2 Theorem 2 from [1] states that the observed DMU stays ecient after changes (1) if and only if 7 b1 ; b2 2 C b1 ; b2 j 1 6 b1 6 ; 2 7 2 2 5 1 6 b 2 6 ; b1 b2 6 1 : 2 9 9
7
If we use any linear combination of b1 and b2 which results with equality in Eq. (5), i.e. 2=9b1 2=9b2 1; we obtain inecient point, which is scale ecient, element of the frontier, but technically inecient. 3. Dominated hyperplane Next, it can be shown that even if we obtain ef®cient projected points with Eq. (3), in some situations Theorem 2 from [2] does not hold. This is the case when the obtained input hyperplane is dominated by some point from the production possibility set. The example of four DMUs with one output and two inputs (see Table 2) will show it. DMU1 , DMU2 and DMU3 are ecient but DMU4 is not. We consider DMU3 . Using Eq. (3) we obtain b1 2 and b2 2. The obtained input hyperplane is of the form 1 1 6 b b 1; 3 1 3 2 and Theorem 2 states that after changes (1) the perturbed DMU3 is ecient if and only if
DMU1
DMU2
DMU3
DMU4
Inputs x1j x2j
1 10
10 1
3 3
4 4
Output y1j
1
1
1
1
b1 ; b2 2 C
b1 ; b2 j 1 6 b1 6 2; 1 1 1 6 b2 6 2; b1 b2 6 1 : 3 3
In the above example this does not hold because if we take b1 b2 32 ; Eq. (7) is satis®ed, but the perturbed DMU3 has the following data: y^13 y13 1; 9 x^13 ; 2 9 x^23 ; 2
8
and it is not ecient because DMU4 dominates it, so Eq. (7) does not represent possible changes to inputs of DMU3 which will not alter its eciency. The same is for the case of output hyperplane, and for the general case where inputs and outputs are changed simultaneously.
4. Case of simultaneous changes of inputs and outputs Zhu in [1] gives also procedure in the case of simultaneous changes of inputs and outputs, resulting in Theorem 5 where sucient conditions were given. Problems of ineciency of projected points, or dominance of obtained hyperplane are also present. But, there are also some dierences in a way hyperplane is obtained. If we look at the example from Table 4 in [2], we have values b1 32 ; b2 32 ; a 45 : The system of equation is
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
3 C1 C2 C3 1; 2 3 9 C1 C2 C3 1; 2 4 C1 C2 C3 1: 5 The solution given in [2] is C1 C2 2 and C5 ÿ5, and the hyperplane is of the form 2b1 2b2 ÿ 5a 1:
10
However, this solution does not satisfy the equations in (9). Substituting the values results with zero right hand side. Next, the determinant of the system (9) is 0. This fact is not the property of the speci®c example. Recall that when we exclude the DMU0 from the reference set, the remaining DMUs form the production possibility set. The hyperplane we want to obtain should be one of the generating hyperplanes of this set, if there is no dominance. Because we have constant returns to scale, these hyperplanes pass through the origin, resulting that we have 0 as a right hand side in Eq. (10). Determinant of the system should also be considered. If the determinant is not 0, what can also be the case, then we should have 1 as right hand side, other ways we will obtain trivial solution. In that case the obtained hyperplane is dominated. So, if Theorem 5 in [2] holds, we should have 0 as right hand side in the equation of the hyperplane. Obtaining hyperplane which passes through the origin does not assure that it is not dominated. Also, the results in [2] should be compared with [6], where the CCR model was used; hence this work is relevant for the analysis of [2], and not with [7], where the additive model was considered. 5. Infeasibility Zhu in [2] states that infeasibility of the modi®ed CCR model is due to certain pattern of zeroes in data. He uses the modi®ed CCR model with all inputs changed and also compares it with the work of Andersen and Petersen [8]. However, the situation with all inputs changed cannot be used as a representative since there is a loss of generality. If certain pattern of zeroes is present (or not present),
243
Table 3 Data for the three DMUs with one output and two inputs ± case of infeasibility DMU1
DMU2
DMU3
Inputs x1j x2j
3 4
4 1
2 2
Output y1j
1
1
1
it can be the reason for the infeasibility of the modi®ed CCR model. This can be seen in [3] where the situation with zeroes in data was considered. Hence, the program (A.1) in [2] cannot be regarded as a representative. But, if some inputs are not changed, then infeasibility can occur even with all data greater than 0. We will use the example where we have 3 DMUs with 1 output and 2 inputs (see Table 3). DMU3 is extreme ecient, all data are greater than 0, but program (3) is infeasible for the second input. Next, if we want to perform the procedure described in [2] for inputs, then no input of DMU0 can be zero. Let us suppose that x1o equals 0. From program (3) it is obvious that b1 x1o 0 x1o for any value of b1 . So, if the program is feasible, this will result that DMU0 can be represented a as linear combination of other DMUs, which contradicts with the assumption that it is extreme ecient. 6. Conclusion The procedure proposed by Zhu in [2] is a new approach to sensitivity analysis in DEA. It can result in sucient and necessary conditions on changes to data that extreme ecient DMU0 can have, Eqs. (1) and (2). However, as shown with previous examples, in certain situations theorems from [2] do not hold. This paper points out some of the shortcomings in order to bypass them in future works. References [1] A. Charnes, W.W. Cooper, E. Rhodes, Measuring the eciency of decision making units, European Journal of Operational Research 2 (1978) 429±444.
244
V. Boljuncic / European Journal of Operational Research 112 (1999) 240±244
[2] J. Zhu, Robustness of the ecient DMUs in data envelopment analysis, European Journal of Operational Research 90 (3) (1996) 451±460. [3] A. Charnes, W.W. Cooper, R.M. Thrall, A structure for classifying and characterizing eciency and ineciency in data envelopment analysis, The Journal of Productivity Analysis 2 (1991) 197±237. [4] R.D. Banker, A. Charnes, W.W. Cooper, Some models for estimating technical and scale ineciencies in data envelopment analysis, Management Science 30 (9) (1984) 1078± 1092. [5] A.I. Ali, L.M. Seiford, The mathematical programming approach to eciency measurement, in: H. Fried, C.A.
Knox Lovell, S. Schmid (Eds.), The Measurement of Productivity Eciency: Techniques and Applications, Oxford University Press, London, 1993. [6] A. Charnes, L. Neralic, Sensitivity analysis in data envelopment analysis 3, Glasnik Matematicki 27 (47) (1992) 191± 201. [7] A. Charnes, L. Neralic, Sensitivity analysis of the additive model in data envelopment analysis, European Journal of Operational Research 48 (1990) 332±341. [8] P. Andersen, N.C. Petersen, A procedure for ranking ecient units in data envelopment analysis, Management Science 39 (1993) 1261±1264.