- Email: [email protected]
Nonparametric Performance Evaluation with Multiple Constituencies
Copyright @ IFAC Management and Control of Production and Logistics, Grenoble, France, 2000
NONPARAMETRIC PERFORMANCE EVALUATION WITH MULTIPLE CONSTITUENCIES 1 M.L. Bougnol, J.H. Dubi,2 N.K. Worner
The University of Mississippi, University, MS 38677
Abstract: We extend the familiar notion in DEA to the case when the assessment of some attributes' desirability is a matter of judgment depending on the constituency. The generalization we present consists of treating the attributes of the entities in the study as potentially desirable and undesirable. An application of this extension is in manufacturing where, a particular process or product attribute may be considered desirable by some constituencies and undesirable by others. The new procedure describes how to ascertain what constituency would find which units efficient and inefficient using standard nonparametric efficient frontier criteria. Copyright © 2000IFAC. Keywords: Linear programming, efficiency enhancement, productivity.
1. INTRODUCTION
DEA to deal with a common set of performance attributes directly rather than "inputs" and "outputs" as in the traditional model. The method is developed from first principles based on proper definitions for dominance with respect to performance criteria and it is shown to be equivalent to the variable returns DEA model. The framework approaches the problem of measuring and comparing relative performance based on multiple attributes directly and intuitively. The paper illustrates the use of this model with an application in manufacturing where Taguchi methods are used to assess different processes.
Data Envelopment Analysis (DEA) is a nonparametric methodology to assess efficiency of a collection of functionally similar "decision making units" to transform inputs into outputs. DEA was introduced by Charnes et al. in 1978 and it has evolved to the point where over a thousand articles have been published about the theory and applications (Seiford [1996]). This methodology requires no assumption about a functional form for the efficient frontier and therefore, no parameter estimation, making it useful in a wide variety of applications.
The present paper extends the idea in Caporaletti et al. [1999] to include constituencies. Constituencies classify the elements in the list of attributes in a nonparametric frontier performance model as either "desirable" or "undesirable." Different constituencies provide different classifications. This extension adapts DEA to address a whole new category of situations where the "inputs" and "outputs" are not inherent to the model but rather depend and vary according to constituencies. Issues
The article by Caporaletti et al. [1999] presents an adaptation of the standard, variable returns, DEA model to compare relative performance using multiple attributes. The idea in that paper is to adapt 1 This work is part of ongoing research funded by grant NOOOl4-99-1-0719 from the Office of Naval Research. 2 Corresponding Author: School of Business, The University of Mississippi, University, MS 38677; email: [email protected].
973
Table 1.
defined by a data set consisting of n points, {a l , ... ,an}, each with m components: thus, a j = (a{, ... , a~). Components correspond to attributes of the model. The value of each component is the magnitude of the attribute with which it is associated.
Attribute Data for Nine Products Product: 1 2 3 4 5 6 7 8 9
Attr. 1 17 9 1 17 19 18 3 1 2
Attr.2 14 3 21 20 5 3 4 19 17
Attr. 3 19 9 14 1 21 1 15 9 13
The third aspect of the model are the 'constituencies.' In an application, a constituency may be a group somehow affected by the n entities, and partial to each of their m attributes. Examples include customers if the entities are products or voters within the relevant electoral district if the entities are, say, government projects. We say that an attribute is 'desirable' for a constituency if the constituency considers greater magnitudes preferable; and conversely, 'undesirable' if less is better. In terms of this work, a constituency has the following formal definition.
that arise include: which entities being compared are dominant for a given constituency; what constituencies consider a given entity dominant; for any entity, which are the potential constituencies, if any, that would dominate it; and, for a given constituency, how should the values of attributes be modified for an entity to become dominant. We will refer to the following simple contrived example in the course of the presentation. Imagine nine products to be marketed and sold worldwide. The markets for the cars will be Asia, Europe, and North America. Table 1 gives the data for three attributes for each of the nine products.
Definition 1. A constituency is an m dimensional vector 8£ composed of 1s and -ls. The presence of a '1' in location i indicates that constituency f judges attribute i as desirable; conversely, a '-1' indicates that this constituency judges this attribute as undesirable. A study involving m attributes can have 2m constituencies although they may not all exist or be considered in an actual application. As long as more than one constituency is affected, the model is relevant. An entity will be considered "dominant" by some constituency if it satisfies the following formal definition.
Imagine further that marketing research reveals that the attributes are valued differently in the three markets. For example, the target customer in North America values Attributes 1 and 2 and considers Attribute 3 undesirable; the customers in Europe consider Attribute 2 and 3 desirable but Attribute 1 is a drawback; finally, customers in Asia value only Attribute 1. The questions faced by the manufacturer are:
Definition 2. An entity is dominant with respect to a constituency, 8£, if it is impossible to find a convex combination of the data of the remaining n - 1 entities such that:
(1) For each constituency, which are the products that are, in some sense, dominant? (2) For each product, which of the three constituencies consider it, in the same sense, dominant? (3) Extending the question above, are there other 'potential' constituencies, beyond the three in the original model, that might prefer a product? (4) Which products, if any, are never dominant in any real or potential market? and (5) For any products dominated for some constituency, what modifications would make them dominant?
(1) For every desirable attribute the value of the combination is greater than or equal to that of the entity being tested; and (2) For every undesirable attribute the value of the combination is less than or equal to that of the entity being tested.
3. THEORETICAL DEVELOPMENT
The model we construct next will address the realistic and interesting questions and issues motivated by this simple example.
Definition 2 in the previous section establishes how entities will be classified with respect to constituencies. In this section we provide the mathematical tools to make this classification.
2. THE MODEL
The classification is based on the solution to the following primal dual pair of linear programs where j* is the index of the entity under examination:
The model involves n entities (e.g., projects, processes, products, etc.) that can be characterized by the same m attributes. The model is 974
(pi) min w(f) = (a i ,7I") 11"./3
s.t.
j
(a ,7I") (cl - al", 71") l5 i 7l"
+ 13 + 13:0::::
dominant: z*(I) = 1.27, z*(2) = 1.06, and z*(3) = 2.50. Next, we introduce the notion of constituency benchmarking.
0; \lj,j i-j*
= 1; :0::::
0;
Definition 3. Let al" be a dominated entity for constituency 8i . Then, its benchmark is the vector aj"' (f) = L:j a j A*(fk
o
+ (a i -
i ai ; a J'. )0 ...... =
As required, any benchmark will score "1" in the appropriate LP. The definition for a benchmark provides the formula for identifying it. So, for instance, in the earlier production example, when Product 9 is evaluated with respect to Constituency 1 (f = 1) using linear program (pi), the optimal objective function value is w*(I) = 1.27 > 1; the indication we need to conclude that Product 9 is dominated for Constituency 1. This product's benchmark becomes al"' (f) = (4.87,20.76,10.85) . A benchmark for a dominated entity depends on the optimal solution to the appropriate LP. This means that different benchmarks are possible depending on the optimal solution.
1;
where the symbol ':.' in (D i ) means that the ith constraint is to be treated as an inequality using the following rule: i _ ......=
{2' :0::::,
if I5f = 1; if I5f = -1.
The point a i is constructed as follows: if I5f if I5f
= 1; = -1;
and all ties are broken arbitrarily.
4. METHODOLOGY
Denote by 7I"*(f) = (7I"i(f), ... , 7I"~(f)), j3(f) and A*(f) = (Ai(f), ... , A~(f)), O*(f) two optimal solutions, the first for (pi) and the other for (Di), respectively; the corresponding optimal objective function values are denoted by w*(f) and z*(f).
An approach based on applying Definition 2 above requires the solution of the LPs (pi)/(Di) satisfy the conditions of Theorem 1. Answering the questions of which entities are dominant for a given constituency and which constituencies consider a given entity dominant is straightforward. We simply formulate the LPs that correspond to the constituencies in the study and use them to test the entities.
The following result based on the optimal solutions to the pair (Pi) / (D i ) offers a necessary and sufficient condition for an entity to be dominant for constituency l5 i in accordance with Definition 2.
Answering the question of whether there are potential constituencies that might prefer a given entity is more complicated. One way to answer this question is to formulate 2ffi LPs, one for each potential constituency, and use them to test the n entities. This constitutes an onerous task. Clearly, evaluating this many LPs is impractical but it is also unnecessary. The procedure can be made more tractable by noting that any point in the strict interior of the convex hull of {a 1 , ... , an} is never dominant for any constituency. Efficient procedures to perform this preprocessing are available, see e.g., Dula [1996]. We can expect that preprocessing for interior points will reduce the number of entities that need to be tested substantially since, in actual practice, the percentage of inefficient DMUs is typically less than 10% with large cardinality problems tend to be 99% interior.
Result 1. The data point aj' corresponds to a dominated entity for constituency l5 i if and only if either
(1) z*(f) = w*(f) > 1; or (2) z*(f) = w*(f) = 1 and there exists an optimal solution such that one of the m constraints in (V) is not binding. Therefore, the data point al" corresponds to a dominant entity if and only if w* (f) = 1 with all constraints in (D i ) binding for any optimal solution.
If we apply the model to the data in Table 1 we obtain the following analysis. The three constituencies are North America: 8 1 = (1,1, -1); Europe: 152 = (-1, 1, 1); and Africa: 15 3 = (1, -1, -1). The results of the LPs reveals that Constituency 1 prefers Products 1, 3, 4, 5, and 6; Constituency 2 prefers Products 1, 3, 5, 7, and 8; and Constituency 3 prefers Products 2, 4, 5, and 6. None of the three constituencies consider Product 9
The question is: Is there a more efficient way to calculate the preference status with respect to all potential constituencies of each of the n points? 975
If such a way exists it will be a method based on strong geometrical properties of the convex hull of the data points. This is a convex bounded polytope the extreme points of which, its frame is a subset of the data points. The sequel to this paper will explore a procedure based on which of the 2m unit directions are contained in the feasibility cone of each of the elements of the frame.
References
1]
2]
3]
1]
i]
i]
Banker, R.D., A. Charnes and W.W. Cooper (1984). Some models for estimating technological and scale inefficiencies in data envelopment analysis. Management Science 30, 9, 10781092. Caporaletti, L.E., J.H. Dula. and N.K Womer (1999). Performance evaluation based on multiple attributes with nonparametric frontiers. Omega 27, 637-645. Charnes, A., W.W. Cooper and E. Rhodes (1978). Measuring the efficiency of decision making units. European Journal of Operational Research 2, 6, 429-444. Dula., J.H. and R.V. Helgason (1996). A new procedure for identifying the frame of the convex hull of a finite collection of points in multidimensional space. European Journal of Operational Research, Vol. 92, 1996, pp. 352-367. Lancaster, KJ. (1966). A new approach to consumer theory, Journal of Political Economics 74,132-157. Seiford, 1. (1996). Data Envelopment Analysis: The evolution of the state of the art (19781995). The Journal of Productivity Analysis 7, 99-137.
976
NONPARAMETRIC PERFORMANCE EVALUATION WITH MULTIPLE CONSTITUENCIES 1 M.L. Bougnol, J.H. Dubi,2 N.K. Worner
The University of Mississippi, University, MS 38677
Abstract: We extend the familiar notion in DEA to the case when the assessment of some attributes' desirability is a matter of judgment depending on the constituency. The generalization we present consists of treating the attributes of the entities in the study as potentially desirable and undesirable. An application of this extension is in manufacturing where, a particular process or product attribute may be considered desirable by some constituencies and undesirable by others. The new procedure describes how to ascertain what constituency would find which units efficient and inefficient using standard nonparametric efficient frontier criteria. Copyright © 2000IFAC. Keywords: Linear programming, efficiency enhancement, productivity.
1. INTRODUCTION
DEA to deal with a common set of performance attributes directly rather than "inputs" and "outputs" as in the traditional model. The method is developed from first principles based on proper definitions for dominance with respect to performance criteria and it is shown to be equivalent to the variable returns DEA model. The framework approaches the problem of measuring and comparing relative performance based on multiple attributes directly and intuitively. The paper illustrates the use of this model with an application in manufacturing where Taguchi methods are used to assess different processes.
Data Envelopment Analysis (DEA) is a nonparametric methodology to assess efficiency of a collection of functionally similar "decision making units" to transform inputs into outputs. DEA was introduced by Charnes et al. in 1978 and it has evolved to the point where over a thousand articles have been published about the theory and applications (Seiford [1996]). This methodology requires no assumption about a functional form for the efficient frontier and therefore, no parameter estimation, making it useful in a wide variety of applications.
The present paper extends the idea in Caporaletti et al. [1999] to include constituencies. Constituencies classify the elements in the list of attributes in a nonparametric frontier performance model as either "desirable" or "undesirable." Different constituencies provide different classifications. This extension adapts DEA to address a whole new category of situations where the "inputs" and "outputs" are not inherent to the model but rather depend and vary according to constituencies. Issues
The article by Caporaletti et al. [1999] presents an adaptation of the standard, variable returns, DEA model to compare relative performance using multiple attributes. The idea in that paper is to adapt 1 This work is part of ongoing research funded by grant NOOOl4-99-1-0719 from the Office of Naval Research. 2 Corresponding Author: School of Business, The University of Mississippi, University, MS 38677; email: [email protected].
973
Table 1.
defined by a data set consisting of n points, {a l , ... ,an}, each with m components: thus, a j = (a{, ... , a~). Components correspond to attributes of the model. The value of each component is the magnitude of the attribute with which it is associated.
Attribute Data for Nine Products Product: 1 2 3 4 5 6 7 8 9
Attr. 1 17 9 1 17 19 18 3 1 2
Attr.2 14 3 21 20 5 3 4 19 17
Attr. 3 19 9 14 1 21 1 15 9 13
The third aspect of the model are the 'constituencies.' In an application, a constituency may be a group somehow affected by the n entities, and partial to each of their m attributes. Examples include customers if the entities are products or voters within the relevant electoral district if the entities are, say, government projects. We say that an attribute is 'desirable' for a constituency if the constituency considers greater magnitudes preferable; and conversely, 'undesirable' if less is better. In terms of this work, a constituency has the following formal definition.
that arise include: which entities being compared are dominant for a given constituency; what constituencies consider a given entity dominant; for any entity, which are the potential constituencies, if any, that would dominate it; and, for a given constituency, how should the values of attributes be modified for an entity to become dominant. We will refer to the following simple contrived example in the course of the presentation. Imagine nine products to be marketed and sold worldwide. The markets for the cars will be Asia, Europe, and North America. Table 1 gives the data for three attributes for each of the nine products.
Definition 1. A constituency is an m dimensional vector 8£ composed of 1s and -ls. The presence of a '1' in location i indicates that constituency f judges attribute i as desirable; conversely, a '-1' indicates that this constituency judges this attribute as undesirable. A study involving m attributes can have 2m constituencies although they may not all exist or be considered in an actual application. As long as more than one constituency is affected, the model is relevant. An entity will be considered "dominant" by some constituency if it satisfies the following formal definition.
Imagine further that marketing research reveals that the attributes are valued differently in the three markets. For example, the target customer in North America values Attributes 1 and 2 and considers Attribute 3 undesirable; the customers in Europe consider Attribute 2 and 3 desirable but Attribute 1 is a drawback; finally, customers in Asia value only Attribute 1. The questions faced by the manufacturer are:
Definition 2. An entity is dominant with respect to a constituency, 8£, if it is impossible to find a convex combination of the data of the remaining n - 1 entities such that:
(1) For each constituency, which are the products that are, in some sense, dominant? (2) For each product, which of the three constituencies consider it, in the same sense, dominant? (3) Extending the question above, are there other 'potential' constituencies, beyond the three in the original model, that might prefer a product? (4) Which products, if any, are never dominant in any real or potential market? and (5) For any products dominated for some constituency, what modifications would make them dominant?
(1) For every desirable attribute the value of the combination is greater than or equal to that of the entity being tested; and (2) For every undesirable attribute the value of the combination is less than or equal to that of the entity being tested.
3. THEORETICAL DEVELOPMENT
The model we construct next will address the realistic and interesting questions and issues motivated by this simple example.
Definition 2 in the previous section establishes how entities will be classified with respect to constituencies. In this section we provide the mathematical tools to make this classification.
2. THE MODEL
The classification is based on the solution to the following primal dual pair of linear programs where j* is the index of the entity under examination:
The model involves n entities (e.g., projects, processes, products, etc.) that can be characterized by the same m attributes. The model is 974
(pi) min w(f) = (a i ,7I") 11"./3
s.t.
j
(a ,7I") (cl - al", 71") l5 i 7l"
+ 13 + 13:0::::
dominant: z*(I) = 1.27, z*(2) = 1.06, and z*(3) = 2.50. Next, we introduce the notion of constituency benchmarking.
0; \lj,j i-j*
= 1; :0::::
0;
Definition 3. Let al" be a dominated entity for constituency 8i . Then, its benchmark is the vector aj"' (f) = L:j a j A*(fk
o
+ (a i -
i ai ; a J'. )0 ...... =
As required, any benchmark will score "1" in the appropriate LP. The definition for a benchmark provides the formula for identifying it. So, for instance, in the earlier production example, when Product 9 is evaluated with respect to Constituency 1 (f = 1) using linear program (pi), the optimal objective function value is w*(I) = 1.27 > 1; the indication we need to conclude that Product 9 is dominated for Constituency 1. This product's benchmark becomes al"' (f) = (4.87,20.76,10.85) . A benchmark for a dominated entity depends on the optimal solution to the appropriate LP. This means that different benchmarks are possible depending on the optimal solution.
1;
where the symbol ':.' in (D i ) means that the ith constraint is to be treated as an inequality using the following rule: i _ ......=
{2' :0::::,
if I5f = 1; if I5f = -1.
The point a i is constructed as follows: if I5f if I5f
= 1; = -1;
and all ties are broken arbitrarily.
4. METHODOLOGY
Denote by 7I"*(f) = (7I"i(f), ... , 7I"~(f)), j3(f) and A*(f) = (Ai(f), ... , A~(f)), O*(f) two optimal solutions, the first for (pi) and the other for (Di), respectively; the corresponding optimal objective function values are denoted by w*(f) and z*(f).
An approach based on applying Definition 2 above requires the solution of the LPs (pi)/(Di) satisfy the conditions of Theorem 1. Answering the questions of which entities are dominant for a given constituency and which constituencies consider a given entity dominant is straightforward. We simply formulate the LPs that correspond to the constituencies in the study and use them to test the entities.
The following result based on the optimal solutions to the pair (Pi) / (D i ) offers a necessary and sufficient condition for an entity to be dominant for constituency l5 i in accordance with Definition 2.
Answering the question of whether there are potential constituencies that might prefer a given entity is more complicated. One way to answer this question is to formulate 2ffi LPs, one for each potential constituency, and use them to test the n entities. This constitutes an onerous task. Clearly, evaluating this many LPs is impractical but it is also unnecessary. The procedure can be made more tractable by noting that any point in the strict interior of the convex hull of {a 1 , ... , an} is never dominant for any constituency. Efficient procedures to perform this preprocessing are available, see e.g., Dula [1996]. We can expect that preprocessing for interior points will reduce the number of entities that need to be tested substantially since, in actual practice, the percentage of inefficient DMUs is typically less than 10% with large cardinality problems tend to be 99% interior.
Result 1. The data point aj' corresponds to a dominated entity for constituency l5 i if and only if either
(1) z*(f) = w*(f) > 1; or (2) z*(f) = w*(f) = 1 and there exists an optimal solution such that one of the m constraints in (V) is not binding. Therefore, the data point al" corresponds to a dominant entity if and only if w* (f) = 1 with all constraints in (D i ) binding for any optimal solution.
If we apply the model to the data in Table 1 we obtain the following analysis. The three constituencies are North America: 8 1 = (1,1, -1); Europe: 152 = (-1, 1, 1); and Africa: 15 3 = (1, -1, -1). The results of the LPs reveals that Constituency 1 prefers Products 1, 3, 4, 5, and 6; Constituency 2 prefers Products 1, 3, 5, 7, and 8; and Constituency 3 prefers Products 2, 4, 5, and 6. None of the three constituencies consider Product 9
The question is: Is there a more efficient way to calculate the preference status with respect to all potential constituencies of each of the n points? 975
If such a way exists it will be a method based on strong geometrical properties of the convex hull of the data points. This is a convex bounded polytope the extreme points of which, its frame is a subset of the data points. The sequel to this paper will explore a procedure based on which of the 2m unit directions are contained in the feasibility cone of each of the elements of the frame.
References
1]
2]
3]
1]
i]
i]
Banker, R.D., A. Charnes and W.W. Cooper (1984). Some models for estimating technological and scale inefficiencies in data envelopment analysis. Management Science 30, 9, 10781092. Caporaletti, L.E., J.H. Dula. and N.K Womer (1999). Performance evaluation based on multiple attributes with nonparametric frontiers. Omega 27, 637-645. Charnes, A., W.W. Cooper and E. Rhodes (1978). Measuring the efficiency of decision making units. European Journal of Operational Research 2, 6, 429-444. Dula., J.H. and R.V. Helgason (1996). A new procedure for identifying the frame of the convex hull of a finite collection of points in multidimensional space. European Journal of Operational Research, Vol. 92, 1996, pp. 352-367. Lancaster, KJ. (1966). A new approach to consumer theory, Journal of Political Economics 74,132-157. Seiford, 1. (1996). Data Envelopment Analysis: The evolution of the state of the art (19781995). The Journal of Productivity Analysis 7, 99-137.
976