Cross efficiency based heuristics to rank decision making units in data envelopment analysis

Computers & Industrial Engineering 111 (2017) 320–330

Contents lists available at ScienceDirect

Computers & Industrial Engineering journal homepage: www.elsevier.com/locate/caie

Cross efficiency based heuristics to rank decision making units in data envelopment analysis Jae-Dong Hong a, Ki-Young Jeong b,⇑ a b

Industrial Engineering, South Carolina State University, 300 College Street NE, Orangeburg, SC 29117, United States Engineering Management, University of Houston at Clear Lake, 2700 Bay Area Blvd, Houston, TX 77058, United States

a r t i c l e

i n f o

Article history: Received 24 October 2015 Received in revised form 4 June 2017 Accepted 10 June 2017 Available online 11 June 2017 Keywords: Heuristic Cross-efficiency evaluation Data envelopment analysis (DEA) Decision making unit (DMU)

a b s t r a c t This paper proposes two cross-efficiency based heuristics (CEHs) for ranking decision making units (DMUs) in Data Envelopment Analysis (DEA). This paper also suggests a stratification-based consistency evaluation framework (SCEF) to evaluate the performance of the proposed heuristics and several DEAbased full ranking methods in terms of consistency. Several cross-efficiency (CE) methods have been developed as a DEA extension to rank efficient and inefficient DMUs with the main idea of using DEA to do peer evaluation. However, it has been well known that they all suffer from lack of discrimination since efficiency scores from those methods may not be unique due to the non-uniqueness of the DEA optimal weights in the Linear Programming (LP) models. The proposed CEH methods overcome this issue since they do not use any LP models but show comparable consistency level to other DEA-based full ranking methods based on SCEF. Numerical examples from the literature and simulation indicate that CEHs are a good surrogate for CE method. Ó 2017 Elsevier Ltd. All rights reserved.

1. Introduction Among many performance evaluation methods, data envelopment analysis (DEA) has been widely used to evaluate the relative performance of a set of peer organizations called decision making units (DMUs) because DEA models need not to recourse to the exact functional behavior of those organizations regarding the transformation of multiple inputs to multiple outputs, often referred to as an operational process. In fact, DEA is a nonparametric approach that uses a Linear Programming (LP) to evaluate the relative performance of DMUs by comparing how well the DMU transforms its inputs into its outputs (Charnes, Cooper, & Rhodes, 1978). Performance evaluation or measurement is one of the most highlighted elements in any performance management. Hence, DEA’s empirical orientation and absence of priori assumptions make DEA as one of the most popular tools for measuring and improving an operational process. In DEA, peer organizations, DMUs, to be assessed should be relatively homogeneous. As the whole technique is based on comparison of each DMU with all the remaining ones, a considerably large number of DMUs compared to the numbers of inputs and outputs are necessary for the assessment to be meaningful (Meza & Jeong, 2013; Ramanathan, 2003). DEA eventually determines which ⇑ Corresponding author. E-mail addresses: [email protected] (J.-D. Hong), [email protected] (K.-Y. Jeong). http://dx.doi.org/10.1016/j.cie.2017.06.015 0360-8352/Ó 2017 Elsevier Ltd. All rights reserved.

DMUs make efficient use of their inputs and which do not. For an inefficient DMU, the analysis can also quantify what levels of improved performance should be attainable and indicate where the inefficient DMU might look for benchmarking as it searches for ways to improve. DEA produces a single, comprehensive measure of performance for each of DMUs. If the situation is simple, and there is just one input and one output, then we would define performance as the ratio of output to input, and we often refer to this ratio as ‘‘productivity” or ‘‘efficiency.” The best ratio among all the DMUs would identify the most efficient DMU, and every other DMU would be rated by comparing its ratio to the best one. However, the DEA-based assessment may suffer from lack of discrimination particularly when multiple DMUs are classified as efficient. To overcome this inability of DEA in discriminating among efficient DMUs, the cross-efficiency (CE, henceforth) evaluation method is suggested (Sexton, Silkman, & Hogan, 1986). CE, investigated by Doyle and Green (1994), is used as a DEA-based full ranking tool among all DMUs. The basic idea of the CE evaluation is to evaluate the overall efficiencies of the DMUs through both selfevaluation and peer-evaluation. It can usually provide a full ranking for the DMUs to be evaluated and eliminate unrealistic weight schemes without requiring the elicitation of weight restrictions from application area experts (Anderson, Hollingsworth, & Inman, 2002). As indicated by Doyle and Green (1994), the non-uniqueness of CE scores often results from the presence of

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

alternative optimal weights in traditional DEA models. The nonuniqueness of CE scores could be concise with the reality that there is an interval for CE of each DMU from the perspective of any other unit. But, for using the CE method as a ranking tool, the nonuniqueness has been criticized as a drawback since it has reduced the usefulness of the CE method. As a result, it is recommended that secondary goals such as aggressive and benevolent models be introduced in CE evaluation. Sun and Lu (2005) propose the cross-efficiency profiling model (CEP) where each input is separately evaluated with respect to the outputs that consume the input. In this way, input-specific CE-based rating gives a profile for each DMU, which is used to improve the discriminating power among DMUs. Liang, Wu, Cook, and Zhu (2008) and Wang and Chin (2010a) extend the models from Doyle and Green (1994) by introducing various secondary objective functions. Regardless the types of the secondary objective functions, the peer-evaluation typically leads to a CE matrix whose diagonal elements are composed of the self-efficiency scores whereas non-diagonal elements are the peer-evaluation scores. Jeong and Ok (2013) suggest that super-efficiency (SE) based diagonal elements add more discriminating power. In the traditional CE, the efficiency score of a DMU under evaluation is maximized as the primary goal while the average CE score of other DMUs is minimized or maximized according to the problem context as the secondary goal. However, Lim (2012) proposes the minimax and maxmin formulation of CE where the secondary goal is replaced with the minimization (or maximization) of the best (or worst) CE score of other DMUs. Wu, Sun, and Liang (2012) propose a CE model based on a weight balanced DEA model to lessen large difference between weighted inputs and outputs and to reduce the number of zero weights. Lam and Bai (2011) suggest a DEA model that minimizes deviations of input and output weights from their means to reduce the chance to choose an extreme or zero weight value. In their model, the mean of an input or output weight is defined as the average of the maximum and minimum of the input or output for a DMU under evaluation can attain while that DMU remains efficient. In many practices, identifying the best ranking is more important than maximizing the individual efficiency score itself. In fact, based on diverse models described above, many authors also propose various CE-based ranking methods. Wang and Chin (2010b) present a neutral DEA CE model by determining the weights for each DMU from its own viewpoint without being aggressive or benevolent to the other DMUs. Then, the neutral DEA model is extended to a cross-weight evaluation through CE matrix. Wang, Chin, and Jiang (2011) improve the neutral DEA CE model by simultaneously considering input-and-output oriented weight determination, resulting in reduction of zero weight for inputs and outputs. Wang and Chin (2011) apply the ordered weighted averaging (OWA) operator to the standard CE matrix in the CE aggregation process to compute the ultimate CE score, and decide the ranking of DMUs. Zerafat Angiz, Mustafa, and Kamali (2013) suggest converting a CE matrix into a cross-ranking matrix to find ultimate CE scores. Recently, Oukil and Amin (2015) propose an OWA composite score-based ranking method by computing the maximum CE for each DMU to increase differentiation in ranking among DMUs, and numerically demonstrate that their method is consistent with respect to the ranking pattern. It is important to recognize that all DEA-based full ranking methods described above address the multiple optimal weight related problem. Although many models suggest diverse ways to improve the discrimination, the fundamental source of the problem still remains since all suggestions are based on LP models. In this paper, we attempt to solve this problem by proposing a completely different approach which has not been discussed in the literature. Specifically, we address the following two objectives: (1)

321

develop CE heuristics (CEHs) as an alternative for CE-based ranking methods and their variations. Note that the concepts for CEHs are based on the structure of CE but they do not require any LP model and (2) develop a systematic consistency evaluation framework to compare the robustness or consistency level of any DEA-related full ranking method. If properly accomplished, the first objective would provide several advantages. First, CEHs will not suffer from the problem caused by the multiple optimal DEA weights; second, they can be easily implemented without any complex optimization software. The second objective, if properly completed, can also significantly contribute to the DEA literature since although many DEA-based ranking methods are presented, there has been no or very little study to our best knowledge on a systematic way to evaluate and compare its performance in terms of robustness or consistency. We adopt the stratification concept suggested by Seiford and Zhu (2003) for the stratification-based consistency evaluation framework (SCEF). Once the CEHs are proposed, their performance compared to other ranking methods will be evaluated using SCEF. The paper is organized as follows. Section 2 gives a brief description of the CE evaluation methods and the formulation. Section 3 presents CEHs as an alternative for CE-based evaluation and its variations. Section 4 proposes the stratification-based consistency evaluation. Section 5 demonstrates CEHs and the consistency evaluation framework with two numerical examples adopted from DEA literature. The paper concludes in Section 6. 2. Cross-efficiency evaluation The fractional or ratio-form DEA model uses the ratio of outputs to inputs to measure the relative efficiency of DMUj as an objective function to be evaluated relative to the ratios of all DMUs. The fractional DEA model (Cooper, Seiford, & Zhu, 2011) is stated as: Objective Function: Maximize the efficiency rating h for DMUk

Ps urk yrk max h ¼ Pr¼1 ; m i¼1 v i kxik

ð1Þ

This is subject to the constraint that when the same set of u and v multipliers (or weights) is applied to all other DMUs being compared, no DMU will be more than 100% efficient as follows: For DMU j, DMUj, the efficiency is

Ps

Pr¼1 m i¼1

urk yrj

v ik xij

6 1;

urk ; v ik P 0;

8j

r ¼ 1; . . . ; s;

ð2Þ i ¼ 1; . . . ; m

where j = number of DMUs being compared in the DEA analysis, j = 1, . . . , n h = efficiency rating of the DMUk being evaluated by DEA yrj = amount of output r generated by DMUj xij = amount of input i used by DMUi i = number of inputs used by DMUs, i = 1, . . . , m r = number of outputs generated by DMUs urk = multipliers or weight assigned by DEA to output r for DMUk vik = multipliers or weight assigned by DEA to input i for DMUk The above fractional DEA model given by (1) and (2) can be transformed to the following linear programming problem, which is called a CRS (Constant Returns-to-Scale) multiplier DEA (mDEA) model:

max

Ek ¼

s X urk yrk ; r¼1

subject to

ð3Þ

322

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

m X

v ik xik ¼ 1;

ð4Þ

i¼1 s m X X urk yrj  v ik xij 6 0; r¼1

j ¼ 1; . . . ; n;

ð5Þ

i¼1

urk ; v ik P 0;

r ¼ 1; . . . ; s;

i ¼ 1; . . . ; m

Ek

Let denote the optimal value of the objective function corresponding to the optimal solution (u⁄, v⁄). DMUk is said to be efficient if Ek ¼ 1. DEA models can be either input-oriented or outputoriented, depending upon the rationale for conducting DEA. The model given by (3)–(5) is called an input-oriented CCR model and Ek is called CRS efficiency score (ES). DEA allows each DMU to be evaluated with its most favorable weights due to its nature of the self-evaluation. The CE method was suggested as a DEA extension to rank DMUs with the main idea being to use DEA for peer evaluation, rather than just for self-evaluation. It consists of two phases. The first phase is the self-evaluation (Phase I) using the model by (3)–(5). The second phase (Phase II) is the peer-evaluation, where the weights/multipliers arising from Phase I are applied to all DMUs to get the cross evaluation score for each of DMUs. To denote the peer-evaluation, let Ekj represent the DEA score for the rated DMUj using the optimal weights /multipliers that a rating DMUk has chosen in model (3)– (5). Now, Ekj is given by

Ps

Ekj ¼ Pr¼1 m i¼1

urk yrj

v ik xij

;

k and j ¼ 1; . . . ; n:

ð6Þ

Note that Ekk , the DEA score for the self-evaluation of Phase I, is the same as Ek and can be obtained using the model by (3)–(5). Then, CE score for DMUj is defined as follows: n X j ¼ 1 E Ekj n k¼1

ð7Þ

Note that the optimal weights obtained from the model in (3)–(5) may not be unique. Thus, Despotis, Stamati, and Smirlis (2010) point out that the CE defined in the above model is arbitrarily obtained, depending on the optimal solution arising from the particular optimization software in use. Liang et al. (2008) propose the game cross-efficiency (GCE) which will maximize the efficiency of DMUj, under the condition that the efficiency of a given DMUk is not less than a given value of efficiency. To calculate the game kcross-efficiency, the following model for DMUj similar to the CE model is used: s X ukrj yrj ;

max

ð8Þ

r¼1

Charnes, Cooper, Seiford, and Stutz (1982) introduce the multiplicative CE (MCE) method. Using MCE method, Cook and Zhu (2014) propose the approach to obtain maximum (and unique) log cross-efficiency (MLCE) scores under the condition that each DMU’s DEA efficiency score remains unchanged. When a DMU under evaluation is not included in the reference set of the envelopment models, the resulting DEA models are called superefficiency (SE) DEA models. Charnes, Haag, Jaska, and Semple (1992) use SE model to study the sensitivity of the efficiency classifications, and Andersen and Petersen (1993) propose using the CRS SE model in ranking efficient DMUs. The above mentioned methods are based upon LP models. In this paper, we propose a completely different approach. Instead of using any LP models, we present a CE based heuristic (CEH) approach to avoid any multiple optimal weight related issue. Then, we use CE, GCE, MCE, MLCE and SE methods to compare corresponding efficiency scores and rankings with those generated by CEH approach under the consistency evaluation framework to be discussed. Since CRS ES (Efficiency Score) method generates efficient DMUs’ SE scores that are greater than one, we normalize each score-we divide each score by the maximum score, and call it a normalized SE (NSE), for comparison’s purpose. 3. Development of CEH methods Sotiros and Despotis (2012) introduce the max-normalized DEA (MN-DEA) by dividing the input/output data by the maxima and show that the MN-DEA model is structurally identical to the regular DEA (R-DEA) model with un-normalized data. In fact, both MNDEA and R-DEA generate the same CE, GCE, MCE, MLCE, and NSE scores for each DMU. Such normalization will prevent round-off errors that imbalanced data may lead to. In this paper, we propose the min-max normalized model (NXN-DEA), where the input variables are divided by their minima, whereas the output variables by the maxima. The NXN-DEA model also finds the same CRS ES and NSE scores that MN-DEA or R-DEA model produces for each DMU, but may yield different GCE, MCE, and MLCE scores from those generated by R-DEA model. The productivity of a DMU is defined as the ratio of the output (s) that it produces to the input(s) that it uses. A simple general functional form for efficiency measure ek defined for DMUk can be written as

ek ¼

ek ¼

v kij xij ¼ 1;

ð9Þ

i¼1 s m X X ukrj yr‘  v kij xi‘ 6 0; r¼1

‘ ¼ 1; . . . ; n;

ð10Þ

i¼1

Ps

v

yk =xk n o; y maxj xj

ð13Þ

j

m X

k r¼1 urj yrk k i¼1 ij xik

ð12Þ

where f k ¼ f ðxik ; yrk Þ: The formulation of ek for a single input-singleoutput is

subject to

Pm

fk ; f max

P ak ;

ukrj ; v kij P 0;

r ¼ 1; . . . ; s;

ð11Þ i ¼ 1; . . . ; m:

This model is similar to the CRS m-DEA model except for Eq. (11), which ensures that the CE score of DMUk should be greater than or equal to ak .

It is quite obvious that the smaller the input is and the greater the output is, the greater the productivity is. The principle of the CEH approach proposed in this paper results from Eq. (13). In CEH approach, we evaluate the ratio of each output to the sum of the inputs and select the best ratio as the DEA method is designed to give the best efficiency score for the DMU under evaluation as shown in (1)–(2) or (3)–(5). See Cooper, Seiford, and Tone (2007) for proof. Suppose that DMUk is a rating DMU. Start normalizing with dividing the input and output of all DMUs by the input and output of the rating DMU, DMUk. That is,

x0ijk ¼ and

xij ; xik

8i and forallj

ð14Þ

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

y0rjk ¼

yrj ; yrk

8r and 8j:

ð15Þ

Note that x0ikk and y0rkk are equal to one. Then, the productivity for the rth output and all inputs for DMUj with respect to DMUk, P rjk , is given by

y0rjk Prjk ¼ Pm 0 ; i xijk

ð16Þ

We propose the following two equations to denote the crossefficiency of DMUj with respect to DMUk as follows: ½1

Ekj ¼

( ) s s X X Prjk =maxt Prtk : r¼1

ð17Þ

r¼1

½2

Ekj ¼ maxr fPrjk g=maxt ½maxr fPrtk g: ½g

ð18Þ ½1

323

4. Consistency evaluation framework Regardless the methods used, rankings generated by a CE method should be consistent and reasonable. For example, suppose that a CE method ranks three DMUs, such as DMU1, DMU2, and DMU3 in this order. Then, DMU2 is supposed to rank ahead of DMU3. If DMU3 ranks ahead of DMU2 by the same CE method when only these two DMUs are compared, we may face difficulty in interpreting or explaining this rank reversal due to its inconsistency. Unfortunately, some CE methods available in the references behave in this way. Further, we also need to evaluate the performance of CEH methods discussed in the previous section. This motivates us to propose a consistency evaluation framework for any DEA-related full ranking method. Seiford and Zhu (2003) propose the context-dependent DEA method to measure the relative attractiveness (AT) score and progress of DMUs when DMUs having worse performance and better, respectively are chosen as the evaluation context. For this, they stratify DMUs into different efficiency levels. Let

We call CEH[g] if Ekj , g = 1, 2, is used to compute CE scores. In Ekj for

J 1 ¼ fDMU j ; j ¼ 1; 2; . . . ; ng be the whole set of n number of DMUs

CEH[1], the overall efficiency of DMUj with respect to DMUk is used in the calculation of the CE, whereas the individual efficiency score

and iteratively define J ‘þ1 ¼ J ‘  E‘ , where E‘ consists of all the effi-

for DMUj is maximized in

½2 Ekj

for CEH[2]. In other words, two numer-

ators in Eqs. (17) and (18) express two different ways to give the best efficiency score for DMUj under evaluation. The denominator of Eq. (17) similar to that of Eq. (13) ensures that CE’s are not greater than one. Thus, CEH[1] might provide more robust CE scores, whereas the CE scores obtained from CEH[2] might show a similar pattern of the CE scores given by a regular DEA method. Substituting Prjk in Eq. (16) into Eqs. (17) and (18) yields ½1

(Ps

Ekj ¼

Pr¼1 m

ð1=yrk Þyrj

i¼1 ð1=xik Þxij

)

 Pm  ð1=xik Þxit  mint Psi¼1 r¼1 ð1=yrk Þyrt

ð19Þ

and

  Pm  ð1=yrk Þyrj ½2 i¼1 ð1=xik Þxit  mint Ekj ¼ maxr Pm ð1=yrk Þyrt i¼1 ð1=xik Þxij

ð20Þ

We can see, from Eqs. (19) and (20), that the structure of CEH approach is same as CE method in DEA methodology. When Eq. (19) of CEH approach is compared with Equation of (6) for CE model, we see that urk and v rk in (6) are replaced by 1/yrk and 1/ xrk , respectively. We also see that Eqs. (19) and (20) are consistent with the principle of self- and peer evaluation used in CE. It implies that a higher yrk and a lower xrk would lead to a higher ES for the self-evaluation of DMUk but to a lower ES for the peer-evaluation of DMUj. In addition, multiplying the first part by the second part of (19) ensures the constraints (5) in CE model. See Appendix B for proof. Now, we can construct a CE matrix similar to that developed by Doyle and Green (1994). The CE score for DMUj is obtained from ½g

the average of all Ekj ,

½g ¼ 1 E j n

n X ½g Ejk

cient DMUs on the ‘th level (or stratification), until J ‘þ1 becomes null. In fact, all DMUs in E‘ are equivalent with the same efficiency score of one from the traditional DEA perspective. Hence, the traditional DEA fails to differentiate these DMUs in its ranking. The attractiveness score for each DMU in the ‘th stratification (E‘ ) is computed against DMUs in the ð‘ þ 1Þth and lower levels as a background. In this way, the context dependent DEA can discriminate all DMUs on each stratification level. We adopt this DEA-based stratification concept to develop a consistency evaluation method, and call it as the stratificationbased consistency evaluation framework (SCEF). The rationale for SCEF is that if a full ranking method is consistent or robust, then its full ranking for DMUs in J 1 (pre-stratification ranking) would be somewhat similar to the ranking for DMUs after all stratifications (post-stratification ranking). Hence, the magnitude or degree of discrepancy between these two rankings can serve as a metric for consistency for that ranking method. Based on this rationale, we propose the following consistency evaluation steps: For a ranking method, rm, Step 1: [pre-stratification ranking] Evaluate DMUs in J l where l = 1 using rm, and decide their ranking; Step 2: [post-stratification ranking] Step 2–1: Construct El by evaluating each DMU in J ‘ ; Then set J ‘þ1 ¼ J ‘  E‘ ; Step 2–2: Evaluate DMUs in El using rm; decide their overall ranking across El ’s; Step 2–3: Stop if J ‘þ1 ¼ £; otherwise set l = l + 1 and go to step 2–1; Step 3: Analyze the consistency between rankings from Step 1 and Step 2.

ð21Þ

k

As described before, the CE scores generated by DEA may not be unique due to multiple optimal weights/multipliers for inputs and outputs obtained from solving LP. But the CE scores generated by the two proposed CEH methods do not require any LP. Hence, these methods do not depend on the multiple optimal weights, but on the ratio of output to the sum of the inputs. To evaluate the proposed CEH methods, we use two numerical examples which can be found in the well-known DEA-related literature.

In Step 3, we use the correlation coefficient (q) or Mean Absolute Deviation (MAD) to measure the similarity between pre-and poststratification ranking patterns. In the pre-stratification ranking, the ranking for all DMUs in the entire set, J 1 , is decided all together without any grouping. However, in the post-stratification ranking, the entire set is classified into several subsets first, each representing a set of efficient frontiers in each stratification level. Then, the ranking for DMUs in each different subset is decided sequentially

324

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

by the ranking method. Eventually, if the ranking method is consistent, the two ranking patterns–represented by q or MAD–between pre-and-post stratification is expected to be similar (large in q and small in MAD).

Before proceeding further, we compute the correlation coefficients between efficiency scores generated from all of these full ranking methods using two data sets. The results show that the lowest value is 0.870 between GCE and CEH[1] in case of R data set, and 0.813 between GCE and MCE in case of NXN data set, and most of the other values are above 0.90, indicating that each of all full ranking methods is a good surrogate for others in terms of the efficiency scores. Since the pre-stratification rankings are provided in Table 1 already (see the number in the bracket next to each efficiency score), we now decide the post-stratification rankings. From Table 1, it is clear that J1 = {DMU1, DMU2, DMU3, DMU4, DMU5}. Further, DMU2 and DMU3 are two efficient frontiers before any stratification in both R and NXN data sets. Hence E1 = {DMU2, DMU3} and J2 = {DMU1, DMU4, DMU5}. The rankings of DMUs in E1 are already provided in Table 1. Now, all DMUs in J2 are evaluated in Table 2, indicating that E2 = {DMU1, DMU4, DMU5} since all of those DMUs are an efficient frontier at this level regardless the data type. Since J3 is null, we stop the step 2–3 here. By combining rankings in E1 and E2, Table 3 displays rankings, MAD, and correlation coefficient (q) for each method between pre-stratification and post-stratification. We can see that the stratification affects the rankings since many methods generate different rankings after the stratification. However, CEHs and NSE show the perfect consistency while CE shows the lowest consistency at both data sets. To investigate the performance of the proposed CEH methods further, we use the data of Zhu (2014, p. 21) that are presented in Table 4, where there are fifteen (15) companies from the Top Fortune Global 500 list in 1995, three inputs: (i) assets ($ millions), (ii) equity ($ millions), and (iii) number of employees, and two outputs: (i) revenue ($ millions) and (ii) profit ($ millions). In Table 5, we present all the scores and rankings generated by all methods again along with the scores of CEHs. We observe that three methods–MCE score with NXN data and MLCE scores with both data sets–generate same rankings, but all other methods generate different rankings and the ranges of the rankings are

5. Numerical examples To evaluate the performance of the two CEH methods, we use two different comparison approaches. In the first approach, we focus on demonstrating the theory and rationales of the CEH methods and the SCEF. Thus, we compare two CEH methods with all other full ranking methods previously discussed in the context of SCEF. For this purpose, the two relatively simple numerical examples from the relevant literature are used. In the second approach, we more focus on demonstrating the performance of the CEH methods through a more extensive comparison. For this purpose, we randomly generate diverse complex numerical examples based on the experimental design, demonstrating the advantages of the CEH methods in terms of the computational time and the correlation with the traditional CE method. 5.1. Examples from literature First, we apply the CEHs and other full ranking methods to the numerical example illustrated by Liang et al. (2008), where there are five DMUs. The data for each DMU consist of three inputs denoted by (x1k, x2k, x3k) and two outputs (y1k, y2k) as shown in Table 1. We demonstrate CEHs with two sets of data. The first set is in the rows for ‘R’, the original or un-normalized data and the second set is in the rows for ‘NXN’, min-max normalized data in the same column. Using DEAFrontier software developed by Zhu (www.deafrontier.net), we compute diverse efficiency scores–CRS ES, CE, GCE, MCE, MLCE, and NSE–to decide rank of each DMU with both data sets. For comparison purpose, we also present two efficiency scores, by the proposed CEH methods using Eqs. (22) and (23), respectively, in the last two columns.

Table 1 Comparison of diverse efficient scores and rankings for five DMUs. DATA

DMU

(x1k, x2k, x3k) (y1k, y2k)

CRS ES

DEA Extensions CE

R

NXN

1 2 3 4 5 1 2 3 4 5

(7, 7, 7)(4, 4) (5, 9, 7)(7, 7) (4, 6, 5)(5, 7) (5, 9, 8)(6, 2) (6, 8, 5)(3, 6) (1.75, 1.16, 1.4)(.571, 0.571) (1.25, 1.5, 1.4)(1, 1) (1, 1, 1)(.714, 1) (1.25, 1.5, 1.6)(.857, 0.285) (1.5, 1.33, 1)(.428, 0.857)

0.685 1.000 1.000 0.857 0.857 0.685 1.000 1.000 0.857 0.857

0.545 0.862 1.000 0.576 0.561 0.447 0.889 0.978 0.550 0.540

GCE [5] [2] [1] [3] [4] [5] [2] [1] [3] [4]

0.638 0.976 1.000 0.798 0.666 0.638 0.976 1.000 0.798 0.667

MCE [5] [2] [1] [3] [4] [5] [2] [1] [3] [4]

0.054 0.670 1.000 0.084 0.076 0.114 0.380 0.714 0.081 0.183

MLCE [5] [2] [1] [3] [4] [4] [2] [1] [5] [3]

0.056 0.760 1.000 0.101 0.076 0.056 0.760 1.000 0.101 0.076

CEH[1]

CEH[2]

0.465 0.844 1.000 0.462 0.587 0.465 0.844 1.000 0.462 0.587

0.459 0.833 0.996 0.601 0.629 0.459 0.833 0.996 0.601 0.629

NSE [5] [2] [1] [3] [4] [5] [2] [1] [3] [4]

0.457 0.746 1.000 0.571 0.571 0.457 0.746 1.000 0.571 0.571

[5] [2] [1] [3] [3] [5] [2] [1] [3] [3]

[4] [2] [1] [5] [3] [4] [2] [1] [5] [3]

[5] [2] [1] [4] [3] [5] [2] [1] [4] [3]

[.] Represents the corresponding ranking.

Table 2 Comparison of efficient scores and rankings for DMU1, DMU4, and DMU5. Data

DMU

CRS ES

DEA extension CE

GCE

MCE

MLCE

NSE

CEH[1]

CEH[2]

R

1 4 5

1.0000 1.0000 1.0000

0.8730 [1] 0.6852 [3] 0.8542 [2]

1.0000 [1] 1.0000 [1] 1.0000 [1]

0.6076 [2] 0.4494 [3] 0.8451 [1]

1.0000 [1] 1.0000 [1] 1.0000 [1]

0.535 [3] 1.000 [1] 1.000 [1]

0.7844 [2] 0.7560 [3] 1.0000 [1]

0.6284 [3] 0.7217 [2] 0.8643 [1]

NXN

1 4 5

1.0000 1.0000 1.0000

0.7460 [2] 0.7361 [3] 0.9333 [1]

1.0000 [1] 1.0000 [1] 1.0000 [1]

0.2267 [2] 0.1620 [3] 0.3649 [1]

0.2267 [2] 0.1620 [3] 0.3646 [1]

0.535 [3] 1.000 [1] 1.000 [1]

0.7844 [2] 0.7560 [3] 1.0000 [1]

0.6284 [3] 0.7217 [2] 0.8643 [1]

[.] Represents the corresponding ranking.

325

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330 Table 3 Comparison of ranking between pre-stratification and post-stratification. Data

DMU

CE pre

post

pre

post

pre

post

pre

post

pre

post

pre

post

pre

post

R

1 2 3 4 5 MAD

5 2 1 3 4 0.80 0.60

3 2 1 5 4

5 2 1 3 4 0.60 0.88

3 2 1 3 3

5 2 1 3 4 0.80 0.70

4 2 1 5 3

5 2 1 3 4 0.60 0.88

3 2 1 3 3

5 2 1 3 3 0.00 1.00

5 2 1 3 3

4 2 1 5 3 0.00 1.00

4 2 1 5 3

5 2 1 4 3 0.00 1.00

5 2 1 4 3

5 2 1 3 4 0.80 0.70

4 2 1 5 3

5 2 1 3 4 0.60 0.88

3 2 1 3 3

5 2 1 3 4 0.80 0.70

4 2 1 5 3

5 2 1 3 4 0.60 0.70

4 2 1 5 3

5 2 1 3 3 0.00 1.00

5 2 1 3 3

4 2 1 5 3 0.00 1.00

4 2 1 5 3

5 2 1 4 3 0.00 1.00

5 2 1 4 3

q NXN

1 2 3 4 5 MAD

q

GCE

MCE

MLCE

NSE

CEH[1]

CEH[2]

Table 4 Fifteen (15) companies from fortune global 500 companies list in 1995. DMU

Company

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Input

Mitsubishi Mitsui Itochu General Motors Sumitomo Marubeni Ford Motor Toyota Motor Exxon Royal Dutch/Shell Wal-Mart Hitachi Nippon Life Nippon T & T AT&T

Output

Assets

Equity

Employees

Revenue

Profit

91920.6 68770.9 65708.9 217123.4 50268.9 71439.3 243,283 106004.2 91296.0 118011.6 37871.0 91620.9 364762.5 127077.3 88,884

10950.0 5553.9 4271.1 23345.5 6681.0 5239.1 24547.0 49691.6 40436.0 58986.4 14762.0 29907.2 2241.9 42240.1 17,274

36,000 80,000 7182 709,000 6193 6702 346,990 146,855 82,000 104,000 675,000 331,852 89,690 231,400 299,300

184365.2 181518.7 169164.6 168828.6 167530.7 161057.4 137137.0 111052.0 110009.0 109833.7 93627.0 84167.1 83206.7 81937.2 79,609

346.2 314.8 121.2 6880.7 210.5 156.6 4139 2662.4 6470.0 6904.6 2740.0 1468.8 2426.6 2209.1 139

Table 5 Comparison of CE scores and rankings. DMU

CRSES

DEA extensions CE

GCE

R 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0.662 1.000 1.000 1.000 1.000 0.971 0.737 0.524 1.000 0.841 1.000 0.386 1.000 0.348 0.270

0.438 0.609 0.659 0.776 0.704 0.587 0.507 0.352 0.900 0.689 0.752 0.274 0.561 0.277 0.137

NXN [11] [7] [6] [2] [4] [8] [10] [12] [1] [5] [3] [14] [9] [13] [15]

0.512 0.695 0.759 0.702 0.835 0.674 0.466 0.393 0.957 0.742 0.808 0.300 0.405 0.296 0.167

R [9] [7] [4] [6] [2] [8] [10] [12] [1] [5] [3] [13] [11] [14] [15]

0.643 0.929 0.983 0.886 0.994 0.888 0.614 0.467 1.000 0.800 0.934 0.358 0.708 0.338 0.231

MCE NXN [10] [5] [3] [7] [2] [6] [11] [12] [1] [8] [4] [13] [9] [14] [15]

R 0.006 0.020 0.021 0.055 0.075 0.022 0.009 0.003 0.189 0.042 0.080 0.001 0.254 0.001 0.000

MLCE NXN [11] [9] [8] [5] [4] [7] [10] [12] [2] [6] [3] [13] [1] [14] [15]

CEH[1]

CEH[2]

0.417 [7] 0.465 [6] 0.671 [3] 0.316 [8] 0.723 [1] 0.592 [4] 0.274[10] 0.255 [11] 0.685 [2] 0.528 [5] 0.221 [12] 0.149 [14] 0.307 [9] 0.181 [13] 0.085 [15]

0.374 0.425 0.635 0.251 0.658 0.545 0.212 0.198 0.598 0.465 0.160 0.113 0.243 0.139 0.078

NSE

R

NXN

R

0.73 [7] 0.77 [6] 4.2 [2] 0.13 [11] 7.0 [1] 4.1 [3] 0.11 [12] 0.16 [10] 0.97 [5] 0.43 [8] 0.28 [9] 0.06 [14] 1.1 [4] 0.06 [13] 0.01 [15]

0.73 [7] 0.77 [6] 4.2 [2] 0.13 [11] 7.0 [1] 4.1 [3] 0.11 [12] 0.16 [10] 0.97 [5] 0.43 [8] 0.28 [9] 0.06 [14] 1.1 [4] 0.06 [13] 0.01 [15]

0.169 0.258 0.328 0.350 0.337 0.248 0.188 0.134 0.342 0.215 0.352 0.099 1.000 0.089 0.069

NXN [11] [7] [6] [3] [5] [8] [10] [12] [4] [9] [2] [13] [1] [14] [15]

[7] [6] [2] [8] [1] [4] [10] [11] [3] [5] [12] [14] [9] [13] [15]

‘A’ represents ‘A’  103.

significant. For example, DMU4 gets the maximum ranking range of 9, where it got ranked #2 by CE (R) and #11 by MCE (NXN) and MLCE (NXN). DMU15 has the minimum range of 0, since it got ranked #15 by all the scores. As shown in the previous example, the efficiency score (ES) has something to do with its rank, but not always in the order of ES shown in Table 5. For example,

DMU4, DMU11, and DMU13 have the perfect ES of one, but the ranking range is 9, 9, and 10, respectively, depending on method used. In fact, these are the top three highest ranges. Note that CEH[1] and CEH[2] generate the same ranks except ranking #2 and #3. All the scores and rankings consistently indicate that DMU12, DMU14 and DMU15 would be ranked the last three, but there is no DMU which

326

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

gets ranked consistently the highest. Only CEH[1] and CEH[2] agree that DMU5, DMU3, and DMU9 would be candidates for the top three and DMU5 would be ranked number one. Note that this example is significantly different from the first example regarding the variation of the correlation coefficient between ranking methods. For example, note the correlation coefficients between ranking methods are displayed in Table 6 where the values before and after ‘/’ symbol represent the correlation coefficient with R and NXN data set, respectively. We can observe that CEH[1] and CEH[2] are a good surrogate with each other. Both GCE and MLCE also show high correlation value with CEHs. While CE is a good surrogate for GCE and GCE is a good surrogate for CEH[1] and CEH[2], CE is not a good surrogate for CEHs. Overall, due to the problem size, the variation of the correlation coefficient is much larger than that in the first example.

and CEH[2] with NAT are very high across data sets. We can see the similar pattern between MCE and MLCE. The result from the first level stratified set suggests that CEH[1] and CEH[2] with NAT may be a good surrogate with each other. Now Tables 7c–7e show the scores and rankings for the DMUs on the next three levels, E2 , E3 ,and E4 , respectively. Note that there is no NAT in Table 7e since E4 is the lowest level subset. Rankings in E2 and E3 consistently show that rankings generated by CEHs are identical to those by NAT. We can also observe that other ranking methods start to show the more consistent pattern than that in E1 probably because of the small data size. Again, Tables 8a and 8b summarize and compare rankings between pre-stratification and post-stratification using MAD and correlation coefficient (q) for all ranking methods. We can see that MADs in these two tables are much larger than those in Table 3 because of the higher complexity in the problem. In Table 8a, we can observe three distinctive consistency clusters in terms of q. MCE and NSE show the best consistency while MLCE the worst consistency. The remaining methods-CEH[1], CEH[2], CE, and GCEshow the intermediate level consistency between these two clusters. Regarding MAD, the performance of CEHs is better than GCE and MLCE. In Table 8b, only two clusters are observed with respect to q. NSE shows the best consistency while all others show a similar value as a single cluster. The same is true regarding MAD. NSE shows the best consistency and all others show a similar value. Eventually, we can make conclusions that NSE shows the best performance across data sets. Further, although CEHs do not show the

We identify the DMUs on the first stratification level, E1 , and report all the scores and corresponding ranks for DMUs in Table 7a. We observe that only CEH[1] and CEH[2] still consistently pick DMU5, DMU3, and DMU9 as the top three DMUs as in the prestratification ranking. We also compute the normalized attractiveness scores (NAT) calculated by dividing each AT score by the greatest AT score for comparison purpose. Now, we compare correlation coefficients between rankings from all methods in Table 7a and display them in Table 7b. The correlation coefficient (q) analysis in Table 7b shows that NAT, CEH[1] and CEH[2] generate very homogeneous ranking patterns compared to others–we can observe that the correlation coefficients of CEH[1]

Table 6 Correlation matrix of efficiency scores for R/NXN where R: regular data without normalization and NXN: Min-max normalized data.

CE GCE MCE MLCE NSE CEH1

GCE

MCE

MLCE

NSE

CEH[1]

CEH[2]

0.893/0.954

0.846/0.764 0.782/0.782

0.568/0.679 0.804/0.804 0.661/0.661

0.832/0.729 0.782/0.782 0.950/0.950 0.629/0.629

0.696/0.793 0.832/0.832 0.596/0.596 0.879/0.879 0.518/0.518

0.679/0.782 0.825/0.825 0.575/0.575 0.889/0.889 0.511/0.511 0.996/0.996

R: Regular (without normalizing) NXN: min–max normalized.

Table 7a Comparison of DMUs in level 1. DMU

ES

DEA extensions

CEH[1]

CE

GCE

R 2 3 4 5 9 11 13 a

1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.538 0.494 1.000 0.476 0.904 0.773 0.764

NXN [5] [6] [1] [7] [2] [3] [4]

0.636 0.626 0.740 0.593 0.687 0.602 0.684

[4] [5] [1] [7] [2] [6] [3]

MCE

R

NXN

R

0.991 1.000 1.000 0.985 0.991 0.974 0.804

[3] [1] [1] [5] [4] [6] [7]

0.003 0.001 0.031 0.006 0.077 0.086 0.170

MLCE

[6] [7] [4] [5] [3] [2] [1]

NXN

R

0.79 [5] 4.2 [2] 0.14 [7] 7.1 [1] 0.99 [4] 0.29 [6] 1.1 [3]

0.010 0.001 0.251 0.006 0.257 0.534 1.000

NSE

[5] [7] [4] [6] [3] [2] [1]

CEH[2]

NAT

NXNa

R NXN

[5] [2] [7] [1] [4] [6] [3]

0.172 0.264 0.233 0.226 1.000 0.235 0.667

[7] [3] [5] [6] [1] [4] [2]

0.137 0.870 0.155 1.000 0.168 0.113 0.569

[6] [2] [5] [1] [4] [7] [3]

0.476 0.669 0.321 0.684 0.648 0.239 [5]

[4] [2] [6] [1] [3] [7]

0.436 0.623 0.247 0.611 0.567 0.164 0.284

[4] [1] [5] [2] [3] [7] [6]

Scores are same as MCE scores of NXN model; ‘A’ represents ‘A’  103.

Table 7b Correlation matrix of rankings in level 1.

CE GCE MCE MLCE NSE NAT CEH1

GCE

MCE

MLCE

NSE

NAT

CEH[1]

CEH[2]

0.099/0.330

0.536/0.536 0.758/0.198

0.607/0.536 0.692/0.198 0.964/1.000

0.357/0.357 0.231/0.231 0.500/0.143 0.464/0.143

0.607/0.250 0.033/0.033 0.286/0.893 0.464/0.893 0.179/0.179

0.714/0.357 0.198/0.198 0.607/0.857 0.714/0.857 0.036/0.036 0.821/0.821

0.571/0.214 0.528/0.528 0.786/0.679 0.857/0.679 0.036/0.036 0.714/0.714 0.929/0.929

327

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330 Table 7c Comparison of DMUs in level 2. DMU

ES

DEA extension CE

6 7 10 a

1.000 1.000 1.000

GCE

MCE

MLCE

NSE

NXN

R

NXNa

R NXN

1.000 [1] 1.000 [1] 1.000 [1]

1.000 [1] 0.171 [3] 0.416 [2]

0.022 [1] 0.001 [3] 0.002 [2]

1.000 [1] 1.000 [1] 1.000 [1]

[1] [3] [2]

1.000 [1] 0.066 [3] 0.237 [2]

1.000 [1] 0.088 [3] 0.146 [2]

GCE

MCE

NSE

NAT

NXN

R

1.000 [1] 0.805 [3] 1.000 [2]

0.753 [3] 0.806 [2] 0.945 [1]

NXN

CEH[2]

0.807 [1] 0.247 [3] 0.558 [2]

0.738 [1] 0.181 [3] 0.468 [2]

CEH[1]

CEH[2]

0.892 [1] 0.748 [2]

0.711 [1] 0.633 [2]

CEH[1]

CEH[2]

0.884 [2] 1.000 [1] 0.476 [3]

0.820 [2] 0.941 [1] 0.665 [3]

NAT

R

R

CEH[1]

Scores are same as MCE scores of NXN model.

Table 7d Comparison of DMUs in level 3. DMU

ES

DEA extension CE

1 8 a

1.000 1.000

R

NXN

R

NXN

1.000 [1] 0.805 [2]

0.795 [1] 0.761 [2]

1.000 [1] 1.000 [1]

MLCE

R

NXN

R

NXNa

R NXN

0.317 [2] 0.353 [1]

0.130 [1] 0.028 [2]

1.000 [1] 1.000 [1]

[1] [2]

1.000 [1] 0.885 [2]

1.000 [1] 0.147 [2]

Scores are same as MCE scores of NXN model.

Table 7e Comparison of DMUs in Level 4. DMU

ES

DEA extension CE

12 14 15 a

1.000 1.000 1.000

GCE

MCE

R

NXN

R

NXN

1.000 [1] 0.805 [3] 1.000 [1]

0.979 [1] 0.951 [2] 0.524 [3]

1.000 [1] 1.000 [1] 1.000 [1]

MLCE

NSE

R

NXN

R

NXNa

R NXN

0.739 [1] 0.646 [2] 0.046 [3]

0.259 [2] 0.278 [1] 0.046 [3]

1.000 [1] 1.000 [1] 1.000 [1]

[2] [1] [3]

0.590 [3] 1.000 [1] 0.759 [2]

NAT

N/A N/A N/A

Scores are same as MCE scores of NXN model.

Table 8a Comparison of ranking between pre- and post-stratification with R. Data

R

DMU

CE

GCE

MCE

MLCE

NSE

CEH[1]

CEH[2]

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

11 7 6 2 4 8 10 12 1 5 3 14 9 13 15

11 5 6 1 7 8 10 12 2 9 3 13 4 15 13

10 5 3 7 2 6 11 12 1 8 4 13 9 14 15

11 3 1 1 5 8 8 11 4 8 6 13 7 13 13

11 9 8 5 4 7 10 12 2 6 3 13 1 14 15

12 6 7 4 5 8 10 11 3 9 2 13 1 14 15

7 6 2 11 1 3 12 10 5 8 9 14 4 13 15

11 5 7 4 6 8 8 11 3 8 2 13 1 13 13

11 7 6 3 5 8 10 12 4 9 2 13 1 14 15

11 7 3 5 6 8 10 12 1 9 4 15 2 13 14

7 6 3 8 1 4 10 11 2 5 12 14 9 13 15

11 4 2 6 1 8 10 12 3 9 7 14 5 13 15

7 6 2 8 1 4 10 11 3 5 12 14 9 13 15

11 4 1 5 2 8 10 12 3 9 7 14 6 14 15

MAD

1.40 0.88

q

2.00 0.83

0.93 0.95

best performance, the gaps between CEHs and other methods are not significant. It indicates that the performance of CEHs is still considered acceptable in terms of the consistency. Considering the simplicity of CEHs that do not require any LP formulation, the results presented here demonstrate that the proposed CEHs could be an alternative method to rank DMUs initially and replace various DEA-based CE ranking methods that require LP formulation.

3.13 0.56

1.07 0.94

1.87 0.82

1.93 0.82

5.2. Simulation Finally, to extensively study the efficiency of the two heuristics vs. CE regarding ranking DMUs, we have implemented the CE model in spreadsheets with VBA (Visual Basic for Applications) and solve it by Excel Analytical Solver Platform V2016-R2 using ‘Gurobi’ Solver Engine (see www.solver.com) on a dual 3.50 GHz Pentium Xeon server with 16 GByte memory. We have randomly

328

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

Table 8b Comparison of ranking between pre- and post-stratification with NXN. Data

DMU

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

PRE

POST

NXN

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

11 7 6 2 4 8 10 12 1 5 3 14 9 13 15

11 4 5 1 7 10 9 12 2 8 6 13 3 14 15

10 5 3 7 2 6 11 12 1 8 4 13 9 14 15

11 3 1 1 5 8 8 11 4 8 6 13 7 13 13

11 9 8 5 4 7 10 12 2 6 3 13 1 14 15

11 5 2 7 1 8 10 12 4 9 6 14 3 13 15

7 6 2 11 1 3 12 10 5 8 9 14 4 13 15

11 5 2 7 1 8 10 12 4 9 6 14 3 13 15

11 7 6 3 5 8 10 12 4 9 2 13 1 14 15

11 7 3 5 6 8 10 12 1 9 4 15 2 13 14

7 6 3 8 1 4 10 11 2 5 12 14 9 13 15

11 4 2 6 1 8 10 12 3 9 7 14 5 13 15

7 6 2 8 1 4 10 11 3 5 12 14 9 13 15

11 4 1 5 2 8 10 12 3 9 7 14 6 13 15

MAD

1.73 0.85

q

CE

GCE

MCE

2.00 0.83

MLCE

1.87 0.83

NSE

1.60 0.86

generated the values of inputs and outputs using uniform distribution with the two sets of minimum and maximum values, U[1, 5] and U[1, 10]. We have also used four sets of the numbers of inputs (I) and outputs (O), {3, 2}, {5, 2}, {3, 5}, and {7, 2} and five sets for the number of DMUs, {20, 40, 60, 80, 100}. We run twenty times for each combination and Tables 9a and 9b display the average computational times (CTÞ in seconds for the problems with any input and output values based on U[1, 5] and U[1, 10], respectively. From ) the same runs, the average values of the correlation coefficients (q for the rankings generated by CE and the two heuristics are calculated and displayed in Tables 10a and 10b for U[1, 5] and U[1, 10], respectively. For each of these computational times and the correlation coefficient, the grand average values are calculated for each case of the inputs and outputs. For simplicity, the term ‘average’ refers to the ‘grand average’ for both computational times and the correlation coefficient while the computational time and the correlation coefficient refer to the average values from the twenty replications.

CEH[1]

1.07 0.94

CEH[2]

1.87 0.82

1.87 0.82

From Tables 9a and 9b, it is apparent that the computational time for CE is significantly higher than that from the two CEH methods7 to 15 times higher. Fig. 1 clearly shows that the computational time for CE sharply increases when the number of DMUs increases while computational times for CEH methods are much less sensitive to the number of DMUs. This is because as the number of DMUs increase, the CE should solve an LP problem for each DMU. We have not observed any visible difference between the two CEH methods in terms of the computational time. Further, we see the possibility that the number of variables may affect the computational time too. For example, when the number of variables varies from 5 with{I, O} = {3, 2} to 9 with {I, O} = {7, 2}, the average of CT in general increases while there is an exception in case of {I, O} = {5, 2}. By comparing the correlation coefficients from Tables 10a and  ’s in Table 10a is larger 10b, we can recognize that the average of q than that in Table 10b, indicating that the wider variation from U [1, 10] may add more discrimination in terms of the ranking

Table 9a Comparison of computational time with U[1, 5]. n

Time (sec)

{I, O} = {3, 2}

{I, O} = {5, 2}

{I, O} = {3, 5}

{I, O} = {7, 2}

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

4.8

62.6

6.8

7.8

61.5

5.5

5.3

20

CT

59.1

4.6

4.6

63.0

4.9

40

CT

120.1

8.3

8.7

135.4

8.6

8.6

139.0

12.6

12.5

131.5

9.1

9.5

60

CT

207.4

11.8

12.7

223.4

13.6

12.7

222.2

17.3

16.0

228.3

14.3

13.8

80

CT

306.7

19.2

18.3

310.9

19.8

21.4

309.9

27.2

25.9

317.2

22.8

22.0

100

CT

382.2

25.0

24.4

387.0

30.1

31.0

389.5

38.5

40.7

396.0

33.2

35.7

Average (CTÞ

215.1

13.8

13.7

233.9

15.4

15.7

224.6

20.5

20.6

226.9

17.0

17.3

Table 9b Comparison of computational time with U[1, 10]. n

Time (sec)

{I, O} = {3, 2}

{I, O} = {5, 2}

{I, O} = {3, 5}

{I, O} = {7, 2}

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

5.2

67.7

6.1

6.0

60.5

5.5

5.5

20

CT

50.2

4.1

4.1

63.5

4.9

40

CT

117.4

8.1

7.5

143.1

8.9

9.1

144.3

10.2

9.7

136.9

9.1

8.8

60

CT

154.6

11.0

11.8

215.1

13.9

13.5

214.9

17.2

15.5

210.1

14.6

15.5

80

CT

281.2

17.4

17.0

290.1

20.4

21.0

294.6

26.9

25.9

289.1

23.6

22.5

100

CT

360.5

23.0

26.8

361.5

33.6

33.4

370.9

39.4

39.4

359.8

31.2

30.5

Average (CTÞ

192.8

12.7

13.4

214.7

16.3

16.4

218.5

19.3

19.3

211.3

16.8

16.6

329

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330 Table 10a Comparison of correlation coefficient of rankings with U[1, 5]. n

Corr coeff

{I, O} = {3, 2}

{I, O} = {5, 2}

{I, O} = {3, 5}

{I, O} = {7, 2}

Average

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

20 40 60 80 100

 q  q  q  q  q

— — — — —

0.898 0.935 0.952 0.962 0.969

0.890 0.913 0.935 0.937 0.945

— — — — —

0.840 0.944 0.950 0.970 0.961

0.864 0.904 0.912 0.945 0.927

— — — — —

0.845 0.921 0.911 0.941 0.936

0.814 0.905 0.890 0.921 0.932

— — — — —

0.901 0.953 0.978 0.970 0.983

0.864 0.926 0.948 0.932 0.950

Þ Average (q

—

0.943

0.924

—

0.933

0.9104

—

0.9108

0.8924

—

0.957

0.924

0.865 0.925 0.935 0.947 0.950

Table 10b Comparison of correlation coefficient of rankings with U[1, 10]. n

Corr coeff

20 40 60 80 100

{I, O} = {3, 2}

{I, O} = {5, 2}

Average

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

CE

CEH[1]

CEH[2]

q q q q q

— — — — —

0.899 0.942 0.957 0.972 0.951

0.898 0.918 0.938 0.947 0.931

— — — — —

0.850 0.972 0.963 0.957 0.980

0.831 0.917 0.939 0.926 0.950

— — — — —

0.853 0.968 0.902 0.950 0.917

0.823 0.957 0.893 0.944 0.904

— — — — —

0.925 0.969 0.960 0.981 0.987

0.897 0.951 0.916 0.947 0.955

Þ Average (q

—

0.944

0.926

—

0.944

0.915

—

0.918

0.904

—

0.964

0.933

CE CEH[1] 300

Time (sec)

{I, O} = {7, 2}

CEH[1]

400

CEH[2]

200

100

0

{I, O} = {3, 5}

CE

20

40

60

80

100

Number of DMUs Fig. 1. Comparison of computational time in case of {I, O} = {7, 2}.

determination. We can also observe that the heuristics generally tend to perform well when the number of DMUs exceeds a certain number. For example, when the number of DMUs, denoted by n, is  ’s generated by the set to 20 and for the first case of UN [1, 5], all q both heuristics are less than 0.901. But in case of U[1, 5], the aver ’s increases as ‘n’ increases. In case of U[1, 10], the increasage of q ing trend is not as consistent as in U[1, 5]. Finally, the averages of q ’s of rankings generated by CE and CEH[1] and CE and CEH[2], turn out to be 0.939 and 0.916, respectively. It strongly implies that the rankings generated by the heuristics are highly related with those by CE.

0.873 0.949 0.934 0.953 0.947

method in DEA methodology. Not like other CE-based ranking methods proposed by many other researchers, the proposed two CEH methods do not require any Linear Programming (LP) formulation/implementation. Consequently, the result does not depend upon LP software or DEA software. We also present the systematic consistency evaluation framework based on the DMU stratification concept, call it as the stratification-based consistency evaluation framework (SCEF) where rankings between pre-stratification and post-stratification are compared. Two empirical examples from literature are used to demonstrate the performance of CEH methods against many other full ranking methods. The SCEF is used to evaluate the consistency of all ranking methods in the examples. Based on the examples from the literature, we observe that CEH methods show the best performance in the first example and the comparable performance in the second example in terms of the consistency compared to other ranking methods. The second example also demonstrates that the ranking pattern generated by CEH methods is consistently similar to that generated by the normalized attractive score (NAT) based ranking method for all stratified levels. From the simulation examples, we confirm that the rankings generated by the heuristics are highly related with those by CE. Considering the simplicity and the comparable performance demonstrated from these examples, we believe that the proposed CEH methods would work as a good surrogate for other complex DEA-based full ranking methods. Further, we also claim that the presented SCEF can be used as a useful tool to evaluate the performance of any DEA-related full ranking methods. Finally, although we numerically demonstrate the performance of CEHs, more extensive future studies are needed to thoroughly investigate their performance under diverse conditions. We claim that our research is a starting point for those future studies.

6. Summary and conclusion Acknowledgement This paper deals with ranking decision making units (DMUs) with data envelopment analysis (DEA) method and its diverse extensions as a full ranking method. We propose two simple heuristics of ranking DMUs, which are in line with the principle and structure of peer evaluation in the cross-efficiency (CE)

This material is based upon work that is supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, under project number SCX 3130315.

330

J.-D. Hong, K.-Y. Jeong / Computers & Industrial Engineering 111 (2017) 320–330

Appendix A

References

List of some acronyms AT CE CEH CEP CRS DEA DMU ES GCE LP MAD MCE MLCE MN NAT NXN NSE OWA R SCEF SE UN VBA

attractiveness score cross-efficiency cross-efficiency based heuristic cross-efficiency profiling constant returns to scale data envelopment analysis decision making unit efficiency score game cross-efficiency linear programming mean absolute deviation multiplicative cross-efficiency maximum log cross-efficiency max-normalized normalized attractiveness score min-max normalized normalized super efficiency ordered weighted averaging regular (without normalizing) stratification-based consistency evaluation framework super efficiency uniform distribution visual basic for applications

Appendix B Proof: multiplying the first part by the second part of Eq. (19) ensures the constraints (5) in CE model. Proof. Eq. (5) can be rewritten as s m X X urk yrj 6 v ik xij ; r¼1

j ¼ 1; . . . ; n;

ðB:1Þ

i¼1

Since

Ps Pr¼1 m i¼1

Pm i¼1

ur yrj

v i xij

v ik xij > 0, dividing both sides yields

6 1;

8j:

ðB:2Þ

Now, replacing ur by 1=yrk and Eq. (19), which is given by

Ps  ð1=yrk Þyrt Pr¼1 ; m i¼1 ð1=xik Þxit

v i by 1=xik becomes the first part of

8t;

ðB:3Þ

which can’t be ensured to be less than or equal to one (1.0). Thus, to ensure (B.3) to be less than or equal to one (1.0), we divide (B.3) by the maximum value of (B.3) for all j. The resulting equation is

(Ps

Pr¼1 m

ð1=yrk Þyrj

i¼1 ð1=xik Þxij

),

Ps  ð1=yrk Þyrt maxt Pr¼1 6 1: m i¼1 ð1=xik Þxit

ðB:4Þ

The fact that (B.4) can be expressed as Eq. (19) completes the proof. h

Andersen, P. M., & Petersen, N. C. (1993). A procedure for ranking efficient units in data envelopment analysis. Management Science, 39, 1261–1264. Anderson, T. R., Hollingsworth, K. B., & Inman, L. B. (2002). The fixed weighting nature of a cross-evaluation model. Journal of Productivity Analysis, 18(1), 249–255. Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operations Research, 2(6), 429–444. Charnes, A., Cooper, W. W., Seiford, L. M., & Stutz, J. (1982). A multiplicative model for efficiency analysis. Socio-Economic Planning Sciences, 16(5), 223–224. Charnes, A., Haag, S., Jaska, P., & Semple, J. (1992). Sensitivity of efficiency classifications in the additive model of data envelopment analysis. International Journal of Systems Science, 23(5), 789–798. Cook, W. D., & Zhu, J. (2014). DEA cobb-douglas frontier and cross-efficiency. Journal of Operational Research Society, 65(2), 265–268. Cooper, W., Seiford, L. M., & Tone, K. (2007). Data envelopment analysis: A comprehensive text with models, applications, references and DEA-solver software (2nd ed.). New York: Springer. Cooper, W., Seiford, L. M., & Zhu, J. (2011). Data envelopment analysis: History, models and interpretations. In W. Cooper, L. M. Seiford, & J. Zhu (Eds.), Handbook on data envelopment analysis (pp. 1–39). New York: Springer. Despotis, D. K., Stamati, L. V., & Smirlis, Y. G. (2010). Data envelopment analysis with nonlinear virtual inputs and outputs. European Journal of Operational Research, 202(2), 604–613. Doyle, J., & Green, R. (1994). Efficiency and cross-efficiency in DEA: Derivations, meanings and the uses. Journal of the Operational Research Society, 45(5), 567–578. Jeong, B. H., & Ok, C. S. (2013). A New ranking approach with a modified cross evaluation matrix. Asia-Pacific Journal of Operational Research, 30(4), 1–17 (Article 1350008). Lam, K. F., & Bai, F. (2011). Minimizing deviations of inputs and output weights form their means in data envelopment analysis. Computers & Industrial Engineering, 60(4), 527–533. Liang, L., Wu, J., Cook, W., & Zhu, J. (2008). The DEA game cross-efficiency model and its Nash equilibrium. Operations Research, 56, 1278–1288. Lim, S. (2012). Minimax and maximin formulations of cross-efficiency in DEA. Computers & Industrial Engineering, 62(3), 726–731. Meza, D., & Jeong, K. (2013). Measuring efficiency of lean six sigma project implementation using data envelopment analysis at NASA. Journal of Industrial Engineering Management, 6(2), 401–422. Oukil, A., & Amin, G. (2015). Maximum appreciative cross-efficiency in DEA: A new ranking method. Computers & Industrial Engineering, 81, 14–21. Ramanathan, R. (2003). An introduction to data envelopment analysis – A tool for performance measurement. New Delhi, India: Sage Publications. Seiford, L. M., & Zhu, J. (2003). Context-dependent data envelopment analysis: Measuring attractiveness and progress. OMEGA, 31(5), 397–480. Sexton, T. R., Silkman, R. H., & Hogan, A. J. (1986). Data envelopment analysis: Critique and extensions. In R. H. Silkman (Ed.). Measuring efficiency: An assessment of data envelopment analysis (Vol. 32, pp. 73–105). San Francisco: Jossey-Bass. Sotiros, D., & Despotis, D. K. (2012). Max-normalized DEA models with an extension to DEA models with nonlinear virtual inputs and outputs. In V. Charles & M. Kumar (Eds.), Data envelopment analysis and its applications to management (pp. 68–86). Newcastle upon Tyne, UK: Cambridge Scholars Publishing. Sun, S., & Lu, W. M. (2005). A cross-efficiency profiling for increasing discrimination in data envelopment analysis. Information Systems and Operational Research, 43 (1), 51–60. Wang, Y. M., & Chin, K. S. (2010a). Some alternative models for DEA cross-efficiency evaluation. International Journal of Production Economics, 128(1), 332–338. Wang, Y. M., & Chin, K. S. (2010b). A neutral DEA model for cross-efficiency evaluation and its extension. Expert Systems with Applications, 37, 3666–3675. Wang, Y. M., & Chin, K. S. (2011). The use of OWA operator weights for crossefficiency aggregation. Omega, 39, 493–503. Wang, Y. M., Chin, K. S., & Jiang, P. (2011). Weight determination in the crossefficiency evaluation. Computers & Industrial Engineering, 61(3), 497–502. Wu, J., Sun, J., & Liang, L. (2012). Cross-efficiency evaluation method based on weight-balanced data envelopment analysis model. Computers & Industrial Engineering, 63(2), 513–519. Zerafat Angiz, M., Mustafa, A., & Kamali, M. J. (2013). Cross-ranking of decision making units in data envelopment analysis. Applied Mathematical Modelling, 37, 398–405. Zhu, J. (2014). Quantitative models for performance evaluation and benchmarking: Data envelopment analysis with spreadsheets (3rd ed.). New York: Springer.