Relative Effectiveness Analysis under Fuzziness

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 102 (2016) 231 – 238

12th International Conference on Application of Fuzzy Systems and Soft Computing, ICAFS 2016, 29-30 August 2016, Vienna, Austria

Relative effectiveness analysis under fuzziness B K Mangaraj* Production, Operations and Decision Sciences Area XLRI-Xavier School of Management Jamshedpur-831001, India

Abstract Effectiveness is an important criterion of performance evaluation that requires the extent to which the goals are achieved. Finding a value for this criterion becomes difficult, when it is considered in the context of fuzzy goals. However, we often require it numerically in a scale to monitor the performance level in a meaningful way. But, there exists no such structured methodology in the literature to compute effectiveness value in a fuzzy environment. This paper defines a fuzzy effectiveness measurement model in a multi-criteria framework to assess this value in the range of 0 to 1. A fuzzy goal programming approach is used to determine the effective point, which also acts as the benchmark for the proposed relative effectiveness analysis. Finally, this approach is demonstrated for its applicability, taking an example of performance evaluation of candidates in a semester examination. The result is then compared with the existing procedure of cumulative- grade- point average. © 2016 2016 The TheAuthors. Authors.Published Published Elsevier byby Elsevier B.V.B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility ofthe Organizing Committee of ICAFS 2016. Peer-review under responsibility of the Organizing Committee of ICAFS 2016 Keywords:Effective point; multi-objective decision-making; fuzzy goal programming; performance evaluation; relative effectiveness; priority.

1. Introduction In Management literature, effectiveness is considered as one of the important criteria of performance evaluation. Broadly, it is defined as the degree to which all the performance criteria achieve their targets1,2,3. Hence, it depends on the achievement of goals and has been explained as a measure of the distance between observed outputs and a set of desired goals4. This implies that higher the effectiveness, lessis the distance of achievement from the target. In this context, Golany and Tamir4extended DEA5 to evaluate effectiveness while classifying the criteria as outputs and inputs. They viewed effectiveness of a decision making unit (DMU) in terms of threshold limits of the corresponding output vector and considered a DMU to be effective, only if, the outputs are greater than or equal to these limits. But,

* Corresponding author. Tel.: +91-657-398-3195; fax: +91-657-222-7814. E-mail : [email protected]

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Organizing Committee of ICAFS 2016 doi:10.1016/j.procs.2016.09.395

232

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238

the question arises as to how to define these goals or threshold limits. In goal programming6, goals are sometimes set as un-attainable targets and so, efforts are made to minimize the deviation of achievements from these goals. In that case, explaining the degree of effectiveness as the measure of the distance between achievements and their corresponding goals becomes meaningless. Though, there is no difficulty in assuming an un-attainable target as a goal, but there is also a need to define an achievable target as an effective goal, so that the degree of effectiveness can be computed. In this context, the interactive multi-objective linear programming (MOLP) procedure of Golany2as an extension of DEA, to obtain an effective point among the alternative efficient points is quite meaningful. For this, he reduced the multi-objective effectiveness problem to a LP, and maximized the objective function as the sum of output variables with respect to the envelopment constraints. For the effectiveness analysis, he determined the output targets for given inputs with respect to the effective point. However, in his model, he did not ensure a common minimum achievement level for all the goals to be qualified for effectiveness measurement. At the same time, the literature has also been completely silent regarding fuzziness of the goals. The purpose of this paper is to propose a fuzzy effectiveness measurement model (FEMM) to assess a set of DMUs having imprecise targets. In this model, the fuzziness of the goals is explained as the level of satisfaction of the criteria as per the decision–maker in the interval [0, 1] A suitable membership function can represent this relationship, which may be linear, piecewise linear or even non-linear. Hence, achieving the maximum overall satisfaction in all the criteria simultaneously becomes the key issue in determining a non-dominated effective point as the frame of reference. Thus, this point becomes the benchmark for effectiveness measurement of all the DMUs under consideration, and is the principle behind relative effective analysis (REA), as discussed in this paper. The major contribution in this paper is that, the value of effectiveness is explained in the range 0 to 1, where the effective point is set at a value 1. Closer the DMU to the effective point, higher is the effectiveness and hence, higher is the effectiveness value. The details of REA is presented in the subsequent sections, In the following section 2, a benchmarking model, as a first step in REA, is described in a fuzzy goal programming form7. Interval priorities8, 9are used to model the membership function of the fuzzy goals in terms of piece-wise linear functions. A two-phase methodology is employed for generating the effective point on the basis of empirical data on the criteria. Section 3utilizes this point to represent the FEMM as a set of fuzzy goals. Based on the achievement values of the DMUs and the effective goal, triangular marginal value functions are used to normalize the effectiveness criteria in the range of[-1, 1]. The model is then extended to a multi-attribute ranking procedure to assess the relative effectiveness of the DMUs in the range of 0 to 1. Section 4provides the application of the whole procedure for the effectiveness evaluation of performance of students in a semester examination and compares the result with their individual cumulative-grade-point-averages. The last section 5 discusses some concluding remarks of the work. 2. Benchmarking model Let there be m DMUs whose effectiveness are to be compared along n effectiveness criteria. In order to obtain the effective point, let us consider the following fuzzy goal programming formulation of FEMM as: Ej ; j = 1, 2, 3…n1;

Find E. To satisfy

(1)

m

Ek =

¦e x

i ik

ˆ Ek ; k = 1, 2, 3…n2

i 1

Subject to m

¦e

i

1 ; and

.( 2)

i 1

ei • 0

: i =, 2… m; and n1 +

n2 = n

233

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238 m

Where

¦e x

i ij

‫ ذ‬E j indicate the jth fuzzy effectiveness goal approximately greater than or equal tothe aspiration

i 1

m

level E j . Similarly,

¦e x

i ik

. ‫ ذ‬Ek represents thekth fuzzy goal approximately lesser than or equal to the aspiration

i 1

m

level Ek . The constraint

¦e

i

1 enforces a convex combination of DMUs as the envelopment agent2. Now the

i 1

objective is to find out the effectiveness weight vector E= ( e1 , e2 ,…., em ) that satisfies inequality system (2.1) under such constraint that maximizes the effectiveness in all the criteria simultaneously. This requires converting model 2.1 into its equivalent precise form by constructing suitable membership functions of the fuzzy effectiveness m

goals. For example, for the goal

¦e x

i ij

‫ ذ‬Ej

, j= 1, 2, 3…n1; with Tj as the tolerance limit, we can specify

i 1

priority levels 1, 2, 3,….., N in ascending order8, 9 , where each priority level N is assigned the maximum overall satisfaction E N on a scale from 0 to 1. Hence, from the definition of priority levels, it follows that, 1=

E1

•

E 2 t E3 t E j

, }. t . E N = 0

(3)

Corresponding to each goal, the interval [ T j , E j ] is divided into sub-intervals [ q j , N , q j , N 1 ] , such that,

Tj = q j , N ” q j , N 1 ” q j , N 2 ” ………. ” q j , 1 = E j

( 4)

This means that, the interval goal [ q j , N , q j , N 1 ] has the maximum overall satisfaction

E1 and any value for

m

the goal in this range has, as the level of satisfaction. Similarly, for the goal

¦e x

i ij

ˆ Ek , k= 1, 2, 3…n2 with ܵ௞

i 1

as the tolerance limit, the priority weight 1= G1 •

G2 ൒ G3

൒

G M ࣅ [0, 1], and k= 1, 2… M satisfy the inequalities as;

E1 ǡ ǥ Ǥ ൒ G M = 0; for which

( 5)

the interval [ܶ௞ ǡ ‫ܧ‬௞ ሿ of the fuzzy goal ‫ܧ‬௞ is divided into sub-intervals [‫ݍ‬௞ǡெ ǡ ‫ݍ‬௞ǡெିଵ ሿ, such that,

Ek = qk , M • qk , M 1 • qk , M  2 •………. • qk , 1 = S k ,

( 6)

The diagrammatic representation of the interval priority structures are shown in fig. 1, where the membership ෪௞ ) is constituted as a monotonically increasing( or decreasing) piece-wise function of the effectiveness goal ‫ܧ‬෩ఫ (or ‫ܧ‬ linear function of the achievement.

234

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238

μ (Ek(x))

μ(Ej (x))

F ȕ1 = 1.0

E

F

ȕ2

į1 = 1.0

į2

E

ȕ3

ȕ4

į3

D

D

C

į4

C

.....

.....

Ɂ୒ିଵ

B

B

ȕ N-1

ȕN = 0.0

A Tj = q jN

qj N-1

qj4

qj3

qj2

qj1

A Ej (x) Sk=qKm

“୏୑ିଵ

“୩ସ

“୩ଷ

“୩ଶ

įM = 0.0

“୩ଵ

Fig.1. Piece-wise linear membership functions of fuzzy goals.

The mathematical formulation of the membership functions are stated as follows: m

P  ENJ

if ­0 ° °° ( E N 1  E N ) m (¦ ei .xij  q j , N ) if T j  ®E N  (q j , N 1  q j , N ) i 1 ° ° if °¯ 1

¦ e .x i

d TJ

ij

i 1

m

¦

 Ej

e .xij

i i

( 7)

i 1

m

¦ e .x i

t Ej

ij

i 1

m

P  EMk

if ­0 ° °° m (G M 1  G M ) (qk , M  ¦ ei .xik ) if Ek  ®G M  (qk , M  qk , M 1 ) i 1 ° ° if °¯ 1

¦ e .x i

ik

t Sk

i 1

m

¦

e .xik

i i

 Sk

( 8)

i 1

m

¦ e .x i

ik

d Ek

i 1

By using the fuzzy decision (max-min) of Bellman and Zadeh10 for the piecewise linear membership functions, the equivalent linear programming model of the multi-objective effectiveness problem for a compromise solution is obtained as:

Max.

O

235

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238

O

````````s.t.

O൑

: j 1, 2,}, n1 ; N

1, 2, }}, N .

d E N 1 : j 1, } , n1; N 1, 2, }} , N .



μ ENj



d μ ENj



μ EMk

1, }, n 2; M

:k



μ EMk d G M 1 : k

1, } , n 2; M

1, 2, }} , M 1, 2, }}, M

m

¦e

1

i

.( 9)

i 1

ei • 0

: i = 1, 2, 3… m.

The solution of the model (2.8) is a compromise solution maximizing all the effectiveness criteria, where the value of O indicates the highest common achievement level of the criteria, and hence a desirable level of overall satisfaction. This means that the solution pushes all of the criteria to a common minimum level,when they are considered simultaneously with equal importance. However, model (8) does not always yield a strongly efficient solution11, 12, 13 and hence may be improved. Based on the two-step approach14, 11, a competitive-cum compensatory model is developed using arithmetic mean as the aggregation operator and the value Ȝ obtained from model 8. The mathematical formulation of the model is stated as: n2

§ 1 · n1 ¨ ¸ (¦O j + ¦ Ok ) ©n¹ j 1 k 1

Max.

O dOj

s.t.

 dE

μ E

j N



μ ENj N 1

O ൑  O ௞ ൌ μ  EMk



μ EMk d G M 1

: j

:j

1, }, n1; N

1, } , n1; N

:k

1, 2, }}, N .

1, 2, }} , N

1, }, n 2; ‫ ܯ‬ൌ ͳǡʹǡ ǥ ǥ ǡ ‫ܯ‬

: ݇ ൌ ͳǡ ǥ ǡ ݊ʹǢ ‫ ܯ‬ൌ ͳǡʹǡ ǥ ǥ ǡ ‫ܯ‬

m

¦e

i

1

( 10)

i 1

ei • 0: i = 1,2…m. The solution E obtained from model (2.9) is a non-dominated compromise solution for the FEMM (2) and can be used to determine the effective target for the DMUs under consideration. ௃ ௄ Definition1 (Effective goal): It is the n-tuple (‫ܤ‬ଵ௃ ǡ ‫ܤ‬ଶ௃ ǡ ǥ ǥ Ǥ ‫ܤ‬௡ଵ ǡ ‫ܤ‬ଵ௄ ǡ ‫ܤ‬ଶ௄ ǡ ǥ ǡ ‫ܤ‬௡ଶ ሻ that represents the value of the neffectiveness criteria with respect to the point E obtained from the model (10). This non-dominated point attains the best possible value that has the highest common minimum satisfaction level in all the effectiveness criteria. 3. Measuring effectiveness The effectiveness value of a DMU depends upon its relative position with respect to the effective goal. Hence, let us use this effective point for standardizing the DMUs, wherelinear interpolation is used to estimate the marginal

236

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238 m

value of the criteria in the range [-1, 1]. For the goal

¦e x

i ij

Ⴭ B j , let I *j and I j be the maximum and minimum

i 1

value of the DMUs and are represented as the positive ideal and negative ideal respectively. Based on these ideals, marginal value function for the criterion is definedas: J

݂‫ܧ‬௜௝ ൌ 

‫ ۓ‬xij  B j ݂‫ܤݎ݋‬௝௃  ൑  ‫ݔ‬௜௝  ൑ ‫ܫ‬௝‫כ‬ ۖ * ۖ I j  B jJ ‫ ۔‬x  BJ j ۖ ij ݂‫ܫݎ݋‬௝ି ൑  ‫ݔ‬௜௝ ൑  ‫ܤ‬௝௃ ۖ J  B  Ij ‫ ە‬j

…

. ( 11)

Clearly, fEij has a value in the range [-1, 1] for all the values of i and j, where the effective goal is at the centre m

(0, 0, 0). Similarly, the marginal value function for the goal

¦e x

i ik

ˆ Ek is expressed as:

i 1

‫ۓ‬ ۖ ۖ

fEik

‫۔‬ ۖ ۖ ‫ە‬

BkK  xij * k

K k

I B

BkK  xij K k

B I

 k

݂‫ܤݎ݋‬௞௄  ൑  ‫ݔ‬௜௞ ൑ ‫ܫ‬௞‫כ‬ …

.( 12a)

݂‫ܫݎ݋‬௞ି  ൑  ‫ݔ‬௜௞ ൑  ‫ܤ‬௞௄

I k* , I k denote the positive ideal and negative ideal respectively for the value function. It may be noted that, at least one of the components of the marginal value tuple of a DMU is negative due to non-dominance nature of the effective point. Based on the above standardization, relative effectiveness is defined as: Definition 2: (Relative effectiveness) It is defined as the degree of congruence of a DMU with the effective point. In order to measure relative effectiveness, it becomes necessary to explain the following concepts. Definition 3: Relative Value ( RV ): It is the arithmetic mean of fEij (i.e., marginal values of ithDMU in n criteria). Hence,

RVi =

1 (¦ fEij fEij  ¦ fEik ) : i 1, 2,3,}.., m n j k

(12b)

Clearly, RVi has a value in the range – 1 to 1 as fEij ࣅ [-1, 1] and fEik ࣅ [-1, 1] for all j and k. Definition 4: Relative Closeness ( RC ): It is the average distance of a DMU from the effective point. Hence,

RCi = 2

1 (¦ fEij2  ¦ fEik2 ) n j k

:i

1, 2, 3,}} , m

( 13).

It can be noted that, RCi , being the average squared distance has the value in the range 0 to 1for all j and k. Based on the above definitions, we define relative effectiveness of a DMU in terms of its degree of congruence with

237

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238

the effective point. Higher the degree better is the congruence and hence, more is the effectiveness value. Mathematically, it is expressed as:

REi =

1  RVi : i 1, 2, 3, }} , m. 1  RCi

( 14)

One finds that, the value of REi is obtained in the range of 0 to 1. Thus, relative effectiveness is maximum at the effective point and has the value as 1. 4. Performance evaluation by relative effectiveness Consider the example of threestudents whose performances are evaluated in a semester examination consisting of three papers, viz., Finance, Marketing and Human Resource Management. Table-1 presents their individual marks and corresponding grades in all the three papers awarded by three professors, who have their own schemes of evaluation (Table-2). Students were awarded marks in these three papers, each out of 100 and graded in a 2-point scale. Table.1. Marks and corresponding grades

Student Id.

Finance (Mark)

Marketing (Mark)

Human Resource (Mark)

Finance (Grade point)

Marketing (Grade point)

Human Resource (Grade point)

S1 S2 S3

98 90 88

90 84 82

50 68 75

2 2 1

2 1 1

1 1 1

Table.2. Schemes of evaluation Letter grade D (Distinction) P (Pass) F ( Fail)

Grade point

Interval priority

Finance

Marketing

Human Resource

2

[0.8, 1]

>= 90

> = 85

>= 80

1

[0.4, 0.8)

[50, 90)

[50, 85)

[50,80)

0

[0.0, 0.4)

< 50

< 50

<50

The grade point system is followed under the assumption that, the rate of performance in the papers is considered not to be uniform in the entire range. Each letter grade has a corresponding grade point and the average grade point of a student in all the papers represents his cumulative-grade-point-average. The grade of a student in a paper depends on his mark which falls in the range as defined by the individual professor. Table 3 presents the performance ranking of these students in terms of their cumulative-grade-point-average (CGPA). It is to be noted that, rank of a student is normally considered as higher for higher CGPA value. But, we observe that, a student gets the same grade point irrespective of his score at any point in the entire range defined by the grade. For example, S1 and S 2 get the same grade point for their marks in Finance, even though they have different individual marks in the paper. In this paper, instead of assuming a constant grade point for a range, piecewise linear function and an interval priority structure are considered, where each student gets a satisfaction index value for each paper in the range of 0 to 1. The average of the index values determines his performance index (PI) value, which can be ranked. In the present scenario, CGPA and PI are calculated with respect to goal score of 100 in each paper individually. However,by looking at the scores of individual papers simultaneously for all the students and by ensuring the highest common minimum in each paper individually, we obtain the effective goal score for the papers in Finance,

238

B.K. Mangaraj / Procedia Computer Science 102 (2016) 231 – 238

Marketing and Human Resource Management as 90.84, 84.16 and 64.95 respectively, using models 2.8 and 2.9. Relative effectiveness (RE) of the students as presented in table 3 reflects the degree of congruence of their performance with respect to this effectiveness goal score. Different ranks for each student obtained by arithmetic average (AA), CGPA, PI and RE are also presented for comparison. Table.3.Performance evaluation by effectiveness versus other measures

Student Id.

Arithmetic Average (AA)

CGPA

PI

S1 S2 S3

79.33 80.66 81.66

1.66 1.33 1

0.742 0.740 0.756

Relative Effectiveness (RE)

Rank based on AA

0.53 0.72 0.26

3 2 1

Rank based on CGPA

Rank based on PI

Rank based on RE

1 2 3

2 3 1

2 1 3

5. Conclusion This paper presents a multi-criteria framework of measuring effectiveness of a set of DMUs under consideration, in a fuzzy environment. The crux of this approach lies with the determination of an effective goal for the DMUs in conjunction with multiple numbers of imprecise effectiveness criteria. Once the effective goal is identified, relative effectiveness is measured as the degree of congruence of the DMUs with respect to this target. Here, a methodology is developed to evaluate the value of this measure in a 0 to 1 scale, so that, similar units can be compared. The validity of this measure is demonstrated by taking into consideration an academic performance evaluation problem. Moreover, this method could be applied to measure effectiveness of DMUs in private, public and governmental systems. References 1. GaetnerGH, Ramnarayan S. Organizational effectiveness: an alternative perspective.Academy of Management Review 1983;8(1): 97-107. 2. Golany B. An Interactive MOLP Procedure for the Extension of DEA to effectiveness analysis.Journal of Operational Research Society1988; 39(8): 725-34. 3. Mouzas S. Efficiency versus effectiveness in business networks.Journal of Business Research 2006; 59: 1124-32. 4. Golany B, Tamir E. Evaluating efficiency-effectiveness-equality trade-offs: A data envelopment analysis. Management Science1995; 41(7): 1172-84. 5. Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units.European Journal of Operations Research1978; 2: 429-44. 6. Charnes A, Cooper WW. Management models and linear programing application of linear programming. New-York: Wiley; 1961. 7. Tiwari RN, Dharmar S, Rao JR. Fuzzy goal programming-an additive model. Fuzzy Sets and Systems1987; 24(1): 27–34. 8. Rubin PA, Narasimhan R. Fuzzy goal programming with nested priorities.Fuzzy Sets and Systems1984; 14(2): 115-29. 9. Sinha SB, Rao KA, Mangaraj BK. Fuzzy goal programming in multi-criteria decision systems: a case study in agriculture planning. SocioEconomic Planning Science 1988; 22(2) : 93–101. 10. Bellman RE, Zadeh LA. Decision-making in a fuzzy environment.Management Science 1970;17(4): B141–B164. 11. Lee ES, Li RJ. Fuzzy multiple objective programming and compromising programming with pareto-optimum.Fuzzy Sets and Systems1993; 53(3): 275-88. 12. Guu SM, Wu YK. Weighted coefficients in two-phase approach for solving multiple objective programming problems.Fuzzy Sets and Systems1997;85(1): 45-48. 13. Wu YK, GuuSM. A compromise model for solving multiple objective problems. Journal of the Chinese Institute of Industrial Engineers 2001; 18(5): 87-93. 14. Li RJ, Lee ES. Fuzzy approaches to multi-criteria de-novo programs. Journal of Mathematical Analysis and Applications 1990; 153: 97-111.