- Email: [email protected]
Ordinal criteria in stochastic multicriteria acceptability analysis (SMAA)
European Journal of Operational Research 147 (2003) 117–127 www.elsevier.com/locate/dsw
Decision Aiding
Ordinal criteria in stochastic multicriteria acceptability analysis (SMAA) Risto Lahdelma a, Kaisa Miettinen b
b,*
, Pekka Salminen
c
a Department of Information Technology, University of Turku, Lemmink€aisenkatu 14 A, FIN-20520 Turku, Finland Department of Mathematical Information Technology, University of Jyv€askyl€a, P.O. Box 35 (Agora), FIN-40351 Jyv€askyl€a, Finland c Department of Economics, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland
Received 22 December 2000; accepted 8 March 2002
Abstract We suggest a method for providing descriptive information about the acceptability of decision alternatives in discrete co-operative group decision-making problems. The new SMAA-O method is a variant of the stochastic multicriteria acceptability analysis (SMAA). SMAA-O is designed for problems where criteria information for some or all criteria is ordinal; that is, experts (or decision-makers) have ranked the alternatives according to each (ordinal) criterion. Considerable savings can be obtained if rank information for some or all the criteria is sufficient for making decisions without significant loss of quality. The approach is particularly useful for group decision making when the group can agree on the use of an additive decision model but only partial preference information, or none at all, is available. 2002 Elsevier Science B.V. All rights reserved. Keywords: Multiobjective; Decision analysis; Decision support systems; Multiple criteria decision-making; Group decision-making; Ordinal data
1. Introduction In many real-life decision-making problems, it is very time consuming and expensive to produce accurate measurements for decision criteria. In fact, even after a costly process, the criteria measurements often still remain very uncertain or inaccurate. Much less work is normally required for *
Corresponding author. Tel.: +358-14-260-2743; fax: +35814-260-2771. E-mail addresses: [email protected].fi (R. Lahdelma), [email protected].fi (K. Miettinen), pekka.salminen@joensuu.fi (P. Salminen).
ranking the alternatives criterion-wise. Experts can often produce rankings for some criteria based on qualitative information about the alternatives. Considerable savings in planning costs can be obtained if rank information for some or all the criteria is sufficient for making decisions without significant loss of quality. Another aspect of real-life decision-making is related to the difficulty of obtaining and using preference information from the decision-makers (DMs). For these reasons, we introduce the SMAA-O method capable of treating mixed cardinal and ordinal criteria data and missing or imprecise preference information.
0377-2217/03/$ - see front matter 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 0 2 ) 0 0 2 6 7 - 9
118
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
The stochastic multicriteria acceptability analysis (SMAA) methods provide descriptive information about the acceptability of decision alternatives in discrete co-operative group decision-making problems. SMAA-methods are designed for situations where the criteria measurements are uncertain or inaccurate and it is for some reason impossible to obtain accurate preference information from the DMs. The SMAA methods are based on exploring the weight space in order to describe the preferences that would make each alternative the most preferred one, or that would give a certain rank for a specified alternative. Related research can be found in the literature. Charnetski [4] and Charnetski and Soland [5] introduced the comparative hypervolume criterion, based on computing for each alternative the volume of the multidimensional weight space that makes the alternative the most preferred. This method can handle preference information in the form of linear constraints for the weights, but is restricted to cardinal and deterministic criteria measurements and an additive utility function. Rietveld [21] and Rietveld and Ouwersloot [22] presented same types of methods for problems with ordinal criteria. In [21], ordinal criteria measurements are suggested to be transformed into cardinal values using alternative mappings whereas the availability of ordinal information about weights is assumed in [22]. The qualiflex method [1,20] approaches similar problems by testing how each possible ranking of alternatives is supported by different criteria in a similar setting to [22]. Bana e Costa [2,3] introduced the overall compromise criterion method for cardinal data for identifying alternatives generating the least conflict between several DMs. This method can handle partial preference information in the form of arbitrary weight distributions. Eiselt and Laporte [7] ranked alternatives using domains in problems with cardinal deterministic criteria. They computed the set of all weight vectors that make a particular alternative the best one using a linear additive value function. Data envelopment analysis (DEA) has also been suggested to be used in multiple criteria decision making problems (see, e.g., [24]). The SMAA method was initially developed based on the overall compromise criterion method.
The motivation for developing methods for problems where preference information (e.g., in the form of weights) is not available is that this situation is common in many real-life decision problems with multiple DMs. Preference information may be difficult to obtain for several reasons: • The DMs do not have time to study the problem carefully enough. • The analysts do not have time or resources to reveal the preferences of a large group of DMs. • The DMs have difficulties in comparing criteria. • The DMs are afraid of revealing their preferences in public. • The DMs do not want to fix their preferences because the preferences may change during the process. Even if preference information (e.g., weights) can be obtained from the DMs, it is unclear how the preferences of several DMs that disagree should be combined. Furthermore, because different weight elicitation methods have been observed to provide different weights for the same problem (see e.g. [25,26]), any subjective weight information should be considered uncertain. For these reasons, SMAA-type inverse methods can often be an appropriate approach in real-life decision problems, see [10,16]. Instead of trying to identify a single best alternative based on precise preference information, SMAA identifies the preferences that make each alternative the best one. Different SMAA variants are based on different decision models. SMAA [9,12] and SMAA-2 [13] assume the existence of a general utility or value function containing weighting coefficients that represent the importance of the criteria (e.g., an additive value function). SMAA-D [15] uses output/input efficiency measures of the DEA type. The DEA efficiency measure can, of course, be considered a special case of a general value function. SMAA, SMAA-2, and SMAA-D model the uncertainty or inaccuracy of criteria data by stochastic distributions. SMAA-3 [8] uses a double threshold model (pseudo-criteria) to model the uncertainty or inaccuracy of data and applies then an outranking procedure involving weighting
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
coefficients for criteria. This model is adopted from the ELECTRE III method (see, e.g., [17] and also [18], the latter being based on non-differentiable multicriteria optimisation). The information provided by the SMAA methods is descriptive. The DMs are given acceptability indices describing the variety of different preferences that support an alternative for the best rank or any particular rank. This information can be used for classifying the alternatives into more or less acceptable ones and those that are not acceptable at all, identifying good compromise candidates, and evaluating the overall acceptability of alternatives using the so-called holistic acceptability indices. SMAA also computes central weights describing the most typical preferences making an alternative the most preferred one. It is also possible to measure with confidence factors whether the problem information is accurate enough for decision-making. In other words, instead of giving exact solutions to the DMs, the SMAA methods aim at describing what kind of preferences correspond to different choices. Because different interest groups usually have different preferences in real-life decision problems, an alternative with a large acceptability index will more likely appeal to a greater number of interest groups than such an alternative that can receive the best rank only with a small variety of preferences. The earlier versions of SMAA are applicable only if the criteria are measured on cardinal scales. Treating ordinal ranks as cardinal data and then using this information in arithmetic computations could lead to a serious distortion. In this paper we introduce the SMAA-O method, which is a new variant of SMAA for problems where some or all of the criteria are measured on ordinal scales. This means that for ordinal criteria we only know which alternative is the best, the second best, and so on, but no absolute measurements are available. Ordinal criteria occur in many real-life problems where the DMs or experts are not able or willing to consider more accurate measurements. The development of SMAA-O is based on the SMAA-2 method [13]. The preliminary ideas of SMAA-O were presented in [19]. In SMAA-O, the ordinalto-cardinal transformation is not forced to be some specific mapping, no information about the
119
weights or preferences in general is assumed, and in addition to volumes of acceptabilities, also other descriptive measures are calculated. Due to the usage of Monte-Carlo simulation, no limits for the number of criteria and/or alternatives are set.
2. Description of SMAA-O 2.1. The problem A discrete multicriteria decision-making problem consists of a set of m alternatives measured in terms of n criteria. Each alternative is thus represented by a row vector xi ¼ ½xi1 ; xi2 ; . . . ; xin . Depending on the information available, the vector consists of deterministic or stochastic criteria measurements, or ordinal (rank) information. In case of deterministic criteria measurements, we assume that the DMsÕ preference structure is represented by a real-valued value function U ðxi ; wÞ involving weighting coefficients w. The value function may have any form jointly accepted by the DMs, but linear or additive forms are often appropriate. The DMsÕ subjective preferences are represented by weight vectors w ¼ ½w1 ; w2 ; . . . ; wn . We assume that the weight vector is non-negative and normalised, that is ) ( n X n w 2 W ¼ w 2 R jwj P 0 and wj ¼ 1 : ð1Þ j¼1
If a single weight vector could be commonly agreed on, the best alternative could be identified by evaluating the value function for each alternative and choosing the alternative i with the largest value U ðxi ; wÞ P U ðxk ; wÞ for k ¼ 1; . . . ; m. The value function could then also be used for ranking the alternatives. However, for the reasons discussed earlier, such a single weight vector may not be available. Therefore, a decision method is needed that can be used without precise weights. 2.2. The SMAA-2 method The SMAA-2 method was developed particularly for situations where neither criteria
120
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
measurements nor weights are precisely known. Uncertain or imprecise criteria measurements are represented by stochastic variables nij (instead of xij ) with an assumed or estimated joint probability distribution and density function f ðnÞ in the space X Rm n . Similarly, the DMsÕ unknown or partially known preferences are represented by a weight distribution with density function f ðwÞ in the weight space W. Total lack of preference information is represented by a uniform weight distribution in W, that is, f ðwÞ ¼ 1=volðW Þ. The value U ðni ; wÞ of each alternative is then a stochastic quantity, and consequently the rank of each alternative also depends on the stochastic criteria measurements and weights. The alternatives are analysed in terms of subsets of the weight space W. Stochastic sets of favourable rank weights are defined as Wi r ðnÞ ¼ fw 2 W jrankði; n; wÞ ¼ rg:
ð2Þ
Given n, the favourable rank weights are those that give the alternative i rank r, that is, make it the rth best. All further analysis is based on the properties of these sets. The first descriptive measure is the rth rank acceptability index for alternative i. The rank acceptability index measures the variety of different preferences that support the alternative i for rank r. Rank acceptability indices are computed as expected volumes of favourable rank weight spaces, that is, as multidimensional integrals over the criteria distributions and the favourable rank weights, by Z Z bri ¼ f ðnÞ f ðwÞ dw dn: ð3Þ X
Wir ðnÞ
In other words, the rth rank acceptability index is the proportion of all the weights (preferences) for which the alternative i obtains the rank r. The most acceptable alternatives are those with high indices for the best ranks. In particular, the first rank acceptability index b1i measures the share of all the weights (preferences) for which the alternative i is the best choice. The first rank acceptability index can thus be used for classifying alternatives (with respect to the assumed value function) into efficient ones ðb1i > 0Þ and those that are inefficient or almost inefficient (b1i is zero or
close to zero). Rank acceptability indices can be examined graphically in order to compare how large shares of the weights place each alternative on each rank. When seeking compromises, alternatives with large indices for the worst ranks should be avoided. SMAA-2 also defines the holistic acceptability index m X ar bri ; ð4Þ ahi ¼ r¼1
which forms an aggregate of the rank acceptability indices of alternative i using meta-weights a1 P a2 P P am P 0. The holistic acceptability index aims at measuring the overall acceptability of alternatives. There are many ways to choose the meta-weights [13]. Conveniently scaled holistic indices in the interval [0, 1] are obtained by setting am ¼ 0 and using m 1-dimensional centroid meta-weights for the remaining ar , scaled such that a1 ¼ 1, that is, m 1 m 1 X 1 X 1 ; r ¼ 1; . . . ; m 1: ð5Þ ar ¼ i i i¼r i¼1 The central weight vector wci is the expected centre of gravity of the favourable first rank weights of an alternative. The central weight vector is computed as an integral of the weight vector over the criteria and weight distributions by Z Z wci ¼ f ðnÞ f ðwÞw dw dn=b1i : ð6Þ X
Wi1 ðnÞ
With the assumed weight distribution, the central weight vector can be considered to represent the preferences of a typical DM who supports alternative i. The central weights can be presented to the DMs in order to help them understand what kinds of preferences correspond to different choices. The confidence factor pic is the probability that alternative i obtains the best rank if the central weight vector is chosen. The confidence factor is computed as an integral over the criteria distributions n by Z c pi ¼ f ðnÞ dn: ð7Þ n2X jrankði;n;wci Þ¼1
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
The confidence factor measures whether the criteria data are accurate enough to discern the alternatives when the central weight vector is used. The confidence factor can be calculated in a similar manner for any given weight vector. In practice, the multidimensional integrals (3), (6) and (7) are computed through Monte-Carlo simulation by generating random numbers for criteria measurements and weight vectors from their distributions and collecting statistics about the ranks that the different alternatives obtain. Note that in group decision making contexts, zero weights must be allowed because some DMs may indeed regard some criteria irrelevant. 2.3. SMAA-O The SMAA-O method is designed for problems where the weights are not precisely known and criteria information is completely or partially ordinal, that is, the experts or DMs have ranked the alternatives criterion-wise for some or all the criteria. Ordinal criteria are thus measured by assigning for each alternative a rank level number rj ¼ 1; . . . ; jmax , where 1 is the best and jmax the worst rank level. Alternatives considered equally good are placed on the same rank level and the rank levels are numbered consecutively. For example, if alternatives 1 and 2 sharing the best rank level for criterion j are followed by alternative 3, we have x1j ¼ x2j ¼ 1 and x3j ¼ 2. Therefore, shared ranks for criterion j result in jmax < m. On an ordinal scale, only the order of the scale values is significant; the scale intervals do not carry any information. In other words, ordinal scales are insensitive to monotone increasing transformations. Ordinal criteria cannot be used directly in a multicriteria value function because an arbitrary transformation of the scale would affect the value function and preference relation among the alternatives. However, we may presume that some unknown cardinal measures correspond to the known ordinal measurements. To model the ordinal criteria correctly, we consider all consistent mappings between ordinal and cardinal scales. The idea is to simulate such mappings numerically by generating random cardinal values corresponding to the known ordinal values.
121
Let cj represent the unknown cardinal values corresponding to the known rank levels rj for criterion j. The ordinal-to-cardinal mapping is thus cj ¼ vj ðrj Þ. Because lower ranks are preferred to higher ranks, we know that vj ð:Þ must be a monotone decreasing function. Without loss of generality we select a linear cardinal scale for cj in the interval [0, 1], where 1 is the best value. Observe that with this choice the sum of the lengths of the scale intervals Dcjr ¼ cj;rþ1 cjr for r ¼ 1; . . . ; jmax 1 is 1. We thus want to simulate all scales whose intervals belong to the valid scale interval space ) ( X j max 1 Cj ¼ Dcj 2 R j Dcjr > 0 and Dcjr ¼ 1 : r
ð8Þ Without any additional knowledge about the scale intervals, we use in the simulation a uniform distribution with a density function fj ðDcj Þ ¼ 1=volðCj Þ;
ð9Þ
where the scale interval space is a jmax 2-dimensional simplex with a volume volðCj Þ ¼ ðjmax 1Þ
1=2
=ðjmax 2Þ!
ð10Þ
The simulation of the cardinal scales is implemented by generating jmax 2 distinct random numbers from the uniform distribution in the interval ]0, 1[ and sorting these numbers along with 1 and 0 in a decreasing order to get 1 ¼ cj1 > cj2 > > cj;j max ¼ 0 (see Section 5.4 in [6]). The distinctness of cjr can be ensured by rejecting sets containing identical values. These numbers are then used as a sample of stochastic cardinal criteria measurements such that for each alternative i, nij is set equal to cjr , where r ¼ xij . Fig. 1 illustrates a sample mapping generated in this process. After the process described above we have stochastic cardinal criteria measurements available for the ordinal criteria. If the problem contains both ordinal and cardinal criteria, the cardinal criteria are drawn from their distributions (as in SMAA-2). The acceptability indices (3), central weight vectors (6) and confidence factors (7) are then computed exactly as in SMAA-2 based on
122
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
Fig. 1. An ordinal-to-cardinal mapping with jmax ¼ 11 rank levels.
statistics collected during the simulation. In SMAA-O, the rank acceptability index measures the variety of different preferences that support the alternative i for rank r considering the simulated cardinal-to-ordinal mappings for ordinal criteria and the uncertainty of cardinal criteria. The central weight vector represents the preferences of a typical DM who supports alternative i. The confidence factor measures whether the criteria data are able to discern the alternatives when the central weight vector is used. Low confidence factors for the best alternatives indicate that either more accurate information is needed or that the alternatives indeed are too similar to be discerned. Additional information about the ordinal scales can also be taken into account. If we know, for example, that the first interval is larger than the second interval for criterion j, we simply discard generated mappings not satisfying this condition. As stated previously, the SMAA methods can be used with any form of value function, as long as all the DMs jointly accept it. However, with SMAA-O we can relax this requirement if the value function is additive. An additive value function is of the form U ðxi ; wÞ ¼
n X j¼1
wj uj ðxij Þ;
ð11Þ
where uj ð Þ is a partial value function for criterion j mapping the cardinal criteria measurements onto partial values uj in [0, 1]. If the DMsÕ partial value functions are unknown, we can, in principle, simulate them in the same way as we did with the ordinal-to-cardinal mappings. However, such additional simulation is not necessary for ordinal criteria, because we can interpret the simulated cardinal values directly as partial values on a linear scale. Therefore, if the DMs accept an additive value function, it is not necessary for the DMs to agree on a common shape for the partial value functions for the ordinal criteria. Nor is it necessary to know these shapes for the ordinal criteria. 2.4. Handling partial preference information SMAA-O can also be used in problems where some preference information is available. In such cases the method can be used iteratively. The problem is first analysed without preference information, and again after more accurate preference information becomes available. The analysis can similarly be repeated with more accurate criteria information. When collecting preference information from the DMs in the form of weights, it is important that the interpretation of the weights is well de-
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
fined. With cardinal criteria and a linear value function the weights are interpreted simply as Ôprice-coefficientsÕ for the different criteria, or equivalently, the ratios between weights can be interpreted as trade-off ratios between criteria. With ordinal criteria, a non-linear value function must be assumed, and the interpretation of the weights is thus more complex. When an additive value function is used, the weights can be interpreted as price-coefficients for the partial values uj on a linear scale. Since only the end-points of the partial value scale correspond to fixed ordinal values (i.e., the best and the worst ranks correspond to 1 and 0, respectively), the DMs must determine their weights based on qualitative properties of the best and the worst alternatives with respect to each criterion. Let us assume, that we have preference information from D different DMs (and possibly other stakeholders) as a set of weight vectors fw1 ; . . . ; wD g, and we want to use this information in SMAA-O. The set of weights can be considered a sample from an unknown weight distribution f ðwÞ. The idea is to construct a weight distribution consistent with this sample and use that distribution in the analysis. There are many different ways to do this. If a fairly large sample of weight vectors is available, it is possible to use the sample directly as a discrete distribution. Another technique is to form a continuous weight distribution asPa convex combination of the sample weightsP w ¼ Dd¼1 bd wd D with random coefficients bd P 0, d¼1 bd ¼ 1. A third technique is to compute weight intervals max ½wmin from the sample and form a uniform j ; wj weight distribution in the restricted weight space ) ( n X 0 n min max W ¼ w 2 R jw 6 w 6 w and wj ¼ 1 : j¼1
ð12Þ The last approach is suitable also if the DMs originally specify their preferences as weight intervals instead of precise weights. 2.5. An example As an example, let us consider the problem of selecting a municipal solid waste management
123
system in the J€ams€a region in Finland (see [11,23]). The goal is to find the best way to deal with waste in the region until the year 2010. Eleven alternatives (listed in [11]) were evaluated based on eight criteria: 1. 2. 3. 4. 5. 6. 7. 8.
Net cost per ton, technical reliability, global effects, local and regional health effects, acidificative releases, surface water dispersed releases, number of employees, amount of recovered waste.
In the original application, ELECTRE III was used. Here we demonstrate the use of SMAA-O on the same problem when only ordinal evaluations of the criteria are available. First, criterion-wise rankings were constructed from the original, inaccurate criteria measurements. Such a ranking could have been formed by experts based on a qualitative analysis of the alternatives without actual measurements. When the difference between criteria measurements was smaller than the originally defined indifference threshold, the alternatives were given the same rank. This strategy resulted in a rather coarse set of rank levels. For example, on criterion 3, only ranks 1 and 2 were obtained. For criterion 5 all the alternatives were within the indifference threshold, and the criterion was thus excluded from the analysis. Table 1
Table 1 The criterion-wise ranks of the alternatives Alternative
C1
C2
C3
C4
C6
C7
C8
IA IB1 IB2 IC1 IC2 IIA IIB IIC IIIA IIIB IIIC
1 2 2 5 5 1 2 5 3 4 6
1 3 3 2 2 1 3 2 4 4 4
2 2 2 1 1 2 2 1 2 2 1
1 1 1 2 2 1 1 2 1 1 2
5 3 3 3 3 5 2 2 4 2 1
6 4 5 2 4 5 3 1 5 3 1
4 4 3 1 1 4 3 1 5 4 2
Criterion C5 is excluded.
124
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
presents the criterion-wise rank levels of the alternatives. First we analyse the problem using the SMAAO method without any preference information. We assume that the DMsÕ preferences are represented by an additive value function. Table 2 presents the holistic acceptability indices (ah ), confidence factors (pc ) and rank acceptability indices (br ) for the alternatives sorted by the holistic acceptability index. The holistic index is computed based on ðm 1Þ-dimensional centroid meta-weights (5). The efficient alternatives appear in boldface. The rank acceptability indices are visualised in Fig. 2.
To save space, we do not present central weights here. As we can see from Table 2 and Fig. 2, alternative IIC receives the highest first rank acceptability and also the highest holistic acceptability. Thus, without any knowledge about the preferences, this alternative is the most widely acceptable alternative. (IIC was also the choice in the original decision process in [11].) The other efficient alternatives are IIB, IIA and IIIC, each with a reasonably high holistic acceptability index. All the efficient alternatives except IIIC have reasonably high confidence factors. These alternatives are thus
Table 2 Descriptive measures (%) without preference information Alternative
ah
pc
b1
b2
b3
b4
b5
b6
b7
b8
b9
b10
b11
IIC IIB IC1 IIA IIIC IC2 IB2 IA IB1 IIIB IIIA
71 50 41 36 33 26 22 22 21 16 3
99 96 – 90 41 – – – – – –
54 23 0 16 7 0 0 0 0 0 0
12 7 35 6 13 0 6 14 3 4 0
8 16 19 6 11 23 4 5 6 3 0
4 12 10 10 17 19 10 5 8 6 0
3 26 6 11 5 11 10 9 11 6 1
6 8 4 9 6 7 17 10 12 18 3
4 8 8 10 4 5 16 8 19 11 7
8 0 5 9 5 9 21 9 21 11 2
1 0 9 18 10 6 10 9 15 12 9
0 0 4 5 8 11 6 21 4 27 14
0 0 0 0 12 9 0 10 0 3 65
Fig. 2. Rank acceptability indices (%) for alternatives. The alternatives are sorted by ah .
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
potentially viable choices under favourable preferences. Interestingly, the inefficient alternative IC1 receives the third largest holistic acceptability index due to high second, third and fourth rank acceptabilities. This indicates that IC1 is potentially a good compromise alternative. IIIC is the most controversial alternative obtaining fairly large indices both for the best and the worst ranks. IIIC is thus not a suitable compromise alternative. Next, we introduce preference information from the 45 DMs to the SMAA-O analysis [11] and carry out the analysis in a restricted weight space, see (12). The weight intervals are presented in Table 3 and the new holistic acceptability indices (ah ), confidence factors (pc ) and rank acceptability indices (br ) in Table 4. In this application, the introduction of partial preference information changes the holistic acceptability indices and the resulting order only moderately. The main observation is that the holistic acceptability of alternative IIA increases significantly. The holistic acceptability of IIIC decreases and the alternative becomes almost inefficient (weakly efficient) with only 1% first rank acceptability. Furthermore, the confidence factor
125
of IIIC is reduced to only 7%, indicating that this alternative is very unlikely to be the most preferred one, even if all the DMs, in consensus, would state that the central weight vector of IIIC represents their preferences. The confidence factor of the most acceptable alternative IIC was very high (99% vs. 94%), indicating that the ordinal criteria data are accurate enough to discern between the alternatives provided that sufficient consensus around the central weight vector is reached. There exists no right solution to this waste management problem against which the goodness of the methods used could be evaluated. However, a large amount of planning costs could have been saved using SMAA-O if the experts had ranked the alternatives according to certain criteria based on qualitative information only. In the J€ams€a application C2 (technical feasibility), C3 (global effects) and C4 (health effects) were examples of criteria that are difficult to measure precisely. Thus, they could have been treated as ordinal and the other criteria as cardinal. 3. Discussion
Table 3 Weight intervals from 45 DMsÕ weights (%) Criterion
C1
C2
C3
C4
C6
C7
C8
wmin wmax
4.8 47.6
3.6 47.6
0.1 14.7
2.0 19.7
4.5 25.0
2.2 26.3
4.8 38.1
It is obvious that if cardinal criteria data is available for each criterion, approaches like SMAA-O are not needed. The strength of SMAAO is that the problem can be analysed in the case when such information is not available for at least some of the criteria. In real-life applications, one
Table 4 Descriptive measures (%) using weight intervals Alternative
ah
pc
b1
b2
b3
b4
b5
b6
b7
b8
b9
b10
b11
IIC IIB IIA IC1 IA IC2 IB2 IIIC IB1 IIIB IIIA
73 54 46 42 29 25 24 19 19 9 1
94 66 72 – – – – 7 – – –
53 24 22 0 0 0 0 1 0 0 0
11 12 10 35 18 0 6 5 3 0 0
11 18 9 14 9 21 5 7 4 1 0
6 16 14 14 8 12 11 10 7 2 0
4 13 10 9 14 15 14 8 12 3 0
10 12 10 5 9 10 15 8 12 7 1
3 5 10 13 10 7 18 8 17 8 1
1 0 7 6 11 15 17 8 22 11 1
0 0 8 4 9 9 8 20 17 21 4
0 0 0 1 12 8 5 15 6 43 11
0 0 0 0 1 3 0 11 0 4 82
126
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
often has to balance between costly cardinal measurements and the possible decline in the quality of the analysis implied by the ordinal data. The quality of the ordinal analysis may decrease significantly in cases where the experts are not able to discriminate the alternatives with respect to some criteria. Without any preference information, SMAA-O (and the other SMAA variants) may end up in a situation where all the alternatives are efficient and have reasonably high holistic acceptability indices. The conclusion then is that all the alternatives are acceptable from some point of view, and the power of the method is limited to providing information about what kind of preferences favour each alternative. For best results in real-life applications, we suggest using SMAA-O iteratively. Initial analyses can be performed using ordinal criteria and without any preference information in order to identify clearly inferior and possibly superior alternatives. The acceptability indices and confidence factors can be used to assess whether more detailed analyses are required. Furthermore, SMAA-O helps to identify which alternatives and criteria need to be measured and analysed more accurately. Such an iterative process based on refining the criteria and preference information gradually may help to avoid unnecessarily precise and expensive measurements. In some cases it is possible to define additional constraints for the ordinal intervals. In practice, it is sometimes possible to determine, for example, that for some criterion the difference between the ranks 1 and 2 is smaller or greater than the difference between the ranks 2 and 3. If this information does not help in breaking the ties and no additional information about the preferences of the DMs is available, the DMs must base their decision on the descriptive information produced so far. The interpretation of weights is different in outranking methods and value function based methods. It is therefore theoretically incorrect to use the same weights in the sample SMAA-O analysis as in the original ELECTRE III analysis. However, when expressing preference information, the DMs should understand the underlying deci-
sion model and the semantics of the preference information in the model. Unfortunately, this is not always the case. Non-expert DMs, in particular, tend to express very similar weights regardless of what the underlying decision model is. Furthermore, recent results [14] show that the weights of pseudo-criteria models and utility function models correlate significantly.
4. Conclusions We have presented the new SMAA-O method for aiding discrete multicriteria decision-making problems. Like the previous variants of SMAA, SMAA-O is intended to assist multicriteria decision making with multiple DMs in situations where little or no preference information is available and criteria measurements are not precisely known. While the previous variants of SMAA require cardinally measured criteria, SMAA-O can be used in situations where some or all of the criteria are measured on ordinal scales. SMAA-O computes several measures for what kind of preferences would make each alternative preferred. The computations are performed using stochastic simulation of the unknown weights and the unknown ordinal-to-cardinal mappings. The advantage of SMAA-O over the previous variants is that the planning costs can in many cases be significantly decreased when criteria that are difficult to measure on cardinal scales are evaluated by experts on ordinal scales based on qualitative information. SMAA-O can also be used iteratively with increasingly accurate preference and criteria information until the analysis indicates that a satisfactory accuracy for decisionmaking has been reached.
Acknowledgements This research was supported in part by the Technology Development Centre of Finland. The research of Kaisa Miettinen was supported by the Academy of Finland, grant no. 65760. Pekka Salminen acted as a senior fellow of the Academy of Finland while this research was undertaken.
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
References [1] J.-P. Ancot, J.H.P. Paelinck, Recent experiences with the qualiflex multicriteria method, in: J.H.P. Paelinck (Ed.), Qualitative and Quantitative Mathematical Economics, Martinus Nijhoff Publishers, The Hague, 1982, pp. 217– 266. [2] C.A. Bana e Costa, A multicriteria decision aid methodology to deal with conflicting situations on the weights, European Journal of Operational Research 26 (1986) 22–34. [3] C.A. Bana e Costa, A methodology for sensitivity analysis in three-criteria problems: A case study in municipal management, European Journal of Operational Research 33 (1988) 159–173. [4] J.R. Charnetski, The multiple attribute problem with partial information: The expected value and comparative hypervolume methods, Ph.D. Thesis, University of Texas at Austin, 1973. [5] J.R. Charnetski, R.M. Soland, Multiple-attribute decision making with partial information: The comparative hypervolume criterion, Naval Research Logistics Quarterly 25 (1978) 279–288. [6] H.A. David, Order Statistics, Wiley, New York, 1970. [7] H.A. Eiselt, G. Laporte, The use of domains in multicriteria decision making, European Journal of Operational Research 61 (1992) 292–298. [8] J. Hokkanen, R. Lahdelma, K. Miettinen, P. Salminen, Determining the implementation order of a general plan by multicriteria method, Journal of Multi-Criteria Decision Analysis 7 (1998) 273–284. [9] J. Hokkanen, R. Lahdelma, P. Salminen, A multiple criteria decision model for analyzing and choosing among different development patterns for the Helsinki cargo harbour, Socio-Economic Planning Sciences 33 (1999) 1–23. [10] J. Hokkanen, R. Lahdelma, P. Salminen, Locating a waste treatment facility by using stochastic multicriteria acceptability analysis with ordinal criteria, European Journal of Operational Research (to appear). [11] J. Hokkanen, P. Salminen, The choice of a solid waste management system by using the ELECTRE III decisionaid method, in: M. Paruccini (Ed.), Applying Multiple Criteria Aid for Decision to Environmental Management, Kluwer Academic Publishers, Dordrecht, 1994, pp. 111– 153. [12] R. Lahdelma, J. Hokkanen, P. Salminen, SMAA – stochastic multiobjective acceptability analysis, European Journal of Operational Research 106 (1998) 137–143. [13] R. Lahdelma, P. Salminen, SMAA-2: Stochastic multicriteria acceptability analysis for group decision making, Operations Research 49 (2001) 444–454.
127
[14] R. Lahdelma, P. Salminen, Pseudo-criteria versus linear utility function in stochastic multicriteria acceptability analysis, European Journal of Operational Research (to appear). [15] R. Lahdelma, P. Salminen, J. Hokkanen, Combining stochastic multiobjective acceptability analysis and DEA, in: D.K. Despotis, C. Zopounidis (Eds.), Integrating Technology and Human Decisions: Global Bridges into the 21st Century, New Technologies Publications, Athens, Greece, 1999, pp. 629–632. [16] R. Lahdelma, P. Salminen, J. Hokkanen, Using multicriteria methods in environmental planning and management, Environmental Management 26 (2000) 595–605. [17] L.Y. Maystre, J. Picted, J. Simos, Methodes Multicriteres ELECTRE, Presses Polytechniques et Universitaires Romandes, Lausanne, Switzerland, 1994. [18] K. Miettinen, P. Salminen, Decision-aid for discrete multiple criteria decision making problems with imprecise data, European Journal of Operational Research 119 (1999) 50–60. [19] K. Miettinen, P. Salminen, R. Lahdelma, Treating ordinal criteria in stochastic weight space analysis, in: M. K€ oksalan, S. Zionts (Eds.), Multiple Criteria Decision Making in the New Millennium, Proceedings of the Fifteenth International Conference on Multiple Criteria Decision Making (MCDM), Springer, Berlin, Heidelberg, 2001, pp. 285– 293. [20] J.H.P. Paelink, Qualitative multiple criteria analysis, environmental protection and multiregional development, Papers of the Regional Science Association 36 (2001) 59–74. [21] P. Rietveld, Multiple Objective Decision Methods and Regional Planning, North Holland, Amsterdam, 1980. [22] P. Rietveld, H. Ouwersloot, Ordinal data in multicriteria decision making, a stochastic dominance approach to siting nuclear power plants, European Journal of Operational Research 56 (1992) 249–262. [23] P. Salminen, J. Hokkanen, R. Lahdelma, Multicriteria decision analysis project on environmental problems, Report 5/1996, University of Jyv€askyl€a, Department of Mathematics, Laboratory of Scientific Computing, 1996. [24] J. Sarkis, A comparative analysis of DEA as a discrete alternative multiple criteria decision tool, European Journal of Operational Research 123 (2000) 543–557. [25] P.J. Schoemaker, C.C. Waid, An experimental comparison of different approaches to determining weights in additive value models, Management Science 28 (1982) 182– 196. [26] M. Weber, K. Borcherding, Behavioral influences on weight judgments in multiattribute decision making, European Journal of Operational Research 67 (1993) 1–12.
Decision Aiding
Ordinal criteria in stochastic multicriteria acceptability analysis (SMAA) Risto Lahdelma a, Kaisa Miettinen b
b,*
, Pekka Salminen
c
a Department of Information Technology, University of Turku, Lemmink€aisenkatu 14 A, FIN-20520 Turku, Finland Department of Mathematical Information Technology, University of Jyv€askyl€a, P.O. Box 35 (Agora), FIN-40351 Jyv€askyl€a, Finland c Department of Economics, University of Joensuu, P.O. Box 111, FIN-80101 Joensuu, Finland
Received 22 December 2000; accepted 8 March 2002
Abstract We suggest a method for providing descriptive information about the acceptability of decision alternatives in discrete co-operative group decision-making problems. The new SMAA-O method is a variant of the stochastic multicriteria acceptability analysis (SMAA). SMAA-O is designed for problems where criteria information for some or all criteria is ordinal; that is, experts (or decision-makers) have ranked the alternatives according to each (ordinal) criterion. Considerable savings can be obtained if rank information for some or all the criteria is sufficient for making decisions without significant loss of quality. The approach is particularly useful for group decision making when the group can agree on the use of an additive decision model but only partial preference information, or none at all, is available. 2002 Elsevier Science B.V. All rights reserved. Keywords: Multiobjective; Decision analysis; Decision support systems; Multiple criteria decision-making; Group decision-making; Ordinal data
1. Introduction In many real-life decision-making problems, it is very time consuming and expensive to produce accurate measurements for decision criteria. In fact, even after a costly process, the criteria measurements often still remain very uncertain or inaccurate. Much less work is normally required for *
Corresponding author. Tel.: +358-14-260-2743; fax: +35814-260-2771. E-mail addresses: [email protected].fi (R. Lahdelma), [email protected].fi (K. Miettinen), pekka.salminen@joensuu.fi (P. Salminen).
ranking the alternatives criterion-wise. Experts can often produce rankings for some criteria based on qualitative information about the alternatives. Considerable savings in planning costs can be obtained if rank information for some or all the criteria is sufficient for making decisions without significant loss of quality. Another aspect of real-life decision-making is related to the difficulty of obtaining and using preference information from the decision-makers (DMs). For these reasons, we introduce the SMAA-O method capable of treating mixed cardinal and ordinal criteria data and missing or imprecise preference information.
0377-2217/03/$ - see front matter 2002 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 - 2 2 1 7 ( 0 2 ) 0 0 2 6 7 - 9
118
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
The stochastic multicriteria acceptability analysis (SMAA) methods provide descriptive information about the acceptability of decision alternatives in discrete co-operative group decision-making problems. SMAA-methods are designed for situations where the criteria measurements are uncertain or inaccurate and it is for some reason impossible to obtain accurate preference information from the DMs. The SMAA methods are based on exploring the weight space in order to describe the preferences that would make each alternative the most preferred one, or that would give a certain rank for a specified alternative. Related research can be found in the literature. Charnetski [4] and Charnetski and Soland [5] introduced the comparative hypervolume criterion, based on computing for each alternative the volume of the multidimensional weight space that makes the alternative the most preferred. This method can handle preference information in the form of linear constraints for the weights, but is restricted to cardinal and deterministic criteria measurements and an additive utility function. Rietveld [21] and Rietveld and Ouwersloot [22] presented same types of methods for problems with ordinal criteria. In [21], ordinal criteria measurements are suggested to be transformed into cardinal values using alternative mappings whereas the availability of ordinal information about weights is assumed in [22]. The qualiflex method [1,20] approaches similar problems by testing how each possible ranking of alternatives is supported by different criteria in a similar setting to [22]. Bana e Costa [2,3] introduced the overall compromise criterion method for cardinal data for identifying alternatives generating the least conflict between several DMs. This method can handle partial preference information in the form of arbitrary weight distributions. Eiselt and Laporte [7] ranked alternatives using domains in problems with cardinal deterministic criteria. They computed the set of all weight vectors that make a particular alternative the best one using a linear additive value function. Data envelopment analysis (DEA) has also been suggested to be used in multiple criteria decision making problems (see, e.g., [24]). The SMAA method was initially developed based on the overall compromise criterion method.
The motivation for developing methods for problems where preference information (e.g., in the form of weights) is not available is that this situation is common in many real-life decision problems with multiple DMs. Preference information may be difficult to obtain for several reasons: • The DMs do not have time to study the problem carefully enough. • The analysts do not have time or resources to reveal the preferences of a large group of DMs. • The DMs have difficulties in comparing criteria. • The DMs are afraid of revealing their preferences in public. • The DMs do not want to fix their preferences because the preferences may change during the process. Even if preference information (e.g., weights) can be obtained from the DMs, it is unclear how the preferences of several DMs that disagree should be combined. Furthermore, because different weight elicitation methods have been observed to provide different weights for the same problem (see e.g. [25,26]), any subjective weight information should be considered uncertain. For these reasons, SMAA-type inverse methods can often be an appropriate approach in real-life decision problems, see [10,16]. Instead of trying to identify a single best alternative based on precise preference information, SMAA identifies the preferences that make each alternative the best one. Different SMAA variants are based on different decision models. SMAA [9,12] and SMAA-2 [13] assume the existence of a general utility or value function containing weighting coefficients that represent the importance of the criteria (e.g., an additive value function). SMAA-D [15] uses output/input efficiency measures of the DEA type. The DEA efficiency measure can, of course, be considered a special case of a general value function. SMAA, SMAA-2, and SMAA-D model the uncertainty or inaccuracy of criteria data by stochastic distributions. SMAA-3 [8] uses a double threshold model (pseudo-criteria) to model the uncertainty or inaccuracy of data and applies then an outranking procedure involving weighting
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
coefficients for criteria. This model is adopted from the ELECTRE III method (see, e.g., [17] and also [18], the latter being based on non-differentiable multicriteria optimisation). The information provided by the SMAA methods is descriptive. The DMs are given acceptability indices describing the variety of different preferences that support an alternative for the best rank or any particular rank. This information can be used for classifying the alternatives into more or less acceptable ones and those that are not acceptable at all, identifying good compromise candidates, and evaluating the overall acceptability of alternatives using the so-called holistic acceptability indices. SMAA also computes central weights describing the most typical preferences making an alternative the most preferred one. It is also possible to measure with confidence factors whether the problem information is accurate enough for decision-making. In other words, instead of giving exact solutions to the DMs, the SMAA methods aim at describing what kind of preferences correspond to different choices. Because different interest groups usually have different preferences in real-life decision problems, an alternative with a large acceptability index will more likely appeal to a greater number of interest groups than such an alternative that can receive the best rank only with a small variety of preferences. The earlier versions of SMAA are applicable only if the criteria are measured on cardinal scales. Treating ordinal ranks as cardinal data and then using this information in arithmetic computations could lead to a serious distortion. In this paper we introduce the SMAA-O method, which is a new variant of SMAA for problems where some or all of the criteria are measured on ordinal scales. This means that for ordinal criteria we only know which alternative is the best, the second best, and so on, but no absolute measurements are available. Ordinal criteria occur in many real-life problems where the DMs or experts are not able or willing to consider more accurate measurements. The development of SMAA-O is based on the SMAA-2 method [13]. The preliminary ideas of SMAA-O were presented in [19]. In SMAA-O, the ordinalto-cardinal transformation is not forced to be some specific mapping, no information about the
119
weights or preferences in general is assumed, and in addition to volumes of acceptabilities, also other descriptive measures are calculated. Due to the usage of Monte-Carlo simulation, no limits for the number of criteria and/or alternatives are set.
2. Description of SMAA-O 2.1. The problem A discrete multicriteria decision-making problem consists of a set of m alternatives measured in terms of n criteria. Each alternative is thus represented by a row vector xi ¼ ½xi1 ; xi2 ; . . . ; xin . Depending on the information available, the vector consists of deterministic or stochastic criteria measurements, or ordinal (rank) information. In case of deterministic criteria measurements, we assume that the DMsÕ preference structure is represented by a real-valued value function U ðxi ; wÞ involving weighting coefficients w. The value function may have any form jointly accepted by the DMs, but linear or additive forms are often appropriate. The DMsÕ subjective preferences are represented by weight vectors w ¼ ½w1 ; w2 ; . . . ; wn . We assume that the weight vector is non-negative and normalised, that is ) ( n X n w 2 W ¼ w 2 R jwj P 0 and wj ¼ 1 : ð1Þ j¼1
If a single weight vector could be commonly agreed on, the best alternative could be identified by evaluating the value function for each alternative and choosing the alternative i with the largest value U ðxi ; wÞ P U ðxk ; wÞ for k ¼ 1; . . . ; m. The value function could then also be used for ranking the alternatives. However, for the reasons discussed earlier, such a single weight vector may not be available. Therefore, a decision method is needed that can be used without precise weights. 2.2. The SMAA-2 method The SMAA-2 method was developed particularly for situations where neither criteria
120
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
measurements nor weights are precisely known. Uncertain or imprecise criteria measurements are represented by stochastic variables nij (instead of xij ) with an assumed or estimated joint probability distribution and density function f ðnÞ in the space X Rm n . Similarly, the DMsÕ unknown or partially known preferences are represented by a weight distribution with density function f ðwÞ in the weight space W. Total lack of preference information is represented by a uniform weight distribution in W, that is, f ðwÞ ¼ 1=volðW Þ. The value U ðni ; wÞ of each alternative is then a stochastic quantity, and consequently the rank of each alternative also depends on the stochastic criteria measurements and weights. The alternatives are analysed in terms of subsets of the weight space W. Stochastic sets of favourable rank weights are defined as Wi r ðnÞ ¼ fw 2 W jrankði; n; wÞ ¼ rg:
ð2Þ
Given n, the favourable rank weights are those that give the alternative i rank r, that is, make it the rth best. All further analysis is based on the properties of these sets. The first descriptive measure is the rth rank acceptability index for alternative i. The rank acceptability index measures the variety of different preferences that support the alternative i for rank r. Rank acceptability indices are computed as expected volumes of favourable rank weight spaces, that is, as multidimensional integrals over the criteria distributions and the favourable rank weights, by Z Z bri ¼ f ðnÞ f ðwÞ dw dn: ð3Þ X
Wir ðnÞ
In other words, the rth rank acceptability index is the proportion of all the weights (preferences) for which the alternative i obtains the rank r. The most acceptable alternatives are those with high indices for the best ranks. In particular, the first rank acceptability index b1i measures the share of all the weights (preferences) for which the alternative i is the best choice. The first rank acceptability index can thus be used for classifying alternatives (with respect to the assumed value function) into efficient ones ðb1i > 0Þ and those that are inefficient or almost inefficient (b1i is zero or
close to zero). Rank acceptability indices can be examined graphically in order to compare how large shares of the weights place each alternative on each rank. When seeking compromises, alternatives with large indices for the worst ranks should be avoided. SMAA-2 also defines the holistic acceptability index m X ar bri ; ð4Þ ahi ¼ r¼1
which forms an aggregate of the rank acceptability indices of alternative i using meta-weights a1 P a2 P P am P 0. The holistic acceptability index aims at measuring the overall acceptability of alternatives. There are many ways to choose the meta-weights [13]. Conveniently scaled holistic indices in the interval [0, 1] are obtained by setting am ¼ 0 and using m 1-dimensional centroid meta-weights for the remaining ar , scaled such that a1 ¼ 1, that is, m 1 m 1 X 1 X 1 ; r ¼ 1; . . . ; m 1: ð5Þ ar ¼ i i i¼r i¼1 The central weight vector wci is the expected centre of gravity of the favourable first rank weights of an alternative. The central weight vector is computed as an integral of the weight vector over the criteria and weight distributions by Z Z wci ¼ f ðnÞ f ðwÞw dw dn=b1i : ð6Þ X
Wi1 ðnÞ
With the assumed weight distribution, the central weight vector can be considered to represent the preferences of a typical DM who supports alternative i. The central weights can be presented to the DMs in order to help them understand what kinds of preferences correspond to different choices. The confidence factor pic is the probability that alternative i obtains the best rank if the central weight vector is chosen. The confidence factor is computed as an integral over the criteria distributions n by Z c pi ¼ f ðnÞ dn: ð7Þ n2X jrankði;n;wci Þ¼1
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
The confidence factor measures whether the criteria data are accurate enough to discern the alternatives when the central weight vector is used. The confidence factor can be calculated in a similar manner for any given weight vector. In practice, the multidimensional integrals (3), (6) and (7) are computed through Monte-Carlo simulation by generating random numbers for criteria measurements and weight vectors from their distributions and collecting statistics about the ranks that the different alternatives obtain. Note that in group decision making contexts, zero weights must be allowed because some DMs may indeed regard some criteria irrelevant. 2.3. SMAA-O The SMAA-O method is designed for problems where the weights are not precisely known and criteria information is completely or partially ordinal, that is, the experts or DMs have ranked the alternatives criterion-wise for some or all the criteria. Ordinal criteria are thus measured by assigning for each alternative a rank level number rj ¼ 1; . . . ; jmax , where 1 is the best and jmax the worst rank level. Alternatives considered equally good are placed on the same rank level and the rank levels are numbered consecutively. For example, if alternatives 1 and 2 sharing the best rank level for criterion j are followed by alternative 3, we have x1j ¼ x2j ¼ 1 and x3j ¼ 2. Therefore, shared ranks for criterion j result in jmax < m. On an ordinal scale, only the order of the scale values is significant; the scale intervals do not carry any information. In other words, ordinal scales are insensitive to monotone increasing transformations. Ordinal criteria cannot be used directly in a multicriteria value function because an arbitrary transformation of the scale would affect the value function and preference relation among the alternatives. However, we may presume that some unknown cardinal measures correspond to the known ordinal measurements. To model the ordinal criteria correctly, we consider all consistent mappings between ordinal and cardinal scales. The idea is to simulate such mappings numerically by generating random cardinal values corresponding to the known ordinal values.
121
Let cj represent the unknown cardinal values corresponding to the known rank levels rj for criterion j. The ordinal-to-cardinal mapping is thus cj ¼ vj ðrj Þ. Because lower ranks are preferred to higher ranks, we know that vj ð:Þ must be a monotone decreasing function. Without loss of generality we select a linear cardinal scale for cj in the interval [0, 1], where 1 is the best value. Observe that with this choice the sum of the lengths of the scale intervals Dcjr ¼ cj;rþ1 cjr for r ¼ 1; . . . ; jmax 1 is 1. We thus want to simulate all scales whose intervals belong to the valid scale interval space ) ( X j max 1 Cj ¼ Dcj 2 R j Dcjr > 0 and Dcjr ¼ 1 : r
ð8Þ Without any additional knowledge about the scale intervals, we use in the simulation a uniform distribution with a density function fj ðDcj Þ ¼ 1=volðCj Þ;
ð9Þ
where the scale interval space is a jmax 2-dimensional simplex with a volume volðCj Þ ¼ ðjmax 1Þ
1=2
=ðjmax 2Þ!
ð10Þ
The simulation of the cardinal scales is implemented by generating jmax 2 distinct random numbers from the uniform distribution in the interval ]0, 1[ and sorting these numbers along with 1 and 0 in a decreasing order to get 1 ¼ cj1 > cj2 > > cj;j max ¼ 0 (see Section 5.4 in [6]). The distinctness of cjr can be ensured by rejecting sets containing identical values. These numbers are then used as a sample of stochastic cardinal criteria measurements such that for each alternative i, nij is set equal to cjr , where r ¼ xij . Fig. 1 illustrates a sample mapping generated in this process. After the process described above we have stochastic cardinal criteria measurements available for the ordinal criteria. If the problem contains both ordinal and cardinal criteria, the cardinal criteria are drawn from their distributions (as in SMAA-2). The acceptability indices (3), central weight vectors (6) and confidence factors (7) are then computed exactly as in SMAA-2 based on
122
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
Fig. 1. An ordinal-to-cardinal mapping with jmax ¼ 11 rank levels.
statistics collected during the simulation. In SMAA-O, the rank acceptability index measures the variety of different preferences that support the alternative i for rank r considering the simulated cardinal-to-ordinal mappings for ordinal criteria and the uncertainty of cardinal criteria. The central weight vector represents the preferences of a typical DM who supports alternative i. The confidence factor measures whether the criteria data are able to discern the alternatives when the central weight vector is used. Low confidence factors for the best alternatives indicate that either more accurate information is needed or that the alternatives indeed are too similar to be discerned. Additional information about the ordinal scales can also be taken into account. If we know, for example, that the first interval is larger than the second interval for criterion j, we simply discard generated mappings not satisfying this condition. As stated previously, the SMAA methods can be used with any form of value function, as long as all the DMs jointly accept it. However, with SMAA-O we can relax this requirement if the value function is additive. An additive value function is of the form U ðxi ; wÞ ¼
n X j¼1
wj uj ðxij Þ;
ð11Þ
where uj ð Þ is a partial value function for criterion j mapping the cardinal criteria measurements onto partial values uj in [0, 1]. If the DMsÕ partial value functions are unknown, we can, in principle, simulate them in the same way as we did with the ordinal-to-cardinal mappings. However, such additional simulation is not necessary for ordinal criteria, because we can interpret the simulated cardinal values directly as partial values on a linear scale. Therefore, if the DMs accept an additive value function, it is not necessary for the DMs to agree on a common shape for the partial value functions for the ordinal criteria. Nor is it necessary to know these shapes for the ordinal criteria. 2.4. Handling partial preference information SMAA-O can also be used in problems where some preference information is available. In such cases the method can be used iteratively. The problem is first analysed without preference information, and again after more accurate preference information becomes available. The analysis can similarly be repeated with more accurate criteria information. When collecting preference information from the DMs in the form of weights, it is important that the interpretation of the weights is well de-
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
fined. With cardinal criteria and a linear value function the weights are interpreted simply as Ôprice-coefficientsÕ for the different criteria, or equivalently, the ratios between weights can be interpreted as trade-off ratios between criteria. With ordinal criteria, a non-linear value function must be assumed, and the interpretation of the weights is thus more complex. When an additive value function is used, the weights can be interpreted as price-coefficients for the partial values uj on a linear scale. Since only the end-points of the partial value scale correspond to fixed ordinal values (i.e., the best and the worst ranks correspond to 1 and 0, respectively), the DMs must determine their weights based on qualitative properties of the best and the worst alternatives with respect to each criterion. Let us assume, that we have preference information from D different DMs (and possibly other stakeholders) as a set of weight vectors fw1 ; . . . ; wD g, and we want to use this information in SMAA-O. The set of weights can be considered a sample from an unknown weight distribution f ðwÞ. The idea is to construct a weight distribution consistent with this sample and use that distribution in the analysis. There are many different ways to do this. If a fairly large sample of weight vectors is available, it is possible to use the sample directly as a discrete distribution. Another technique is to form a continuous weight distribution asPa convex combination of the sample weightsP w ¼ Dd¼1 bd wd D with random coefficients bd P 0, d¼1 bd ¼ 1. A third technique is to compute weight intervals max ½wmin from the sample and form a uniform j ; wj weight distribution in the restricted weight space ) ( n X 0 n min max W ¼ w 2 R jw 6 w 6 w and wj ¼ 1 : j¼1
ð12Þ The last approach is suitable also if the DMs originally specify their preferences as weight intervals instead of precise weights. 2.5. An example As an example, let us consider the problem of selecting a municipal solid waste management
123
system in the J€ams€a region in Finland (see [11,23]). The goal is to find the best way to deal with waste in the region until the year 2010. Eleven alternatives (listed in [11]) were evaluated based on eight criteria: 1. 2. 3. 4. 5. 6. 7. 8.
Net cost per ton, technical reliability, global effects, local and regional health effects, acidificative releases, surface water dispersed releases, number of employees, amount of recovered waste.
In the original application, ELECTRE III was used. Here we demonstrate the use of SMAA-O on the same problem when only ordinal evaluations of the criteria are available. First, criterion-wise rankings were constructed from the original, inaccurate criteria measurements. Such a ranking could have been formed by experts based on a qualitative analysis of the alternatives without actual measurements. When the difference between criteria measurements was smaller than the originally defined indifference threshold, the alternatives were given the same rank. This strategy resulted in a rather coarse set of rank levels. For example, on criterion 3, only ranks 1 and 2 were obtained. For criterion 5 all the alternatives were within the indifference threshold, and the criterion was thus excluded from the analysis. Table 1
Table 1 The criterion-wise ranks of the alternatives Alternative
C1
C2
C3
C4
C6
C7
C8
IA IB1 IB2 IC1 IC2 IIA IIB IIC IIIA IIIB IIIC
1 2 2 5 5 1 2 5 3 4 6
1 3 3 2 2 1 3 2 4 4 4
2 2 2 1 1 2 2 1 2 2 1
1 1 1 2 2 1 1 2 1 1 2
5 3 3 3 3 5 2 2 4 2 1
6 4 5 2 4 5 3 1 5 3 1
4 4 3 1 1 4 3 1 5 4 2
Criterion C5 is excluded.
124
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
presents the criterion-wise rank levels of the alternatives. First we analyse the problem using the SMAAO method without any preference information. We assume that the DMsÕ preferences are represented by an additive value function. Table 2 presents the holistic acceptability indices (ah ), confidence factors (pc ) and rank acceptability indices (br ) for the alternatives sorted by the holistic acceptability index. The holistic index is computed based on ðm 1Þ-dimensional centroid meta-weights (5). The efficient alternatives appear in boldface. The rank acceptability indices are visualised in Fig. 2.
To save space, we do not present central weights here. As we can see from Table 2 and Fig. 2, alternative IIC receives the highest first rank acceptability and also the highest holistic acceptability. Thus, without any knowledge about the preferences, this alternative is the most widely acceptable alternative. (IIC was also the choice in the original decision process in [11].) The other efficient alternatives are IIB, IIA and IIIC, each with a reasonably high holistic acceptability index. All the efficient alternatives except IIIC have reasonably high confidence factors. These alternatives are thus
Table 2 Descriptive measures (%) without preference information Alternative
ah
pc
b1
b2
b3
b4
b5
b6
b7
b8
b9
b10
b11
IIC IIB IC1 IIA IIIC IC2 IB2 IA IB1 IIIB IIIA
71 50 41 36 33 26 22 22 21 16 3
99 96 – 90 41 – – – – – –
54 23 0 16 7 0 0 0 0 0 0
12 7 35 6 13 0 6 14 3 4 0
8 16 19 6 11 23 4 5 6 3 0
4 12 10 10 17 19 10 5 8 6 0
3 26 6 11 5 11 10 9 11 6 1
6 8 4 9 6 7 17 10 12 18 3
4 8 8 10 4 5 16 8 19 11 7
8 0 5 9 5 9 21 9 21 11 2
1 0 9 18 10 6 10 9 15 12 9
0 0 4 5 8 11 6 21 4 27 14
0 0 0 0 12 9 0 10 0 3 65
Fig. 2. Rank acceptability indices (%) for alternatives. The alternatives are sorted by ah .
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
potentially viable choices under favourable preferences. Interestingly, the inefficient alternative IC1 receives the third largest holistic acceptability index due to high second, third and fourth rank acceptabilities. This indicates that IC1 is potentially a good compromise alternative. IIIC is the most controversial alternative obtaining fairly large indices both for the best and the worst ranks. IIIC is thus not a suitable compromise alternative. Next, we introduce preference information from the 45 DMs to the SMAA-O analysis [11] and carry out the analysis in a restricted weight space, see (12). The weight intervals are presented in Table 3 and the new holistic acceptability indices (ah ), confidence factors (pc ) and rank acceptability indices (br ) in Table 4. In this application, the introduction of partial preference information changes the holistic acceptability indices and the resulting order only moderately. The main observation is that the holistic acceptability of alternative IIA increases significantly. The holistic acceptability of IIIC decreases and the alternative becomes almost inefficient (weakly efficient) with only 1% first rank acceptability. Furthermore, the confidence factor
125
of IIIC is reduced to only 7%, indicating that this alternative is very unlikely to be the most preferred one, even if all the DMs, in consensus, would state that the central weight vector of IIIC represents their preferences. The confidence factor of the most acceptable alternative IIC was very high (99% vs. 94%), indicating that the ordinal criteria data are accurate enough to discern between the alternatives provided that sufficient consensus around the central weight vector is reached. There exists no right solution to this waste management problem against which the goodness of the methods used could be evaluated. However, a large amount of planning costs could have been saved using SMAA-O if the experts had ranked the alternatives according to certain criteria based on qualitative information only. In the J€ams€a application C2 (technical feasibility), C3 (global effects) and C4 (health effects) were examples of criteria that are difficult to measure precisely. Thus, they could have been treated as ordinal and the other criteria as cardinal. 3. Discussion
Table 3 Weight intervals from 45 DMsÕ weights (%) Criterion
C1
C2
C3
C4
C6
C7
C8
wmin wmax
4.8 47.6
3.6 47.6
0.1 14.7
2.0 19.7
4.5 25.0
2.2 26.3
4.8 38.1
It is obvious that if cardinal criteria data is available for each criterion, approaches like SMAA-O are not needed. The strength of SMAAO is that the problem can be analysed in the case when such information is not available for at least some of the criteria. In real-life applications, one
Table 4 Descriptive measures (%) using weight intervals Alternative
ah
pc
b1
b2
b3
b4
b5
b6
b7
b8
b9
b10
b11
IIC IIB IIA IC1 IA IC2 IB2 IIIC IB1 IIIB IIIA
73 54 46 42 29 25 24 19 19 9 1
94 66 72 – – – – 7 – – –
53 24 22 0 0 0 0 1 0 0 0
11 12 10 35 18 0 6 5 3 0 0
11 18 9 14 9 21 5 7 4 1 0
6 16 14 14 8 12 11 10 7 2 0
4 13 10 9 14 15 14 8 12 3 0
10 12 10 5 9 10 15 8 12 7 1
3 5 10 13 10 7 18 8 17 8 1
1 0 7 6 11 15 17 8 22 11 1
0 0 8 4 9 9 8 20 17 21 4
0 0 0 1 12 8 5 15 6 43 11
0 0 0 0 1 3 0 11 0 4 82
126
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
often has to balance between costly cardinal measurements and the possible decline in the quality of the analysis implied by the ordinal data. The quality of the ordinal analysis may decrease significantly in cases where the experts are not able to discriminate the alternatives with respect to some criteria. Without any preference information, SMAA-O (and the other SMAA variants) may end up in a situation where all the alternatives are efficient and have reasonably high holistic acceptability indices. The conclusion then is that all the alternatives are acceptable from some point of view, and the power of the method is limited to providing information about what kind of preferences favour each alternative. For best results in real-life applications, we suggest using SMAA-O iteratively. Initial analyses can be performed using ordinal criteria and without any preference information in order to identify clearly inferior and possibly superior alternatives. The acceptability indices and confidence factors can be used to assess whether more detailed analyses are required. Furthermore, SMAA-O helps to identify which alternatives and criteria need to be measured and analysed more accurately. Such an iterative process based on refining the criteria and preference information gradually may help to avoid unnecessarily precise and expensive measurements. In some cases it is possible to define additional constraints for the ordinal intervals. In practice, it is sometimes possible to determine, for example, that for some criterion the difference between the ranks 1 and 2 is smaller or greater than the difference between the ranks 2 and 3. If this information does not help in breaking the ties and no additional information about the preferences of the DMs is available, the DMs must base their decision on the descriptive information produced so far. The interpretation of weights is different in outranking methods and value function based methods. It is therefore theoretically incorrect to use the same weights in the sample SMAA-O analysis as in the original ELECTRE III analysis. However, when expressing preference information, the DMs should understand the underlying deci-
sion model and the semantics of the preference information in the model. Unfortunately, this is not always the case. Non-expert DMs, in particular, tend to express very similar weights regardless of what the underlying decision model is. Furthermore, recent results [14] show that the weights of pseudo-criteria models and utility function models correlate significantly.
4. Conclusions We have presented the new SMAA-O method for aiding discrete multicriteria decision-making problems. Like the previous variants of SMAA, SMAA-O is intended to assist multicriteria decision making with multiple DMs in situations where little or no preference information is available and criteria measurements are not precisely known. While the previous variants of SMAA require cardinally measured criteria, SMAA-O can be used in situations where some or all of the criteria are measured on ordinal scales. SMAA-O computes several measures for what kind of preferences would make each alternative preferred. The computations are performed using stochastic simulation of the unknown weights and the unknown ordinal-to-cardinal mappings. The advantage of SMAA-O over the previous variants is that the planning costs can in many cases be significantly decreased when criteria that are difficult to measure on cardinal scales are evaluated by experts on ordinal scales based on qualitative information. SMAA-O can also be used iteratively with increasingly accurate preference and criteria information until the analysis indicates that a satisfactory accuracy for decisionmaking has been reached.
Acknowledgements This research was supported in part by the Technology Development Centre of Finland. The research of Kaisa Miettinen was supported by the Academy of Finland, grant no. 65760. Pekka Salminen acted as a senior fellow of the Academy of Finland while this research was undertaken.
R. Lahdelma et al. / European Journal of Operational Research 147 (2003) 117–127
References [1] J.-P. Ancot, J.H.P. Paelinck, Recent experiences with the qualiflex multicriteria method, in: J.H.P. Paelinck (Ed.), Qualitative and Quantitative Mathematical Economics, Martinus Nijhoff Publishers, The Hague, 1982, pp. 217– 266. [2] C.A. Bana e Costa, A multicriteria decision aid methodology to deal with conflicting situations on the weights, European Journal of Operational Research 26 (1986) 22–34. [3] C.A. Bana e Costa, A methodology for sensitivity analysis in three-criteria problems: A case study in municipal management, European Journal of Operational Research 33 (1988) 159–173. [4] J.R. Charnetski, The multiple attribute problem with partial information: The expected value and comparative hypervolume methods, Ph.D. Thesis, University of Texas at Austin, 1973. [5] J.R. Charnetski, R.M. Soland, Multiple-attribute decision making with partial information: The comparative hypervolume criterion, Naval Research Logistics Quarterly 25 (1978) 279–288. [6] H.A. David, Order Statistics, Wiley, New York, 1970. [7] H.A. Eiselt, G. Laporte, The use of domains in multicriteria decision making, European Journal of Operational Research 61 (1992) 292–298. [8] J. Hokkanen, R. Lahdelma, K. Miettinen, P. Salminen, Determining the implementation order of a general plan by multicriteria method, Journal of Multi-Criteria Decision Analysis 7 (1998) 273–284. [9] J. Hokkanen, R. Lahdelma, P. Salminen, A multiple criteria decision model for analyzing and choosing among different development patterns for the Helsinki cargo harbour, Socio-Economic Planning Sciences 33 (1999) 1–23. [10] J. Hokkanen, R. Lahdelma, P. Salminen, Locating a waste treatment facility by using stochastic multicriteria acceptability analysis with ordinal criteria, European Journal of Operational Research (to appear). [11] J. Hokkanen, P. Salminen, The choice of a solid waste management system by using the ELECTRE III decisionaid method, in: M. Paruccini (Ed.), Applying Multiple Criteria Aid for Decision to Environmental Management, Kluwer Academic Publishers, Dordrecht, 1994, pp. 111– 153. [12] R. Lahdelma, J. Hokkanen, P. Salminen, SMAA – stochastic multiobjective acceptability analysis, European Journal of Operational Research 106 (1998) 137–143. [13] R. Lahdelma, P. Salminen, SMAA-2: Stochastic multicriteria acceptability analysis for group decision making, Operations Research 49 (2001) 444–454.
127
[14] R. Lahdelma, P. Salminen, Pseudo-criteria versus linear utility function in stochastic multicriteria acceptability analysis, European Journal of Operational Research (to appear). [15] R. Lahdelma, P. Salminen, J. Hokkanen, Combining stochastic multiobjective acceptability analysis and DEA, in: D.K. Despotis, C. Zopounidis (Eds.), Integrating Technology and Human Decisions: Global Bridges into the 21st Century, New Technologies Publications, Athens, Greece, 1999, pp. 629–632. [16] R. Lahdelma, P. Salminen, J. Hokkanen, Using multicriteria methods in environmental planning and management, Environmental Management 26 (2000) 595–605. [17] L.Y. Maystre, J. Picted, J. Simos, Methodes Multicriteres ELECTRE, Presses Polytechniques et Universitaires Romandes, Lausanne, Switzerland, 1994. [18] K. Miettinen, P. Salminen, Decision-aid for discrete multiple criteria decision making problems with imprecise data, European Journal of Operational Research 119 (1999) 50–60. [19] K. Miettinen, P. Salminen, R. Lahdelma, Treating ordinal criteria in stochastic weight space analysis, in: M. K€ oksalan, S. Zionts (Eds.), Multiple Criteria Decision Making in the New Millennium, Proceedings of the Fifteenth International Conference on Multiple Criteria Decision Making (MCDM), Springer, Berlin, Heidelberg, 2001, pp. 285– 293. [20] J.H.P. Paelink, Qualitative multiple criteria analysis, environmental protection and multiregional development, Papers of the Regional Science Association 36 (2001) 59–74. [21] P. Rietveld, Multiple Objective Decision Methods and Regional Planning, North Holland, Amsterdam, 1980. [22] P. Rietveld, H. Ouwersloot, Ordinal data in multicriteria decision making, a stochastic dominance approach to siting nuclear power plants, European Journal of Operational Research 56 (1992) 249–262. [23] P. Salminen, J. Hokkanen, R. Lahdelma, Multicriteria decision analysis project on environmental problems, Report 5/1996, University of Jyv€askyl€a, Department of Mathematics, Laboratory of Scientific Computing, 1996. [24] J. Sarkis, A comparative analysis of DEA as a discrete alternative multiple criteria decision tool, European Journal of Operational Research 123 (2000) 543–557. [25] P.J. Schoemaker, C.C. Waid, An experimental comparison of different approaches to determining weights in additive value models, Management Science 28 (1982) 182– 196. [26] M. Weber, K. Borcherding, Behavioral influences on weight judgments in multiattribute decision making, European Journal of Operational Research 67 (1993) 1–12.