- Email: [email protected]
Chapter 17 Choosing the decision-making method
Chapter 17 Choosing the decision-making method FC, FP and RSS When there are m conflicting indicators, even in the case in which there is only one Decision Maker (DM), in order to determine the best compromise alternative it is first necessary to evaluate the alternatives. This evaluation can be executed with different methods, called decision-making methods, which provide a procedure for coming to a decision in a transparent and repeatable way. The choice of the decision-making method to be adopted is critical, because it can influence the final decision. In this chapter we will compare several of the most commonly used decision-making methods: Multi-Attribute Value Theory (MAVT); Analytic Hierarchy Process (AHP); and the family of ELECTRE methods. In our presentation we will assume that the system is not affected by random disturbances, that the number of alternatives considered is finite and that the effects that each one of them produces are known. We will begin our analysis by considering what it means to ‘choose’ an alternative and which kind of characteristics the decision-making method and the DM must have. Then, we will describe the three methods one by one; we will give some general indications about how to choose among them, and we will look at the difficulties that arise when the number of alternatives to be compared is infinite and/or the system is affected by random disturbances. We will then identify the method most suited for our context and conclude that it is the MAVT.
17.1
Rankings and ordinal scales
Assume that the set A = |A0, . . . , AnA | of the nA alternatives from which the DM must choose and the effects that each one produces are known. These latter are expressed through the vector i of the indicators. Assume for the moment that the system is not affected by random disturbances and thus these vectors are deterministic; we will remove this hypothesis in the last section. On the basis of this information the DM must select the alternative that she prefers. To do so, she must rank the alternatives in set A according to decreasing values of her satisfaction: the first alternative in the order is the one chosen. Thus, choosing means ranking, and thus how to create the ranking is the object of our attention. A ranking can be expressed in various ways, according to various scales (or paradigms), and the most simple and direct one is the ordinal scale. According to this scale a ranking only expresses the 379
380
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Figure 17.1: Satisfaction associated with the surface area of a wetland. DM ’s1
preferences about the alternatives. For example, given the alternatives A1, A2, A3 and A4, a (ordinal) ranking is expressed in the following form {A4, A1 ∼ A3, A2}
(17.1)
This states that A4 is preferred over A1 (in the following we denote this fact with A4 A1), A1 is equivalent (∼) to A3, which is preferred over A2. This is clearly sufficient to conclude that the chosen alternative is A4. However, it is not easy for the DM to establish such an order, since the information that she has about each alternative is the corresponding vector of indicators. She must therefore compare these vectors, which is not a simple task for two reasons. The first reason is that the value of an indicator quantifies an effect in physical units and this is quite different from the level of satisfaction that the DM attributes to that effect: in fact, her satisfaction is not always proportional to the value that the indicator assumes. Consider for example a project in which the canalization of a river channel reduces the surface area of a wetland, which is potentially malarial, where migratory birds nest and mosquitoes reproduce. A good indicator of the contraction of the wetlands is its surface area. The environmentalists’ satisfaction is not proportional to the area since, if its value is too high, there could be an annoying and dangerous swarm of mosquitoes; their satisfaction is not inversely proportional to the area either, because reducing it too much would pose a risk to the survival of the migratory birds. To conclude, their satisfaction is a bell-shaped function of the area (Figure 17.1), because the maximum satisfaction is obtained with intermediate values. It is thus necessary to find a way to associate a value of satisfaction to each value of the indicator. Once this has been achieved, it is possible to rank the alternatives with respect to the satisfaction that the indicator produces: in this way one obtains a partial ranking for each indicator. Now the second difficulty arises: the partial rankings for the different indicators are generally different, in fact, the first alternative is rarely the same in all of them, because the evaluation criteria are usually in conflict. The task of the DM is thus to reduce this plurality of partial rankings, by defining, on the basis of these, a global ranking, in which the alternatives are listed from the best to the worst according to the ‘global satisfaction’ that each one produces. To define this satisfaction, value judgements come into play, which 1 As already said, in this chapter we assume that there is just one Decision Maker and that the Stakeholders do not have a decision role. The person that conducts the evaluation is therefore the DM. The considerations that are developed, however, are also applicable to the case in which the evaluation is conducted by several people at the same time, for example a group of DMs or Stakeholders. In Chapter 21 we will explain how to recompose their independent evaluations in one ranking.
17.1 R ANKINGS AND ORDINAL SCALES
381
are, by nature, subjective. A transparent decision thus requires that these subjective aspects be well highlighted and kept separate from the objective aspects (the effects produced). So the ordering rule, i.e. the procedure with which the DM passes from the partial rankings to the global ranking, must be explicitly formulated.
17.1.1
Arrow’s Theorem
An ordering rule does not only require an explicit formulation: it must also be democratic, because only in this case will it be accepted by the Stakeholders. But what does ‘democratic’ mean? An interesting definition of ‘democratic’ comes from Arrow,2 according to whom an ordering rule is democratic if, in non-trivial cases, i.e. when there are at least three alternatives to be ordered, it satisfies the following conditions: (1) unanimity: if in all the partial rankings A1 is preferred over A2 (A1 A2), then in the global ranking there is A1 A2; (2) non-imposition (or citizen sovereignty): the global ranking depends exclusively on the partial rankings; (3) non-dictatorship: the global ranking does not always coincide with one of the single partial rankings while ignoring all the others; (4) independence from irrelevant alternatives: this condition is valid when the alternatives are designed, evaluated and compared in two or more subsequent phases. Suppose that in the first phase alternatives A1 and A2 are generated, and that the global ranking derived from the rule is the following: {A1, A2}. If in the second phase alternative A3 is generated, the global ranking of the three alternatives must be such that the preference between A1 and A2 remains unvaried: the global ranking could be {A1, A3, A2}, or {A1, A2, A3}, or {A3, A1, A2}, but it cannot be {A2, A3, A1}. This condition is important because it prevents alternatives from being inserted in the set A with the sole aim of manipulating the order of the others. Different rules have been proposed for generating a global ranking from the partial rankings when they are expressed with ordinal scales. The most famous rules were elaborated in the second half of the 1700s by Borda (1781) and Condorcet (1785). The simplest is the rule of simple majority, which is the one upon which most electoral systems are based, and which we now illustrate to give an idea of how such rules can be formulated. The rule of simple majority establishes that, given a set of alternatives, the relationship between a pair of alternatives in the global ranking is equivalent to the one they have in the simple majority of the m partial rankings. For example, consider a decision problem with three alternatives A1, A2 and A3 and three indicators relative to three evaluation criteria: flooding, irrigation and hydropower production. Suppose that, on the basis of values of the three corresponding indicators, the DM has expressed the following three partial rankings: • flooding: {A1, A3, A2}; • irrigation: {A2, A1, A3}; • hydropower production: {A2, A3, A1}. 2 Nobel Prize for Economics in 1971.
382
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
We note that • A1 A3 in the simple majority of the partial rankings (flooding and irrigation) and thus in a global ranking based on the rule of simple majority A1 A3; • A2 A1 in the simple majority of the partial rankings (irrigation and hydropower production) and thus A2 A1 also in the global ranking; • A2 A3 in the simple majority of the partial rankings (irrigation and hydropower production), and thus A2 A3. The global ranking will therefore be the following: {A2, A1, A3} In order to accept the rule of simple majority we must make sure that it is democratic, but this is useless, since in 1951 Arrow identified the following paradox, today known as Arrow’s Paradox (Arrow, 1951), which shows that in some cases the rule of simple majority fails to produce a meaningful global ranking and therefore it is not an acceptable ordering rule. Let us suppose that in the previous example the DM expresses the following partial rankings: • flooding: {A1, A2, A3}; • irrigation: {A2, A3, A1}; • hydropower production: {A3, A1, A2}. By aggregating these orders with the simple-majority rule, the following relations are obtained: • A1 A2 in two of the three partial rankings (flooding and hydropower production): thus A1 A2 in the global ranking; • A2 A3 in two of the three partial rankings (flooding and irrigation): thus A2 A3; • A3 A1 in two of the three partial rankings (irrigation and hydropower production): thus A3 A1 from which the following global ranking results A1 A2 A3 A1 Clearly this ranking is paradoxical, because it does not respect the property of transitivity.3 One might think that the problem is in the simple-majority rule and that a different ordering rule should be found; but this conjecture must also be abandoned, since in 1963 Arrow proved that when the ordinal scale is adopted, it is not possible to obtain a global ranking from the partial rankings by using democratic ordering rules, i.e., that an ordering rule that respects all four conditions on page 381 cannot be found when the ordinal scale is adopted (Impossibility Theorem) (Arrow, 1963). 3 For a formal definition of this term see Section 17.2.
17.1 R ANKINGS AND ORDINAL SCALES
17.1.2
383
Absolute and interval scales
In the previous section we have seen that, when looking for a democratic (in the Arrow sense) ordering rule, ordinal scales are inadequate. Therefore, it is necessary to refer to scales that express not only qualitative preference judgements but also the intensity of the preference (cardinal scales), as, for example, the absolute scale. These scales generate cardinal rankings that can be expressed with the following notation {18, 15, 18, 21}
(17.2)
where the first number is the value attributed to alternative A1, the second to A2, and so on. Note that from the cardinal rankings an ordinal ranking can always be extracted; for example, from the cardinal ranking (17.2) it is possible to extract the ordinal ranking defined by (17.1). The adoption of an absolute scale requires us to fix the unit of measurement of the values and the zero, which represents total lack of value; this is not a simple task, since these elements are very subjective. Therefore, often an interval scale is preferred, i.e. a scale in which preference intensities are not related to the absolute values, but to the differences in value between the alternatives. On an interval scale, the value can be defined in an infinite number of equivalent ways, each related to the other by a linear transformation with positive coefficients. For example, by adopting an interval scale, the ranking (17.2) is equivalent to the following {1810, 1510, 1810, 2110}
(17.3)
which is obtained by applying the transformation i = 100i + 10 to (17.2); but it is also equivalent to the following {2.8, 2.5, 2.8, 3.1}
(17.4)
which is obtained by applying the transformation i = 0.1i + 1 and so on. For instance, temperature scales are interval scales: they associate different values to the two physical states of water freezing and water boiling, 0 and 100 in the Celsius scale (◦ C), and 32 and 212 in the Fahrenheit scale (◦ F). These values are linked by the following relation F = 1.8C + 32 Zero degrees on the Celsius scale corresponds to 32 degrees on the Fahrenheit scale, but both measure the same effect: the freezing temperature of water. One could apply any other monotonic transformation with positive coefficients to the values of temperature indicators to produce an infinite number of other temperature scales, all of which would be equivalent. In conclusion, by adopting an interval scale, one can accept that a value is a subjective quantity, without it influencing the course of the decision-making process. This is why interval scales are very suitable for expressing intensity of preference. The adoption of an interval scale also offers practical advantages, as we will see in Section 20.4 when describing more deeply the MAVT method, which adopts an interval scale.
384
17.2
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Preference axioms
In the previous section we analysed what to make a choice means and we said that choosing means ranking; then, we introduced several ways in which ranking can be expressed. Now, we will consider how to make a choice. The aim of the evaluation, in fact, is to formalize the logical procedure through which the DM makes a choice. In this way the decision is made transparent, in the sense that the objective and subjective motivations at the basis of the decision are made explicit and repeatable. This is the equivalent of identifying the so called preference structure of the DM (we will return to this subject in Section 20.2). However, a DM does not necessarily have a preference structure: if, for example, she decides randomly, she would not have one. Therefore we must firstly establish under which conditions this structure exists. The majority of authors hold that the existence of preference structure requires the satisfaction of the following axioms: Axiom 1 (Completeness): Given two alternatives A1 and A2, the DM is always able to say which of the two she prefers, and by how much, or whether they are equivalent to each other. Axiom 2 (Transitivity): Given three alternatives A1, A2 and A3, if the DM prefers A1 to A2 and A2 to A3, then she must necessarily prefer A1 to A3, i.e. A1 A2
and
A2 A3
⇒
A1 A3
(17.5)
Axiom 3 (Independence from irrelevant alternatives): Given a set A of alternatives, the order of preference that the DM establishes between any two of them does not vary if A is enlarged with a set A of new alternatives. We have already encountered this axiom among the conditions required for an ordering rule to be democratic (see page 381). Axiom 2 adopts ordinal expressions and it is thus suitable when an ordinal scale is adopted. Therefore, when working with the interval scale, it must be substituted by the following axiom, which is its cardinal form: Axiom 2b (Consistency): Given three alternatives A1, A2 and A3, if the DM affirms that A1 is m times more preferable to A2 and A2 is n times more preferable to A3, then she must necessarily affirm that A1 is m · n times more preferable to A3, i.e. A1 ∼ m A2
and
A2 ∼ n A3
⇒
A1 ∼ m · n A3
with m, n > 0
(17.6)
Evaluation methods based on the hypothesis that the DM satisfies the preference axioms listed above are called normative methods (or prescriptive methods) (French, 1988). On the other hand, descriptive methods do not require a priori that the DM’s preference structure satisfies any given axioms. Instead, they try to follow the DM’s own way of reasoning. The price of the malleability of these methods is a loss of mathematical rigour; the advantage is a more realistic representation of the decision-making process. The choice between the two methods is left to the Analyst, who must assess whether a rigorous mathematical approach would be helpful to organize the information provided by the DM, or would be a strong constriction on her way of reasoning.
17.3 M ULTI -ATTRIBUTE VALUE T HEORY
385
Table 17.1. The four forms of problem Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
Form 1 Form 3
Form 2 Form 4
The evaluation method should be chosen not only on the basis of the DM’s needs: it should also take into account the characteristics of the problem that is being tackled. Certain methods can in fact be used only with problems of a given form and not with others. The characteristics that define the form of a problem are the nature of the set (either discretefinite or continuous) of the alternatives to be considered and the nature of the indicators (either quantitative or quantitative–qualitative) used to estimate the effects. The four forms of problem that derive from these characteristics are classified in Table 17.1. Note that a real problem can be formalized in different forms, according to how one decides to formulate it (e.g. by adopting only quantitative indicators or not). Now we will illustrate three decision-making methods: Multi-Attribute Value Theory, Analytic Hierarchy Process and ELECTRE methods. We chose these three both because they are the most widely employed, and because together they cover the first three forms of problem in Table 17.1, which are the only ones that can be solved with the knowledge we have today. Our description will not be exhaustive, but it will simply aim at providing the elements necessary for a comparison of the three methods. Since, as we will see in the following, Multi-Attribute Value Theory is the most suitable method for the problems we deal with, Chapter 20 will be dedicated to a more detailed description of it.
17.3
Multi-Attribute Value Theory
Multi-Attribute Value Theory (MAVT) (Keeney and Raiffa, 1976) is based on the identification of a value function V (·). This function, the argument of which is the vector of the indicators that quantify the effects of an alternative, returns a single value that expresses the DM ’s satisfaction for the alternative. Increasing values of V correspond to increasing levels of satisfaction. Once the value function has been identified, it is possible to order any set of alternatives. The value V is thus the Project Index that we introduced in Section 3.4. The MAVT can be applied to Form 1, 2 and 3 problems (see Table 17.2). Since it is extremely difficult to identify the value function directly, the problem is led back to the definition of a partial value function for each one of the indicators. This is possible only if the DM’s choices satisfy the axioms of Completeness, Consistency and Independence from irrelevant alternatives. The MAVT is thus a normative method. The (global) Table 17.2. The forms of problem to which the MAVT can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
386
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Figure 17.2: Steps in identifying the value function, given the values i1 , . . . , im of the indicators, according to the MAVT.
value function is then obtained through the composition of the partial value functions, generally through a weighted sum (see Figure 17.2). The values of the coefficient (weights) in the sum depend on the relative importance that the DM associates to each of the indicators. Notice that, by computing the value function as a weighted sum of the partial value functions, one implicitly assumes that a poor performance of one of the indicators can be balanced by good performances of others. In other words one implicitly assumes that Compensation is allowed among the criteria. The MAVT thus requires that compensation exists among all the indicators (with the exception of one at the most, Keeney and Raiffa, 1976). In order to guarantee coherence among the values provided by the partial value functions and the weights, it is important that the latter are estimated, with the procedure that we will define in Section 20.6, on the basis of the values that each indicator can assume and not of abstract considerations, far from the specific context examined. Also the definition of the partial value functions requires a series of complex operations, in which the DM has to express her evaluation of the effects of an alternative in a formalized way. Any attempt at simplifying these operations by adopting quick hypotheses should be absolutely avoided: for example, by hypothesizing a priori, i.e. without due verification, that the partial value functions are linear. By doing thus in fact, the results are seriously invalidated. We will return to this discussion in Chapter 20.
17.4
Analytic Hierarchy Process
Just as in the MAVT, in the Analytic Hierarchy Process (AHP) (Saaty, 1980, 1992) the performance of an alternative is obtained through the weighted sum of the performances that it attains with respect to the single indicators. Therefore, compensation must be admitted among the criteria. However, the procedure used to define the weights and the performances, given the effects of each alternative, is very different from the one used in the MAVT: it follows that the types of problems to which it can be applied are not the same (see Table 17.3). Once the hierarchy of criteria (Chapter 3) has been defined, the AHP requires that all the alternatives are appended to each one of the leaf criteria. For example, Figure 17.3 shows the hierarchy of the Upstream Tourism sector in the Verbano Project (see Chapter 4 of PRACTICE) in the form required by the AHP. The evaluation is then performed in two steps: pairwise comparison and hierarchical recomposition.
17.4 A NALYTIC H IERARCHY P ROCESS
387
Table 17.3. The types of problem to which the AHP can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
First, the DM is required to compare, for each criterion in the hierarchy, all the elements that depend on it at the immediately lower level. For example, when the leaf criterion Reduced access to beaches is considered in the hierarchy of Figure 17.3, the elements of the lower level are the alternatives; when the sector criterion Loss of activity due to vacating tourists is considered, the elements of the lower level are the three criteria Reduced landscape aesthetics, Reduced access to beaches, Discomfort produced by mosquitoes; etc. The comparison is aimed at identifying a ranking vector for each criterion, whose ith component quantifies how much the ith element of the lower level satisfies that criterion. The ranking vector can thus be used to rank the lower-level elements with respect to the criterion being considered. The identification of the ranking vector for each criterion is performed by asking the DM to make pairwise comparisons of the elements at the lower level, filling in a square matrix (called pairwise comparisons matrix), whose elements aij express the intensity of preference for the element in the ith row compared to the element in the j th column, with respect to the criterion under examination. The ranking vector is then extracted from this matrix by means of a suitable algorithm. Note that, depending on the level in the hierarchy, the pairwise comparisons matrix expresses preferences either among alternatives or criteria. In the first case, the ith component of the resulting ranking vector measures how much the ith alternative satisfies a given criterion; in the latter, it measures the relative importance of the ith criterion with respect to the upper level criterion, i.e. the ranking vector is a vector of weights. It is important to underline that, when filling in the pairwise comparison matrix, the DM might not satisfy the axiom of Consistency: the AHP, in fact, allows the ranking vectors to be derived even when there are slight4 inconsistencies. Unlike the MAVT, the AHP is therefore
Figure 17.3: The hierarchy for the Upstream Tourism sector of the Verbano Project, in the case when three alternatives were to be compared. 4 An inconsistency is slight when it does not cause a rank-reversal between two alternatives. For example, if the DM
says that A1 ∼ 2 · A2 and that A2 ∼ 2 · A3, it is not necessary that she affirm that A1 ∼ 4 · A3 (as the axiom
388
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
a descriptive method. It requires only that the condition of Reciprocity be satisfied, i.e. that aj i = 1/aij , a condition that is so obvious that it is almost always satisfied. Now, once a pairwise comparison matrix has been filled in for every criterion in the hierarchy, it is possible to derive the ranking vectors of the alternatives with respect to each leaf criterion, and the ranking vectors of the criteria with respect to each upper-level criterion. Through a sequence of matrix products a final ranking vector of the alternatives with respect to the root criterion of the hierarchy is obtained; this operation is called hierarchical recomposition. Note that deriving the final ranking of the alternatives as a linear combination of the alternatives’ ranking vectors with respect to the single leaf criteria, implies assuming compensation among the criteria. With respect to the MAVT, the AHP makes the interaction with the DM simpler, because she is no longer asked to identify functions that express how her degree of satisfaction varies with the indicator values, but only to express her own preferences through a sequence of pairwise comparisons. The questions that are posed are always the same, regardless of the elements in the hierarchy that are taken into consideration: “Do you prefer criterion (alternative) X or criterion (alternative) Y , with respect to criterion Z? And what’s the intensity of your preference?” Answers to such questions are usually expressed through Saaty’s verbal scale, which allows nine possible judgments, ranging from “very weak” to “very strong” preferences, which are then transposed into numbers between 1 and 9. However, simplifying the interaction with the DM has three disadvantages: (1) the number of questions to be posed increases: the DM must express her preference between all possible pairs of elements at every level of the hierarchy; (2) the complexity of the method is increased from a mathematical point of view; (3) there is the risk that the questions posed to the DM will be poorly formulated. Let us analyse these issues one at a time. 1. The number of questions increases significantly as the number of alternatives and criteria increases. For example, consider a hierarchy with only two levels, n alternatives and m criteria: the number of questions that must be posed is given by the following relation5 n · (n − 1) m · (m − 1) + (17.7) 2 2 It is clear that, as n and m increase, it rapidly becomes impossible to pose all the questions. # questions = m ·
2. The increase in mathematical complexity would not of itself be a problem but for the fact that it introduces degrees of freedom into the methodology which can be used by the Analyst but not by the DM (for example the choice of the algorithm used to obtain the scoring vector from the pairwise comparisons, in the event that the matrix provided by the DM is not consistent). of Consistency would have done), but only that she judge A1 to be better than A3. This is the same as requiring that the axiom of Transitivity be satisfied. 5 To derive it, remember that for the pairwise comparison matrices the condition of Reciprocity must be fulfilled and that the elements along the diagonal do not need to be provided by the DM, because they are equal to 1 by definition.
17.5 ELECTRE METHODS
389
Table 17.4. The types of problem to which the ELECTRE methods can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
3. Questions to the DM are poorly formulated because, when comparing the criteria, the DM does not refer to the range of values actually assumed by the indicators: she expresses her judgements in the abstract. To provide correct responses, instead, it would be necessary to consider the range of possible indicator values, since the importance of criterion X with respect to criterion Y might depend on the value of the indicator that measures the satisfaction of X. To clarify this point, consider again the hierarchy in Figure 17.3. If the indicator associated to the criterion Reduced landscape aesthetics takes on values between ‘unacceptable’ and ‘sufficient’, while the one associated to the criterion Discomfort produced by mosquitoes assumes values between ‘good’ and ‘optimal’, one naturally tends to give the first more importance, because it is the more worrying one. However, the judgement could be reversed if the alternatives would produce very positive effects for the first indicator and negative ones for the second. Finally, note that unlike the MAVT, which foresees defining value functions that can be later on be used to order any set of alternatives, with the AHP only the ranking of the alternatives that were directly compared by the DM is obtained. This means that if a new alternative is introduced, in order to know its position in the ranking it would be necessary to ask the DM to compare it with all the alternatives that have already been considered. Furthermore, since the ranking provided by the AHP depends on the alternatives being considered, the method does not satisfy the axiom of Independence from irrelevant alternatives. Therefore, the introduction of a new alternative could change the order of the other ones. This effect could be exploited to manipulate the order itself: alternatives could be introduced with the specific aim of sabotaging a given alternative in the final ranking.
17.5
ELECTRE methods
The ELECTRE methods (ELimination Et Choix Traduisant la REalité) (Roy, 1978, 1996) are only applicable to problems with a discrete set of alternatives (see Table 17.4). The literature speaks of ELECTRE methods because a number of methods have been developed from the same founding idea. They all intend to be as close as possible to the actual decision-making process, which means that they strive not to introduce any elements that oblige the DM to follow a line of reasoning that is not her own. ELECTRE methods are thus descriptive methods. As in the AHP, Consistency of preferences is not required; moreover, ELECTRE methods also admits Incomparability among the alternatives and Lack of transitivity in the rankings. As far as incomparability is concerned, note that both the MAVT and the AHP assume that an alternative’s bad performance with respect to a specific criterion can be compensated for by good performances with respect to the other criteria, whatever the gravity of the bad performance. It follows that it is always possible to compare two alternatives and identify which is preferable: this is the very reason why these two
390
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
methods satisfy the axiom of Completeness. On the contrary, in the ELECTRE methods it is possible for the DM to establish thresholds over which compensation among criteria is no longer possible: a circumstance that can often occur in reality. For example, think of a comparison between two cars, a utility passenger car and a limousine. The latter is for sure better than the former for many criteria, such as comfort, velocity, and perhaps also aesthetics. However, since it is much more expensive than the utility passenger car, it is not possible to express a clean preference for it. In other words, better performances on nearly all the criteria cannot compensate for the bad performance on the price criterion. In this situation, the DM is not able to express her preference between the two alternatives, which the ELECTRE methods would classify as incomparable among each other. As far as transitivity is concerned with, the ELECTRE methods acknowledge that, being human, the DM has limited discrimination capacity. This means that, when the difference between two indicator values is lower than a certain discrimination threshold, the DM does not perceive those values as being different. The values of the discrimination thresholds, which differ from one indicator to another, must obviously be defined by the DM. For example, when the DM has to compare alternatives from the point of view of supplying water to an irrigation district, she might not perceive the difference between supplies that differ by less than 2 m3 /s, while the difference could become appreciable for values of 5 m3 /s and substantial over 10 m3 /s. However, allowing for limited discrimination capacity may lead to intransitivity, as in the case of the “coffee paradox”. Think of two cups of coffee, of which one is bitter, the other is sweetened with one sugar grain only: since her discrimination capacity is limited, the DM is not able to express a preference among them and judges them as indistinguishable. Considering the sweetened cup and comparing it with a cup containing two grains of sugar, again the DM judges them undistinguishable. Thus, for transitivity, the bitter cup is judged undistinguishable from the sweetened with two grains cup. Keeping in comparing two cups which only differ for one sugar grain, the DM ends for comparing a completely saturated of sugar cup with a nearly saturated of sugar cup, and, again, judges them as undistinguishable. Applying transitivity to the whole chain of comparisons, the bitter cup results undistinguishable from the saturated of sugar cup. But this makes no sense, since they taste completely different. This paradox shows that accepting limited discrimination capacity implies the ranking might lack in transitivity. From these considerations it follows that the ranking of the alternatives obtained with an ELECTRE method might be incomplete and/or non-transitive.6 This means that, given an alternative, it might be impossible classifying all the other alternatives as better or worse than it, since some could be non-comparable and/or indistinguishable. In these conditions, it might be only possible understanding whether between two alternatives there is an outranking relationship, i.e. if one of them is clearly preferable to the other (Roy, 1991). If such a relationship exists, then the ranking reflects it by putting the outranking alternative in a better position than the one that is outranked (such as A2 with respect to A4 in Figure 17.4a). In order that an alternative outrank another, it is necessary that the reasons in its favour (or at least those that do not oppose it) be sufficiently strong with respect to the ones to the contrary. If an outranking relationship does not exist between two alternatives, they are either indistinguishable or not-comparable (for example A2 and A1 in Figure 17.4b). The ELECTRE methods all share this framework. Some of them allow to identify only a core set of alternatives, which are judged incomparable or indistinguishable among each 6 A ranking is said to be complete when all the alternatives being considered have a place in it and each position in the order is occupied by only one alternative (Figure 17.4); it is said to be transitive when it satisfies the axiom of Transitivity.
17.6 C HOICE OF THE METHOD
391
Figure 17.4: Comparison between a complete and transitive ranking (a) and a partial one (b). In the first, A4 is worse than A3, which in turn is worse than A2, and so on. In the second, A3 and A4 are worse than A1 and A2, and these latter are indistinguishable; nothing can be said about the relation between A3 and A4, which are not-comparable.
other. The so-called ELECTRE III method provides also a ranking of the alternatives, though partial. It requires the DM to provide a set of discrimination thresholds, one for each criterion considered, and a vector of weights for the different criteria. With this information, the ranking of the alternatives is produced through an automatic procedure, which we cannot describe here, but which the reader may find in Roy (1996). We can thus conclude that: 1. Since the outranking relationships are derived for pairs of alternatives, the ranking obtained with the ELECTRE methods depends on the set of alternatives being considered. Just as in the AHP, Independence from irrelevant alternatives is not guaranteed. 2. The number of questions for the DM does not depend on the number of alternatives, but only on the number of criteria (both the thresholds and the weights are defined for the criteria). The introduction of a new alternative thus does not lead to new questions. 3. Even if the method was born with the aim of designing a procedure that was as close as possible to the real decision-making process, the algorithm for extracting the alternatives’s ranking contains elements that are difficult for the DM to manage and understand.
17.6
Choice of the method
To facilitate the comparison of the three methods that have been illustrated, Table 17.5 lists, for each one, the preference axioms that it presupposes, while Table 17.6 lists the methods that are applicable for each type of problem. The choice of the method must take into account which axioms are satisfied by the preference structures of the actors involved (DM(s) and/or Stakeholders), and the characteristics of the problem. Before adopting a method it is necessary to check if the axioms that it presupposes are verified in the context examined. ELECTRE methods do not pose any conditions for the DM ’s preference structure (see Table 17.5). The AHP requires that the axiom of Completeness be satisfied; the MAVT also calls for Consistency and Independence from irrelevant
392
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD Table 17.5. The axioms required by each method
Completeness Transitivity Consistency Independence
MAVT
AHP
ELECTRE
Table 17.6. Forms of problem and solution methods Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite
MAVT AHP ELECTRE
MAVT AHP ELECTRE
Continuous
MAVT
alternatives. Both the MAVT and the AHP require that there can be compensation among the criteria and that the alternatives be comparable. The validity of these axioms can be checked by questioning the actors and analysing their responses. If the Project involves multiple DMs, none of the methods listed above can be applied as such, because they were all derived in the context of a problem with only one DM. As we will explain in Chapter 21, however, some of the tools developed in the MAVT can be used as a starting point for negotiations. When the Project foresees the identification of the alternatives in several steps, both the AHP and the ELECTRE methods are not very suitable, because the results that they provide depend on the set of alternatives considered. Often, in fact, to identify the whole set of alternatives, one goes through several steps, in each of which a subset of alternatives is designed, evaluated and compared (for an example see Chapters 11–14 of PRACTICE). The information that emerges during a comparison is used to identify the alternatives to be generated in the next step, for example, by identifying the mitigation actions for the most disfavoured sectors. In order that this procedure be concluded successfully, it is necessary that an alternative that is judged to be preferable to another in a given step continue to be so in the following steps; this is guaranteed only if the evaluation is performed with the MAVT method. If, instead, the AHP or the ELECTRE methods were used, at each new generation the evaluation would have to be repeated ex-novo with all the alternatives, not just those from the last generation; further, adding new alternatives could modify the order of those generated in the previous step, because the Independence from irrelevant alternatives is not guaranteed. When there is a large number of alternatives to be evaluated, the AHP, although applicable in theory, cannot be applied in practice, because the number of questions that must be posed to the DM grows with the square of the number nA of alternatives. A high number of alternatives does not constitute, however, a limit for the applicability of ELECTRE methods, as long as the number is finite. Finally, when using the MAVT, it is theoretically possible to deal even with an infinite number of alternatives.
17.6 C HOICE OF THE METHOD
393
Problems with an infinite number of alternatives are the ones most frequently encountered in practice: the number of feasible alternatives is, in fact, infinite when, for example, at least one component of the vector up of planning decisions assumes a value in an infinite set, or when one wants to design a management policy with a functional approach. In both the cases it is impossible to evaluate all the alternatives in an exhaustive way and so it is necessary to consider only the ‘most interesting’ ones. Then, the problem arises of how to identify these ‘most interesting’ alternatives. In Chapter 8 we explained that one can choose among infinite alternatives by formulating and solving an optimization problem (the Design Problem). However, in that chapter we dealt with full rationality conditions, i.e. with only one criterion, and therefore the concept of optimality was well defined. With more than one criterion that concept is, instead, no longer evident. We know, in fact, that, with conflicting criteria, the different viewpoints will consider different alternatives as optimal. It is thus clear that, in order to formulate the Design Problem, we must, first of all, define what we mean by ‘optimal’, i.e. we must define how to recompose the different viewpoints. For this it is, first of all, necessary to choose the evaluation method and then to formulate a Multi-Objective (MO) Design Problem according to the evaluation method adopted. The presence of an infinite number of alternatives determines the choice of the evaluation method: Table 17.6 shows, in fact, that only the MAVT allows for the evaluation of infinite alternatives. This method also has an interesting extension, known by the acronym MAUT , i.e. Multi-Attribute Utility Theory (Keeney and Raiffa, 1976), by means of which it is possible to deal, in a theoretically rigorous way, also with random indicators. By adopting the MAUT, the value function V (·) is replaced by the utility function U (·), which was introduced in Section 9.1.3. Thanks to this function, both the DM’s satisfaction and her risk aversion can be taken into account at the same time. The MAUT would therefore be the appropriate method to deal with the problems most frequently encountered, if it were not for two difficulties, which we have already met: 1. To identify the utility functions, the range of indicator values in correspondence with the alternatives to be compared must be known; but, to identify the alternatives to be compared, a Design Problem, which includes the utility functions, must be formulated. We talked about this vicious circle when dealing with full rationality conditions in Section 9.1.3, and there we proposed to break it through a recursive procedure. An analogous procedure can be applied in partial rationality conditions. 2. When there are management actions among the actions being considered and we want to solve the Design Problem with algorithms based on SDP, the objectives cannot be defined through the utility function. They must be defined with filtering criteria so that separability is guaranteed (see page 262). It follows that the Design Problem must be formulated and solved in the space of the objectives, i.e. the indicators that are obtained by applying a filtering criterion to the random indicators (see page 133). Thus the evaluation will be performed with the MAVT, however not at the same time as Designing Alternatives, but afterwards (and after Estimating Effects), when the values of the evaluation indicators are known. In conclusion, for Pure Planning Problems the most appropriate method seems to be the if no random disturbances act on the system, and the MAUT in the opposite case. For Pure Management (or Mixed) Problems it is always necessary to turn to the MAVT, and filter the randomness of the indicators beforehand, by applying filtering criteria. Since our interest is more oriented to this second class of Problems, in Chapter 20 we will go into MAVT ,
394
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
greater detail about the application of the MAVT. The reader interested in the MAUT can usefully refer to Keeney and Raiffa (1976).
17.1
Rankings and ordinal scales
Assume that the set A = |A0, . . . , AnA | of the nA alternatives from which the DM must choose and the effects that each one produces are known. These latter are expressed through the vector i of the indicators. Assume for the moment that the system is not affected by random disturbances and thus these vectors are deterministic; we will remove this hypothesis in the last section. On the basis of this information the DM must select the alternative that she prefers. To do so, she must rank the alternatives in set A according to decreasing values of her satisfaction: the first alternative in the order is the one chosen. Thus, choosing means ranking, and thus how to create the ranking is the object of our attention. A ranking can be expressed in various ways, according to various scales (or paradigms), and the most simple and direct one is the ordinal scale. According to this scale a ranking only expresses the 379
380
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Figure 17.1: Satisfaction associated with the surface area of a wetland. DM ’s1
preferences about the alternatives. For example, given the alternatives A1, A2, A3 and A4, a (ordinal) ranking is expressed in the following form {A4, A1 ∼ A3, A2}
(17.1)
This states that A4 is preferred over A1 (in the following we denote this fact with A4 A1), A1 is equivalent (∼) to A3, which is preferred over A2. This is clearly sufficient to conclude that the chosen alternative is A4. However, it is not easy for the DM to establish such an order, since the information that she has about each alternative is the corresponding vector of indicators. She must therefore compare these vectors, which is not a simple task for two reasons. The first reason is that the value of an indicator quantifies an effect in physical units and this is quite different from the level of satisfaction that the DM attributes to that effect: in fact, her satisfaction is not always proportional to the value that the indicator assumes. Consider for example a project in which the canalization of a river channel reduces the surface area of a wetland, which is potentially malarial, where migratory birds nest and mosquitoes reproduce. A good indicator of the contraction of the wetlands is its surface area. The environmentalists’ satisfaction is not proportional to the area since, if its value is too high, there could be an annoying and dangerous swarm of mosquitoes; their satisfaction is not inversely proportional to the area either, because reducing it too much would pose a risk to the survival of the migratory birds. To conclude, their satisfaction is a bell-shaped function of the area (Figure 17.1), because the maximum satisfaction is obtained with intermediate values. It is thus necessary to find a way to associate a value of satisfaction to each value of the indicator. Once this has been achieved, it is possible to rank the alternatives with respect to the satisfaction that the indicator produces: in this way one obtains a partial ranking for each indicator. Now the second difficulty arises: the partial rankings for the different indicators are generally different, in fact, the first alternative is rarely the same in all of them, because the evaluation criteria are usually in conflict. The task of the DM is thus to reduce this plurality of partial rankings, by defining, on the basis of these, a global ranking, in which the alternatives are listed from the best to the worst according to the ‘global satisfaction’ that each one produces. To define this satisfaction, value judgements come into play, which 1 As already said, in this chapter we assume that there is just one Decision Maker and that the Stakeholders do not have a decision role. The person that conducts the evaluation is therefore the DM. The considerations that are developed, however, are also applicable to the case in which the evaluation is conducted by several people at the same time, for example a group of DMs or Stakeholders. In Chapter 21 we will explain how to recompose their independent evaluations in one ranking.
17.1 R ANKINGS AND ORDINAL SCALES
381
are, by nature, subjective. A transparent decision thus requires that these subjective aspects be well highlighted and kept separate from the objective aspects (the effects produced). So the ordering rule, i.e. the procedure with which the DM passes from the partial rankings to the global ranking, must be explicitly formulated.
17.1.1
Arrow’s Theorem
An ordering rule does not only require an explicit formulation: it must also be democratic, because only in this case will it be accepted by the Stakeholders. But what does ‘democratic’ mean? An interesting definition of ‘democratic’ comes from Arrow,2 according to whom an ordering rule is democratic if, in non-trivial cases, i.e. when there are at least three alternatives to be ordered, it satisfies the following conditions: (1) unanimity: if in all the partial rankings A1 is preferred over A2 (A1 A2), then in the global ranking there is A1 A2; (2) non-imposition (or citizen sovereignty): the global ranking depends exclusively on the partial rankings; (3) non-dictatorship: the global ranking does not always coincide with one of the single partial rankings while ignoring all the others; (4) independence from irrelevant alternatives: this condition is valid when the alternatives are designed, evaluated and compared in two or more subsequent phases. Suppose that in the first phase alternatives A1 and A2 are generated, and that the global ranking derived from the rule is the following: {A1, A2}. If in the second phase alternative A3 is generated, the global ranking of the three alternatives must be such that the preference between A1 and A2 remains unvaried: the global ranking could be {A1, A3, A2}, or {A1, A2, A3}, or {A3, A1, A2}, but it cannot be {A2, A3, A1}. This condition is important because it prevents alternatives from being inserted in the set A with the sole aim of manipulating the order of the others. Different rules have been proposed for generating a global ranking from the partial rankings when they are expressed with ordinal scales. The most famous rules were elaborated in the second half of the 1700s by Borda (1781) and Condorcet (1785). The simplest is the rule of simple majority, which is the one upon which most electoral systems are based, and which we now illustrate to give an idea of how such rules can be formulated. The rule of simple majority establishes that, given a set of alternatives, the relationship between a pair of alternatives in the global ranking is equivalent to the one they have in the simple majority of the m partial rankings. For example, consider a decision problem with three alternatives A1, A2 and A3 and three indicators relative to three evaluation criteria: flooding, irrigation and hydropower production. Suppose that, on the basis of values of the three corresponding indicators, the DM has expressed the following three partial rankings: • flooding: {A1, A3, A2}; • irrigation: {A2, A1, A3}; • hydropower production: {A2, A3, A1}. 2 Nobel Prize for Economics in 1971.
382
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
We note that • A1 A3 in the simple majority of the partial rankings (flooding and irrigation) and thus in a global ranking based on the rule of simple majority A1 A3; • A2 A1 in the simple majority of the partial rankings (irrigation and hydropower production) and thus A2 A1 also in the global ranking; • A2 A3 in the simple majority of the partial rankings (irrigation and hydropower production), and thus A2 A3. The global ranking will therefore be the following: {A2, A1, A3} In order to accept the rule of simple majority we must make sure that it is democratic, but this is useless, since in 1951 Arrow identified the following paradox, today known as Arrow’s Paradox (Arrow, 1951), which shows that in some cases the rule of simple majority fails to produce a meaningful global ranking and therefore it is not an acceptable ordering rule. Let us suppose that in the previous example the DM expresses the following partial rankings: • flooding: {A1, A2, A3}; • irrigation: {A2, A3, A1}; • hydropower production: {A3, A1, A2}. By aggregating these orders with the simple-majority rule, the following relations are obtained: • A1 A2 in two of the three partial rankings (flooding and hydropower production): thus A1 A2 in the global ranking; • A2 A3 in two of the three partial rankings (flooding and irrigation): thus A2 A3; • A3 A1 in two of the three partial rankings (irrigation and hydropower production): thus A3 A1 from which the following global ranking results A1 A2 A3 A1 Clearly this ranking is paradoxical, because it does not respect the property of transitivity.3 One might think that the problem is in the simple-majority rule and that a different ordering rule should be found; but this conjecture must also be abandoned, since in 1963 Arrow proved that when the ordinal scale is adopted, it is not possible to obtain a global ranking from the partial rankings by using democratic ordering rules, i.e., that an ordering rule that respects all four conditions on page 381 cannot be found when the ordinal scale is adopted (Impossibility Theorem) (Arrow, 1963). 3 For a formal definition of this term see Section 17.2.
17.1 R ANKINGS AND ORDINAL SCALES
17.1.2
383
Absolute and interval scales
In the previous section we have seen that, when looking for a democratic (in the Arrow sense) ordering rule, ordinal scales are inadequate. Therefore, it is necessary to refer to scales that express not only qualitative preference judgements but also the intensity of the preference (cardinal scales), as, for example, the absolute scale. These scales generate cardinal rankings that can be expressed with the following notation {18, 15, 18, 21}
(17.2)
where the first number is the value attributed to alternative A1, the second to A2, and so on. Note that from the cardinal rankings an ordinal ranking can always be extracted; for example, from the cardinal ranking (17.2) it is possible to extract the ordinal ranking defined by (17.1). The adoption of an absolute scale requires us to fix the unit of measurement of the values and the zero, which represents total lack of value; this is not a simple task, since these elements are very subjective. Therefore, often an interval scale is preferred, i.e. a scale in which preference intensities are not related to the absolute values, but to the differences in value between the alternatives. On an interval scale, the value can be defined in an infinite number of equivalent ways, each related to the other by a linear transformation with positive coefficients. For example, by adopting an interval scale, the ranking (17.2) is equivalent to the following {1810, 1510, 1810, 2110}
(17.3)
which is obtained by applying the transformation i = 100i + 10 to (17.2); but it is also equivalent to the following {2.8, 2.5, 2.8, 3.1}
(17.4)
which is obtained by applying the transformation i = 0.1i + 1 and so on. For instance, temperature scales are interval scales: they associate different values to the two physical states of water freezing and water boiling, 0 and 100 in the Celsius scale (◦ C), and 32 and 212 in the Fahrenheit scale (◦ F). These values are linked by the following relation F = 1.8C + 32 Zero degrees on the Celsius scale corresponds to 32 degrees on the Fahrenheit scale, but both measure the same effect: the freezing temperature of water. One could apply any other monotonic transformation with positive coefficients to the values of temperature indicators to produce an infinite number of other temperature scales, all of which would be equivalent. In conclusion, by adopting an interval scale, one can accept that a value is a subjective quantity, without it influencing the course of the decision-making process. This is why interval scales are very suitable for expressing intensity of preference. The adoption of an interval scale also offers practical advantages, as we will see in Section 20.4 when describing more deeply the MAVT method, which adopts an interval scale.
384
17.2
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Preference axioms
In the previous section we analysed what to make a choice means and we said that choosing means ranking; then, we introduced several ways in which ranking can be expressed. Now, we will consider how to make a choice. The aim of the evaluation, in fact, is to formalize the logical procedure through which the DM makes a choice. In this way the decision is made transparent, in the sense that the objective and subjective motivations at the basis of the decision are made explicit and repeatable. This is the equivalent of identifying the so called preference structure of the DM (we will return to this subject in Section 20.2). However, a DM does not necessarily have a preference structure: if, for example, she decides randomly, she would not have one. Therefore we must firstly establish under which conditions this structure exists. The majority of authors hold that the existence of preference structure requires the satisfaction of the following axioms: Axiom 1 (Completeness): Given two alternatives A1 and A2, the DM is always able to say which of the two she prefers, and by how much, or whether they are equivalent to each other. Axiom 2 (Transitivity): Given three alternatives A1, A2 and A3, if the DM prefers A1 to A2 and A2 to A3, then she must necessarily prefer A1 to A3, i.e. A1 A2
and
A2 A3
⇒
A1 A3
(17.5)
Axiom 3 (Independence from irrelevant alternatives): Given a set A of alternatives, the order of preference that the DM establishes between any two of them does not vary if A is enlarged with a set A of new alternatives. We have already encountered this axiom among the conditions required for an ordering rule to be democratic (see page 381). Axiom 2 adopts ordinal expressions and it is thus suitable when an ordinal scale is adopted. Therefore, when working with the interval scale, it must be substituted by the following axiom, which is its cardinal form: Axiom 2b (Consistency): Given three alternatives A1, A2 and A3, if the DM affirms that A1 is m times more preferable to A2 and A2 is n times more preferable to A3, then she must necessarily affirm that A1 is m · n times more preferable to A3, i.e. A1 ∼ m A2
and
A2 ∼ n A3
⇒
A1 ∼ m · n A3
with m, n > 0
(17.6)
Evaluation methods based on the hypothesis that the DM satisfies the preference axioms listed above are called normative methods (or prescriptive methods) (French, 1988). On the other hand, descriptive methods do not require a priori that the DM’s preference structure satisfies any given axioms. Instead, they try to follow the DM’s own way of reasoning. The price of the malleability of these methods is a loss of mathematical rigour; the advantage is a more realistic representation of the decision-making process. The choice between the two methods is left to the Analyst, who must assess whether a rigorous mathematical approach would be helpful to organize the information provided by the DM, or would be a strong constriction on her way of reasoning.
17.3 M ULTI -ATTRIBUTE VALUE T HEORY
385
Table 17.1. The four forms of problem Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
Form 1 Form 3
Form 2 Form 4
The evaluation method should be chosen not only on the basis of the DM’s needs: it should also take into account the characteristics of the problem that is being tackled. Certain methods can in fact be used only with problems of a given form and not with others. The characteristics that define the form of a problem are the nature of the set (either discretefinite or continuous) of the alternatives to be considered and the nature of the indicators (either quantitative or quantitative–qualitative) used to estimate the effects. The four forms of problem that derive from these characteristics are classified in Table 17.1. Note that a real problem can be formalized in different forms, according to how one decides to formulate it (e.g. by adopting only quantitative indicators or not). Now we will illustrate three decision-making methods: Multi-Attribute Value Theory, Analytic Hierarchy Process and ELECTRE methods. We chose these three both because they are the most widely employed, and because together they cover the first three forms of problem in Table 17.1, which are the only ones that can be solved with the knowledge we have today. Our description will not be exhaustive, but it will simply aim at providing the elements necessary for a comparison of the three methods. Since, as we will see in the following, Multi-Attribute Value Theory is the most suitable method for the problems we deal with, Chapter 20 will be dedicated to a more detailed description of it.
17.3
Multi-Attribute Value Theory
Multi-Attribute Value Theory (MAVT) (Keeney and Raiffa, 1976) is based on the identification of a value function V (·). This function, the argument of which is the vector of the indicators that quantify the effects of an alternative, returns a single value that expresses the DM ’s satisfaction for the alternative. Increasing values of V correspond to increasing levels of satisfaction. Once the value function has been identified, it is possible to order any set of alternatives. The value V is thus the Project Index that we introduced in Section 3.4. The MAVT can be applied to Form 1, 2 and 3 problems (see Table 17.2). Since it is extremely difficult to identify the value function directly, the problem is led back to the definition of a partial value function for each one of the indicators. This is possible only if the DM’s choices satisfy the axioms of Completeness, Consistency and Independence from irrelevant alternatives. The MAVT is thus a normative method. The (global) Table 17.2. The forms of problem to which the MAVT can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
386
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
Figure 17.2: Steps in identifying the value function, given the values i1 , . . . , im of the indicators, according to the MAVT.
value function is then obtained through the composition of the partial value functions, generally through a weighted sum (see Figure 17.2). The values of the coefficient (weights) in the sum depend on the relative importance that the DM associates to each of the indicators. Notice that, by computing the value function as a weighted sum of the partial value functions, one implicitly assumes that a poor performance of one of the indicators can be balanced by good performances of others. In other words one implicitly assumes that Compensation is allowed among the criteria. The MAVT thus requires that compensation exists among all the indicators (with the exception of one at the most, Keeney and Raiffa, 1976). In order to guarantee coherence among the values provided by the partial value functions and the weights, it is important that the latter are estimated, with the procedure that we will define in Section 20.6, on the basis of the values that each indicator can assume and not of abstract considerations, far from the specific context examined. Also the definition of the partial value functions requires a series of complex operations, in which the DM has to express her evaluation of the effects of an alternative in a formalized way. Any attempt at simplifying these operations by adopting quick hypotheses should be absolutely avoided: for example, by hypothesizing a priori, i.e. without due verification, that the partial value functions are linear. By doing thus in fact, the results are seriously invalidated. We will return to this discussion in Chapter 20.
17.4
Analytic Hierarchy Process
Just as in the MAVT, in the Analytic Hierarchy Process (AHP) (Saaty, 1980, 1992) the performance of an alternative is obtained through the weighted sum of the performances that it attains with respect to the single indicators. Therefore, compensation must be admitted among the criteria. However, the procedure used to define the weights and the performances, given the effects of each alternative, is very different from the one used in the MAVT: it follows that the types of problems to which it can be applied are not the same (see Table 17.3). Once the hierarchy of criteria (Chapter 3) has been defined, the AHP requires that all the alternatives are appended to each one of the leaf criteria. For example, Figure 17.3 shows the hierarchy of the Upstream Tourism sector in the Verbano Project (see Chapter 4 of PRACTICE) in the form required by the AHP. The evaluation is then performed in two steps: pairwise comparison and hierarchical recomposition.
17.4 A NALYTIC H IERARCHY P ROCESS
387
Table 17.3. The types of problem to which the AHP can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
First, the DM is required to compare, for each criterion in the hierarchy, all the elements that depend on it at the immediately lower level. For example, when the leaf criterion Reduced access to beaches is considered in the hierarchy of Figure 17.3, the elements of the lower level are the alternatives; when the sector criterion Loss of activity due to vacating tourists is considered, the elements of the lower level are the three criteria Reduced landscape aesthetics, Reduced access to beaches, Discomfort produced by mosquitoes; etc. The comparison is aimed at identifying a ranking vector for each criterion, whose ith component quantifies how much the ith element of the lower level satisfies that criterion. The ranking vector can thus be used to rank the lower-level elements with respect to the criterion being considered. The identification of the ranking vector for each criterion is performed by asking the DM to make pairwise comparisons of the elements at the lower level, filling in a square matrix (called pairwise comparisons matrix), whose elements aij express the intensity of preference for the element in the ith row compared to the element in the j th column, with respect to the criterion under examination. The ranking vector is then extracted from this matrix by means of a suitable algorithm. Note that, depending on the level in the hierarchy, the pairwise comparisons matrix expresses preferences either among alternatives or criteria. In the first case, the ith component of the resulting ranking vector measures how much the ith alternative satisfies a given criterion; in the latter, it measures the relative importance of the ith criterion with respect to the upper level criterion, i.e. the ranking vector is a vector of weights. It is important to underline that, when filling in the pairwise comparison matrix, the DM might not satisfy the axiom of Consistency: the AHP, in fact, allows the ranking vectors to be derived even when there are slight4 inconsistencies. Unlike the MAVT, the AHP is therefore
Figure 17.3: The hierarchy for the Upstream Tourism sector of the Verbano Project, in the case when three alternatives were to be compared. 4 An inconsistency is slight when it does not cause a rank-reversal between two alternatives. For example, if the DM
says that A1 ∼ 2 · A2 and that A2 ∼ 2 · A3, it is not necessary that she affirm that A1 ∼ 4 · A3 (as the axiom
388
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
a descriptive method. It requires only that the condition of Reciprocity be satisfied, i.e. that aj i = 1/aij , a condition that is so obvious that it is almost always satisfied. Now, once a pairwise comparison matrix has been filled in for every criterion in the hierarchy, it is possible to derive the ranking vectors of the alternatives with respect to each leaf criterion, and the ranking vectors of the criteria with respect to each upper-level criterion. Through a sequence of matrix products a final ranking vector of the alternatives with respect to the root criterion of the hierarchy is obtained; this operation is called hierarchical recomposition. Note that deriving the final ranking of the alternatives as a linear combination of the alternatives’ ranking vectors with respect to the single leaf criteria, implies assuming compensation among the criteria. With respect to the MAVT, the AHP makes the interaction with the DM simpler, because she is no longer asked to identify functions that express how her degree of satisfaction varies with the indicator values, but only to express her own preferences through a sequence of pairwise comparisons. The questions that are posed are always the same, regardless of the elements in the hierarchy that are taken into consideration: “Do you prefer criterion (alternative) X or criterion (alternative) Y , with respect to criterion Z? And what’s the intensity of your preference?” Answers to such questions are usually expressed through Saaty’s verbal scale, which allows nine possible judgments, ranging from “very weak” to “very strong” preferences, which are then transposed into numbers between 1 and 9. However, simplifying the interaction with the DM has three disadvantages: (1) the number of questions to be posed increases: the DM must express her preference between all possible pairs of elements at every level of the hierarchy; (2) the complexity of the method is increased from a mathematical point of view; (3) there is the risk that the questions posed to the DM will be poorly formulated. Let us analyse these issues one at a time. 1. The number of questions increases significantly as the number of alternatives and criteria increases. For example, consider a hierarchy with only two levels, n alternatives and m criteria: the number of questions that must be posed is given by the following relation5 n · (n − 1) m · (m − 1) + (17.7) 2 2 It is clear that, as n and m increase, it rapidly becomes impossible to pose all the questions. # questions = m ·
2. The increase in mathematical complexity would not of itself be a problem but for the fact that it introduces degrees of freedom into the methodology which can be used by the Analyst but not by the DM (for example the choice of the algorithm used to obtain the scoring vector from the pairwise comparisons, in the event that the matrix provided by the DM is not consistent). of Consistency would have done), but only that she judge A1 to be better than A3. This is the same as requiring that the axiom of Transitivity be satisfied. 5 To derive it, remember that for the pairwise comparison matrices the condition of Reciprocity must be fulfilled and that the elements along the diagonal do not need to be provided by the DM, because they are equal to 1 by definition.
17.5 ELECTRE METHODS
389
Table 17.4. The types of problem to which the ELECTRE methods can be applied Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite Continuous
3. Questions to the DM are poorly formulated because, when comparing the criteria, the DM does not refer to the range of values actually assumed by the indicators: she expresses her judgements in the abstract. To provide correct responses, instead, it would be necessary to consider the range of possible indicator values, since the importance of criterion X with respect to criterion Y might depend on the value of the indicator that measures the satisfaction of X. To clarify this point, consider again the hierarchy in Figure 17.3. If the indicator associated to the criterion Reduced landscape aesthetics takes on values between ‘unacceptable’ and ‘sufficient’, while the one associated to the criterion Discomfort produced by mosquitoes assumes values between ‘good’ and ‘optimal’, one naturally tends to give the first more importance, because it is the more worrying one. However, the judgement could be reversed if the alternatives would produce very positive effects for the first indicator and negative ones for the second. Finally, note that unlike the MAVT, which foresees defining value functions that can be later on be used to order any set of alternatives, with the AHP only the ranking of the alternatives that were directly compared by the DM is obtained. This means that if a new alternative is introduced, in order to know its position in the ranking it would be necessary to ask the DM to compare it with all the alternatives that have already been considered. Furthermore, since the ranking provided by the AHP depends on the alternatives being considered, the method does not satisfy the axiom of Independence from irrelevant alternatives. Therefore, the introduction of a new alternative could change the order of the other ones. This effect could be exploited to manipulate the order itself: alternatives could be introduced with the specific aim of sabotaging a given alternative in the final ranking.
17.5
ELECTRE methods
The ELECTRE methods (ELimination Et Choix Traduisant la REalité) (Roy, 1978, 1996) are only applicable to problems with a discrete set of alternatives (see Table 17.4). The literature speaks of ELECTRE methods because a number of methods have been developed from the same founding idea. They all intend to be as close as possible to the actual decision-making process, which means that they strive not to introduce any elements that oblige the DM to follow a line of reasoning that is not her own. ELECTRE methods are thus descriptive methods. As in the AHP, Consistency of preferences is not required; moreover, ELECTRE methods also admits Incomparability among the alternatives and Lack of transitivity in the rankings. As far as incomparability is concerned, note that both the MAVT and the AHP assume that an alternative’s bad performance with respect to a specific criterion can be compensated for by good performances with respect to the other criteria, whatever the gravity of the bad performance. It follows that it is always possible to compare two alternatives and identify which is preferable: this is the very reason why these two
390
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
methods satisfy the axiom of Completeness. On the contrary, in the ELECTRE methods it is possible for the DM to establish thresholds over which compensation among criteria is no longer possible: a circumstance that can often occur in reality. For example, think of a comparison between two cars, a utility passenger car and a limousine. The latter is for sure better than the former for many criteria, such as comfort, velocity, and perhaps also aesthetics. However, since it is much more expensive than the utility passenger car, it is not possible to express a clean preference for it. In other words, better performances on nearly all the criteria cannot compensate for the bad performance on the price criterion. In this situation, the DM is not able to express her preference between the two alternatives, which the ELECTRE methods would classify as incomparable among each other. As far as transitivity is concerned with, the ELECTRE methods acknowledge that, being human, the DM has limited discrimination capacity. This means that, when the difference between two indicator values is lower than a certain discrimination threshold, the DM does not perceive those values as being different. The values of the discrimination thresholds, which differ from one indicator to another, must obviously be defined by the DM. For example, when the DM has to compare alternatives from the point of view of supplying water to an irrigation district, she might not perceive the difference between supplies that differ by less than 2 m3 /s, while the difference could become appreciable for values of 5 m3 /s and substantial over 10 m3 /s. However, allowing for limited discrimination capacity may lead to intransitivity, as in the case of the “coffee paradox”. Think of two cups of coffee, of which one is bitter, the other is sweetened with one sugar grain only: since her discrimination capacity is limited, the DM is not able to express a preference among them and judges them as indistinguishable. Considering the sweetened cup and comparing it with a cup containing two grains of sugar, again the DM judges them undistinguishable. Thus, for transitivity, the bitter cup is judged undistinguishable from the sweetened with two grains cup. Keeping in comparing two cups which only differ for one sugar grain, the DM ends for comparing a completely saturated of sugar cup with a nearly saturated of sugar cup, and, again, judges them as undistinguishable. Applying transitivity to the whole chain of comparisons, the bitter cup results undistinguishable from the saturated of sugar cup. But this makes no sense, since they taste completely different. This paradox shows that accepting limited discrimination capacity implies the ranking might lack in transitivity. From these considerations it follows that the ranking of the alternatives obtained with an ELECTRE method might be incomplete and/or non-transitive.6 This means that, given an alternative, it might be impossible classifying all the other alternatives as better or worse than it, since some could be non-comparable and/or indistinguishable. In these conditions, it might be only possible understanding whether between two alternatives there is an outranking relationship, i.e. if one of them is clearly preferable to the other (Roy, 1991). If such a relationship exists, then the ranking reflects it by putting the outranking alternative in a better position than the one that is outranked (such as A2 with respect to A4 in Figure 17.4a). In order that an alternative outrank another, it is necessary that the reasons in its favour (or at least those that do not oppose it) be sufficiently strong with respect to the ones to the contrary. If an outranking relationship does not exist between two alternatives, they are either indistinguishable or not-comparable (for example A2 and A1 in Figure 17.4b). The ELECTRE methods all share this framework. Some of them allow to identify only a core set of alternatives, which are judged incomparable or indistinguishable among each 6 A ranking is said to be complete when all the alternatives being considered have a place in it and each position in the order is occupied by only one alternative (Figure 17.4); it is said to be transitive when it satisfies the axiom of Transitivity.
17.6 C HOICE OF THE METHOD
391
Figure 17.4: Comparison between a complete and transitive ranking (a) and a partial one (b). In the first, A4 is worse than A3, which in turn is worse than A2, and so on. In the second, A3 and A4 are worse than A1 and A2, and these latter are indistinguishable; nothing can be said about the relation between A3 and A4, which are not-comparable.
other. The so-called ELECTRE III method provides also a ranking of the alternatives, though partial. It requires the DM to provide a set of discrimination thresholds, one for each criterion considered, and a vector of weights for the different criteria. With this information, the ranking of the alternatives is produced through an automatic procedure, which we cannot describe here, but which the reader may find in Roy (1996). We can thus conclude that: 1. Since the outranking relationships are derived for pairs of alternatives, the ranking obtained with the ELECTRE methods depends on the set of alternatives being considered. Just as in the AHP, Independence from irrelevant alternatives is not guaranteed. 2. The number of questions for the DM does not depend on the number of alternatives, but only on the number of criteria (both the thresholds and the weights are defined for the criteria). The introduction of a new alternative thus does not lead to new questions. 3. Even if the method was born with the aim of designing a procedure that was as close as possible to the real decision-making process, the algorithm for extracting the alternatives’s ranking contains elements that are difficult for the DM to manage and understand.
17.6
Choice of the method
To facilitate the comparison of the three methods that have been illustrated, Table 17.5 lists, for each one, the preference axioms that it presupposes, while Table 17.6 lists the methods that are applicable for each type of problem. The choice of the method must take into account which axioms are satisfied by the preference structures of the actors involved (DM(s) and/or Stakeholders), and the characteristics of the problem. Before adopting a method it is necessary to check if the axioms that it presupposes are verified in the context examined. ELECTRE methods do not pose any conditions for the DM ’s preference structure (see Table 17.5). The AHP requires that the axiom of Completeness be satisfied; the MAVT also calls for Consistency and Independence from irrelevant
392
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD Table 17.5. The axioms required by each method
Completeness Transitivity Consistency Independence
MAVT
AHP
ELECTRE
Table 17.6. Forms of problem and solution methods Set of alternatives
Quantitative indicators
Quantitative and qualitative indicators
Discrete and finite
MAVT AHP ELECTRE
MAVT AHP ELECTRE
Continuous
MAVT
alternatives. Both the MAVT and the AHP require that there can be compensation among the criteria and that the alternatives be comparable. The validity of these axioms can be checked by questioning the actors and analysing their responses. If the Project involves multiple DMs, none of the methods listed above can be applied as such, because they were all derived in the context of a problem with only one DM. As we will explain in Chapter 21, however, some of the tools developed in the MAVT can be used as a starting point for negotiations. When the Project foresees the identification of the alternatives in several steps, both the AHP and the ELECTRE methods are not very suitable, because the results that they provide depend on the set of alternatives considered. Often, in fact, to identify the whole set of alternatives, one goes through several steps, in each of which a subset of alternatives is designed, evaluated and compared (for an example see Chapters 11–14 of PRACTICE). The information that emerges during a comparison is used to identify the alternatives to be generated in the next step, for example, by identifying the mitigation actions for the most disfavoured sectors. In order that this procedure be concluded successfully, it is necessary that an alternative that is judged to be preferable to another in a given step continue to be so in the following steps; this is guaranteed only if the evaluation is performed with the MAVT method. If, instead, the AHP or the ELECTRE methods were used, at each new generation the evaluation would have to be repeated ex-novo with all the alternatives, not just those from the last generation; further, adding new alternatives could modify the order of those generated in the previous step, because the Independence from irrelevant alternatives is not guaranteed. When there is a large number of alternatives to be evaluated, the AHP, although applicable in theory, cannot be applied in practice, because the number of questions that must be posed to the DM grows with the square of the number nA of alternatives. A high number of alternatives does not constitute, however, a limit for the applicability of ELECTRE methods, as long as the number is finite. Finally, when using the MAVT, it is theoretically possible to deal even with an infinite number of alternatives.
17.6 C HOICE OF THE METHOD
393
Problems with an infinite number of alternatives are the ones most frequently encountered in practice: the number of feasible alternatives is, in fact, infinite when, for example, at least one component of the vector up of planning decisions assumes a value in an infinite set, or when one wants to design a management policy with a functional approach. In both the cases it is impossible to evaluate all the alternatives in an exhaustive way and so it is necessary to consider only the ‘most interesting’ ones. Then, the problem arises of how to identify these ‘most interesting’ alternatives. In Chapter 8 we explained that one can choose among infinite alternatives by formulating and solving an optimization problem (the Design Problem). However, in that chapter we dealt with full rationality conditions, i.e. with only one criterion, and therefore the concept of optimality was well defined. With more than one criterion that concept is, instead, no longer evident. We know, in fact, that, with conflicting criteria, the different viewpoints will consider different alternatives as optimal. It is thus clear that, in order to formulate the Design Problem, we must, first of all, define what we mean by ‘optimal’, i.e. we must define how to recompose the different viewpoints. For this it is, first of all, necessary to choose the evaluation method and then to formulate a Multi-Objective (MO) Design Problem according to the evaluation method adopted. The presence of an infinite number of alternatives determines the choice of the evaluation method: Table 17.6 shows, in fact, that only the MAVT allows for the evaluation of infinite alternatives. This method also has an interesting extension, known by the acronym MAUT , i.e. Multi-Attribute Utility Theory (Keeney and Raiffa, 1976), by means of which it is possible to deal, in a theoretically rigorous way, also with random indicators. By adopting the MAUT, the value function V (·) is replaced by the utility function U (·), which was introduced in Section 9.1.3. Thanks to this function, both the DM’s satisfaction and her risk aversion can be taken into account at the same time. The MAUT would therefore be the appropriate method to deal with the problems most frequently encountered, if it were not for two difficulties, which we have already met: 1. To identify the utility functions, the range of indicator values in correspondence with the alternatives to be compared must be known; but, to identify the alternatives to be compared, a Design Problem, which includes the utility functions, must be formulated. We talked about this vicious circle when dealing with full rationality conditions in Section 9.1.3, and there we proposed to break it through a recursive procedure. An analogous procedure can be applied in partial rationality conditions. 2. When there are management actions among the actions being considered and we want to solve the Design Problem with algorithms based on SDP, the objectives cannot be defined through the utility function. They must be defined with filtering criteria so that separability is guaranteed (see page 262). It follows that the Design Problem must be formulated and solved in the space of the objectives, i.e. the indicators that are obtained by applying a filtering criterion to the random indicators (see page 133). Thus the evaluation will be performed with the MAVT, however not at the same time as Designing Alternatives, but afterwards (and after Estimating Effects), when the values of the evaluation indicators are known. In conclusion, for Pure Planning Problems the most appropriate method seems to be the if no random disturbances act on the system, and the MAUT in the opposite case. For Pure Management (or Mixed) Problems it is always necessary to turn to the MAVT, and filter the randomness of the indicators beforehand, by applying filtering criteria. Since our interest is more oriented to this second class of Problems, in Chapter 20 we will go into MAVT ,
394
C HAPTER 17. C HOOSING THE DECISION - MAKING METHOD
greater detail about the application of the MAVT. The reader interested in the MAUT can usefully refer to Keeney and Raiffa (1976).