- Email: [email protected]
Incorporating negative values into the Analytic Hierarchy Process
Computers & Operations Research 32 (2005) 3163 – 3173 www.elsevier.com/locate/cor
Incorporating negative values into the Analytic Hierarchy Process Ido Milleta,∗ , Bertram Schonerb a Penn State Erie, Black School of Business, Station Road, Erie, PA 16563-1400, USA b Simon Fraser University, Faculty of Business Administration, Burnaby B.C., Canada V5A 1S6
Available online 2 July 2004
Abstract This paper describes why and how ratio scaled multiple criteria analysis techniques, such as the Analytic Hierarchy Process (AHP), should allow for a subtraction mechanism whereby negative preference (aversion) is combined with positive preference. The main contribution of the paper is a demonstration of how the current imposition of a strictly positive additive value structure can lead to incorrect preference ratios and even incorrect ranking of alternatives. The proposed preference elicitation and computation method solves these problems in a simple and intuitive manner. Scope and purpose We focus in this paper on common cases where the AHP is used to evaluate alternatives in light of multiple criteria and where at least some of the considerations involve negative rather than positive effects from the point of view of the decision maker. The main objective is to correct how such negative effects are evaluated within the AHP. 䉷 2004 Elsevier Ltd. All rights reserved. Keywords: Analytic hierarchy process (AHP); Multi-criteria decision making (MCDM); Cost benefit analysis; Utility; Ratio scale
1. Introduction It is useful to indicate what this paper is not about. We do not address the question of whether the Analytic Hierarchy Process (AHP) should be additive or multiplicative. We examine the additive AHP [1,2], and address the major problems arising from restricting it to non-negative values. In passing, we note that this is equivalent to restricting multiplicative priorities to be greater than or equal to one. ∗ Corresponding author. Tel.: +1-814-898-6262; fax: +1-814-898-6223.
E-mail addresses: [email protected] (I. Millet), [email protected] (B. Schoner). 0305-0548/$ - see front matter 䉷 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.cor.2004.05.005
3164
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Like any system of measurement on a ratio scale, the AHP is predicated on the existence of a natural zero [3]. We show that the AHP treats negative preferences in a manner that ignores the natural zero preference point and hence abdicates the ratio property of the scale. By restricting all values of alternatives on criteria to be positive, even where the alternative is known to have an undesirable attribute, the resulting scale of measurement may not even be ordinal. It is important to address this issue because the AHP has been widely used for more than two decades [4]. We investigate the problem and outline the solution.
2. The natural zero A true ratio scale has a natural zero, a zero that, as the name implies, signifies the lack of magnitude [3]. Examples abound—volume, weight, assets, profits, etc. Some scales, such as those for volume and weight, are constrained to be nonnegative; but others, such as scales for assets or profit are not—negative assets are termed debts and negative profits are termed losses. To establish a clear definition of what it means for an alternative to have a true zero value under a given criterion, let the descriptive value of alternative Ai on criterion Cj be represented by xij . For example, if a particular restaurant is alternative A1 and décor is criterion C1 , then x11 may be “abysmal décor”, and the partial priority u11 = u(x11 ) of restaurant A1 on criterion C1 should then reflect negative desirability. If the overall priority of an alternative Ai is derived from a linear composite of partial priorities uij under criteria/attributes Cj then Ui = U (Ai ) =
n
wj uij
(1)
j =1
where j = 1, 2, . . . , n, and wj is the weight for criterion Cj . Eq. (1) is standard in the AHP. However, standard AHP, while attempting preference measurement on a ratio scale, fails to recognize and define the true zero preference point. We define uij = 0 if and only if xij neither adds to nor detracts from the overall desirability of alternative Ai to the decision maker. For example, a restaurant with décor that neither added to nor subtracted from its overall desirability would be at the natural zero in the value of its décor.
3. Handling of negative values in the AHP Measurement in the AHP is based on comparing the desirability of two alternatives on particular criteria with the imposed restriction that both are positive. For example, in selecting a breakfast menu one might be asked to compare the desirability of a raw egg to that of an omelet. For some of us, no positive ratio is possible since the raw egg has worse than zero desirability—it is very undesirable. Still, most decision makers are content to simply indicate a strong or very strong preference for an omelet over raw eggs. Behind the scene, the AHP converts such responses to numbers and treats them as if they were from a strictly positive ratio scale. We end up with the omelet getting a high positive value and raw eggs enjoying a low but positive value. This is clearly wrong since, given the ratio scale assumption, it implies that the decision maker should be willing to pay a positive, though small, amount
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3165
for the pleasure of consuming the raw egg. In reality, the decision maker would require payment before agreeing to consume the raw egg. The problem has escaped the attention of AHP researchers and practitioners because our intuition and language for expressing preferences are geared more for ordinal and interval scales than for ratio scales. Furthermore, the biasing effect is not very noticeable in the context of a single criterion. For example, in the case of an omelet versus raw egg we clearly get the correct ordinal rankings. It is only in the context of multiple criteria that the biasing effects of this procedure become evident even to the point of rank reversals. Treating negative profits or assets as if they were small positive profits or assets is clearly unacceptable to our accounting and banking systems. Treating negative desirability as if it were small positive desirability, as is done in the AHP, should be equally unacceptable to any system of decision-making claiming results on a ratio scale. Two approaches have been suggested to handle negative values in the AHP, but neither is satisfactory. The first is to segregate criteria into positive and negative categories, to handle the two separately, and to calculate a benefit to cost ratio. There are several problems with this approach. First and foremost, benefit to cost ratios are appropriate only when the decision maker is interested in leveraging scarce resources to produce maximum benefits. For example, the benefit to cost ratio is a correct selection heuristic when funding multiple projects subject to a budget constraint. If alternatives are mutually exclusive and nonrepeating then net benefits are likely to be more relevant than the benefit to cost ratio. This explains why most decision makers would prefer an investment of $100 yielding a $200 return (a 2:1 ratio) over an investment of $1 yielding a $20 return (a 20:1 ratio). Modeling benefits and costs in separate AHP hierarchies is also problematic when alternatives under a single criterion span negative as well as positive values. Another problem with this approach is that the resulting benefit to cost ratios provide no indication of whether benefits actually exceed costs [5]. The second approach that has been suggested to handle negative values in the AHP is to invert them into positive values. This is currently a standard procedure within the AHP. Saaty states that “in buying a car, the reciprocal of the actual dollar cost of the car can be included because the less costly a car is, the more it is preferred on the criterion of cost” [6, p. 9]. Expert Choice [7], a software implementation of the AHP, even provides a special option that allows decision makers to enter cost values and easily invert them to positive preferences. Such an inversion procedure amounts to dividing a ratio scale measurements by a negative value even though only division (or multiplication) by a positive constant is a permissible transformation. Furthermore, the value we are dividing by is not even a constant since each value gets divided by the negative of its own squared value. The effect is not only an arbitrary shifting of the natural zero point below values that should be negative, but also a reversal of strong and weak effects. Large costs become small positive effects while small costs become large positive effects. Hence, while the AHP claims it produces value measurements on a ratio scale, its value inversion procedure leads to a clear loss of the ratio scale property. A quantitative example of the distortion resulting from the AHP inversion procedure is presented in Section 4 below. The problem is exacerbated when under a single criterion some alternatives are desirable while others are not. In choosing between restaurants, one alternative with the tastiest food and most reasonable prices may possess abysmal décor. Assigning it a small positive value for décor understates the negative effect this restaurant’s décor has on the dining experience. Nevertheless, the typical AHP analysis would force
3166
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
the decision maker to state by how many times the pleasing décor of one restaurant is better than the abysmal décor of the restaurant in question. This is equivalent to asking by how many times four is bigger than minus two and forcing the answer to be a positive number. Such a positive number does not exist and the result cannot be on a ratio scale. Many decision-making situations involve tradeoffs between positive and negative effects. The restaurant example mentioned above is one such case. Another example lies in the choice of a medical procedure [8] or test [9] where riskier or unpleasant interventions may generate better long-term results. A more business-oriented example would be the evaluation of expensive equipment alternatives [10] where better functionality and features are frequently predicated on higher training, support, and purchasing costs. A subjective review of 34 AHP sample models that accompany the Expert Choice software [7] reveals that at least 19 (56%) contain negative as well as positive criteria. These 19 models included a wide range of negative criteria such as: Cost, Price, Effort, Time, Risk, Environmental Effects, Pollution, Erosion, Waste, Effects of Failure, Negative Attitude, Anxiety, Financial Risk, Market Competitiveness, Complexity, Initial Investment, Interest Rate, and Space Limitations. Clearly, it is common practice to use the AHP to model positive and negative effects within a single model.
4. An example of incorrect results using standard AHP Consider the case of John, a home renovator who has to select one of three projects to fill a two-month gap in his schedule. John bases his decision on two criteria: profit and effort. John is not interested in maximizing profit per unit of effort—instead he is interested in selecting the project with the highest net benefits. John thinks of effort in terms the difficulty of the job and the working conditions. For illustrative purposes, we will assume that John, although not consciously aware of the fact, implicitly converts effort to dollar cost. Hence, from our point of view the problem can be collapsed into a single criterion of net value. However, from John’s point of view the two criteria of profit and effort are evaluated on non-commensurate scales of profit (positive utility) and effort (negative utility). Setting up the situation in this manner allows us to directly compute absolute as well as relative net values. We can then contrast these results to those derived by the AHP. As shown by Table 1, from our all-knowing point of view, although Project A has the highest benefit to cost ratio, since John is interested in selecting the project with the highest net value, project B is best. What results would John, without knowledge of the effort-to-cost transformation, get by using the current AHP approach for such a situation? Using standard AHP, John would be directed to use the Ideal mode since the decision maker is concerned with the performance rather than the dominance of each alternative [11]. The ideal mode of synthesis uses the best alternative under each criterion as a linking Table 1 Project characteristics
Project A Project B Project C
Profit
Implicit cost of effort
Net value
Relative net value
$3000 $6000 $12,000
$1000 $3000 $15,000
$2000 $3000 $(3000)
1 1.5 −1.5
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3167
Table 2 Partial priorities for revenue and effort in standard AHP
Project A Project B Project C
Profit
Effort
0.25 0.5 1
1 0.333 0.067
pin [12] with a relative value of 1. Although other alternatives could also be used to link across criteria [13], the result would be unchanged. It is easy to demonstrate that transforming negative values into positive priorities (the term employed for desirability in standard AHP) leads to distortions in the overall priority vector. The vectors of partial priorities for John’s project choice problem, as generated by the AHP approach, are given in Table 2. Note that since Project A has the lowest effort, it receives the highest positive priority under that criterion. Let wp and we represent the criteria weights for profit and effort. What are the values for these criteria weights (between zero and one, summing to one) for which the rankings of the alternatives conform to our prior analysis in Table 1? For Project B to be preferred to project A, we have 0.5wp + 0.333we > 0.25wp + 1we
(2)
and for Project B to be preferred to Project C we have 0.5wp + 0.333we > 1wp + 0.067we .
(3)
From (1) we have wp > 2.67we , and from (2) we have wp < 0.53we . Hence, although Project B is clearly preferred, there are no criteria weights that generate the correct solution. It is easily shown that Project A is preferred for values of wp less than 0.555, and project C, in spite of the fact that it has a negative overall value, is selected for any value of wp above 0.555. This demonstrates that the current AHP approach produces not only incorrect preference ratios, but also incorrect rankings.
5. The bipolar AHP Relatively simple changes to software implementations of the AHP can support the identification and elicitation of preferences for cases where whole criteria (such as cost) or individual alternatives (such as the restaurant with abysmal décor) are associated with negative values to the decision maker. This paper will refer to this proposed approach as the Bipolar AHP (BAHP), reflecting the accommodation of both positive and negative desirability as opposed to the strictly positive view of the AHP. The BAHP requires several relatively simple modifications to the AHP user interface and computational process. First, users should designate each node in the criteria and alternative hierarchy as associated with negative or positive preferences. Based on these designations, we should change the way we elicit preferences from the user, process the information, and present the results.
3168
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
5.1. Indicating negative criteria and alternatives Software implementations of the AHP already maintain certain attributes such as name, comments and location for each node in the criteria and alternatives hierarchy. The BAHP requires that we add one more attribute designating each node as positive or negative. The user interface should reflect this attribute by coloring negative nodes in red or providing similar cues. Once a criterion node is designated as negative, all lower nodes branching from it should inherit that designation. For example, once a criterion of Maintenance in Fig. 2 is designated as negative, its sub-criteria (Insurance, Warranty, Fuel Economy, and Service) and the alternatives listed under those sub-criteria should all be designated as negative as well. The user should be allowed to override such inherited designations by directly changing the designation of a lower-level criterion or alternative. Users should designate criterion nodes as negative when the criterion reflects performance aspects that, in the majority of the alternatives, cause a reduction in preference. Typical examples include criteria such as cost, time, distance, risk, pain, and effort. 5.2. Elicitation of preferences Since under negative criteria, we aim to assess the negative priority of each alternative, we should open up the AHP process, which currently supports only preference elicitation, to support also aversion (negative preference) elicitation. Under a positive criterion, we should use the current elicitation process whereby the user is asked to express by how much one alternative seems better than another in terms of its contribution to that criterion. However, under a negative criterion, we should use an aversion elicitation process whereby the user is asked to express by how much one alternative seems worse than another. 5.2.1. Pairwise preference elicitation for mixed cases In the project selection case, the alternatives were all either positive or negative under each criterion. How should we elicit preferences in cases where under the same criterion some alternatives are desirable while some are not? For example, suppose that three restaurants are being considered, A1 , A2 and A3 , and that we are deriving the priority vector for décor. We first ask the respondent if any restaurants have a décor that actually diminishes the dining experience. Suppose A1 is identified and that paired comparisons of the other alternatives reveal that restaurant A2 is three times preferred to restaurant A3 . We cannot directly ask for a paired comparison of the ratio of desirability of the décor of restaurant A1 to either of the others, inasmuch as they are of opposite signs. However, we may ask the respondent to compare the absolute value (desirability impact) of the décor of restaurant A2 to the absolute value (undesirability impact) of the décor of restaurant A1 . As an alternative to this “absolute value” elicitation procedure, an equivalent “gradient value” elicitation procedure would ask the decision maker to compare the value of 10% improvement in the décor of restaurant A2 to the value of 10% improvement in the décor of restaurant A1 . Suppose the answer is a ratio of 2:1. From Step 2 we can assign a value of 1 to A2 and 0.333 to A3 ; and from Step 3 a value of −0.5 to A1 . Special handling must be applied when the decision maker indicates zero preference (neither positive nor negative) for an alternative under a given criterion. Clearly, such an alternative should not participate in pairwise comparisons under that criterion and its partial preference under that criterion should be set to zero.
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3169
Fig. 1. Graphical elicitation and presentation of preferences.
It should be noted that in some cases the same effect could be framed as either negative or positive. For example, the choice of a medical procedure may be framed positively using a ‘Contribution to Life Expectancy’ criterion or negatively using a ‘Mortality Rate’ criterion [8]. The proposed procedure does not remove the biasing effects resulting from the framing of criteria as either negative or positive. The goal here is merely to model negative effects once the decision maker recognizes them as such. 5.2.2. Graphical preference elicitation While pairwise comparisons may be the most accurate elicitation technique, they are not perceived as the easiest [14]. Expert Choice [7] and Hipre [15] are AHP software packages that incorporate graphical methods whereby the decision maker may interactively adjust the relative lengths of bars representing the performance of the alternatives on each criterion. In effect, this elicitation method involves comparing all alternatives simultaneously, rather than in pairs. As illustrated in Fig. 1, a simple modification would allow the decision maker to indicate negative preferences and their relative distance below the true zero preference line. 5.3. Presentation of preferences Just as the user interface should reflect negative designations of nodes, so should negative preferences for alternatives be reflected in the final results presented to the decision maker. For numeric presentations this would simply require using negative numbers. For graphical presentations, we could use the same type of display shown in Fig. 1, except that the values would reflect the combined synthesis results across all criteria. 5.4. Computational modifications The Bipolar AHP approach recognizes negative desirability as negative numbers and normalizes all partial values to the most extreme alternative under each criterion. For John’s problem of selecting a renovation project, Table 3 shows that the negative values under Effort were derived by identifying Project C as the alternative with the most extreme value and normalizing to that value. By coincidence, Project C happens to be the extreme alternative under both criteria.
3170
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Table 3 Positive and negative priorities for profit and effort
Project A Project B Project C
Profit
Effort
0.25 0.5 1
−0.067 −0.2 −1
5.4.1. Relative criteria weights Although within criteria the relative preferences (vij ) are in commensurate units, between criteria they are not. The values for each project in Table 3 cannot be summed because each criterion is using a different scale. We must elicit criteria weights wj such that the products wj uij are in commensurate units for all uij and can be summed across criteria. The classical AHP procedure derives criteria weights by asking the decision maker to compare the importance of the criteria. These weights are assumed to be independent of the actual alternatives. Previous literature has already pointed out the ambiguity [16] and shortcomings of such unanchored comparisons of criteria weights [12,17]. Recent literature [11] reflects an agreement that, in cases where the decision maker is concerned with the performance rather than the dominance of each alternative, elicitation of criteria weights must be anchored to specific alternatives or performance benchmarks. Since we stretch the scale used under each criterion to the point where the extreme alternative receives a value of 1 (or −1 if negative), anchoring the elicitation of relative weights to these extreme values provides a simple and accurate method for unifying these scales. It has previously been shown that, with positive preference values, the relative criteria weights must be proportional to the relative preference values of the extreme alternative under each criterion [13]. The correct criteria weights should produce weighted part values (wj uij )=Uij that are in commensurate units of measurement, reflecting the relative priority values across alternatives and criteria. Assume that we have an extreme positive priority value on criterion Cp associated with alternative Af (ufp = 1) and an extreme negative priority value on criterion Cq associated with alternative Ag (ugq = 1). Hence the ratio of the relative priority values is given by: Ufp wp ufp wp (1) wp = = (4) U w u w (−1) = w . gq q gq q q We can conclude that, with extreme values assigned partial priority values of one or minus one, relative criteria weights can be defined as the absolute value of the relative preference of the extreme priorities values under each criterion. Criteria weights may also be interpreted as the relative marginal priority, trade-off gradients, or scaling factors [18]. All of these views produce the same results, but give rise to different elicitation protocols. With the “absolute value” elicitation procedure, we would ask decision makers to compare the absolute value of the extreme alternative under each criterion. With the “gradient value” elicitation procedure, we would ask decision makers to compare the value of 10% improvement in these extreme alternatives. 5.4.2. Weighted sums for the project selection case In the project selection case, any of these weight elicitation methods should produce an effort/profit weight ratio of 45 reflecting the absolute ratio of $15,000 in implicit cost of effort to $12,000 of profit for
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3171
Fig. 2. A sample AHP model for the selection of a car.
the extreme alternatives under these two criteria. The resulting weights are 49 for profit and 59 for effort. Applying these weights, while treating negative priorities as negative numbers, yields a weighted sum of 0.074 for Project A ( 49 ∗ 0.25 + 59 ∗ −0.067), 0.111 for Project B ( 49 ∗ 0.5 + 59 ∗ −0.2), and −0.111 for Project C ( 49 ∗ 1 + 59 ∗ −1). Normalizing to the first alternative, the vector of relative overall preference is (1, 1.5, −1.5), which corresponds exactly to the vector of net value (2, 3, −3) in Table 1. This demonstrates that, in the presence of alternatives with negative values, Bipolar AHP produces correct results while AHP does not. Furthermore, unlike AHP, Bipolar AHP not only recognizes partial negative priorities, but can also identify alternatives with negative overall priority. For example, Project C above is shown to have negative net value to the decision maker. 5.5. Another case in point The model shown in Fig. 2 is one of the sample models included with the Expert Choice software [7]. This model prioritizes five automobile types according to the criteria of Price, Maintenance, Prestige, and Quality. It is reasonable to treat Price and Maintenance as negative criteria. While maintenance may produce positive effects on an automobile, this model is concerned with the desirability of different car models given different levels of required maintenance. Typically, maintenance requirements have negative effects (cost, time, and effort) on the decision maker. Fig. 3 contrasts the results generated for this model by AHP with those generated by BAHP using the same information but treating negative preferences as such. It is easy to see that the results are quite different. While the two top alternatives retained their ranking, Volvo increased its lead over Mercedes by a significant margin. This is because BAHP assigns Mercedes significant negative scores (rather than the merely small positive scores assigned by AHP) under the criteria of Price, Insurance, Warranty, and Service costs. The desirability of Thunder is much lower according to BAHP, to the point of rank reversal with Maxima. Note also that Grand Am is shown to be undesirable (negative utility)—an outcome that is not possible with AHP. The Thunder and Grand Am alternatives received large positive AHP scores just for being the least negative alternatives under criteria such as Price, Insurance, Warranty, and Service costs. We can see that AHP and BAHP produce significantly different results. Furthermore, for our project selection case AHP produced incorrect alternative rankings under any possible set of criteria weights while BAHP produced results that precisely matched the numerical benchmark.
3172
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Fig. 3. Comparison of AHP versus BAHP results.
6. Conclusions This paper demonstrates that typical decision making situations, where tradeoffs are required between positive and negative effects, cannot be handled correctly by current AHP methods. The proposed modifications to the AHP methodology provide a simple solution to accommodating negative preferences within the AHP while maintaining a true zero reference point, and hence a true ratio scale. While personal observations suggest that decision makers would welcome the option to specify negative preferences, future research should investigate how user interface options and preference elicitation protocols may best accommodate the added complexity of negative numbers while maintaining the intuitive appeal of the process. Any approach with an explicit recognition of both negative and positive values, must contend with known framing effects. Prior research has clearly shown that alternatives can be framed as losses or gains and that framing of choice situations can have a biasing effect on decision makers [8,19–23]. The Bipolar AHP approach would have to contend with these issues. It would also provide opportunities for studying framing effects more directly. Another area left for future research is the comparison of the “absolute value” and “gradient value” methods for eliciting relative values across negative and positive alternatives and for eliciting relative weights across positive and negative criteria. As discussed earlier in this paper, these methods are conceptually equivalent, but may result in different levels of accuracy and perceived ease of use. Notwithstanding the potential value of such future investigations, given the popularity of the AHP [4], our first priority should be to alert practitioners to the hazards posed by using this methodology to evaluate alternatives with negative effects. Our second priority should be to correct the AHP and its widely used software implementations by incorporating the proposed bipolar approach.
References [1] Saaty TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 1977;15 (3): 234–81. [2] Saaty TL. The analytic hierarchy process. New York: McGraw-Hill; 1980. [3] Stevens SS. On the theory of scales of measurement. Science 1946;103 (2684):677–80.
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3173
[4] Wasil E, Golden B. Celebrating 25 years of AHP-based decision making. Computers and Operations Research 2003;30 (10):1419–20. [5] Wedley WC, Choo EU, Schoner B. Magnitude adjustment for AHP benefit/cost ratios. European Journal of Operational Research 2001;133 (2):342–51. [6] Saaty TL. The analytic network process. Pittsburgh: RWS Publications; 1996. [7] Expert Choice. Expert Choice Inc., (www.expertchoice.com.), 2002. [8] McNeil JB. et al. On the elicitation of preferences for alternative therapies. The New England Journal of Medicine 1982;306 (21):1259–62. [9] Liberatore MJ. et al. Decision counseling for men considering prostate cancer screening. Computers & Operations Research 2003;30 (10):1421–34. [10] Sloane EB. et al. Using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment. Computers & Operations Research 2003;30 (10):1447–65. [11] Millet I, Saaty TL. On the relativity of relative measures—accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research 2000;121 (1):205–12. [12] Belton V, Gear AE. On a shortcoming of Saaty’s method of analytic hierarchies. Omega 1983;11 (3):228–30. [13] Schoner B, Wedley WC, Choo EU. A unified approach to AHP with linking pins. European Journal of Operational Research 1993;64 (3):384–92. [14] Millet I. The effectiveness of alternative preference elicitation methods in the analytic hierarchy process. Journal of MultiCriteria Decision Analysis 1997;6 (1):41–51. [15] Hämäläinen RP, Lauri H. HIPRE 3+ User’s Guide. System Analysis Laboratory, Helsinki University of Technology, 1992. [16] Watson SR, Freeling ANS. Assessing attributes weights by ratios. Omega 1982;10 (6):582–5. [17] Schoner B, Wedley WC. Ambiguous criteria weights in AHP: consequences and solutions. Decision Sciences 1989;20 (3):462–75. [18] Choo EU, Schoner B, Wedley WC. Interpretation of criteria weights in multicriteria decision making. Computers and Industrial Engineering 1999;37 (3):527–41. [19] Bostic R, Herrnstein R, Luce RD. The effect on the preference-reversal phenomenon of using choice indifferences. Journal of Economic Behavior and Organization 1990;13 (2):193–212. [20] Farquhar P. Utility assessment methods. Management Science 1984;30 (11):1283–300. [21] Hershey JC, Schoemaker PJH. Probability versus certainty equivalence methods in utility measurement: are they equivalent? Management Science 1985;31 (10):1213–31. [22] Johnson E, Schkade D. Bias in utility assessments: further evidence and explanations. Management Science 1989;35 (4):406–24. [23] Tversky A, Kahneman D. Rational choice and the framing of decisions. Journal of Business 1986;59 (4):251–78.
Incorporating negative values into the Analytic Hierarchy Process Ido Milleta,∗ , Bertram Schonerb a Penn State Erie, Black School of Business, Station Road, Erie, PA 16563-1400, USA b Simon Fraser University, Faculty of Business Administration, Burnaby B.C., Canada V5A 1S6
Available online 2 July 2004
Abstract This paper describes why and how ratio scaled multiple criteria analysis techniques, such as the Analytic Hierarchy Process (AHP), should allow for a subtraction mechanism whereby negative preference (aversion) is combined with positive preference. The main contribution of the paper is a demonstration of how the current imposition of a strictly positive additive value structure can lead to incorrect preference ratios and even incorrect ranking of alternatives. The proposed preference elicitation and computation method solves these problems in a simple and intuitive manner. Scope and purpose We focus in this paper on common cases where the AHP is used to evaluate alternatives in light of multiple criteria and where at least some of the considerations involve negative rather than positive effects from the point of view of the decision maker. The main objective is to correct how such negative effects are evaluated within the AHP. 䉷 2004 Elsevier Ltd. All rights reserved. Keywords: Analytic hierarchy process (AHP); Multi-criteria decision making (MCDM); Cost benefit analysis; Utility; Ratio scale
1. Introduction It is useful to indicate what this paper is not about. We do not address the question of whether the Analytic Hierarchy Process (AHP) should be additive or multiplicative. We examine the additive AHP [1,2], and address the major problems arising from restricting it to non-negative values. In passing, we note that this is equivalent to restricting multiplicative priorities to be greater than or equal to one. ∗ Corresponding author. Tel.: +1-814-898-6262; fax: +1-814-898-6223.
E-mail addresses: [email protected] (I. Millet), [email protected] (B. Schoner). 0305-0548/$ - see front matter 䉷 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.cor.2004.05.005
3164
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Like any system of measurement on a ratio scale, the AHP is predicated on the existence of a natural zero [3]. We show that the AHP treats negative preferences in a manner that ignores the natural zero preference point and hence abdicates the ratio property of the scale. By restricting all values of alternatives on criteria to be positive, even where the alternative is known to have an undesirable attribute, the resulting scale of measurement may not even be ordinal. It is important to address this issue because the AHP has been widely used for more than two decades [4]. We investigate the problem and outline the solution.
2. The natural zero A true ratio scale has a natural zero, a zero that, as the name implies, signifies the lack of magnitude [3]. Examples abound—volume, weight, assets, profits, etc. Some scales, such as those for volume and weight, are constrained to be nonnegative; but others, such as scales for assets or profit are not—negative assets are termed debts and negative profits are termed losses. To establish a clear definition of what it means for an alternative to have a true zero value under a given criterion, let the descriptive value of alternative Ai on criterion Cj be represented by xij . For example, if a particular restaurant is alternative A1 and décor is criterion C1 , then x11 may be “abysmal décor”, and the partial priority u11 = u(x11 ) of restaurant A1 on criterion C1 should then reflect negative desirability. If the overall priority of an alternative Ai is derived from a linear composite of partial priorities uij under criteria/attributes Cj then Ui = U (Ai ) =
n
wj uij
(1)
j =1
where j = 1, 2, . . . , n, and wj is the weight for criterion Cj . Eq. (1) is standard in the AHP. However, standard AHP, while attempting preference measurement on a ratio scale, fails to recognize and define the true zero preference point. We define uij = 0 if and only if xij neither adds to nor detracts from the overall desirability of alternative Ai to the decision maker. For example, a restaurant with décor that neither added to nor subtracted from its overall desirability would be at the natural zero in the value of its décor.
3. Handling of negative values in the AHP Measurement in the AHP is based on comparing the desirability of two alternatives on particular criteria with the imposed restriction that both are positive. For example, in selecting a breakfast menu one might be asked to compare the desirability of a raw egg to that of an omelet. For some of us, no positive ratio is possible since the raw egg has worse than zero desirability—it is very undesirable. Still, most decision makers are content to simply indicate a strong or very strong preference for an omelet over raw eggs. Behind the scene, the AHP converts such responses to numbers and treats them as if they were from a strictly positive ratio scale. We end up with the omelet getting a high positive value and raw eggs enjoying a low but positive value. This is clearly wrong since, given the ratio scale assumption, it implies that the decision maker should be willing to pay a positive, though small, amount
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3165
for the pleasure of consuming the raw egg. In reality, the decision maker would require payment before agreeing to consume the raw egg. The problem has escaped the attention of AHP researchers and practitioners because our intuition and language for expressing preferences are geared more for ordinal and interval scales than for ratio scales. Furthermore, the biasing effect is not very noticeable in the context of a single criterion. For example, in the case of an omelet versus raw egg we clearly get the correct ordinal rankings. It is only in the context of multiple criteria that the biasing effects of this procedure become evident even to the point of rank reversals. Treating negative profits or assets as if they were small positive profits or assets is clearly unacceptable to our accounting and banking systems. Treating negative desirability as if it were small positive desirability, as is done in the AHP, should be equally unacceptable to any system of decision-making claiming results on a ratio scale. Two approaches have been suggested to handle negative values in the AHP, but neither is satisfactory. The first is to segregate criteria into positive and negative categories, to handle the two separately, and to calculate a benefit to cost ratio. There are several problems with this approach. First and foremost, benefit to cost ratios are appropriate only when the decision maker is interested in leveraging scarce resources to produce maximum benefits. For example, the benefit to cost ratio is a correct selection heuristic when funding multiple projects subject to a budget constraint. If alternatives are mutually exclusive and nonrepeating then net benefits are likely to be more relevant than the benefit to cost ratio. This explains why most decision makers would prefer an investment of $100 yielding a $200 return (a 2:1 ratio) over an investment of $1 yielding a $20 return (a 20:1 ratio). Modeling benefits and costs in separate AHP hierarchies is also problematic when alternatives under a single criterion span negative as well as positive values. Another problem with this approach is that the resulting benefit to cost ratios provide no indication of whether benefits actually exceed costs [5]. The second approach that has been suggested to handle negative values in the AHP is to invert them into positive values. This is currently a standard procedure within the AHP. Saaty states that “in buying a car, the reciprocal of the actual dollar cost of the car can be included because the less costly a car is, the more it is preferred on the criterion of cost” [6, p. 9]. Expert Choice [7], a software implementation of the AHP, even provides a special option that allows decision makers to enter cost values and easily invert them to positive preferences. Such an inversion procedure amounts to dividing a ratio scale measurements by a negative value even though only division (or multiplication) by a positive constant is a permissible transformation. Furthermore, the value we are dividing by is not even a constant since each value gets divided by the negative of its own squared value. The effect is not only an arbitrary shifting of the natural zero point below values that should be negative, but also a reversal of strong and weak effects. Large costs become small positive effects while small costs become large positive effects. Hence, while the AHP claims it produces value measurements on a ratio scale, its value inversion procedure leads to a clear loss of the ratio scale property. A quantitative example of the distortion resulting from the AHP inversion procedure is presented in Section 4 below. The problem is exacerbated when under a single criterion some alternatives are desirable while others are not. In choosing between restaurants, one alternative with the tastiest food and most reasonable prices may possess abysmal décor. Assigning it a small positive value for décor understates the negative effect this restaurant’s décor has on the dining experience. Nevertheless, the typical AHP analysis would force
3166
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
the decision maker to state by how many times the pleasing décor of one restaurant is better than the abysmal décor of the restaurant in question. This is equivalent to asking by how many times four is bigger than minus two and forcing the answer to be a positive number. Such a positive number does not exist and the result cannot be on a ratio scale. Many decision-making situations involve tradeoffs between positive and negative effects. The restaurant example mentioned above is one such case. Another example lies in the choice of a medical procedure [8] or test [9] where riskier or unpleasant interventions may generate better long-term results. A more business-oriented example would be the evaluation of expensive equipment alternatives [10] where better functionality and features are frequently predicated on higher training, support, and purchasing costs. A subjective review of 34 AHP sample models that accompany the Expert Choice software [7] reveals that at least 19 (56%) contain negative as well as positive criteria. These 19 models included a wide range of negative criteria such as: Cost, Price, Effort, Time, Risk, Environmental Effects, Pollution, Erosion, Waste, Effects of Failure, Negative Attitude, Anxiety, Financial Risk, Market Competitiveness, Complexity, Initial Investment, Interest Rate, and Space Limitations. Clearly, it is common practice to use the AHP to model positive and negative effects within a single model.
4. An example of incorrect results using standard AHP Consider the case of John, a home renovator who has to select one of three projects to fill a two-month gap in his schedule. John bases his decision on two criteria: profit and effort. John is not interested in maximizing profit per unit of effort—instead he is interested in selecting the project with the highest net benefits. John thinks of effort in terms the difficulty of the job and the working conditions. For illustrative purposes, we will assume that John, although not consciously aware of the fact, implicitly converts effort to dollar cost. Hence, from our point of view the problem can be collapsed into a single criterion of net value. However, from John’s point of view the two criteria of profit and effort are evaluated on non-commensurate scales of profit (positive utility) and effort (negative utility). Setting up the situation in this manner allows us to directly compute absolute as well as relative net values. We can then contrast these results to those derived by the AHP. As shown by Table 1, from our all-knowing point of view, although Project A has the highest benefit to cost ratio, since John is interested in selecting the project with the highest net value, project B is best. What results would John, without knowledge of the effort-to-cost transformation, get by using the current AHP approach for such a situation? Using standard AHP, John would be directed to use the Ideal mode since the decision maker is concerned with the performance rather than the dominance of each alternative [11]. The ideal mode of synthesis uses the best alternative under each criterion as a linking Table 1 Project characteristics
Project A Project B Project C
Profit
Implicit cost of effort
Net value
Relative net value
$3000 $6000 $12,000
$1000 $3000 $15,000
$2000 $3000 $(3000)
1 1.5 −1.5
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3167
Table 2 Partial priorities for revenue and effort in standard AHP
Project A Project B Project C
Profit
Effort
0.25 0.5 1
1 0.333 0.067
pin [12] with a relative value of 1. Although other alternatives could also be used to link across criteria [13], the result would be unchanged. It is easy to demonstrate that transforming negative values into positive priorities (the term employed for desirability in standard AHP) leads to distortions in the overall priority vector. The vectors of partial priorities for John’s project choice problem, as generated by the AHP approach, are given in Table 2. Note that since Project A has the lowest effort, it receives the highest positive priority under that criterion. Let wp and we represent the criteria weights for profit and effort. What are the values for these criteria weights (between zero and one, summing to one) for which the rankings of the alternatives conform to our prior analysis in Table 1? For Project B to be preferred to project A, we have 0.5wp + 0.333we > 0.25wp + 1we
(2)
and for Project B to be preferred to Project C we have 0.5wp + 0.333we > 1wp + 0.067we .
(3)
From (1) we have wp > 2.67we , and from (2) we have wp < 0.53we . Hence, although Project B is clearly preferred, there are no criteria weights that generate the correct solution. It is easily shown that Project A is preferred for values of wp less than 0.555, and project C, in spite of the fact that it has a negative overall value, is selected for any value of wp above 0.555. This demonstrates that the current AHP approach produces not only incorrect preference ratios, but also incorrect rankings.
5. The bipolar AHP Relatively simple changes to software implementations of the AHP can support the identification and elicitation of preferences for cases where whole criteria (such as cost) or individual alternatives (such as the restaurant with abysmal décor) are associated with negative values to the decision maker. This paper will refer to this proposed approach as the Bipolar AHP (BAHP), reflecting the accommodation of both positive and negative desirability as opposed to the strictly positive view of the AHP. The BAHP requires several relatively simple modifications to the AHP user interface and computational process. First, users should designate each node in the criteria and alternative hierarchy as associated with negative or positive preferences. Based on these designations, we should change the way we elicit preferences from the user, process the information, and present the results.
3168
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
5.1. Indicating negative criteria and alternatives Software implementations of the AHP already maintain certain attributes such as name, comments and location for each node in the criteria and alternatives hierarchy. The BAHP requires that we add one more attribute designating each node as positive or negative. The user interface should reflect this attribute by coloring negative nodes in red or providing similar cues. Once a criterion node is designated as negative, all lower nodes branching from it should inherit that designation. For example, once a criterion of Maintenance in Fig. 2 is designated as negative, its sub-criteria (Insurance, Warranty, Fuel Economy, and Service) and the alternatives listed under those sub-criteria should all be designated as negative as well. The user should be allowed to override such inherited designations by directly changing the designation of a lower-level criterion or alternative. Users should designate criterion nodes as negative when the criterion reflects performance aspects that, in the majority of the alternatives, cause a reduction in preference. Typical examples include criteria such as cost, time, distance, risk, pain, and effort. 5.2. Elicitation of preferences Since under negative criteria, we aim to assess the negative priority of each alternative, we should open up the AHP process, which currently supports only preference elicitation, to support also aversion (negative preference) elicitation. Under a positive criterion, we should use the current elicitation process whereby the user is asked to express by how much one alternative seems better than another in terms of its contribution to that criterion. However, under a negative criterion, we should use an aversion elicitation process whereby the user is asked to express by how much one alternative seems worse than another. 5.2.1. Pairwise preference elicitation for mixed cases In the project selection case, the alternatives were all either positive or negative under each criterion. How should we elicit preferences in cases where under the same criterion some alternatives are desirable while some are not? For example, suppose that three restaurants are being considered, A1 , A2 and A3 , and that we are deriving the priority vector for décor. We first ask the respondent if any restaurants have a décor that actually diminishes the dining experience. Suppose A1 is identified and that paired comparisons of the other alternatives reveal that restaurant A2 is three times preferred to restaurant A3 . We cannot directly ask for a paired comparison of the ratio of desirability of the décor of restaurant A1 to either of the others, inasmuch as they are of opposite signs. However, we may ask the respondent to compare the absolute value (desirability impact) of the décor of restaurant A2 to the absolute value (undesirability impact) of the décor of restaurant A1 . As an alternative to this “absolute value” elicitation procedure, an equivalent “gradient value” elicitation procedure would ask the decision maker to compare the value of 10% improvement in the décor of restaurant A2 to the value of 10% improvement in the décor of restaurant A1 . Suppose the answer is a ratio of 2:1. From Step 2 we can assign a value of 1 to A2 and 0.333 to A3 ; and from Step 3 a value of −0.5 to A1 . Special handling must be applied when the decision maker indicates zero preference (neither positive nor negative) for an alternative under a given criterion. Clearly, such an alternative should not participate in pairwise comparisons under that criterion and its partial preference under that criterion should be set to zero.
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3169
Fig. 1. Graphical elicitation and presentation of preferences.
It should be noted that in some cases the same effect could be framed as either negative or positive. For example, the choice of a medical procedure may be framed positively using a ‘Contribution to Life Expectancy’ criterion or negatively using a ‘Mortality Rate’ criterion [8]. The proposed procedure does not remove the biasing effects resulting from the framing of criteria as either negative or positive. The goal here is merely to model negative effects once the decision maker recognizes them as such. 5.2.2. Graphical preference elicitation While pairwise comparisons may be the most accurate elicitation technique, they are not perceived as the easiest [14]. Expert Choice [7] and Hipre [15] are AHP software packages that incorporate graphical methods whereby the decision maker may interactively adjust the relative lengths of bars representing the performance of the alternatives on each criterion. In effect, this elicitation method involves comparing all alternatives simultaneously, rather than in pairs. As illustrated in Fig. 1, a simple modification would allow the decision maker to indicate negative preferences and their relative distance below the true zero preference line. 5.3. Presentation of preferences Just as the user interface should reflect negative designations of nodes, so should negative preferences for alternatives be reflected in the final results presented to the decision maker. For numeric presentations this would simply require using negative numbers. For graphical presentations, we could use the same type of display shown in Fig. 1, except that the values would reflect the combined synthesis results across all criteria. 5.4. Computational modifications The Bipolar AHP approach recognizes negative desirability as negative numbers and normalizes all partial values to the most extreme alternative under each criterion. For John’s problem of selecting a renovation project, Table 3 shows that the negative values under Effort were derived by identifying Project C as the alternative with the most extreme value and normalizing to that value. By coincidence, Project C happens to be the extreme alternative under both criteria.
3170
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Table 3 Positive and negative priorities for profit and effort
Project A Project B Project C
Profit
Effort
0.25 0.5 1
−0.067 −0.2 −1
5.4.1. Relative criteria weights Although within criteria the relative preferences (vij ) are in commensurate units, between criteria they are not. The values for each project in Table 3 cannot be summed because each criterion is using a different scale. We must elicit criteria weights wj such that the products wj uij are in commensurate units for all uij and can be summed across criteria. The classical AHP procedure derives criteria weights by asking the decision maker to compare the importance of the criteria. These weights are assumed to be independent of the actual alternatives. Previous literature has already pointed out the ambiguity [16] and shortcomings of such unanchored comparisons of criteria weights [12,17]. Recent literature [11] reflects an agreement that, in cases where the decision maker is concerned with the performance rather than the dominance of each alternative, elicitation of criteria weights must be anchored to specific alternatives or performance benchmarks. Since we stretch the scale used under each criterion to the point where the extreme alternative receives a value of 1 (or −1 if negative), anchoring the elicitation of relative weights to these extreme values provides a simple and accurate method for unifying these scales. It has previously been shown that, with positive preference values, the relative criteria weights must be proportional to the relative preference values of the extreme alternative under each criterion [13]. The correct criteria weights should produce weighted part values (wj uij )=Uij that are in commensurate units of measurement, reflecting the relative priority values across alternatives and criteria. Assume that we have an extreme positive priority value on criterion Cp associated with alternative Af (ufp = 1) and an extreme negative priority value on criterion Cq associated with alternative Ag (ugq = 1). Hence the ratio of the relative priority values is given by: Ufp wp ufp wp (1) wp = = (4) U w u w (−1) = w . gq q gq q q We can conclude that, with extreme values assigned partial priority values of one or minus one, relative criteria weights can be defined as the absolute value of the relative preference of the extreme priorities values under each criterion. Criteria weights may also be interpreted as the relative marginal priority, trade-off gradients, or scaling factors [18]. All of these views produce the same results, but give rise to different elicitation protocols. With the “absolute value” elicitation procedure, we would ask decision makers to compare the absolute value of the extreme alternative under each criterion. With the “gradient value” elicitation procedure, we would ask decision makers to compare the value of 10% improvement in these extreme alternatives. 5.4.2. Weighted sums for the project selection case In the project selection case, any of these weight elicitation methods should produce an effort/profit weight ratio of 45 reflecting the absolute ratio of $15,000 in implicit cost of effort to $12,000 of profit for
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3171
Fig. 2. A sample AHP model for the selection of a car.
the extreme alternatives under these two criteria. The resulting weights are 49 for profit and 59 for effort. Applying these weights, while treating negative priorities as negative numbers, yields a weighted sum of 0.074 for Project A ( 49 ∗ 0.25 + 59 ∗ −0.067), 0.111 for Project B ( 49 ∗ 0.5 + 59 ∗ −0.2), and −0.111 for Project C ( 49 ∗ 1 + 59 ∗ −1). Normalizing to the first alternative, the vector of relative overall preference is (1, 1.5, −1.5), which corresponds exactly to the vector of net value (2, 3, −3) in Table 1. This demonstrates that, in the presence of alternatives with negative values, Bipolar AHP produces correct results while AHP does not. Furthermore, unlike AHP, Bipolar AHP not only recognizes partial negative priorities, but can also identify alternatives with negative overall priority. For example, Project C above is shown to have negative net value to the decision maker. 5.5. Another case in point The model shown in Fig. 2 is one of the sample models included with the Expert Choice software [7]. This model prioritizes five automobile types according to the criteria of Price, Maintenance, Prestige, and Quality. It is reasonable to treat Price and Maintenance as negative criteria. While maintenance may produce positive effects on an automobile, this model is concerned with the desirability of different car models given different levels of required maintenance. Typically, maintenance requirements have negative effects (cost, time, and effort) on the decision maker. Fig. 3 contrasts the results generated for this model by AHP with those generated by BAHP using the same information but treating negative preferences as such. It is easy to see that the results are quite different. While the two top alternatives retained their ranking, Volvo increased its lead over Mercedes by a significant margin. This is because BAHP assigns Mercedes significant negative scores (rather than the merely small positive scores assigned by AHP) under the criteria of Price, Insurance, Warranty, and Service costs. The desirability of Thunder is much lower according to BAHP, to the point of rank reversal with Maxima. Note also that Grand Am is shown to be undesirable (negative utility)—an outcome that is not possible with AHP. The Thunder and Grand Am alternatives received large positive AHP scores just for being the least negative alternatives under criteria such as Price, Insurance, Warranty, and Service costs. We can see that AHP and BAHP produce significantly different results. Furthermore, for our project selection case AHP produced incorrect alternative rankings under any possible set of criteria weights while BAHP produced results that precisely matched the numerical benchmark.
3172
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
Fig. 3. Comparison of AHP versus BAHP results.
6. Conclusions This paper demonstrates that typical decision making situations, where tradeoffs are required between positive and negative effects, cannot be handled correctly by current AHP methods. The proposed modifications to the AHP methodology provide a simple solution to accommodating negative preferences within the AHP while maintaining a true zero reference point, and hence a true ratio scale. While personal observations suggest that decision makers would welcome the option to specify negative preferences, future research should investigate how user interface options and preference elicitation protocols may best accommodate the added complexity of negative numbers while maintaining the intuitive appeal of the process. Any approach with an explicit recognition of both negative and positive values, must contend with known framing effects. Prior research has clearly shown that alternatives can be framed as losses or gains and that framing of choice situations can have a biasing effect on decision makers [8,19–23]. The Bipolar AHP approach would have to contend with these issues. It would also provide opportunities for studying framing effects more directly. Another area left for future research is the comparison of the “absolute value” and “gradient value” methods for eliciting relative values across negative and positive alternatives and for eliciting relative weights across positive and negative criteria. As discussed earlier in this paper, these methods are conceptually equivalent, but may result in different levels of accuracy and perceived ease of use. Notwithstanding the potential value of such future investigations, given the popularity of the AHP [4], our first priority should be to alert practitioners to the hazards posed by using this methodology to evaluate alternatives with negative effects. Our second priority should be to correct the AHP and its widely used software implementations by incorporating the proposed bipolar approach.
References [1] Saaty TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 1977;15 (3): 234–81. [2] Saaty TL. The analytic hierarchy process. New York: McGraw-Hill; 1980. [3] Stevens SS. On the theory of scales of measurement. Science 1946;103 (2684):677–80.
I. Millet, B. Schoner / Computers & Operations Research 32 (2005) 3163 – 3173
3173
[4] Wasil E, Golden B. Celebrating 25 years of AHP-based decision making. Computers and Operations Research 2003;30 (10):1419–20. [5] Wedley WC, Choo EU, Schoner B. Magnitude adjustment for AHP benefit/cost ratios. European Journal of Operational Research 2001;133 (2):342–51. [6] Saaty TL. The analytic network process. Pittsburgh: RWS Publications; 1996. [7] Expert Choice. Expert Choice Inc., (www.expertchoice.com.), 2002. [8] McNeil JB. et al. On the elicitation of preferences for alternative therapies. The New England Journal of Medicine 1982;306 (21):1259–62. [9] Liberatore MJ. et al. Decision counseling for men considering prostate cancer screening. Computers & Operations Research 2003;30 (10):1421–34. [10] Sloane EB. et al. Using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment. Computers & Operations Research 2003;30 (10):1447–65. [11] Millet I, Saaty TL. On the relativity of relative measures—accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research 2000;121 (1):205–12. [12] Belton V, Gear AE. On a shortcoming of Saaty’s method of analytic hierarchies. Omega 1983;11 (3):228–30. [13] Schoner B, Wedley WC, Choo EU. A unified approach to AHP with linking pins. European Journal of Operational Research 1993;64 (3):384–92. [14] Millet I. The effectiveness of alternative preference elicitation methods in the analytic hierarchy process. Journal of MultiCriteria Decision Analysis 1997;6 (1):41–51. [15] Hämäläinen RP, Lauri H. HIPRE 3+ User’s Guide. System Analysis Laboratory, Helsinki University of Technology, 1992. [16] Watson SR, Freeling ANS. Assessing attributes weights by ratios. Omega 1982;10 (6):582–5. [17] Schoner B, Wedley WC. Ambiguous criteria weights in AHP: consequences and solutions. Decision Sciences 1989;20 (3):462–75. [18] Choo EU, Schoner B, Wedley WC. Interpretation of criteria weights in multicriteria decision making. Computers and Industrial Engineering 1999;37 (3):527–41. [19] Bostic R, Herrnstein R, Luce RD. The effect on the preference-reversal phenomenon of using choice indifferences. Journal of Economic Behavior and Organization 1990;13 (2):193–212. [20] Farquhar P. Utility assessment methods. Management Science 1984;30 (11):1283–300. [21] Hershey JC, Schoemaker PJH. Probability versus certainty equivalence methods in utility measurement: are they equivalent? Management Science 1985;31 (10):1213–31. [22] Johnson E, Schkade D. Bias in utility assessments: further evidence and explanations. Management Science 1989;35 (4):406–24. [23] Tversky A, Kahneman D. Rational choice and the framing of decisions. Journal of Business 1986;59 (4):251–78.