- Email: [email protected]
A sensitivity analysis of multicriteria choice-methods
A sensitivity analysis of multicriteria choice-methods An application on the basis of the optimal site selection for a nuclear power plant J. A. Hartog, E. Hinloopen and P. Nijkamp
Three classes of qualitative multicriteria methods are dealt with in this paper. An empirical problem, the choice of the best site for a nuclear power plant, is used to investigate the extent to which the results of the analysis are influenced by the choice of specific methods. The conclusion is that a certain sensitivity is unavoidable but that the final ranking of the alternatives - on the basis of a set of good methods is nevertheless reasonably stable. Keywords: Multicriteriaanalysis;Nuclearpower plants; Robustness
Plan and project evaluation is an essential part of all efficiency-orientedpolicy-making. However, as long as no unambiguous efficiency criteria are formulated and accepted by everybody, policy analysis most often is a form of conflict analysis in which dissimilar considerations have to be envisaged simultaneously. The latter situation especially has led to the rise of so-called multicriteria methods. Multicriteria methods have been applied repeatedly for complex planning problems in many countries. Many applications can be found in the field of physical planning and environmental management (see Voogd
[82). All these plans and projects have in common that they have to reconcile several conflicting objectives. Many of these studies were complicated by an additional difficultyin that they had to incorporate effects that are difficult to quantify. Clearly at the moment a
P. Nijkamp is with Vrije Universiteit, Faculteit der Economische Wetenschappen, Postbox 7161, 1007 MC, Amsterdam. A. Hartog is an emeritus professor of Erasmus University Rotterdam. E. Hinloopen is associated with the NMB Bank, Amsterdam. Final manuscriptreceived 10 November 1988.
great variety of multicriteria methods is in existence, but there is only a limited number that is able to treat ordinal, lexicographical or binary information in a logically consistent manner. Some decision problems have a mixed character and deal with some effects that are 'hard' while other effects are only measurable on a lower scale. There are only a few methods that are able to cope with this class of problems. This paper is mainly concerned with the latter class of problems. The sensitivity of the results when applying different 'qualitative' evaluation methods will be investigated on the basis of an actual policy choice problem in the Netherlands, viz the problem of locating a new nuclear power plant.
Three methods for multicriteria analysis Many statistical methods are in use to analyse problems that are in essence marked by a choice between several discrete alternatives. In this paper we will only deal with three of these methods: the HinloopenNijkamp regime method, the Isracls-Kcller regime method and the numerical interpretation method. The first method has been described in the international literature various times (see Hinloopen et al [2]; Hinloopcn and Nijkamp [3]; and Nijkamp [5]). The other two arc not published in English as far as we
0140-9883/89/040293-08 © 1989 Butterworth & Co (Publishers) Ltd
293
A .sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al
know. In the appendix we include a short description of the three methods. It requires some explanation why when choosing three methods of multicriteria analysis we selected two methods which at first sight are rather similar, ie the Israels-Keller and Hinloopen-Nijkamp regime methods. These two methods have indeed a lot in common. The basis of both methods is a pairwise comparison of alternatives and both distill out of each comparison a regime, that is a vector consisting of pluses and minuses (or ones and zeros) as explained in the Appendix. Both methods use the assumption that the ordinal character of priorities and effects is caused by lack of precise knowledge. The ordinal measurement of phenomena may gradually be transformed into cardinal information with increasing knowledge. A wellknown example, that is often quoted in this context, is the concept of temperature. In the early days of physics the only observation on temperature was warmer or colder. Temperature was only measurable on an ordinal scale. With progress in physics, however, it became possible to measure temperature on a ratio scale. The advance of science expresses itself in an improvement of measurableness so goes the argument. This point of view is of course not generally accepted, and certainly not in the humanities in which latent and fuzzy concepts like the beauty of a literary style are often used. It seems to us that a reasonable point of departure in this context is that under certain circumstances it is not unreasonable to accept that behind the ordinal character of a variable a cardinal measurability is hidden and that under other circumstances this is not the case. The borderline is often not so difficult to determine, however. The peculiarity of the temperature example is that in general everybody who has to determine which one of two objects is the warmest will come to the same conclusion. In all circumstances where this is the case it seems reasonable to suppose that the character of the variable is in essence a cardinal one. In such a case it is often not difficult to devise a unit of measurement into which the magnitude of the variable can be expressed. Insofar as it is not reasonable to suppose that everybody will have the same opinion, it does not seem reasonable to suppose a cardinal background behind ordinal data. Finally a common element of both above-mentioned methods is that they manage the uncertainty element by means of the calculus of probability and translate 'lack of knowledge' into a rectangular probability distribution. Both methods, however, do also show differences in essential aspects. The Israels-KeUer method translates the concept of dominance in such a way that the 294
method has a somewhat lexicographic character. 'A consequence of this trait is that an alternative that does not score strongly on an important criterion is not likely to come out on top irrespective of its perhaps high - score on less important criteria. In certain circumstances this can analytically be an attractive property, but there exist differences of opinion on whether the restricted possibility of compensation in a lexicographic ordering is not too severely restrictive. However, what is of importance here is that it is reasonable to expect that the results of the HinloopenNijkamp method might differ from those of the Israels-Keller method. This expectation is based partly on a comparison of both methods in another context (Albers [1]) where it was shown on the basis of an extensive dataset that the Israels-Keller method punishes weakness in an important criterion mercillessly.
Choice of location of nuclear power plants The Dutch government presented a first memorandum on the location of nuclear reactors to parliament on 11 January 1985. The Advisory Council for Physical Planning (RARO) used this report as a basis for extensive discussion. The RARO also submitted a report in which multicriteria methods were extensively used (RARO [6]). This report did not discuss the problem of whether nuclear reactors ought to be built, but only dealt with the problem of where a nuclear reactor should be built if it was decided to increase Dutch nuclear capacity. The RARO constructed an extensive list of potential sites which was finally reduced to nine alternative locations in the Netherlands. These locations are Bath/Hoedekenskerke(1), Borssele(2), Eems(3), Flevonoord(4), Ketelmeer(5), Maasvlakte(6), Moerdijk(7), Noordoostpolder (NOP)(8) and Wieringermeer(9). Fifteen relevant criteria were distinguished to structure the judgment process: population density (A), facilities for evacuation/emergency assistance (B), pollution of agricultural soil (C), threat to industrial areas (D), probability of polluting drinking water (E), quantitative availability of cooling water (F), quality cooling water (G), decrease existing air pollution (H), potential use of waste heat (I), indirect claims on scarce land (J), effects on the landscape (K), consequences for the environment (L), linkage to the existing energy network (M), presence of physical infrastructure (N) and suitability for a coal-fired power station(O). In the RARO report each location was evaluated for each criterion. Apart from criterion A, only ordinal information was used. These ordinal data, however, do not provide a complete ranking, but only a partial ENERGY ECONOMICS
October 1989
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al
one in sequential classes. The data therefore, contain many ties. The information on criterid~ ~A (weighted population density) on the other hand was included as (standardized) cardinal information. Criterion O (suitability of a location for coal-fired plant) deserves some special attention. The RARO started from the assumption that in time it would be necessary to assign new locations for the use of coal-fired installations. They were, however, of the opinion that a strong concentration of power has so many disadvantages that the fact that a nuclear reactor was situated at a site that was also suitable for a coal-fired station might preclude the establishment of such an energy source at a future data. This constitutes the background of the procedure to rate a location that is eminently suitable for the use of coal in a reversed way (ie low) as a location for a nuclear reactor. Clearly possible advantages of large-scale energy production units are not taken into consideration here. This raises the necessity to determine also the suitability of a site as a location for a coal-fired power station. To solve this problem a separate multicriteria analysis was first executed (MCA-coal). The results of this analysis were used as input for the analysis of the multicriteria analysis for nuclear reactors (MCAnuclear). As potential locations for MCA-coal, locations (1)-(9) were used as well as the following additional locations: Ijmuiden(10), Velsen(ll), Hemweg(12), Rotterdam(13), Nijmegen(14), Diemen(15) and Merwede/Dordreeht(16). 1 The following criteria were used in the MCA-eoal: cooling water (quantitative) (a), effect on landscape (b), linkage to the energy network structure (c), accessibility for transport of coal (d), distance from urban conglomerations (e), distance from agricultural and horticultural areas (f), distance from natural and recreational areas (g) and an agglomeration effect (h). All these data were measured on an ordinal scale. The effect matrices of MCA-coal and MCA-nuclear are presented in Table 1 and 2, respectively. In all cases the convention has been adopted that higher rates indicate a more preferred value: the higher the better. The RARO distinguished four different policy variants by introducing four sets of weights to indicate the importance of the criteria. These four variants are: (i) base variant; (ii) financial-economic variant; (iii) environmental variant; and (iv) spatial variant. Un-
1One might question whether it is sensible to take additional coal-fired plants into consideration because only the ranking of (1)-(9) is of importance for MCA-nuclear. The only analytically interesting reason for this procedure is to be found in the so-called 'Independence of Irrelevant Alternatives' (IIA) discussion, a problem well known in the discrete choice analysis. There it is shown that the final ranking of a set of alternatives may change if new, irrelevant alternatives are included.
ENERGY E C O N O M I C S
October 1989
Table 1. Input data for MCA-coal. Location Criterion
(a) (b) (c) (d) (e) (0 (g) (h)
12345678910111213141516
344444144 4 2 133113311 3 3 332333321 2 3 355215311 3 4 323333333 2 2 112112211 2 1 3 3 13 3 3 3 3 3 2 3 1 2 2 1 12 2 1 1 2 2
2 3 3 4 2 2 3 1
2 3 2 4 1 2 3 2
2 3 3 4 1 1 3 1
2 2 3 3 2 2 2 1
1 3 3 4 1 1 3 1
Table 2. Input data for IdCA.-nuclear. Location Criterion A AI" B C D E F G H I J K L M N
1
2
3
4
5
6
49 51 84 73 70 57 2 2 4 3 3 2 2 1 2 2 2 1 2 2 2 2 2 3 5 2 3 4 5 1 2 2 2 1 1 2 2 3 3 3 3 3 2 3 2 1 1 3 1 1 1 1 1 2 1 2 2 2 I 3 3 2 2 3 4 3 1 3 3 3 1 3 1 3 1 3 2 3 2 2 1 3 3 2 1 2 2 1 1 2
7
8
9
0 81 79 1 4 4 2 2 2 1 2 2 3 5 5 1 1 1 1 3 3 2 1 1 I 1 1 2 1 1 1 4 4 2 1 1 3 3 3 2 2 1 2 1 1
"A1 represents the ordinal ranking constructed on the basis of criterion A after transforming the cardinal information into ordinal information. bThe ranking in the last row has to be computed on the basis of the results of the MCA-coal per variant (see Table 5).
fortunately the RARO report is not very clear as to the special characteristics of the base variant. This is inconvenient because of the fact that in this variant the weights of the criteria that are used in MCA-coal as well as in MCA-nuclear (the pairs (a) and (F), (b) and (K), and (c) and (M)) are not exactly equal, as can be seen in Tables 3 and 4. Here too the convention, the higher the better, has been adopted.
Results of the MCA In the above mentioned RARO report a stepwise MCA was executed: first MCA-coal and then MCA-nuclear. The method used was the Qualiflex-method developed by the Netherlands Economic Institute (Rotterdam) while a check on the basis of an ordinal concordance analysis was also carried out. It was a pity that by using the Qualiflex method one or more of the 15 criteria had to be left out of consideration because for technical reasons the Qualiflex computer program can
295
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al Table 3. Rankings of criteria per variant for MCA-COUl. Criterion (a) Variant (i) (ii) (iii) (iv)
(b)
(c)
(d)
(e)
(t)
Table 6. Results per variant for MCA-nuclear.
(g)
(h)
3 5 6 2
2 2 2 5
1 5 1 5
45 4 3
5 4 5 4
2 4 2 1
3 3 6 2
3 4 3 2
(i) (ii) (iii) (iv)
A B C D E F G H
I
5 5 5 5
2 1 1 1 1 1 4 1 1 1 4 4 2 1 1 4 1 1 2 4 4 1 4 4
1 1 1 1
3 4 3 3
4 4 4 4
3 2 2 1 3 4 2 4 3 4 4 2 3 2 2 1
J K L M N O 2 4 2 4
Table 5. Results per variant for MCA..cnaL Ordinal concordance
Qnaliflex Location Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer
Variant i ii 5 2 3 6 7 1 4 8 9
5 2 3 4 7 1 6 8 9
iii
iv
8 3 9 4 5 1 2 6 7
5 2 4 6 7 1 3 8 9
Variant i ii 6 3 2 5 7 1 4 8 9
6 2 3 5 7 1 4 8 9
iii
iv
9 3 4 5 6 1 2 7 8
6 2 4 5 7 1 3 8 9
"Only the nine relevant locations for MCA-nuclear arc included here. These results constitute the input for criterion 0 in Table 2; a low value in this table indicates that the location is not suitable for a coal-fired plant and therefore very suitable from this point of view for a nuclear power station.
not manage more than 14 criteria. This serious restriction does not hold in the case of the ordinal concordance analysis, although this method suffers from some methodological impurities as regards the use of ordinal weights. Furthermore, in this case the quantitatively measured criterion A had to be transformed into a ranking with some loss of information as a consequence. The results of both methods (ie Qualiflex and ordinal concordance), distilled with some effort from the RARO report, are presented in Table 5 (MCA-coal) and Table 6 (MCA-nuclear) for all four variants. A striking characteristic of these results is the lack of robustness of the final ranking of the locations with respect to changes in the variants and the variance of the individual results between the Qualiflex-method and the ordinal concordance analysis, be it that the overall pattern is reasonably similar. This sensitivity
296
Variant i ii
Location
Table 4. Rlmkings of criteria per variant for MCA-nuclear. Criterion Variant
Ordinal concordance
Qnaliflex
Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer
2 4 9 5 3 8 1 7 6
2 3 5 8 7 9 1 6 4
iii
iv
1 4 7 6 3 9 2 8 5
2 3 7 9 8 6 i 5 4
Variant i ii 2 4 9 5 3 7 1 8 6
2 3 6 7 5 9 1 8, 4
iii
iv
2 6 7 4 3 9 1 8 5
2 3 5 7 6 9 1 8 4
is partly connected with the fact that the input data are not purely ordinal, but represent in essence a classification in size classes or relevance categories. Having described now the main elements of the choice procedure, it is no doubt an interesting exercise to investigate the extent to which the above results are confirmed by means of other methods. As mentioned before, we have chosen three methods for a robustness test: the Israels-Keller regime method, the Hinloopen-Nijkamp regime method and the numerical interpretation method. It has to be noted that from these three methods only the Hinloopen-Nijkamp regime method is able to deal with cardinal and mixed information (the 'mixed' option). This is especially important for criterion A in our MCA-nuclear. The results of these tests are discussed below. R o b u s t n e s s of three M C A ' s The results of Israels-Keller regime method (Regime I-K), the Hinloopen-Nijkamp regime method (Regime H-N) and the numerical interpretation method (Num Inter) will be presented and discussed. The outcomes for MCA-coal and MCA-nuclear will be given for each of the four variants mentioned above. The results for MCA-coal are to be found in Table 7. The complete set of alternatives, 16 in total, is used in the derivation of these results. The ranking of only locations 1 to 9 is used as input for criterion O in the computation of MCA-nuclear. Our computations showed that the IIA-problem, alluded to in footnote 1, was indeed present. If the nine relevant locations were selected from the beginning, the resulting ranking was sometimes slightly different from the one obtained when using the complete set of 16 locations. Because of the reasons mentioned above we nevertheless opted for the complete set. The main conclusion to be derived from Table 7 is that for each of the four variants separately the three
ENERGY ECONOMICS
October 1989
,4 sensitivity analysis of multicriteria choice methods." J.A. Hartog et al Table 7. Results per variant for MCA-coal for ~
methods. Regime H - N
Regime I-K Location Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer Ijmuiden Velsen Hemweg Rotterdam Nijmegen Diemen Merwede/D
-
Variant i ii 9 14 15 12 6 16 13 5 4 8 11 10 7 3 2 1
9 15 14 13 12 16 7 5 4 8 11 10 6 3 2 1
iii 8 15 14 13 7 16 6 5 4 12 10 9 11 3 2 1
Numerical iatorl~tafion
iv
Variant i fi
fii
iv
Variant i fi
fii
iv
8 15 12 9 4 16 13 2 1 7 14 11 6 10 3 5
11 14 15 7 6 16 13 5 4 10 12 8 9 3 2 1
8 15 14 13 7 16 ~ 5 4 12 10 9 11 3 2 1
8 15 13 5 3 16 14 2 1 7 12 11 6 10 4 9
7 14 14 8 6 16 13 5 3 9 12 11 10 4 1 2
6 15 14 I0 9 16 13 5 4 7 12 11 8 3 1 2
8 15 13 9 5 16 14 2 1 6 12 11 10 7 3 4
7 14 15 8 6 16 10 2 1 11 13 12 9 5 4 3
7 15 14 9 6 16 13 2 1 10 11 12 8 5 4 3
MCA-nuclear (see Table 8). These results again show a reasonable similarity in outcomes of the computations of the three methods. The correlation of the results of two arbitrary methods is always extremely high. Nevertheless it is undeniable that the methods also show clear differences connected with their structure. The Israels-Keller regime method for instance never indicates Maasvlakte as the best location. The numerical interpretation method chooses Maasvlakte as the best alternative in three out of the four cases, whilst the Hinloopen-Nijkamp regime method chooses Maasvlakte as the best location for two variants. The cause of these differences is to be found in the fact that the Maasvlakte location scores poorly on the criterion population density, which happens to be the criterion that gets the highest priority in all four variants. The somewhat lexicographic character of the Israels-Keller method makes it very difficult to compensate weakness in such an important criterion by strength in other criteria. The method gives the place of honour in three
methods under consideration did produce rather similar results, ie rather similar rankings of the alternatives. Within variant 1, for instance, the locations Maasvlakte and Eems are identified as the best ones in each of the three methods, and the locations Diemen and Merwede/Dordrecht as the worst ones. Similar patterns can be found in the other variants. This leads to the conclusion that the results are rather robust as regards the choice of one of these three methods. Only in the middle region can shifts in rankings be observed more frequently. The choice of variants on the other hand does seem to exert a definite influence on the ranking of alternatives, although the Maasvlakte is the best location for a coal-fired power station in each variant. Comparison with Table 5 shows that some differences do exist, especially with respect to the Qualiflex-method, although these differences certainly cannot be called dramatic. Next the results of each of the four variants and each of the three methods are used as input for
Table 8. Results per variant for MCA-nudear for three methods.
l_~.adon
Regime I-K
Regime H - N
Variant i ii
Variant i ii
2 3 9 5 4 7 1 8 6
iii
iv
2 4 8 6 3 7 1 9 5
2 3 6 5 4 8 1 9 7
Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Mocrdijk NOP Wieringermeer
2 5 7 4 3 8 1 9 6
ENERGY ECONOMICS
October 1989
1 3 9 5 4 6 2 8 7
2 3 8 6 4 9 I 7 5
Numerical iatorl~tatioa iii
iv
2 4 8 5 3 9 1 7 6
2 3 7 6 4 8 1 9 5
Variant i ii 2 4 6 7 3 9 1 8 5
2 3 6 8 5 9 1 7 4
iii
iv
2 4 7 6 3 9 1 8 5
2 3 6 9 5 8 1 7 4
297
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog
et al
of the four variants to the N O P alternative and to the Eems alternative in the financial-economic variant. Figure 1 illustrates the influence of the choice of the priorities on the final results of the HinloopenNijkamp method, whilst Figure 2 pictures the method sensitivity in the case of variant (i). If we include in the analysis the results of the Qualiflex and the Ordinal concordance methods from Table 6, it becomes obvious that the results do not differ extremely from those of the three methods of Table 8, although the differences are larger than those between the three methods we used. Such a robustness strengthens the confidence in such methods in any case.
sen voor Kerncentrales [6]) to select provisionally three
Concluding remarks After a long discussion the Dutch government has decided (Regeringsbeslissing (deel D), Vestigingsplaat-
locations as a possible site for a nuclear power plant. These places are: Borssele, Eems and the Maasvlakte. The choice of Maasvlakte is obvious. In many cases-10 out of 20-this site is the best one. The choice of Eems too is a plausible one. This location comes out on top in four cases: within variant (i) by the Qualiflex, Ordinal concordance and Hinloopen-Nijkamp methods and within variant (ii) by the Israels-Keller method. The choice of Borssele, however, cannot be defended on the basis of the criteria and priorities used by the RARO. Furthermore it is interesting to note that the location Moerdijk-strongly propagated for regional interests by the province of Brabant-is in none of the variants an acceptable or defensible option.
Appendix
IOC t~
[ ~
8C ~
tmr.I
Brief description of three multicriteria methods
vm,2
The lsraeis-Keller method
2
i
Both/H
i
i
I~1
Eeme Ket~amm B~x'mele Fle~o~ ~
Mo~iik
NOP
Wierle Irmeer
Figure 1. Nine nuclear power plants (Hinloopen-Nijkamp regime method, four variants).
i
~ ~
i [~
Numericalinterpmtotio~t method
[~
Isroele- K e l l e r
Hk~lOOl~n-NijkomO
['~
Ordinary electricity
The starting point of this approach (see Israels and Keller [4]) is that behind the observed ordinal priorities of criteria unknown cardinal weights are assumed. In our brief illustrative description it is assumed that the criteria are ranked according to diminishing priority. Fundamental for this approach is a set of cardinal weights, called the extreme weight set. If we have three criteria ranked according to diminishing priorities, the extreme weight set consists evidently of the following three sets 1
1/2
0
0
0
1/3 1/3 1/3
'07 90
"~,e°1 N 4o
PI i
Both/H
i
aorMele
Eem
i
i
i
i
i
Ke~ MoerdlJk Wlerln Ivrmoer Flevo-Noord klooe~dclk~ NOP
Figure 2. Nine nuclear power plants (four methods compared for variant (i)).
298
An alternative is assumed to dominate another one if and only if it does not lose any comparison on the basis of the extreme weight sets and gains at least one. The underlying theoretical idea of this method is that each weight set that places the criteria in a correct sequence is a linear combination of the extreme weight sets. A consequence of this idea is that if an alternative dominates another one on the basis of the extreme weight set, it dominates the other one for all weight sets that place the criteria in the correct sequence. The numerical calculations start by comparing each alternative with respect to each of the other ones. Then a square matrix is constructed, the so-called dominance matrix. The elements of this matrix are either
ENERGY ECONOMICS
October 1989
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et ai 1 or 0. The element (ij) equals one if alternative i dominates alternative j. Otherwise the element is zero. Summation of the elements of row i gives the number of alternatives dominated by i. The sum of the elements of column i gives the number ofelements that dominate i. The difference between these two sums is the score of alternative i. Finally the alternatives are ranked according to their score.
The Hinioopen-Nijkamp method Like in the lsraels-Keller method, regimes are here also constructed by means of a pairwise comparison of the ordinal scores of the alternatives. It is furthermore assumed that the importance of criteria differs, so that the criteria can be ordered in a series with decreasing importance. The priority of the criteria is assumed to be measured on an ordinal scale as a consequence of lack of knowledge. It is assumed that behind the soft (ordinal) weights hard but unknown criteria weights do exist. The Israels-Keller and Hinloopen-Nijkamp methods are similar up to this point. A further point of similarity between the two methods is that both use probability concepts to model lack of knowledge. But here the similarity ends. Hinloopen and Nijkamp translate 'lack of prior knowledge' on the state of a system by 'equally probable'. 'In other words, the weight vector w (the set of numbers representing the hard weights) can adopt with equal probability each value that is in agreement with the ordinal information implied by w* (the set of numbers representing the ranking of the weights)'. This argument is essentially based on the 'principle of insufficient reason', which also constitutes the foundation stone for the so-called Laplace criteflon for 'decision making under uncertainty' (see Taha [7]). The computations are commenced by computing the regime matrix. We shall clarify the rest of the computations by means of an example. A full explanation of the method is to be found in the literature referred to above. Suppose the regime + + - - , which is constructed by comparing the scores of alternative I and II. We know that these symbols belong to criteria of diminishing importance, because that is the way the criteria are sorted. Therefore wl > We > w3 > w4, in which the wi are the unknown cardinal weights. Alternative I is now defined to be 'more important' than alternative II ff the sum of the cardinal weights of those criteria for which alternative I shows a higher score than II is greater than the sum of the cardinal weights of the criteria for which I scores lower than II. In our example I is more important than II if wl + w2 > w3 + w,. Here this equality has probability
ENERGY E C O N O M I C S
October 1989
one because of the fact that the criteria are ranked by diminishing priority which implies that the inequality is Itrue. T h e situation is more ambiguous for other regimes. Suppose the regime + - - +. Here it is not certain which alternative 'wins'. In fact the probability of anyone of both winning equals 1/2. The rest of the method can be summarized as follows. Each alternative is confronted with each of the other alternatives. In this way we obtain n times (n-l) regimes where n is the number of alternatives. Then the probability is computed that - given each regime- the unknown weights are such that alternative i is more important than the other member of the pair. This leads to n - 1 probabilities. The average of these probabilities is the probability Pi alternative i wins a random pairwise comparison. The alternatives are then ranked according to diminishing values of their Pi- The alternative with the highest value of pi is considered to be the 'most important' alternative. Many options exist to make the method more flexible. One of these options is that ties in the sequence of priorities can be incorporated.
The numerical interpretation method This method is much simpler, philosophically as well as mathematically, than the other two. This method too is based on the pairwise comparison of the alternatives but here the resemblance stops. Take for example two alternatives A 1 and A2. Select all possible pairs of two criteria, eg C1 and C2, Suppose the priorities of these criteria are wl and w2. The scores of A1 on these criteria are e(1,1) and e(1,2) and those of A2 are e(2,1) and e(2,2) respectively. Then A1 wins one point in the comparison with A2 in 11 situations, viz: e(1,1) > e(1,1) > e(1,1) = e(1,1) > when e(1,1) <
e(2,1) e(2,1) e(2,1) e(2,1)
and e(1,2) > e(2,2), or when and e(1,2) = e(2,2), or when and e(1,2)> e(2,2), or when and e(l,2) < e(2,2) and wl > w2, or
e(2,1) and e(1,2) > e(2,2) and wl < w2
A1 and A2 score equal when e(1,1) = e(2,1) and e(1,2) = e(2,2), or when e(1,1) > e(2,1) and e(l,2) < e(2,2) and wl = w2 or when e(1,1) < e(2,1) and e(1,2) > e(2,2) and wl = w2 A1 loses in the remaining 11 cases. The final result for each alternative is the algebraic sum of its score points. The appealing feature of this
299
A sensitivity analysis ofmalticriteria choice method~: J,A. Hartog et al
method is its simplicity. Of course the method has some disadvantages. The most important one is that the method is not able to process cardinal data without loss of information, a disadvantage the method has in common with the Israels-Keller method; it is evident that this information may cause differences in actual practice. In Albers [1] it is shown that when the priorities of various criteria are the same, the numerical interpretation method and the Hinloopen-Nijkamp regime method (which method has the option of unknown priorities - an assumption more or less similar to the case of equal priorities) produce similar scores. Differences in the results of the methods become evident after the introduction of the assumption of different priorities. Then the results indicate that the numerical interpretation method assigns the least importance to differences in priorities, whilst the Hinloopen-Kijkamp method attaches more and the Israels-Keller method most importance to these differences (a result which is of course intuitively plausible).
300
References 1 L.H. Albers, Het Gewichtloze Gewogen; cultuurhistoris. che betekenis van landgoederen go~valueerd met behulp van multicriteria analyse, Delftse Universitaire Pets, Delft, 1987. 2 E. Hinloopen, P. Nijkamp and P. Rietveld, 'Qualitative discrete multiple criteria choice models in regional planning,' Regional Science and Urban Economics, Vol 13, 1983, pp 77-102. 3 E. Hinloopcn and P. Nijkamp, 'Qualitative multiple criteria choice analysis; the dominant regime method', Quality and Quantity, 1989, forthcoming. 4 A. Z Israels and W. J. Keller, 'Multicriteria analyse voor ordinale data', Kwantitatieve Methoden, Voi 7, No 22, 1986, pp 49-74. 5 P. Nijkamp, 'Culture and region', Environment and Planning, Vol 15, No 1, 1988, pp 5-14. 6 RARO (Raad van Advies voor de Ruimtelijke Ordening), Vestigingsplaatsen voor kerncentrales, Staatsuitgeverij, The Hague, 1986. 7 H.A. Taha, Operations Research, MacMillan, New York, 1976. 8 H. Voogd, Multicritieria Evaluation for Urban and Regional Planning, Pergamon, London, 1982.
ENERGY ECONOMICS
October 1989
Three classes of qualitative multicriteria methods are dealt with in this paper. An empirical problem, the choice of the best site for a nuclear power plant, is used to investigate the extent to which the results of the analysis are influenced by the choice of specific methods. The conclusion is that a certain sensitivity is unavoidable but that the final ranking of the alternatives - on the basis of a set of good methods is nevertheless reasonably stable. Keywords: Multicriteriaanalysis;Nuclearpower plants; Robustness
Plan and project evaluation is an essential part of all efficiency-orientedpolicy-making. However, as long as no unambiguous efficiency criteria are formulated and accepted by everybody, policy analysis most often is a form of conflict analysis in which dissimilar considerations have to be envisaged simultaneously. The latter situation especially has led to the rise of so-called multicriteria methods. Multicriteria methods have been applied repeatedly for complex planning problems in many countries. Many applications can be found in the field of physical planning and environmental management (see Voogd
[82). All these plans and projects have in common that they have to reconcile several conflicting objectives. Many of these studies were complicated by an additional difficultyin that they had to incorporate effects that are difficult to quantify. Clearly at the moment a
P. Nijkamp is with Vrije Universiteit, Faculteit der Economische Wetenschappen, Postbox 7161, 1007 MC, Amsterdam. A. Hartog is an emeritus professor of Erasmus University Rotterdam. E. Hinloopen is associated with the NMB Bank, Amsterdam. Final manuscriptreceived 10 November 1988.
great variety of multicriteria methods is in existence, but there is only a limited number that is able to treat ordinal, lexicographical or binary information in a logically consistent manner. Some decision problems have a mixed character and deal with some effects that are 'hard' while other effects are only measurable on a lower scale. There are only a few methods that are able to cope with this class of problems. This paper is mainly concerned with the latter class of problems. The sensitivity of the results when applying different 'qualitative' evaluation methods will be investigated on the basis of an actual policy choice problem in the Netherlands, viz the problem of locating a new nuclear power plant.
Three methods for multicriteria analysis Many statistical methods are in use to analyse problems that are in essence marked by a choice between several discrete alternatives. In this paper we will only deal with three of these methods: the HinloopenNijkamp regime method, the Isracls-Kcller regime method and the numerical interpretation method. The first method has been described in the international literature various times (see Hinloopen et al [2]; Hinloopcn and Nijkamp [3]; and Nijkamp [5]). The other two arc not published in English as far as we
0140-9883/89/040293-08 © 1989 Butterworth & Co (Publishers) Ltd
293
A .sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al
know. In the appendix we include a short description of the three methods. It requires some explanation why when choosing three methods of multicriteria analysis we selected two methods which at first sight are rather similar, ie the Israels-Keller and Hinloopen-Nijkamp regime methods. These two methods have indeed a lot in common. The basis of both methods is a pairwise comparison of alternatives and both distill out of each comparison a regime, that is a vector consisting of pluses and minuses (or ones and zeros) as explained in the Appendix. Both methods use the assumption that the ordinal character of priorities and effects is caused by lack of precise knowledge. The ordinal measurement of phenomena may gradually be transformed into cardinal information with increasing knowledge. A wellknown example, that is often quoted in this context, is the concept of temperature. In the early days of physics the only observation on temperature was warmer or colder. Temperature was only measurable on an ordinal scale. With progress in physics, however, it became possible to measure temperature on a ratio scale. The advance of science expresses itself in an improvement of measurableness so goes the argument. This point of view is of course not generally accepted, and certainly not in the humanities in which latent and fuzzy concepts like the beauty of a literary style are often used. It seems to us that a reasonable point of departure in this context is that under certain circumstances it is not unreasonable to accept that behind the ordinal character of a variable a cardinal measurability is hidden and that under other circumstances this is not the case. The borderline is often not so difficult to determine, however. The peculiarity of the temperature example is that in general everybody who has to determine which one of two objects is the warmest will come to the same conclusion. In all circumstances where this is the case it seems reasonable to suppose that the character of the variable is in essence a cardinal one. In such a case it is often not difficult to devise a unit of measurement into which the magnitude of the variable can be expressed. Insofar as it is not reasonable to suppose that everybody will have the same opinion, it does not seem reasonable to suppose a cardinal background behind ordinal data. Finally a common element of both above-mentioned methods is that they manage the uncertainty element by means of the calculus of probability and translate 'lack of knowledge' into a rectangular probability distribution. Both methods, however, do also show differences in essential aspects. The Israels-KeUer method translates the concept of dominance in such a way that the 294
method has a somewhat lexicographic character. 'A consequence of this trait is that an alternative that does not score strongly on an important criterion is not likely to come out on top irrespective of its perhaps high - score on less important criteria. In certain circumstances this can analytically be an attractive property, but there exist differences of opinion on whether the restricted possibility of compensation in a lexicographic ordering is not too severely restrictive. However, what is of importance here is that it is reasonable to expect that the results of the HinloopenNijkamp method might differ from those of the Israels-Keller method. This expectation is based partly on a comparison of both methods in another context (Albers [1]) where it was shown on the basis of an extensive dataset that the Israels-Keller method punishes weakness in an important criterion mercillessly.
Choice of location of nuclear power plants The Dutch government presented a first memorandum on the location of nuclear reactors to parliament on 11 January 1985. The Advisory Council for Physical Planning (RARO) used this report as a basis for extensive discussion. The RARO also submitted a report in which multicriteria methods were extensively used (RARO [6]). This report did not discuss the problem of whether nuclear reactors ought to be built, but only dealt with the problem of where a nuclear reactor should be built if it was decided to increase Dutch nuclear capacity. The RARO constructed an extensive list of potential sites which was finally reduced to nine alternative locations in the Netherlands. These locations are Bath/Hoedekenskerke(1), Borssele(2), Eems(3), Flevonoord(4), Ketelmeer(5), Maasvlakte(6), Moerdijk(7), Noordoostpolder (NOP)(8) and Wieringermeer(9). Fifteen relevant criteria were distinguished to structure the judgment process: population density (A), facilities for evacuation/emergency assistance (B), pollution of agricultural soil (C), threat to industrial areas (D), probability of polluting drinking water (E), quantitative availability of cooling water (F), quality cooling water (G), decrease existing air pollution (H), potential use of waste heat (I), indirect claims on scarce land (J), effects on the landscape (K), consequences for the environment (L), linkage to the existing energy network (M), presence of physical infrastructure (N) and suitability for a coal-fired power station(O). In the RARO report each location was evaluated for each criterion. Apart from criterion A, only ordinal information was used. These ordinal data, however, do not provide a complete ranking, but only a partial ENERGY ECONOMICS
October 1989
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al
one in sequential classes. The data therefore, contain many ties. The information on criterid~ ~A (weighted population density) on the other hand was included as (standardized) cardinal information. Criterion O (suitability of a location for coal-fired plant) deserves some special attention. The RARO started from the assumption that in time it would be necessary to assign new locations for the use of coal-fired installations. They were, however, of the opinion that a strong concentration of power has so many disadvantages that the fact that a nuclear reactor was situated at a site that was also suitable for a coal-fired station might preclude the establishment of such an energy source at a future data. This constitutes the background of the procedure to rate a location that is eminently suitable for the use of coal in a reversed way (ie low) as a location for a nuclear reactor. Clearly possible advantages of large-scale energy production units are not taken into consideration here. This raises the necessity to determine also the suitability of a site as a location for a coal-fired power station. To solve this problem a separate multicriteria analysis was first executed (MCA-coal). The results of this analysis were used as input for the analysis of the multicriteria analysis for nuclear reactors (MCAnuclear). As potential locations for MCA-coal, locations (1)-(9) were used as well as the following additional locations: Ijmuiden(10), Velsen(ll), Hemweg(12), Rotterdam(13), Nijmegen(14), Diemen(15) and Merwede/Dordreeht(16). 1 The following criteria were used in the MCA-eoal: cooling water (quantitative) (a), effect on landscape (b), linkage to the energy network structure (c), accessibility for transport of coal (d), distance from urban conglomerations (e), distance from agricultural and horticultural areas (f), distance from natural and recreational areas (g) and an agglomeration effect (h). All these data were measured on an ordinal scale. The effect matrices of MCA-coal and MCA-nuclear are presented in Table 1 and 2, respectively. In all cases the convention has been adopted that higher rates indicate a more preferred value: the higher the better. The RARO distinguished four different policy variants by introducing four sets of weights to indicate the importance of the criteria. These four variants are: (i) base variant; (ii) financial-economic variant; (iii) environmental variant; and (iv) spatial variant. Un-
1One might question whether it is sensible to take additional coal-fired plants into consideration because only the ranking of (1)-(9) is of importance for MCA-nuclear. The only analytically interesting reason for this procedure is to be found in the so-called 'Independence of Irrelevant Alternatives' (IIA) discussion, a problem well known in the discrete choice analysis. There it is shown that the final ranking of a set of alternatives may change if new, irrelevant alternatives are included.
ENERGY E C O N O M I C S
October 1989
Table 1. Input data for MCA-coal. Location Criterion
(a) (b) (c) (d) (e) (0 (g) (h)
12345678910111213141516
344444144 4 2 133113311 3 3 332333321 2 3 355215311 3 4 323333333 2 2 112112211 2 1 3 3 13 3 3 3 3 3 2 3 1 2 2 1 12 2 1 1 2 2
2 3 3 4 2 2 3 1
2 3 2 4 1 2 3 2
2 3 3 4 1 1 3 1
2 2 3 3 2 2 2 1
1 3 3 4 1 1 3 1
Table 2. Input data for IdCA.-nuclear. Location Criterion A AI" B C D E F G H I J K L M N
1
2
3
4
5
6
49 51 84 73 70 57 2 2 4 3 3 2 2 1 2 2 2 1 2 2 2 2 2 3 5 2 3 4 5 1 2 2 2 1 1 2 2 3 3 3 3 3 2 3 2 1 1 3 1 1 1 1 1 2 1 2 2 2 I 3 3 2 2 3 4 3 1 3 3 3 1 3 1 3 1 3 2 3 2 2 1 3 3 2 1 2 2 1 1 2
7
8
9
0 81 79 1 4 4 2 2 2 1 2 2 3 5 5 1 1 1 1 3 3 2 1 1 I 1 1 2 1 1 1 4 4 2 1 1 3 3 3 2 2 1 2 1 1
"A1 represents the ordinal ranking constructed on the basis of criterion A after transforming the cardinal information into ordinal information. bThe ranking in the last row has to be computed on the basis of the results of the MCA-coal per variant (see Table 5).
fortunately the RARO report is not very clear as to the special characteristics of the base variant. This is inconvenient because of the fact that in this variant the weights of the criteria that are used in MCA-coal as well as in MCA-nuclear (the pairs (a) and (F), (b) and (K), and (c) and (M)) are not exactly equal, as can be seen in Tables 3 and 4. Here too the convention, the higher the better, has been adopted.
Results of the MCA In the above mentioned RARO report a stepwise MCA was executed: first MCA-coal and then MCA-nuclear. The method used was the Qualiflex-method developed by the Netherlands Economic Institute (Rotterdam) while a check on the basis of an ordinal concordance analysis was also carried out. It was a pity that by using the Qualiflex method one or more of the 15 criteria had to be left out of consideration because for technical reasons the Qualiflex computer program can
295
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et al Table 3. Rankings of criteria per variant for MCA-COUl. Criterion (a) Variant (i) (ii) (iii) (iv)
(b)
(c)
(d)
(e)
(t)
Table 6. Results per variant for MCA-nuclear.
(g)
(h)
3 5 6 2
2 2 2 5
1 5 1 5
45 4 3
5 4 5 4
2 4 2 1
3 3 6 2
3 4 3 2
(i) (ii) (iii) (iv)
A B C D E F G H
I
5 5 5 5
2 1 1 1 1 1 4 1 1 1 4 4 2 1 1 4 1 1 2 4 4 1 4 4
1 1 1 1
3 4 3 3
4 4 4 4
3 2 2 1 3 4 2 4 3 4 4 2 3 2 2 1
J K L M N O 2 4 2 4
Table 5. Results per variant for MCA..cnaL Ordinal concordance
Qnaliflex Location Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer
Variant i ii 5 2 3 6 7 1 4 8 9
5 2 3 4 7 1 6 8 9
iii
iv
8 3 9 4 5 1 2 6 7
5 2 4 6 7 1 3 8 9
Variant i ii 6 3 2 5 7 1 4 8 9
6 2 3 5 7 1 4 8 9
iii
iv
9 3 4 5 6 1 2 7 8
6 2 4 5 7 1 3 8 9
"Only the nine relevant locations for MCA-nuclear arc included here. These results constitute the input for criterion 0 in Table 2; a low value in this table indicates that the location is not suitable for a coal-fired plant and therefore very suitable from this point of view for a nuclear power station.
not manage more than 14 criteria. This serious restriction does not hold in the case of the ordinal concordance analysis, although this method suffers from some methodological impurities as regards the use of ordinal weights. Furthermore, in this case the quantitatively measured criterion A had to be transformed into a ranking with some loss of information as a consequence. The results of both methods (ie Qualiflex and ordinal concordance), distilled with some effort from the RARO report, are presented in Table 5 (MCA-coal) and Table 6 (MCA-nuclear) for all four variants. A striking characteristic of these results is the lack of robustness of the final ranking of the locations with respect to changes in the variants and the variance of the individual results between the Qualiflex-method and the ordinal concordance analysis, be it that the overall pattern is reasonably similar. This sensitivity
296
Variant i ii
Location
Table 4. Rlmkings of criteria per variant for MCA-nuclear. Criterion Variant
Ordinal concordance
Qnaliflex
Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer
2 4 9 5 3 8 1 7 6
2 3 5 8 7 9 1 6 4
iii
iv
1 4 7 6 3 9 2 8 5
2 3 7 9 8 6 i 5 4
Variant i ii 2 4 9 5 3 7 1 8 6
2 3 6 7 5 9 1 8, 4
iii
iv
2 6 7 4 3 9 1 8 5
2 3 5 7 6 9 1 8 4
is partly connected with the fact that the input data are not purely ordinal, but represent in essence a classification in size classes or relevance categories. Having described now the main elements of the choice procedure, it is no doubt an interesting exercise to investigate the extent to which the above results are confirmed by means of other methods. As mentioned before, we have chosen three methods for a robustness test: the Israels-Keller regime method, the Hinloopen-Nijkamp regime method and the numerical interpretation method. It has to be noted that from these three methods only the Hinloopen-Nijkamp regime method is able to deal with cardinal and mixed information (the 'mixed' option). This is especially important for criterion A in our MCA-nuclear. The results of these tests are discussed below. R o b u s t n e s s of three M C A ' s The results of Israels-Keller regime method (Regime I-K), the Hinloopen-Nijkamp regime method (Regime H-N) and the numerical interpretation method (Num Inter) will be presented and discussed. The outcomes for MCA-coal and MCA-nuclear will be given for each of the four variants mentioned above. The results for MCA-coal are to be found in Table 7. The complete set of alternatives, 16 in total, is used in the derivation of these results. The ranking of only locations 1 to 9 is used as input for criterion O in the computation of MCA-nuclear. Our computations showed that the IIA-problem, alluded to in footnote 1, was indeed present. If the nine relevant locations were selected from the beginning, the resulting ranking was sometimes slightly different from the one obtained when using the complete set of 16 locations. Because of the reasons mentioned above we nevertheless opted for the complete set. The main conclusion to be derived from Table 7 is that for each of the four variants separately the three
ENERGY ECONOMICS
October 1989
,4 sensitivity analysis of multicriteria choice methods." J.A. Hartog et al Table 7. Results per variant for MCA-coal for ~
methods. Regime H - N
Regime I-K Location Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Moerdijk NOP Wieringermeer Ijmuiden Velsen Hemweg Rotterdam Nijmegen Diemen Merwede/D
-
Variant i ii 9 14 15 12 6 16 13 5 4 8 11 10 7 3 2 1
9 15 14 13 12 16 7 5 4 8 11 10 6 3 2 1
iii 8 15 14 13 7 16 6 5 4 12 10 9 11 3 2 1
Numerical iatorl~tafion
iv
Variant i fi
fii
iv
Variant i fi
fii
iv
8 15 12 9 4 16 13 2 1 7 14 11 6 10 3 5
11 14 15 7 6 16 13 5 4 10 12 8 9 3 2 1
8 15 14 13 7 16 ~ 5 4 12 10 9 11 3 2 1
8 15 13 5 3 16 14 2 1 7 12 11 6 10 4 9
7 14 14 8 6 16 13 5 3 9 12 11 10 4 1 2
6 15 14 I0 9 16 13 5 4 7 12 11 8 3 1 2
8 15 13 9 5 16 14 2 1 6 12 11 10 7 3 4
7 14 15 8 6 16 10 2 1 11 13 12 9 5 4 3
7 15 14 9 6 16 13 2 1 10 11 12 8 5 4 3
MCA-nuclear (see Table 8). These results again show a reasonable similarity in outcomes of the computations of the three methods. The correlation of the results of two arbitrary methods is always extremely high. Nevertheless it is undeniable that the methods also show clear differences connected with their structure. The Israels-Keller regime method for instance never indicates Maasvlakte as the best location. The numerical interpretation method chooses Maasvlakte as the best alternative in three out of the four cases, whilst the Hinloopen-Nijkamp regime method chooses Maasvlakte as the best location for two variants. The cause of these differences is to be found in the fact that the Maasvlakte location scores poorly on the criterion population density, which happens to be the criterion that gets the highest priority in all four variants. The somewhat lexicographic character of the Israels-Keller method makes it very difficult to compensate weakness in such an important criterion by strength in other criteria. The method gives the place of honour in three
methods under consideration did produce rather similar results, ie rather similar rankings of the alternatives. Within variant 1, for instance, the locations Maasvlakte and Eems are identified as the best ones in each of the three methods, and the locations Diemen and Merwede/Dordrecht as the worst ones. Similar patterns can be found in the other variants. This leads to the conclusion that the results are rather robust as regards the choice of one of these three methods. Only in the middle region can shifts in rankings be observed more frequently. The choice of variants on the other hand does seem to exert a definite influence on the ranking of alternatives, although the Maasvlakte is the best location for a coal-fired power station in each variant. Comparison with Table 5 shows that some differences do exist, especially with respect to the Qualiflex-method, although these differences certainly cannot be called dramatic. Next the results of each of the four variants and each of the three methods are used as input for
Table 8. Results per variant for MCA-nudear for three methods.
l_~.adon
Regime I-K
Regime H - N
Variant i ii
Variant i ii
2 3 9 5 4 7 1 8 6
iii
iv
2 4 8 6 3 7 1 9 5
2 3 6 5 4 8 1 9 7
Bath/H Borssele Eems Flevo-Noord Ketelmeer Maasvlakte Mocrdijk NOP Wieringermeer
2 5 7 4 3 8 1 9 6
ENERGY ECONOMICS
October 1989
1 3 9 5 4 6 2 8 7
2 3 8 6 4 9 I 7 5
Numerical iatorl~tatioa iii
iv
2 4 8 5 3 9 1 7 6
2 3 7 6 4 8 1 9 5
Variant i ii 2 4 6 7 3 9 1 8 5
2 3 6 8 5 9 1 7 4
iii
iv
2 4 7 6 3 9 1 8 5
2 3 6 9 5 8 1 7 4
297
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog
et al
of the four variants to the N O P alternative and to the Eems alternative in the financial-economic variant. Figure 1 illustrates the influence of the choice of the priorities on the final results of the HinloopenNijkamp method, whilst Figure 2 pictures the method sensitivity in the case of variant (i). If we include in the analysis the results of the Qualiflex and the Ordinal concordance methods from Table 6, it becomes obvious that the results do not differ extremely from those of the three methods of Table 8, although the differences are larger than those between the three methods we used. Such a robustness strengthens the confidence in such methods in any case.
sen voor Kerncentrales [6]) to select provisionally three
Concluding remarks After a long discussion the Dutch government has decided (Regeringsbeslissing (deel D), Vestigingsplaat-
locations as a possible site for a nuclear power plant. These places are: Borssele, Eems and the Maasvlakte. The choice of Maasvlakte is obvious. In many cases-10 out of 20-this site is the best one. The choice of Eems too is a plausible one. This location comes out on top in four cases: within variant (i) by the Qualiflex, Ordinal concordance and Hinloopen-Nijkamp methods and within variant (ii) by the Israels-Keller method. The choice of Borssele, however, cannot be defended on the basis of the criteria and priorities used by the RARO. Furthermore it is interesting to note that the location Moerdijk-strongly propagated for regional interests by the province of Brabant-is in none of the variants an acceptable or defensible option.
Appendix
IOC t~
[ ~
8C ~
tmr.I
Brief description of three multicriteria methods
vm,2
The lsraeis-Keller method
2
i
Both/H
i
i
I~1
Eeme Ket~amm B~x'mele Fle~o~ ~
Mo~iik
NOP
Wierle Irmeer
Figure 1. Nine nuclear power plants (Hinloopen-Nijkamp regime method, four variants).
i
~ ~
i [~
Numericalinterpmtotio~t method
[~
Isroele- K e l l e r
Hk~lOOl~n-NijkomO
['~
Ordinary electricity
The starting point of this approach (see Israels and Keller [4]) is that behind the observed ordinal priorities of criteria unknown cardinal weights are assumed. In our brief illustrative description it is assumed that the criteria are ranked according to diminishing priority. Fundamental for this approach is a set of cardinal weights, called the extreme weight set. If we have three criteria ranked according to diminishing priorities, the extreme weight set consists evidently of the following three sets 1
1/2
0
0
0
1/3 1/3 1/3
'07 90
"~,e°1 N 4o
PI i
Both/H
i
aorMele
Eem
i
i
i
i
i
Ke~ MoerdlJk Wlerln Ivrmoer Flevo-Noord klooe~dclk~ NOP
Figure 2. Nine nuclear power plants (four methods compared for variant (i)).
298
An alternative is assumed to dominate another one if and only if it does not lose any comparison on the basis of the extreme weight sets and gains at least one. The underlying theoretical idea of this method is that each weight set that places the criteria in a correct sequence is a linear combination of the extreme weight sets. A consequence of this idea is that if an alternative dominates another one on the basis of the extreme weight set, it dominates the other one for all weight sets that place the criteria in the correct sequence. The numerical calculations start by comparing each alternative with respect to each of the other ones. Then a square matrix is constructed, the so-called dominance matrix. The elements of this matrix are either
ENERGY ECONOMICS
October 1989
A sensitivity analysis ofmulticriteria choice methods: J.A. Hartog et ai 1 or 0. The element (ij) equals one if alternative i dominates alternative j. Otherwise the element is zero. Summation of the elements of row i gives the number of alternatives dominated by i. The sum of the elements of column i gives the number ofelements that dominate i. The difference between these two sums is the score of alternative i. Finally the alternatives are ranked according to their score.
The Hinioopen-Nijkamp method Like in the lsraels-Keller method, regimes are here also constructed by means of a pairwise comparison of the ordinal scores of the alternatives. It is furthermore assumed that the importance of criteria differs, so that the criteria can be ordered in a series with decreasing importance. The priority of the criteria is assumed to be measured on an ordinal scale as a consequence of lack of knowledge. It is assumed that behind the soft (ordinal) weights hard but unknown criteria weights do exist. The Israels-Keller and Hinloopen-Nijkamp methods are similar up to this point. A further point of similarity between the two methods is that both use probability concepts to model lack of knowledge. But here the similarity ends. Hinloopen and Nijkamp translate 'lack of prior knowledge' on the state of a system by 'equally probable'. 'In other words, the weight vector w (the set of numbers representing the hard weights) can adopt with equal probability each value that is in agreement with the ordinal information implied by w* (the set of numbers representing the ranking of the weights)'. This argument is essentially based on the 'principle of insufficient reason', which also constitutes the foundation stone for the so-called Laplace criteflon for 'decision making under uncertainty' (see Taha [7]). The computations are commenced by computing the regime matrix. We shall clarify the rest of the computations by means of an example. A full explanation of the method is to be found in the literature referred to above. Suppose the regime + + - - , which is constructed by comparing the scores of alternative I and II. We know that these symbols belong to criteria of diminishing importance, because that is the way the criteria are sorted. Therefore wl > We > w3 > w4, in which the wi are the unknown cardinal weights. Alternative I is now defined to be 'more important' than alternative II ff the sum of the cardinal weights of those criteria for which alternative I shows a higher score than II is greater than the sum of the cardinal weights of the criteria for which I scores lower than II. In our example I is more important than II if wl + w2 > w3 + w,. Here this equality has probability
ENERGY E C O N O M I C S
October 1989
one because of the fact that the criteria are ranked by diminishing priority which implies that the inequality is Itrue. T h e situation is more ambiguous for other regimes. Suppose the regime + - - +. Here it is not certain which alternative 'wins'. In fact the probability of anyone of both winning equals 1/2. The rest of the method can be summarized as follows. Each alternative is confronted with each of the other alternatives. In this way we obtain n times (n-l) regimes where n is the number of alternatives. Then the probability is computed that - given each regime- the unknown weights are such that alternative i is more important than the other member of the pair. This leads to n - 1 probabilities. The average of these probabilities is the probability Pi alternative i wins a random pairwise comparison. The alternatives are then ranked according to diminishing values of their Pi- The alternative with the highest value of pi is considered to be the 'most important' alternative. Many options exist to make the method more flexible. One of these options is that ties in the sequence of priorities can be incorporated.
The numerical interpretation method This method is much simpler, philosophically as well as mathematically, than the other two. This method too is based on the pairwise comparison of the alternatives but here the resemblance stops. Take for example two alternatives A 1 and A2. Select all possible pairs of two criteria, eg C1 and C2, Suppose the priorities of these criteria are wl and w2. The scores of A1 on these criteria are e(1,1) and e(1,2) and those of A2 are e(2,1) and e(2,2) respectively. Then A1 wins one point in the comparison with A2 in 11 situations, viz: e(1,1) > e(1,1) > e(1,1) = e(1,1) > when e(1,1) <
e(2,1) e(2,1) e(2,1) e(2,1)
and e(1,2) > e(2,2), or when and e(1,2) = e(2,2), or when and e(1,2)> e(2,2), or when and e(l,2) < e(2,2) and wl > w2, or
e(2,1) and e(1,2) > e(2,2) and wl < w2
A1 and A2 score equal when e(1,1) = e(2,1) and e(1,2) = e(2,2), or when e(1,1) > e(2,1) and e(l,2) < e(2,2) and wl = w2 or when e(1,1) < e(2,1) and e(1,2) > e(2,2) and wl = w2 A1 loses in the remaining 11 cases. The final result for each alternative is the algebraic sum of its score points. The appealing feature of this
299
A sensitivity analysis ofmalticriteria choice method~: J,A. Hartog et al
method is its simplicity. Of course the method has some disadvantages. The most important one is that the method is not able to process cardinal data without loss of information, a disadvantage the method has in common with the Israels-Keller method; it is evident that this information may cause differences in actual practice. In Albers [1] it is shown that when the priorities of various criteria are the same, the numerical interpretation method and the Hinloopen-Nijkamp regime method (which method has the option of unknown priorities - an assumption more or less similar to the case of equal priorities) produce similar scores. Differences in the results of the methods become evident after the introduction of the assumption of different priorities. Then the results indicate that the numerical interpretation method assigns the least importance to differences in priorities, whilst the Hinloopen-Kijkamp method attaches more and the Israels-Keller method most importance to these differences (a result which is of course intuitively plausible).
300
References 1 L.H. Albers, Het Gewichtloze Gewogen; cultuurhistoris. che betekenis van landgoederen go~valueerd met behulp van multicriteria analyse, Delftse Universitaire Pets, Delft, 1987. 2 E. Hinloopen, P. Nijkamp and P. Rietveld, 'Qualitative discrete multiple criteria choice models in regional planning,' Regional Science and Urban Economics, Vol 13, 1983, pp 77-102. 3 E. Hinloopcn and P. Nijkamp, 'Qualitative multiple criteria choice analysis; the dominant regime method', Quality and Quantity, 1989, forthcoming. 4 A. Z Israels and W. J. Keller, 'Multicriteria analyse voor ordinale data', Kwantitatieve Methoden, Voi 7, No 22, 1986, pp 49-74. 5 P. Nijkamp, 'Culture and region', Environment and Planning, Vol 15, No 1, 1988, pp 5-14. 6 RARO (Raad van Advies voor de Ruimtelijke Ordening), Vestigingsplaatsen voor kerncentrales, Staatsuitgeverij, The Hague, 1986. 7 H.A. Taha, Operations Research, MacMillan, New York, 1976. 8 H. Voogd, Multicritieria Evaluation for Urban and Regional Planning, Pergamon, London, 1982.
ENERGY ECONOMICS
October 1989