Multi-Criteria Decision Making techniques for the management of public procurement tenders: A case study

Journal Pre-proof Multi-Criteria Decision Making techniques for the management of public procurement tenders: A case study Mariagrazia Dotoli, Nicola Epicoco, Marco Falagario

PII: DOI: Reference:

S1568-4946(20)30004-1 https://doi.org/10.1016/j.asoc.2020.106064 ASOC 106064

To appear in:

Applied Soft Computing Journal

Received date : 5 September 2018 Revised date : 20 December 2019 Accepted date : 2 January 2020 Please cite this article as: M. Dotoli, N. Epicoco and M. Falagario, Multi-Criteria Decision Making techniques for the management of public procurement tenders: A case study, Applied Soft Computing Journal (2020), doi: https://doi.org/10.1016/j.asoc.2020.106064. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2020 Published by Elsevier B.V.

*Manuscript Click here to view linked References

Journal Pre-proof

Multi-Criteria Decision Making Techniques for the Management of Public Procurement Tenders: A Case Study

pro of

Mariagrazia Dotolia, Nicola Epicocoa*, Marco Falagariob a Dept. of Electrical and Information Engineering, Politecnico di Bari, Bari, Italy b Dept. of Mechanics, Mathematics and Management, Politecnico di Bari, Bari, Italy {mariagrazia.dotoli, nicola.epicoco, marco.falagario}@poliba.it ABSTRACT

KEYWORDS

urn a

lP

re-

Multi-Criteria Decision Making (MCDM) techniques are mathematical tools that help decision makers evaluating and ranking in an automatic way many possible alternatives over multiple conflicting criteria in highly complex situations. Several MCDM approaches exist, and their application fields are numerous, including the Supplier Selection Problem (SSP), which is an important problem in the management field. The aim of this paper is to perform a comparative analysis among some selected well-known MCDM techniques to show how they can properly support the specific decision making process of Public Procurement (PP) tenders, which is a particular type of the SSP, characterized by very stringent rules, thus requiring a specific assessment. Indeed, PP is a field characterized by the need for transparency, objectivity, and non-discrimination, which requires tendering organizations to explicitly state the adopted awarding method, the chosen decision criteria, and their relative importance in the call for proposals. However, this field has been seldomly investigated in the pertinent literature and thus the aim of this paper is to overcome such a limitation. In particular, this work focuses on the most commonly adopted methods in the field of supplier selection, namely the Analytic Hierarchy Process (AHP), the Preference Ranking Organization METHod for Enrichment of Evaluations (PROMETHEE), the Multi Attribute Utility Theory (MAUT), and the Data Envelopment Analysis (DEA). First, we adapt these techniques to the PP problem and its requirements. Then, by means of some real tenders at a European Institution, the selected techniques are compared with each other and with the currently adopted methodology in their classical deterministic setting, to identify which method best suits the specific requirements of PP tenders. Hence, since nowadays uncertainty is inherent in data from real applications, and can be modeled by expert evaluations through fuzzy logic, the comparison is extended to the fuzzy counterparts of two of the most promising selected approaches, i.e., the Fuzzy AHP and the Fuzzy DEA, showing that these methods can be effectively applied to the PP sector also in the presence of uncertainty on the tenders data.

Jo

Multi-Criteria Decision Making, Public Procurement, Compensatory models. 1. INTRODUCTION

Public Procurement (PP) is the process by which public authorities purchase goods and/or services from the private sector. On average, governments spend more than 30% of their total expenditure on PP [1], hence governments are expected to carry it out efficiently and with high standard levels to safeguard the public interest. The PP process is subject to very specific rules *

Corresponding author: Prof. M. Dotoli, DEI – Politecnico di Bari, Via Re David, 200, 70125 – Bari, Italy. Phone: +39 080 5963667.

1

Journal Pre-proof

Jo

urn a

lP

re-

pro of

and policies covering how the relevant decisions are made, with the final aim of taking advantage from competition between suppliers and reducing corruption risks [2]. Since there are no universally accepted international standards for PP, some differences from country to country can be observed. Nonetheless, there is a common harmonization in terms of basic principles of this sector and on how its functioning can be improved [3]. One of the most fundamental aspects in PP is to ensure that the bidding process and the contract awarding respect the principles of non-discrimination, free competition, and transparency of the procedures. For instance, to create a level playing field for all businesses across Europe, and to prevent fraud, waste, corruption, or local protectionism, the European Union (EU) sets out some harmonized PP rules, which are transposed into each EU member legislation and apply to all those tenders whose monetary value exceeds a certain amount, while for tenders of lower value only the national rules apply, provided the compliance with the general principles of EU law. In particular, in the EU the PP is strictly regulated by Directive 2014/24/EU [4], which states the procedures for the awarding of public works, public supply, and public service contracts, and by Directive 2014/25/EU [5], aimed at increasing the efficiency of public spending, facilitating in particular the participation of small and mediumsized enterprises in PP, and to enable procurers to make better use of PP in support of common societal goals. Both these directives are aimed at ensuring a competitive bidding process by establishing common rules for advertising procurement needs, invitations to tender, and contract award, respecting the recalled principles of non-discrimination, free competition, and transparency in the awarding process. Usually, at the contract awarding stage, all the received offers have been already preliminary evaluated in terms of eligibility and compliance with norms. Hence, decision makers have to award the tender to the best bidder. The recalled European directives suggest some general guidelines on how to properly choose the evaluating criteria and their weights. However, no specific method is suggested for the tenders’ evaluation. To this aim, depending on the specific tenders’ requirements, two approaches are commonly adopted, namely the Lowest Price (LP) and the Most Economically Advantageous Tender (MEAT, also known as best price-quality ratio) criteria. The LP criterion awards the cheapest offer, but abnormally low offers may be rejected by the contracting authority. This criterion is generally suitable only for simple procurements of short-term, low-level services/goods of a standard specification, or when there are mandatory aspects that allow evaluating only the lowest price. On the contrary, the MEAT criterion also takes into account some technical or qualitative aspects. Consequently, the awarding process typically consists in a Multi-Criteria Decision Making (MCDM) method, in which the contracting entities have to explicitly indicate in the call for proposals the chosen criteria and their importance level for the award, so as to respect the recalled principle of transparency of the awarding procedure. In addition, since usually several decision makers are involved in the tenders awarding process, MCDM approaches appear suitable methods to support the decision making process while respecting its regulations and rules. Given the large amount of available techniques, the proper choice of which MCDM method to use is a very difficult task. In general, the choice depends on multiple factors, such as the required effort, desired level of accuracy, computational time, users’ skills and knowledge, preference and assumption of the decision maker(s), and so on. The problem is further complicated by the facts that different MCDM methods often produce different outcomes solution and that some methods may be strongly sensitive to slight variations in the input data. In addition, in the PP case, the proper selection of which MCDM technique to adopt is further complicated by the need to comply with the mentioned principles of non-discrimination, free competition, and transparency of the awarding procedures. Summing up, the proper selection of which tender awarding techniques to adopt is not a trivial task.

2

Journal Pre-proof

pro of

In this context, the aim of this work is to analyze some well-known MCDM techniques, identify the necessary modifications to make them suitable for the specific PP requirements, and compare them on several real tenders so as to select which method best fits the PP requirements. In addition, since uncertainty characterizes the problem, to a greater or lesser intensity, in almost any PP project, the comparison is also extended to the case of uncertain data and vague judgments, by considering the fuzzy versions of the selected approaches. The analysis is conducted by evaluating real tenders from the procurement office of a European Institution (we omit the name due to privacy reason). The work aims at supporting decision makers in the PP sector in properly selecting tenderers under multiple conflicting criteria while ensuring the aforementioned and mandatory rules in the awarding process. The motivation for this investigation is to overcome the lacks encountered in the literature, where, to the best of the authors’ knowledge, no previous work compares several MCDM techniques, from different methodological classes, in the PP field, providing, where needed, the necessary modifications to make them applicable to the PP context. In the following section, through a detailed analysis of the current state of the art, we recall the basic notions on MCDM techniques, detailing the selected methods and their corresponding mathematical formulations. Section 3 provides some applications to the EI tenders and summarizes the main findings of our research.

re-

2. MULTI-CRITERIA DECISION MAKING (MCDM) TECHNIQUES FOR PUBLIC PROCUREMENT (PP)

lP

The PP purchasing problem is a particular supplier selection problem that, in addition to the typical objectives of the private sector (i.e., to acquire the right goods/services, from the right source, at the right price, at the highest service level, and taking into account norms requirements and environmental impact), also requires the compliance with the principles of non-discrimination, free competition, and transparency of the awarding procedures. Hence, in this section we define the problem statements of the selected MCDM techniques for the specific PP context. More precisely, with reference to the main MCDM original techniques, we suitably modify the approaches to cope with the mentioned PP requirements.

Jo

urn a

2.1. Deterministic MCDM approaches for PP MCDM techniques are mathematical tools concerned with structuring and solving decision and planning problems characterized by many alternatives to be evaluated against multiple conflicting criteria [6, 7]. Depending on the characteristics of the decision alternatives, MCDM models can be divided into Multi Attribute Decision Making (MADM) problems, with a predetermined and limited number of alternatives (as is the case of selection problem and PP tenders), or Multi Objective Decision Making (MODM) problems, in which the decision alternatives are not given, but their set is explicitly defined by constraints using multiple objective programming (e.g., in design problems) [8]. The state of the art review highlights that a huge body of publications on MCDM techniques is available (see, e.g., the recent review in [9]). All these approaches have some similarities [10], but some relevant differences and peculiarities can also be observed. Briefly said, two broad classes can be identified [8]: the non-compensatory methods (i.e., when a reduction on the value of a criterion is not compensated by the gain of units on other criteria), which are simple but have limited application, and the compensatory models (i.e., when a reduction in the value of a criterion is somehow compensated by the gain of units on other criteria), which are very popular and can be further divided into four groups [11]: 1) pair-wise comparisons methods; 2) distance based methods; 3) outranking methods; 4) other methods. Pair-wise comparisons methods ([6, 7, 11]) rely on comparing alternatives in pairs to judge which one is preferred on the basis of the chosen evaluating criteria. AHP (Analytic 3

Journal Pre-proof

Jo

urn a

lP

re-

pro of

Hierarchy Process) and ANP (Analytic Network Process) are very popular methods in this category. Distance based (or compromise) methods ([6, 7, 11]), such as VIKOR (VIsekriterijumska optimizacija i KOmpromisno Resenje, i.e., Multicriteria Optimization and Compromise Solution) and TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), consist in calculating the “closeness to” or “distance from” a defined ideal best and worst solutions or reference points. In outranking (or concordance) methods ([6, 7, 11]), the output of the analysis is not a value for each alternative, but rather an outranking relation on their set (i.e., an alternative outranks another alternative if, according to the decision maker’s preferences, there are strong enough arguments to state that it is at least as good as the other alternative and no strong argument to state the contrary). Among these models the most common techniques are MAUT (Multi-Attribute Utility Theory), PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation), and ELECTRE (ELimination Et Choice Translating REality). Finally, the last group includes all the other MCDM methods that do not belong to one of the previous classes and usually adopt mathematical models. In this group, the most commonly adopted techniques are DEA (Data Envelopment Analysis), which is commonly used for benchmarking in operations management and in supplier selection problems (see, e.g., [12]), DEMATEL (DEcision MAking Trial and Evaluation Laboratory), which is an effective method for the identification of cause-effect relations in complex systems (see, e.g., the review in [13]), and OWA (Ordered Weighted Averaging), which is able to deal with multiple stakeholders’ constructive participation in decision-making processes trading-offs between different objectives (see, e.g., the work in [14]). The literature review also highlights that MCDM techniques have been applied in several fields, among which one of the most investigated is the Supplier Selection Problem (SSP) (see, e.g., the reviews in [9, 15, 16]). In effect, the SSP presents some peculiarities that also hold for PP problem, that is, the need to evaluate different predetermined alternatives (i.e., suppliers in the SSP or bidders in the PP problem) under multiple conflicting criteria (typically cost and service level requirements), so as to purchase the right goods/services, from the right source, in the right quantity, at the right time and cost, and with the right quality. Since each MCDM method clearly has its own strengths and weaknesses, many literature contributions propose to integrate some MCDM methods to improve the SSP outcomes (e.g., ANP and TOPSIS [17], DEMATEL and TOPSIS [18], DEA and TOPSIS [19], or ANP and DEA [20]). However, only few works compare with each other the MCDM methods, and usually the comparison is conducted on just few techniques, typically without considering methods belonging to different methodological classes (see, e.g., the work in [21], comparing AHP and DEA with a new method called Operational Competitiveness Rating (OCRA) and that in [22], comparing PROMETHEE with MAUT). In effect, more detailed comparisons among different MCDM techniques exist only for some specific fields (e.g., for seismic structural retrofitting [23], for connecting road selection [24], for sustainable housing affordability [25], and for employee placement [26]), which clearly are characterized by their specific peculiarities and requirements, thus making their findings unsuitable for the PP field. To the best of the authors’ knowledge, no previous contribution compares different MCDM techniques in this particular field, investigating if and how each method is able to guarantee a delicate balancing between the need to achieve the best outcome for the organization and the need to ensure the basic principles of transparency, non-discrimination, equality, and proportionality [27]. In this paper, we overcome this limitation analyzing some MCDM approaches from different methodological classes to identify if they are suitable to ensure the compliance with the PP 4

Journal Pre-proof

Jo

urn a

lP

re-

pro of

requirements, modify them in case they are not, and compare them when applied to real PP tenders both under deterministic and uncertain data. More in detail, taking advantage of the similarities between the SSP and the PP problem, but also considering the mentioned peculiarities of the latter problem, in this work we investigate the application to PP of some selected well-known techniques in the SSP area. With respect to the classification proposed above, we select the most commonly adopted methods from each group. More in detail, the comparison concerns AHP (a pair-wise comparisons method), PROMETHEE and MAUT (both outranking methods), and DEA (a mathematical method). We do not take into account distance based methods (such as TOPSIS) since defining a priori in the call for tenderers (i.e., before having received the offers) the ideal best and worst solutions is non-trivial (particularly in the case of new tenders and for group decision making processes) and too subjective, which may lead to mismatching the non-discrimination requirements of the PP. These techniques are selected in this work since they are the most commonly employed in SSP, which, as mentioned above, has some analogy with respect to PP. As a matter of fact, AHP has been widely applied to MADM problems (see, e.g., the review in [28]) and it is actually the most frequently used methodology to solve the SSP [9] (see, e.g., contributions [29–31] Moreover, although the frequency in using DEA in the SSP literature is decreased in the recent years, DEA is still one of the most commonly adopted mathematical methods in this field [9] (see, e.g., the recent contributions in [12, 32] and the reviews in [15, 33]). In addition, DEA is recognized as a useful tool to examine the efficiency of government service providers [33]. Among the outranking methods, the ELECTRE method uses a straightforward preference relation, whereas both the PROMETHEE and MAUT techniques advance ELECTRE by further using pairwise comparisons [9]. In addition, ELECTRE is not reported in the recent literature on the SSP [9], whereas PROMETHEE has been successfully applied to decision making and supplier selection problems (see, e.g., the review in [16] and the recent contributions in [35, 36]). Similarly, the MAUT technique is commonly used to help decision makers gain further knowledge and understanding of the problem and is often used as an ideal method to solve complex problems in contexts characterized by multiple criteria (such as SSP and PP), particularly when the environment is complicated and risky [36] (such as in the PP field). In the first phase of this research (which lasted twelve months) we analyzed several real tenders at a European Institution (EI) to compare the currently adopted EI method with the selected deterministic MCDM techniques, suitably modified (when needed) to cope with the PP requirements. The final aim of this phase was to identify which method best fits the PP sector. In the second phase we focused on the two most promising techniques (i.e., the AHP and the DEA), also including in the comparison their fuzzy counterparts to take into account the uncertainty that inevitably afflicts the evaluations. We remark that this phase is still occurring to allow a long-term period evaluation and to properly form the EI employees. We also remark that, nowadays, the EI has started to adopt the AHP method. In the next subsections we recall the basics of the selected MCDM techniques. For each method we also highlight the main strengths and weaknesses and we propose some modifications to overcome such drawbacks and make these approaches best fit the PP problem (that is, to ensure the mandatory requirements in terms of transparency, nondiscrimination, equality, and proportionality of the awarding). In addition, since multiple individuals are involved in the decision making process (as these requirements imply), we also provide an analysis on how to take into account this aspect when the MCDM techniques are applied to the tenderers evaluation.

5

Journal Pre-proof 2.1.1. The PP problem statement The PP problem is composed by a finite set of I alternatives (in our case the bidders) denoted as B  b1 ,..., bi ,..., bI  , that are to be evaluated according to N conflicting criteria, denoted as

C  c1 ,..., cn ,...cN  . Such criteria can have different domains, and, referring to the PP sector,

urn a

lP

re-

pro of

they usually include the price offer (which is desirable to minimize) and some quality requirements (to be maximized). The decision makers assign to each criterion an importance weight, whose set is W  w1 ,...wn ,..., wN  and whose sum equals to 1. The objective of the PP problem is thus to automatically determine a performance index vector PI   pi1 ,..., pii ,..., piI  that collects the performance indices pii of each i-th bidder, such that the highest performance index pii singles out the winner of the tender. 2.1.2. The Analytic Hierarchy Process (AHP) for PP Description: The AHP [37] is a MCDM method to derive ratio-scales importance of alternatives from the pair-wise comparisons of criteria and alternatives by means of a weighted sum approach. The classical AHP procedure requires several steps to be iteratively solved (see [37]). Since in the PP sector weights on criteria and sub-criteria are to be preliminarily stated in the call for proposals, the classical AHP procedure can be modified as follows. 1) First the complex, unstructured problem is divided into its component parts according to a hierarchical design (see Fig. 1) with a main goal (objective) at the top of the hierarchy (i.e., selecting the best bidder in the tender), criteria (and sub-criteria) at levels (and sub-levels), and decision alternatives (the bidders) at the bottom of the structure. Typically, the main criteria are price and quality. We remark that, while in its traditional statement AHP may have numerous decision levels [37], in PP an excessive number of levels may lead to extra analytical effort in assessing data and can lead to difficulties in the analysis of the outcomes. It is also important to build a model in which the really relevant criteria and alternatives are identified, thus ensuring completeness and avoiding redundancy. Unlike ordinary optimization methods, which necessarily require the availability of actual measures, in the AHP the evaluating criteria can be referred both to tangible measurements (such as the price) and intangible features (i.e., subjective opinions, such as decision maker(s) preferences and estimates). Select the Best Bidder

c1

c2

b2

…

Jo

b1

c3

…

cN

bI

Figure 1 - Basic hierarchical structure of the AHP decision making problem in PP.

2) Then, the pair-wise comparison matrices on the relative importance of each item are determined. To this aim, each element in an upper level is used to compare the elements in the level immediately below, then proceeding downwards following the principle of hierarchical composition [37]. Hence, the scores obtained from the comparisons at each level are used to weight the items at the level below. However, with respect to the classical AHP, in the PP sector the weights on criteria are already imposed in the tender. Therefore, the importance of each criterion with respect to the overall goal is already known, and as such the pair-wise comparison refers to the priority of each alternative in reaching each criterion. Referring to 6

Journal Pre-proof the basic hierarchical structure of the PP problem reported in Fig. 1, the pair-wise matrix Ci,j(n) of dimension IxI is obtained comparing the impact of the i-th alternative with respect to the j-th alternative (with i,j=1,…,I) in reaching criterion n (with n=1,…,N): a1,2 ( n )  1  a2 ,1( n ) 1 C (n)   ... ...  aI ,1( n ) aI ,2 ( n )  i,j

... a1,I ( n )  ... a2 ,I ( n ) ... ...  ... 1 

(1)

pro of

This result of each comparison (i.e., the generic term ai,j(n) in Ci,j(n)) is expressed in a range [1-9] by the Saaty’s scale, reported in Table I, in which even numbers represents intermediate values [37]. In particular, AHP assigns a 1 to all the elements on the diagonal of the matrix (meaning that when we compare any alternative against itself, there is no preference), a value ai,j(n)>1 if alternative i is more important than the j-th one, and a value ai.j(n)<1, if the i-th alternative is less important than j. In addition, the preference rating of aj,i(n) is obtained by computing the reciprocal of ai,j(n) [37]. TABLE I. SAATY’S AHP SCALE FOR THE PAIR-WISE COMPARISON OF ALTERNATIVES [37]. Value Preference Equal importance Weak or slight Moderate importance Moderate plus Strong importance Strong plus Very strong importance Very, very strong Extreme importance

Two alternatives contribute equally to the criterion in the upper level Experience and judgement moderately favour one alternative over another

re-

1 2 3 4 5 6 7 8 9

Meaning

Experience and judgement strongly favour one alternative over another An alternative is very strongly favoured over another

Jo

urn a

lP

It is to be noticed that AHP allows for inconsistencies in pair-wise judgments. As an example, given a criterion n, the decision maker considers alternative i extremely important with respect to alternative j (that is aij(n)=9), and equally important with respect to alternative f (that is, aif(n)=1). Then, one would expect that f is extremely important with respect to j (that is, afj(n)=9), but no specific rule on that is provided by the AHP methodology, and the comparison between f and j is left to the decision maker(s), which may lead to inconsistent assessments, particularly when the number of elements to be compared is relevant. Overall, N matrices of dimension IxI are obtained in this step. 3) To identify the alternative having the highest priority, several methods are available in the literature, the most common of which are the eigenvalue and the row geometric mean methods (see [37] for details). However, a simplified procedure can be applied, consisting in summing the values in each column of the pair-wise comparison matrix Ci,j(n), dividing each element in the pair-wise matrix by its column total as in eq. (2) (the resulting matrix is the normalized pair-wise comparison matrix), and computing the average of the elements in each row of the normalized matrix as in eq. (3) (the resulting I-dimensional column vector is the score vector of the I alternatives under the n-th criterion S(n)=[s1(n),…,si(n),…sI(n)]). I

ai , j ( n )  ai , j ( n ) /  a f , j ( n )

(2)

 I  si ( n )    ai, f ( n )  / I  f 1 

(3)

f 1

Hence, these relative weights sum up to one. 4) A score matrix S=[S(1), …,S(n),…,S(N)] of dimension IxN is then determined, collecting the N score vectors. The overall performance index vector PI is finally determined by

7

Journal Pre-proof

Jo

urn a

lP

re-

pro of

weighting the score matrix with the weights imposed in the call for tenders, representing the rating of the alternatives in achieving the goal of the problem: (4) PI  S W The i-th element pii in the I-dimensional vector PI represents the global score of the i-th alternative, and, according to these values, bidders are ranked to award the tender. 5) Due to the inconsistency typical of human judgements, it is also important to verify whether a pair-wise comparisons matrix is consistent or not. To this aim, AHP requires the computation of the Consistency Ratio (CR) [37]. We omit here the description of how to determine it (see [37] for details). We just recall that, for each pair-wise comparisons matrix Ci,j(n), if CR(Ci,j(n)) is higher than a threshold value (usually 0.1), then the judgements in Ci,j(n) are not consistent and they have to be elicited again [37]. Strengths and weaknesses: The AHP technique has several advantages for the PP, including the identification of inconsistencies in the assessments, which reduces bias in the decision making process [38]. The main advantage is that AHP leads to a detailed, structured, and systematic decomposition of the problem into its components, with a large degree of flexibility, thus allowing the analysis of problems with complex structures [39]. In addition, AHP helps capturing both subjective and objective evaluation measures, using both qualitative and quantitative data, which is typical in PP. Another strength of AHP lies in its ease of use, since it is not required a deep mathematical knowledge. AHP also easily supports group decision making. Finally, although the technique is deterministic, AHP helps modelling some kind of uncertainty and risk, since it is capable of deriving scales even where deterministic measures are not available [40]. However, AHP also presents a series of weaknesses. A major disadvantage of the AHP is that the number of required pair-wise comparisons may be very large, particularly in case of inconsistencies, thus significantly increasing the computational effort and the uncertainty of the decision process [39]. Moreover, compensation between good and bad scores may occur because of the aggregation of judgments. Furthermore, AHP has been criticised for its limited reliability and subjectivity due to the decision makers’ preferences, as well as for the effectiveness of the 1-9 Saaty’s scale [41]. In fact, the decision makers might find it difficult to make an exact judgment about a comparison distinguish among these 9 points, or they may like to express different values. AHP also suffers from the so-called rank reversal problem [42], that is, even adding (or deleting) irrelevant alternatives or criteria may lead to changes in the final ranking. Still, when the priority values are close to each other, then the final ranking may be inaccurate [43]. Clearly, these shortcomings may have a significant impact on the practical usability of AHP in the PP problem, due to the need for transparency in this sector. Modification to cope with the PP context: We conclude this section remarking again that the classical AHP procedure in [37] is unsuitable for direct application to the PP context, since weights at each level are obtained ex post. Hence, since in the PP sector weights on criteria are stated a priori in the call for tenders, we have partially modified the classical AHP method to take into account such an issue. 2.1.3. The Preference Ranking Organization METHod for Enrichment Evaluations (PROMETHEE) for PP Description: The PROMETHEE method [16, 45] is an MCDM approach based on the concept of outranking relation, i.e., a binary relation defined between each pair of alternatives (i,j), with i,j=1,…,I, so that, if i is preferred to j, then i outranks j. In particular, PROMETHEE II provides a complete ranking of a finite set of feasible alternatives from the best to the worst one. Just like AHP, also PROMETHEE is essentially based on a pair-wise comparison of the alternatives against each criterion [39]. However, it also requires the definition of a preference function for each criterion, in order to transform comparisons into a preference degree ranging 8

Journal Pre-proof between zero and one [44]. After identifying the goal, the evaluating criteria, their preference weights, and the alternatives to be evaluated, the basic steps of the method are the following [44]: 1) Determine the differences d(n) (for each n=1,…,N) between the values assigned to two alternatives i and j on a given criterion n, based on pair-wise comparisons of the alternatives:

di , j ( n )  fi ( n )  f j ( n )

I

i    i, j , j 1

re-

pro of

(5) 2) Model the decision maker’s preferences for the n-th criterion by applying the preference function Pi,j(n), which is a function of di,j(n). For each preference function, the value of a strict preference threshold pi,j(n), the value of an indifference threshold qi,j(n), and the value of an intermediate value si,j(n) between pi,j(n) and qi,j(n) have to be determined according to the decision maker(s) preferences. Consequently, two alternatives are indifferent for criterion n if di,j(n) does not exceed the indifference threshold qi,j(n). Conversely, if the difference is greater than pi,j(n), then there is a strict preference for alternative i. 3) Determine the multi-criteria preference index (or global preference index) for each pair of alternatives i and j (i, j=1,…,I):  N   N   i , j    wn  Pi , j ( n )  /   wn  (6)  n 1   n 1  4) Determine the incoming (or positive) and the outcoming (or negative) outranking flows for the i-th alternative, whose difference is the net outranking flow of alternative I (i.e., the performance index PIi): I

i   j ,i j 1

 PI i  i  i

(7)

Jo

urn a

lP

The maximum value of PIi indicates the best alternative. Strengths and weaknesses: The success of PROMETHEE is mainly due to its mathematical properties and to its ease of use. Moreover, it is particularly suitable to deal with ranking problems, managing both qualitative and quantitative measures. In addition, its decision logic that is not fully compensatory and its ability to partially take into account uncertainty/imprecision (deriving from inaccurate data or disagreement of decision makers on evaluations) make PROMETHEE suitable to face PP purchasing problems. However, like AHP, PROMETHEE suffers from the rank reversal problem when a new alternative is introduced (or deleted). With respect to AHP, PROMETHEE does not provide the possibility to structure the decision problem, and hence, in case of many criteria and alternatives, it may become difficult for the decision makers to obtain a clear view of the problem and to evaluate the results. Finally, the way in which the preference functions are processed is complicated and particularly hard to explain to non-specialists. Modification to cope with the PP context: Differently from the AHP, the PROMETHEE in its classical form is able to cope with the PP requirements, and as such no modification is needed. 2.1.4. The Multi-Attribute Utility Theory (MAUT) for PP Description: The MAUT method [6,45] is grounded on the hypothesis that decision makers try to optimize (consciously or implicitly) a function that aggregates all points of view. After identifying the main objective, criteria, sub-criteria, the corresponding weights, and the alternatives to be evaluated, this function can be determined as follows [45]. 1) The decision maker is required to identify a marginal utility function ui(n) representing his preference on the i-th alternative (i=1,…,I) with respect to the n-th criterion (n=1,…,N). The shapes of this function are determined by the decision maker on the basis of his attitudes to risk. Typically, concave utility functions are associated with a risk-averse attitude and convex utility functions with risk-prone behavior. In the simpler form of MAUT (the so-called Simple 9

Journal Pre-proof Multi-Attribute Rating Technique (SMART) [45]), the marginal utility functions are transformed into marginal utility scores, such that the best and worst alternatives on a specific criterion n respectively have a marginal utility score of 1 and 0. 2) Each couple of alternatives (i,j) is compared over a given criterion n by evaluating the preference and indifference relations amongst them: i is preferred to j  ui ( n )  u j ( n )   i, j  1,...,I :   i and j are indifferent  ui ( n )  u j ( n )

(8)

pro of

3) The performance index of the i-th bidder (the so-called global utility function) PIi is computed by combining the single marginal utilities. To this aim, the most commonly adopted form of utility function is the additive one, which corresponds to make independence assumptions on the variables (that is, an attribute can be expressed independently from the values of others) and consists in a weighted sum of the marginal utilities [45]: N

PI i   wn  ui ( n ) n 1

(9)

subdivided

urn a

lP

re-

Since weights are normalized (i.e., they sum to 1), PIi is always a number between 0 and 1. 4) A sensitivity analysis is performed to examine how a change in an attribute’s value would influence the overall utility. Basically, this phase consists in changing some of the values in the analysis to check whether or not conclusions change, and if so, by how much [46]. However, the method is rather complex, particularly in the utility function elicitation process. Moreover, since the initial computations are time demanding, the sensitivity analysis is even much more complex [46]. In addition, some empirical studies have demonstrated that decision makers may choose alternatives that imply a violation of the MAUT independence assumption [45]. Although some non-linear utility functions can be adopted in such cases, this further complicates the overall resolution problem. Modification to cope with the PP context: Just like the PROMETHEE, also the MAUT technique in its classical form can be applied to the PP context without requiring any modification to its formulation. 2.1.5. The Data Envelopment Analysis (DEA) for PP Description: DEA [47] is a ratio-based mathematical technique to determine how efficiently the alternatives (bidders) satisfy the conflicting criteria C  c1 ,..., cn ,...cN  , which are usually into

H

C H  c1 ,..., ch ,..., cH 

C  C H  CG and

input criteria (e.g., resources needed to obtain outcomes) and G output criteria (e.g., outcomes) CG  cH 1 ,..., cg ,..., cH G  , being

H+G=N. The efficiency of each alternative is the ratio between the weighted

Jo

sum of outputs and that of inputs. For each alternative i, given its g-th output performance value yi(g) against cg and its h-th input performance value xi(h) against ch, the final aim is to determine the corresponding non-negative input and output coefficients v(h) and u(g) that maximize the efficiency of the i-th bidder, while imposing that the efficiencies of all alternatives do not overcome the unitary value. Hence, for each bidder i=1,…,I the following problem is to be solved [47]:  G G   u ( g )  yi ( g ) u ( g )  yi ( g )   g H1 1 g 1 max Ei  H subject to:  (10) v(h)  xi (h)   v(h)  xi (h)   h 1 h 1 u ( g ), v(h)  0 g  H  1,..., H  G, h  1,..., H

10

Journal Pre-proof

Jo

urn a

lP

re-

pro of

Alternatives achieving an efficiency value equal to 1 are referred to as the relatively efficient bidders, whereas alternatives having an efficiency score less than 1 are referred to as inefficient ones. Efficient bidders identify an efficient frontier, and the efficiencies of other alternatives are measured by the deviation of their points from this frontier. Since the efficiency is a measure of how good an alternative is in transforming its inputs in its outputs, that is, in maximizing its outputs while keeping limited its inputs, the DEA approach is particularly suitable for production systems. Nonetheless, its formulation can be easily generalized considering the input criteria as criteria to be minimized and the outputs criteria as those to be maximized. Strengths and weaknesses: The success of DEA is mainly due to its robustness and simplicity of application and to the possibility to easily cope with multiple inputs and outputs (even with different data units of measurement) and to perform homogeneous evaluations (with no need for weights ponderation and normalization of data). DEA is also applicable in case of partially missing data [7]. Moreover, the technique leads to combining all inputs and outputs into a single ratio. Another advantage of the DEA method is that, with respect to other MCDM techniques (e.g., MAUT) it does not require an assumption of a functional form relating inputs to outputs. In addition, possible sources of inefficiency can be determined, which make DEA also useful for self-diagnosis and benchmarking [31]. However, since DEA is a so-called extreme point technique, even symmetrical noise with zero mean (that is, errors that perfectly compensate each other) may cause significant problems. In addition, being DEA a nonparametric technique, it is difficult to perform statistical hypothesis tests. Moreover, being a deterministic technique, DEA produces results that are sensitive to measurement error, which means that if the inputs of an alternative are understated (or its outputs overstated), then that bidder can become an outlier that distorts the shape of the frontier and therefore the overall efficiency scores [31]. Furthermore, DEA scores are sensitive to the size of the sample, requiring that a sufficient number of alternatives are evaluated (whose number typically depends on the number of input/output criteria according to same empiric rules) [48]. Finally, since DEA requires solving a separate problem (10) for each bidder, large PP problems can be computationally intensive. Modification to cope with the PP context: Despite these limitations, most of the highlighted weaknesses can be easily overcome. For instance, although the classical DEA consists in a non-linear problem, the Charnes-Cooper transformation [47] allows linearizing it, keeping constant the weighted sum of outputs while minimizing that of inputs (input-oriented resolution), or keeping fixed the weighted sum of inputs while maximizing that of outputs (output-oriented method). By applying the latter method, problem (10) is modified as follows (for each i=1,…,I): H G u ( g )  y ( g )  v(h)  xi (h)  0  i  g 1 h 1 G  max Ei   u ( g )  yi ( g ) s.t.:  H (11) g 1  v(h)  xi (h)  1  h 1 u ( g ), v(h)  0 g  H  1,..., H  G, h  1,..., H However, both the traditional DEA and its linearized version do not allow to directly impose weights of inputs and outputs, which is mandatory in PP problems. Hence, to this aim we introduce Assurance Regions (ARs) [50, 51], which allow adding some constraints to (11) to emphasize the importance of some criteria on others. For instance, to impose that the generic input weight v(h1) has to range between l(h1, h2) and u(h1, h2) times weight v(h2) (with h1, h2=1,…,H), the following constraint is applied to (11): l (h1 , h2 )   v(h1 ) / v(h2 )   u(h1 , h2 ) (12) 11

Journal Pre-proof

urn a

lP

re-

pro of

Another drawback of the classical DEA is that of distinguishing only between efficient and inefficient alternatives, without further discriminating among the latter group. For this reason, traditional DEA is mainly useful in a pre-evaluation phase [51], but it does not fully fit the PP sector, where a unique ranking outcome is required. Nonetheless, in order to increase the traditional DEA discriminative power, several procedure are available in the literature (see, e.g., [50]), among which the most popular is the cross-efficiency DEA [52], that has been already applied in the PP context (see [53]). The cross-efficiency DEA seeks to maximize the average efficiency scores of all alternatives from the perspective of a given bidder. Hence, the coefficients resulting from the maximization of the efficiency of each alternative (as obtained by applying the classical DEA) are then used to estimate the efficiency of all others. In other words, I efficiency estimates are associated to each i-th bidder (with i=1,…,I) and a crossefficiency matrix CE={CEi,j} is built. The generic diagonal element CEi,i of CE is obtained as the solution of classical DEA problem, while each extra-diagonal value CEi,j (i,j=1,… ,I, i≠j} is the efficiency of the i-th bidder calculated with the most favorable weights (vj(h) and uj(g)) of the j-th competing alternative, chosen each time as a reference [52]. Hence, in this version of DEA, IxI problems are solved. While in the traditional DEA each alternative can maximize its efficiency by increasing the weight of its most favourable criteria, the crossefficiency approach takes into account for each bidder the most favourable weight set of all its competitors. In addition, in order to overcome the frequent occurrences of multiple optimal solutions of the classical DEA and to obtain the uniqueness of scores, a second-level optimization procedure is executed for each alternative i=1,…I [54]: H  I  v ( h )     xi (h)   1  j 1, j i   h 1 G H  I G    u ( g )  y j ( g )  E j   v(h)  x j (h)  0 j (  i )  1,.., I max  u ( g )    y j ( g )  s.t.:  (13) g 1 h 1 g 1  j 1, j i  G H  u ( g )  yi ( g )   v(h)  xi (h)  0  g 1 h 1  u ( g ), v(h)  0 g  H  1,..., H  G, h  1,..., H In this way, all alternatives are evaluated according to the same set of weights. The overall cross-efficiency (that is, the final performance index) of each bidder is finally determined by averaging the i-th row elements of the cross-efficiency matrix CE, and alternatives can be ranked according to such values:

 I  PIi    CEi , j  / I  j 1 

(14)

Jo

We remark that the DEA technique, both in its traditional version and in its cross-efficiency extension, is deterministic. However, in real world data are often imprecise or vague, and missing data, judgment data, or predictive ones may have to be dealt with. To this aim, stochastic (see, e.g., the review in [55]) or fuzzy (see, e.g., the review in [56]) DEA approaches are also available. In particular, referring to the SSP, we recall here the stochastic cross-efficiency DEA model in [50] and the fuzzy cross-efficiency DEA models in [51]. Since stochastic approaches would require the availability of reliable historical data on all the bidders for each tender, in this study we choose the fuzzy framework, whose formulation is presented in Section 2.2.3. We conclude this section remarking that, to allow for application of DEA to the PP problem, weights on criteria have to be stated a priori in the call for tenders. Hence, here we combine DEA with the ARs to take into account such an issue. In addition, to guarantee the ease of use (as required by the mentioned European directives), we consider the output-oriented 12

Journal Pre-proof

pro of

linearization (i.e., we evaluate which tenderer is able to provide the maximum quality level with the same price offer of his competitors). Finally, to ensure the non-discrimination requested by the general rules of PP, we implement the cross-efficiency DEA variant. 2.1.6. The currently adopted EI method for PP Description: This section recalls the currently adopted methodology at the procurement office of the considered EI. Basically, it consists in a simple MEAT approach that provides a financial ranking through a weighted average assessing the n criteria of the bids, divided into price and (n-1) quality sub-criteria. For each bidder i=1,…,I, the overall score is obtained as: PI i  N ip  N iq (15) being N ip the price score and N iq the overall quality score. The former is determined as: N ip  wp   Pmin / Pi 

z

urn a

lP

re-

(16) where: wp is the [0-100] weight assigned to the price criterion depending on the specific case, Pi the price offer of the i-th bidder, Pmin the price offer of the cheapest received bid, and z is a factor depending on the type of tender under evaluation. In particular, eq. (16) assumes a quadratic formulation (i.e., z=2) for service tenders, while it is formulated as a cubic function (i.e., z=3) for goods and supplies markets, to better highlight price differences among the bids. The quality score is then determined as: n 1 w ( k )  N ( k ) i Niq ( n )   q (17a) N ( k ) k 1 max being: wq(k) the [0-100] weight assigned to the k-th quality criterion, Ni(k) the score assigned to the i-th bidder for the k-th (sub)-criterion (assigned by the board of experts and expressed in a [0,10] range), Nmax(k) the maximum score assigned to the k-th (sub)-criterion. Note that in case the k-th quality criterion is composed by T sub-criteria, then formula (17a) is modified as follows (we avoid describing in detail the apparent meaning of variables): n 1 n 1 T w ( t )  N ( t ) i Niq ( n )   Niq ( t )   q (17b) N max ( t ) k 1 k 1 t 1 According to the values of PIi in eq. (15) all bidders are then ranked to award the tender. Strengths and weaknesses: Despite the ease of use of this methodology and its compliance with the PP rules, it exhibits very significant limitations: a lack of discrimination among bidders, a strong influence of price on the final score (even when the quadratic form is used), a non-linear framework (which causes loss of information), a deterministic setting (which does not allow to take into account uncertainty or imprecision in data). Modification to cope with the PP context: Being a technique specifically developed to fulfill the PP requirements, the currently adopted EI method can be directly applied to public tenders, in accordance with the European directives 2014/24/EU [4] and 2014/25/EU [5] rules and requirements.

Jo

2.2. MCDM approaches for PP under uncertainty All the described MCDM techniques have the main limitation of being deterministic. However, uncertain data, vagueness, and imprecision, are common in any decision making process and are typical of human’s judgments. To this aim, stochastic and fuzzy approaches can be adopted. Stochastic approaches rely on the idea that uncertain values can be modelled by suitable probability distributions, given that historical reliable data are available. Hence, these methods need to collect a large amount of historical data, assuming that what has happened in the past will also similarly occur in the future. In addition, due to the business markets’ volatility, stochastic approaches may lead to unfair or even irregular tenders awarding. In such cases, uncertainty or imprecision can be effectively modeled through the 13

Journal Pre-proof

lP

re-

pro of

fuzzy logic framework [57], which well mimics the natural language and the way of reasoning of humans, also allowing to consider the specific risk attitude of decision maker(s), thus reducing errors and improving the decision making results. In particular, triangular fuzzy numbers can be used to model uncertain values, since they are intuitive, easy to use, computationally simple, well mimicking the human reasoning, and they often show better performance compared to other membership functions [12]. Several fuzzy MCDM techniques have been proposed in the literature (see, e.g., the reviews in [11, 61], among which in the subsequent subsections we recall the Fuzzy AHP (FAHP) (see, e.g., [59]) and the crossefficiency Fuzzy DEA (FDEA) (see, e.g., [51]). 2.2.1. The Fuzzy Analytic Hierarchy Process (FAHP) for PP Description: In the FAHP approach, the pair-wise comparisons of alternatives are performed through linguistic variables represented by triangular numbers [59]. Therefore, the judgemental Saaty’s scale is now fuzzified. Table II shows a FAHP semantic scale of relative importance of the i-th alternative as opposed to the j-th one [60]. The AHP procedure described in Section 2.1.2 is then applied by the outlined steps where operations are now made on triangular fuzzy numbers following their arithmetic properties [57]. At the end of the procedure, the resulting performance index vector still collects fuzzy numbers that can be defuzzified to rank alternatives (e.g. by determining the Centre Of the Area (COA) of the distribution, that is, a value dividing the area under the triangular membership function curve into two approximately equal parts [57]). Finally, the consistency of the judgements can be verified, for instance following the procedure suggested in [61]. Strengths and weaknesses: Overall, the FAHP procedure presents the same advantages and disadvantages of the classical AHP method described in Section 2.1.2, with the main difference of being able to manage uncertain data. Modification to cope with the PP context: Just as the AHP, also the FAHP requires the same modification described in Section 2.1.2 to impose weights at the call for tenders and avoid determining them at the end of the procedure. TABLE II. THE FUZZIFIED SAATY-S SCALE FOR THE PAIR-WISE COMPARISON OF ALTERNATIVES (ADAPTED FROM [60]). Triangular score Preference

(1,1,3) (1,3,5) (3,5,7) (5,7,9) (7,9,9) Intermediate

Just equal

Alternatives i and j are just equally important

urn a

(1,1,1)

Meaning

Equal importance

Alternatives i and j are equally important

Weak importance

Experience and judgement slightly favour i over j

Essential or strong importance Experience and judgement strongly favour i over j Very strongly importance

i is very strongly favoured over j

Extreme importance

The evidence favouring i over j is of the highest possible order of affirmation

When compromise is needed, values between two adjacent judgements can be used.

If i has one of the above judgements assigned to it when compared with j, then j has the reciprocal value when compared with i.

Jo

2.2.2. The Fuzzy Data Envelopment Analysis (FDEA) for PP Description: In the FDEA formulation, the generic i-th bidder biB is evaluated against uncertain input/output criteria, each defined as a triangular fuzzy number collecting the corresponding pessimistic, modal, and optimistic values, i.e., o m p 3 p m o 3 xi (h)  ( xi (h), xi (h), xi (h))  R and yi ( g )  ( yi ( g ), yi ( g ), yi ( g ))  R . These triples can assume different values with a degree of possibility [0,1], expressed by suitable membership functions representing the triangularly-shaped possibility function (see [51] for details). As the traditional DEA approach, the FDEA technique is aimed at maximizing the efficiency of tenderers (i.e., the ratio between the weighted sum of outputs and that of inputs). The same 14

Journal Pre-proof

pro of

procedure described in Section 2.1.5 can be applied, together with the arithmetic properties of fuzzy number [57], to solve the arising fuzzy problem. Strengths and weaknesses: With respect to the DEA approach, both in its classical formulation and in the cross-efficiency variant, the FDEA is able the overcome the limitation of being applicable only to deterministic frameworks. However, such a methodology requires solving several problems in which the weight set varies at each step, thus implying a high computational effort. Modification to cope with the PP context: In order to use only one set of weights (i.e., to perform a homogeneous evaluation of alternatives, as imposed by the PP rules) and keep uncertainty up to the final stage (so as to get possibility distributions of the efficiencies), we recall here the formulation proposed in [51], where the fuzzy cross-efficiencies CEi  (CEip , CEim , CEio )  R 3 (for i=1,…,I) are derived as a compromise between conflicting

urn a

lP

re-

objectives aimed at maximizing the most probable value while increasing the possibility of getting better results: a) the maximization of the median value; b) the minimization of the distance of the pessimistic value from the modal one; c) the maximization of the distance of the optimistic value from the modal one. Thus, for each bidder i, a Positive Ideal Solution (PIS) is defined as the ideal solution that simultaneously satisfies the three objectives above:  F1,PIS  max CEim i  PIS m p (18)  F2,i  min[CEi  CEi ]  PIS o m  F3,i  max[CEi  CEi ] Three optimization problems subject to the constraints in eq. (13) (modified to cope with the fuzziness of variables) are to be solved (see [51]). Since in practice objectives (a)-(c) can never be reached simultaneously by the same weight set, a Negative Ideal Solution (NIS) is also defined as the ideal solution missing all these objectives:  F1,NIS  min CEim i   NIS m p (19)  F2,i  max[CEi  CEi ]  NIS o m   F3,i  min[CEi  CEi ] Then, from the PIS/NIS of each alternative i a compromise triple ( Fi a , Fi b , Fi c ) is determined

Jo

solving a fuzzy multi-objective linear programming model. The fuzzy cross-efficiency of each actor is obtained from this triple and according to eqs. (18) and (19) as follows: CEim  Fi a  p a b (20) CEi  Fi  Fi  o a c CEi  Fi  Fi The defuzzified value of each cross-efficiency is finally determined through the COA to rank all the DMUs. PI i  (CEip  CEim  CEio ) / 3 (21) We refer the reader to the contribution in [51] for more details. 2.3. The Group Decision Making Process for PP In most activities, multiple individuals are involved in the decision making process. This is particularly true in the PP sector due to the need for ensuring impartiality and transparency to the tenders. To this aim, a Group Decision Making (GDM) process is required to aggregate the multiple judgements (e.g., using a weighted mean). Basically, the GDM procedure can be carried out in two different ways [62]: 15

Journal Pre-proof

pro of

1) Aggregating Individual Judgements (AIJ): the aggregation occurs in the preliminary phase, assuming that decision makers act as a single manager. 2) Aggregating Individual Priorities (AIP): the aggregation occurs in the final phase, assuming that each group member acts individually. The same classification also holds for fuzzy approaches [63], with the only difference that in these cases the aggregation concerns fuzzy numbers. Both procedures allow applying different weights depending on the specific skills or level of expertise of each decision maker in the group. Clearly, the choice between the AIJ and the AIP procedures depends on whether the group members either engage in discussion to achieve a consensus or separately express their own preferences. Since in the PP field decisions are usually shared among experts, the AIJ method is adopted in this paper. We conclude this section highlighting that, according to the state of the art review, the group decision making process has been seldomly investigated when MCDM techniques are compared. 3. THE CASE STUDY AT THE EUROPEAN INSTITUTION (EI) 3.1. Comparison of the deterministic MCDM approaches for PP

urn a

lP

re-

In the first phase of this study we analyzed several public tenders at the PP office of the case study EI. This phase lasted for twelve months and was aimed at comparing the selected deterministic MCDM techniques and the currently adopted EI method. In particular, the analysis concerned some previous tenders from the past, suitably selected to address different settings (in terms of type of call, its hierarchical structure, number of participating bidders, and overall number of criteria), so as to evaluate the performance of each method under different contexts. All the new tenders occurring during this phase were also analyzed, for a total of 11 tenders. For the sake of brevity we only describe here a single tender that we call Tender 1, while Section 3.1.1 summarizes and discusses the main findings of the whole phase. The considered public tender refers to a goods market and consists in selecting the best supplier for some office furniture. This tender deserves a special focus since the bidder awarded according to the EI method successively went bankrupt, causing a re-awarding of the tender and thus a loss of further resources. Nine eligible tenderers (whose names are here omitted due to privacy reasons) applied their offers (i.e., I=9, B  b1 ,..., b9  ). Select the Best Bidder

c1=price

quality

Certifications

Technical

c3

Jo

c2

c5

c4

c7

c6

c8

c9

c11

c10 c13

b1

Conditions

c14

c12 c15

c16

c17

b9

…

Figure 2 – The specific hierarchical structure of the AHP decision making problem for Tender 1.

Offers were evaluated based on two main criteria, namely the price and the overall quality, and the latter was subdivided into 3 sub-criteria, i.e., “Technical” value, “Certifications”, and “Conditions”. Each sub-criterion was furtherly characterized by some attributes as reported in 16

Journal Pre-proof

pro of

the third column of Table III. It is important to notice that, in accordance with the modern approaches to the SSP, some risk factors are also taken in into account in this tender. Indeed, risk is evaluated in terms of low quality (i.e., criteria from c7 to c9), injuries (that is, the safety criterion c6), environmental impact (from c10 to c12), and disruption risks (incorporated in the warranty conditions in c13). Figure 2 adapts the generic AHP hierarchical structure of Fig. 1 to the specific decision making problem of Tender 1. For each item (i.e., criterion, sub-criterion, and attribute), the EI board of managers (composed by 5 experts, having the same level of importance) selected the corresponding weights (see Table III). Table IV shows the data for each tenderer: while the prices arise from the received offers, the scores assigned to other items arise from the judgments of the EI experts (expressed in a 0-10 scale and aggregated through the AIJ). According to the DEA approach, the overall problem is characterized by N=17 criteria, with H=1 input (i.e., the price) and G=16 outputs (i.e., the whole attributes set), thus C H  c1 and

CG  c2 ,..., c17  . A proper analysis of data in Table IV while taking into account weights and sub-weights in Table III requires MCDM approaches.

TABLE III. CRITERIA, SUB-CRITERIA, ATTRIBUTES, AND WEIGHTS FOR TENDER 1. 40.00%

Ergonomics (26.67%)

8.00%

Functionality (26.67%)

8.00%

Aesthetics (20.00%)

6.00%

Assortment (16.67%)

5.00%

Safety (10.00%)

3.00%

Product Quality (30.00%)

3.00%

Production Quality (20.00%)

2.00%

Distribution Quality (10.00%)

1.00%

Products Environmental (20.00%)

2.00%

Production Environmental (10.00%)

1.00%

Distribution Environmental (10.00%)

1.00%

lP

Technical (50%)

Certifications (16.67%)

urn a

Quality (60%)

Final weights

re-

Criteria Sub-criteria Attributes (weights) (sub-weights) (sub-weights) Price (40%)

Conditions (33.33%)

Warranty (30.00%)

6.00%

Post-sales (20.00%)

4.00%

Delivery (10.00%)

2.00%

Transport & Assembly (30.00%)

6.00%

Packaging (10.00%)

2.00%

TABLE IV. DATA OF TENDER 1.

Criteria/Attributes

CG

b2

b3

b4

b5

b6

b7

b8

b9

93.58 103.52 88.27 106.08 65.96 125.90 93.55 139.51 101.41

Jo

CH c1 Price [k€]

b1

c2 Ergonomics

8.0

8.0

8.0

8.0

6.0

8.0

8.0

8.0

8.0

c3 Functionality

8.0

6.0

10.0

10.0

4.0

8.0

8.0

8.0

8.0

c4 Aesthetics

4.0

6.0

8.0

8.0

4.0

8.0

8.0

6.0

6.0

c5 Assortment

6.0

6.0

9.0

1.0

1.0

1.0

1.0

6.0

1.0

c6 Safety

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

c7 Product Quality.

8.0

8.0

8.0

8.0

8.0

4.0

4.0

8.0

8.0

c8 Production Quality

8.0

8.0

8.0

8.0

8.0

4.0

4.0

8.0

8.0

c9 Distribution Quality

8.0

4.0

8.0

4.0

8.0

4.0

4.0

8.0

8.0

c10 Products Environmental

8.0

8.0

8.0

8.0

4.0

4.0

8.0

8.0

8.0

c11 Production Environmental

8.0

8.0

8.0

8.0

4.0

4.0

8.0

8.0

8.0

17

Journal Pre-proof c12 Distribution Environmental

8.0

4.0

8.0

4.0

8.0

4.0

4.0

8.0

8.0

c13 Warranty

6.0

1.0

10.0

8.0

1.0

1.0

1.0

8.0

6.0

c14 Post-sales

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

c15 Delivery

4.0

4.0

10.0

4.0

4.0

4.0

10.0

4.0

4.0

c16 Transport & Assembly

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

8.0

c17 Packaging

8.0

8.0

8.0

6.0

8.0

6.0

8.0

8.0

8.0

re-

pro of

It is important to remark that, while the EI, the MAUT, and the PROMETHEE methods can be directly applied to the case study in their classical formulation, the DEA and the AHP techniques require some modifications. In particular, although the DEA is able to directly process the data to evaluate the I=9 bidders against the N=17 criteria, in order to properly state the established weights reported in Table III, the ARs on all the pairs of criteria are imposed. For instance, since the final weights on price (c1) and ergonomics (c2) are respectively 40% and 8%, the former is about 5 times more important than the latter, that is, according to eq. (12) the ratio between their weights is set 4.9≤v(1)/u(2)≤5.1. Similarly, all the other ARs can be imposed for each pairs of criteria and the cross-efficiency DEA procedure in Section 2.1.5 is applied to rank the tenderers. Conversely, before applying the AHP, a preprocessing of data is required to normalize the price values and to convert the absolute-scaled judgements values from the adopted 0-10 scale into the 1-9 Saaty’s scale. To this aim, we apply a min-max method to directly determine the value of each element ai,j(n) in the pair-wise comparison matrix that compares tenderers i and j (i,j=1,…,9) with respect to the generic n-th criterion.

   8   Aj ( n )  Ai ( n )    1 1 /   Amax ( n )  Amin ( n )     ai , j ( n )    8   Ai ( n )  Aj ( n ) 1    Amax ( n )  Amin ( n )

lP

if Ai ( n )  Aj ( n ) for n  h  1

(22a)

if Ai ( n )  Aj ( n )

Jo

urn a

 8   Aj ( n )  Ai ( n ) 1 if Ai ( n )Aj ( n ) 1 /    Amax ( n )  Amin ( n )  where Ai(n) and Aj(n) respectively denote the aggregated judgmental values of bidders i and j related to criterion n (i.e., the values in Table V). Note that eq. (22a) assumes that the given criterion has a range whose lower level indicates good performance, while its upper level stands for poor performance, that is, this formula stands for the criterion to be minimized (i.e., price). Conversely, eq. (22b) holds for criteria having increasing function, and as such it is used for those criteria to be maximized (i.e., all the quality attributes). Also note that in case a criterion cn assumes equal values for all bidders (i.e., if Amax(n)=Amin(n)), then the relative importance is set to ai,j(n)=1 for each couple of bidders (i,j) by convention. As an example, Table V shows the obtained pair-wise comparison matrix for the price criterion. Similarly, all the other pair-wise comparison matrices are obtained, and the AHP procedure described in Section 2.1.2 is applied to rank the bidders and award the tender. Table VI reports, for each method, the obtained scores and the corresponding ranking. First, it is to be noticed that all methods, except MAUT, rank b3 and b5 in the first two positions. MAUT ranks as second bidder b1, which is on average ranked as third (fourth according to the DEA). In order to properly analyse such conflicting results, we compute two indices. In particular, we first determine a robustness index, namely the percentage adherence level index 18

Journal Pre-proof

re-

pro of

(ALI%), to define how much each ranking is close to the average classification provided by all the methods (that is, how each method is able to provide a good compromise between the advantages of all techniques). To this aim, we determine the most frequently occupied position of each bidder (i.e., the percentage of occurrence reported in the second-last column of Table VI), and then the ALI% index is obtained by determining for each method how many bidders are exactly in the same position of the corresponding average ranking reported in the last column of Table VI. The second-last row of Table VI reports the obtained values for each method, showing that the EI, MAUT, and PROMETHEE methods have the highest ALI% index and provide very similar rankings. On the contrary, the DEA approach has a poor ALI% index, i.e., it provides a very different ranking with respect to the other methods. In addition, we compute a percentage difference index (DIFF%) to evaluate, for each method, how far each bidder is from the one that is ranked as first. Note that, while the ALI% is focused on the differences in the ranking between the methods, the DIFF% index is focused on the differences in the ranking within the same method, thus providing different insights. Since we are mainly interested in the awarding process, we focus our analysis on the first and second positions of the ranking by only reporting (in the last row of Table VI) the DIFF%(1,2) index. The highest discriminative power is guaranteed by the DEA, while by the EI and the AHP the scores of the first and the second ranked bidders are really close each other. This is a really important task since in the PP transparency is to be ensured in the awarding process. We remark that the bidder awarded by the EI method (i.e., b3) successively went bankrupt, causing a re-awarding of the tender to b5, that is, the bidder selected as best by the AHP and DEA methods. Hence, by adopting such methods the re-awarding of the tender would not have been necessary. TABLE V. PAIR-WISE MATRIX OF THE BIDDERS WRT PRICE FOR TENDER 1. b2

b3

b4

b5

b6

b7

b8

b9

lP

b1

1.00 0.48 1.58 0.42 4.00 0.22 1.00 0.17 0.54 2.08 1.00 2.66 0.78 5.09 0.29 2.08 0.20 1.23 0.63 0.38 1.00 0.34 3.43 0.20 0.63 0.15 0.41 2.36 1.28 2.94 1.00 5.36 0.32 2.36 0.22 1.51 0.25 0.20 0.29 0.19 1.00 0.13 0.25 0.11 0.21 4.52 3.43 5.09 3.16 7.52 1.00 4.52 0.40 3.66

urn a

Bidders b1 b2 b3 b4 b5 b6 b7 b8 b9

1.00 0.48 1.58 0.42 4.00 0.22 1.00 0.17 0.54 6.00 4.91 6.57 4.64 9.00 2.48 6.00 1.00 5.14 1.85 0.81 2.43 0.66 4.86 0.27 1.85 0.19 1.00

TABLE VI. SCORES AND FINAL RANKING OF EACH METHOD FOR TENDER 1. Method Bidders

score rank

63.04 3 54.90 8 76.70 1 60.80 4 76.10 2 46.50 9 58.00 6 56.50 7 58.90 5 77.8% -0.78%

Jo

b1 b2 b3 b4 b5 b6 b7 b8 b9 ALI% DIFF%(1,2)

EI

AHP

Score (·102)

rank

11.42 3 8.57 7 17.06 2 10.74 4 17.33 1 6.28 9 10.73 5 8.11 8 9.74 6 55.6% -1.56%

DEA

Score (·102)

rank

70.49 4 63.72 6 74.73 2 62.18 7 100.00 1 52.39 8 70.51 3 47.28 9 65.04 5 11.1% -25.27%

19

MAUT PROMETHEE score (·102)

rank

62.56 2 50.36 7 69.90 1 54.79 4 58.69 3 37.45 9 53.87 6 41.62 8 53.96 5 77.8% -10.50%

score (·102)

rank

59.58 3 43.20 7 74.86 1 45.24 6 60.13 2 29,.43 9 54.85 4 34.18 8 48.57 5 77.8% -19.68%

Average % of rank occurrence 60% 3 60% 7 60% 1 60% 4 40% 2 80% 9 40% 6 60% 8 80% 5

Journal Pre-proof

Jo

urn a

lP

re-

pro of

3.1.1. Overall comparison of the deterministic MCDM approaches for PP We evaluated many additional tenders, either occurred before our evaluation or occurring during the year of observation (leading, in some cases, to a renegotiation of the offers, with all the deriving benefits). In this section, we summarize the main finding of the whole analysis. More in detail, we asked the board of experts of the EI to give their judgements (and relevant weights) on some suitably defined Key Performance Indicators (KPIs) characterizing the selected MCDM techniques and able to take into account the specific, and in some cases mandatory, requirements of the PP awarding process. According to their evaluations, we compare and rank the considered methods by applying in turn the EI, the AHP, and the DEA methods (where the alternatives are now represented by the adopted MCDM methods). Six different criteria have been identified by the experts as relevant to judge and compare the methods in terms of overall performance and compliance to the PP requirements. Each criterion is characterized by its importance weight (in brackets), as pointed out by the EI experts. The identified relevant criteria are the following: - “Parametrization” (weighted 16.0%), which evaluates the need for additional parameters, the arising subjectivity, and the overall computational time (including the eventually required preprocessing of data). The highest score was assigned to DEA, which only requires the bidders’ data and the ARs, while the lowest scores were for MAUT and AHP, as they need further decisions (respectively the utility functions and the pair-wise comparisons) and computations (respectively the sensitivity analysis and the consistency ratio). - “Consistency” (weighted 8.0%), which evaluates the robustness of the ranking, that is, the capability of each method in providing the same ranking even when some parameters are slightly modified (e.g., a rank reversal due to a slight change of weights is negatively considered). According to this criterion, the best method is DEA, followed by AHP (thanks to its consistency ratio analysis). Conversely, the worst method is EI, due to the strong influence given to the price criterion on the final score. - “Perspectives” (weighted 4.0%), which takes into account the possible further improvements of each method, based on a literature review. In this regard, AHP and DEA are widely diffused, with a huge body of literature and research aimed at improving their models. On the other hand, there are no applications of the EI method in the related literature. - “Ease” (weighted 18.0%), which evaluates whether the method is easily accessible and understandable by all the stakeholders (i.e., the EI employees and the tenderers) and can be implemented in the PP process respecting the principles of transparency and clarity of public tenders. In this regard, the EI method is obviously the simplest one, while DEA is quite complex in terms of formulation. - “Information Loss” (weighted 14.0%), which evaluates the quantity of data that can be lost during their processing, thus affecting the final score. While DEA allows ranking all the bidders by making a sole ratio and hence keeping all the data, the EI approach uses two different formulas for price and quality, considering the relative ranking instead of an overall assessment. Similarly, through the pair-wise comparisons of AHP, some data are lost. - “PP Compliance” (weighted 40.0%), which assesses how much each method fits the regulation of public tenders (and as such it is weighted much more than other KPIs). While AHP can be customized case by case based on the specific tender’s requirements, DEA shows problems in setting the weights trough the ARs and in providing information on the incidence of each criterion on the overall score. Note that the time complexity of the methods is here neglected. In effect, we have verified that all the examined techniques have very limited computational times on standard personal computers (we run the simulations in the MATLAB environment on a PC with 3.40 GHz processor and 8.00 GB of RAM). More in details, the computation time is typically up to a few minutes only in the case of very complex tenders with dozens of criteria and several 20

Journal Pre-proof

lP

re-

pro of

dozens of bidders. Therefore, the considered methods are surely applicable to the PP sector (where the number of bidders and the selected criteria do not typically exceed such numbers). In addition, we remark that several days (or months in some cases) typically pass between the deadline of the call for tenders and the awarding of the winner. Moreover, since the PP awarding process is based on qualitative evaluations performed by the decision makers (price is typically the sole quantitative criterion), subjectivity is inevitably inherent in the evaluations. Nonetheless, among the criteria, subjectivity is one of the aspects evaluated through the so-called “Parametrization” criterion. Table VII reports, for each method, the corresponding EI experts’ evaluation (in a 0-10 scale) of the above KPIs. For each method and each KPI, the table also proposes a graphical scale of the corresponding experts’ evaluation, where a full circle indicates two points, an empty circle stands for one point, and a small empty circle for half a point. Table VII also shows the final scores and ranks of the analysed methods obtained by respectively applying the EI, the AHP, and the DEA techniques. Note that, when the EI method is applied, only eq. (17b) is now applied, since no price score is to be assessed. In addition, the “Parametrization” and the “Information Loss” criteria are to be minimized, and as such, when applying the DEA, they are considered as inputs, and their values are converted into the complement to ten of the experts’ judgement. Finally, as already seen in the previous section, before applying the DEA, the ARs on all the pairs of criteria are to be imposed (see eq. (12)), while before applying the AHP, the absolute-scaled judgements values are to be converted from the adopted 0-10 scale into the 1-9 Saaty’s scale (see eq. (22b)). By analyzing Table VII, it is to be noticed that the AHP is always ranked first and the DEA is the second most frequently ranked technique (last column). Moreover, the AHP has the highest ALI% index (see the second-last row of Table VII), showing a more robust ranking and a good compromise between the advantages of the other techniques. Finally, the DEA exhibits the best DIFF%(1,2) index (see the last row of Table VII), meaning that it shows the highest discrimination in the ranking (i.e., it ensures the highest transparency in the final ranking). TABLE VII. EVALUATION OF EACH DETERMINISTIC APPROACH.

●●

4.5

AHP

●●◦

7.5

DEA

urn a

EI AHP DEA average Inf. PP KPIs Param. Consist. Perpect. Ease Loss Compl. score score % of (16%) (8%) (4%) (18%) Methods (14%) (40%) score rank (·102) rank (·102) rankoccurrence rank 4 1 0 9 3 6 45.25 5 13.79 3 68.85 2 33% 3 EI

●●●○◦

5

MAUT

●●○

PROMETHEE

●●●●○

●○

7

9

5

2

●●●

8

●●●○

●●●●○

●●○

●

●●●●

8

9

2

10

3

●●●●

●●●●○

●

●●●●●

●○

6

4

7

5

4

●●●

●●

●●●○

●●○

●●

5

6

6

5

5

●●○

●●●

●●●

●●○

●●○

61.40

1

25.60

1 100.00 1

100%

1

51.60

2

18.51

2

19.45

5

66%

2

51.00

4

12.59

5

47.55

3

33%

5

51.20

3

12.78

4

44.07

4

66%

4

40% -15.96%

Jo

ALI% DIFF%(1,2)

5.5

●●○◦

○

100% -27.70%

40% -31.15%

Further tests were also carried out to assess the changes in the ranking when slightly different weights are imposed, but the no relevant changes were observed, particularly in the first positions. We also determined the overall ALI% and DIFF%(1,2) indices described in Section 3.1, whose values are reported in the last two rows of Table VII, confirming that the EI method has a poor discriminative power compared with the AHP and the DEA. Overall, the performed analyses show that AHP and DEA are the MCDM approaches that best fit the PP problem, while the currently adopted EI method is unsuitable for the scope, mainly due to its low ability to discriminate among the tenderers. The MAUT and PROMETHEE methods partially lose their main strengths when applied to public tenders. In 21

Journal Pre-proof

lP

re-

pro of

fact, they respectively require the definition of utility and preference functions on each criterion, which, according to the PP requirements in terms of transparency, should be declared in the call for tenders. At the same time, they allow to explicitly state weights, thus guaranteeing a strict compliance with those preliminarily indicated in the call. Nonetheless, thanks to the proposed modifications, the same advantage is also ensured by the proposed variants of the AHP and DEA techniques, thus overcoming the main drawback of their classical formulation. On the other hand, DEA still has relevant limitations in the fact that it does not allow evaluating the incidence of each criterion on the overall score, and in the fact that it requires a deep mathematical knowledge. Further, the AHP approach is characterized by subjectivity and has to be partially modified to ensure the weights compliance with the call for tender. Nevertheless, it currently reveals to be the most suitable method for the EI case, as it fully improves the performance provided by the EI method without strongly altering the currently adopted procedures. For the above reasons, the EI has started to adopt AHP in the current tenders and the board of experts started to express their judgments directly in the 1-9 Saaty’s scale so as to facilitate the implementation of the method. Under our guidance, the EI has continued monitoring and analyzing the new tenders to compare DEA and AHP, also including in the comparison their fuzzy counterparts to take into account the uncertainty that inevitably afflicts the evaluations. This phase is still occurring to allow a long-term evaluation and to properly form the EI employees. We conclude this section underlining that we also carried out some indication on when (i.e. in which kind of tender) each method is preferable. In particular, the DEA approach is mainly suitable for production services, where inputs and outputs can be easily associated to the process. Instead, the AHP best fits structured services with a complex hierarchy, but with a limited number of items at each level (so as to reduce the required pair-wise comparisons), such as, for instance, projects that are structured in work packages and sub-activities. PROMETHEE is applicable only to simple services with a high number of alternatives. Finally, the MAUT is suitable for simple and standard services (where the definition of the marginal utility function can be replicated each time). 3.2. Comparison of the fuzzy MCDM approaches for PP

urn a

This section presents one of the public tenders analyzed in a fuzzy setting (here called Tender 2). The tender concerns a service market (i.e., the provision of office cleaning services), with six bidders (two additional bidders were excluded since they do not comply with one of the requirements) to be ranked against the price offer (weighted 40%) and the overall quality (weighted 60%), which was furtherly subdivided into 5 sub-criteria, reported in the second column of Table VIII. The hierarchical structure of this tender is described in Fig. 3. Select the Best Bidder

Jo

c1=price

quality

c3=Qualifications

c2=Work. hours

b1

c4=Conditions

…

c5=Control

c6=Environm.

b6

Figure 3 – The specific hierarchical structure of the AHP decision making problem for Tender 2.

22

Journal Pre-proof TABLE VIII. CRITERIA, SUB-CRITERIA, WEIGHTS, AND DATA OF TENDER 2.

CH

CG

CRITERIA SUB-CRITERIA FINAL b1 b2 b3 b4 b5 b6 (and weights) (and sub-weights) WEIGHTS 40.00% 430.509 458.136 437.305 443.519 462.908 459.233 Price [k€] (40%) c1 c2 Work. Hours [h] (25.00%) 15.00% 21,540 21,686 20,994 22,095 23,606 21,553 15.00% (7;8;9) (5;5;5) (6;6;7) (6;6.5;7) (8;8;8) (5;6;7) c3 Qualifications (25.00%) Conditions (16.67%) 10.00% (7;7;9) (4;5;5) (5;5;6) (7;7;7) (7;7.5;9) (6;6;7) Quality (60%) c4 10.00% (7;7;8) (5;5;5) (6;6;8) (6;6;6) (7;8;9) (6;6;6) c5 Control (16.67%) 10.00% (6;7;8) (6;6;6.5)(8;8;8.5)(6.5;7;7) (8;8;9) (7;7;8) c6 Environmental (16.67%)

pro of

Experts provided the pessimistic, the modal, and the optimistic values of their evaluations (directly in the fuzzified 1-9 Saaty’s scale in Table II). Their judgements were then aggregated through the AIJ procedure (see Section 2.3). The modal values were used to implement the deterministic AHP, DEA, and EI methods. Table VIII sums up the criteria, their relative weights, sub-weights, and the corresponding final weights, as well as the scores assigned to the participating bidders. The price and the number of working hours guaranteed by contract arise from the offers (i.e., they are crisp data), while other scores are fuzzy triples obtained by aggregating the experts’ evaluations. Overall, the MCDM problem consists in I=6 tenderers ( B  b1 ,..., b6  ), to be assessed against N=6 criteria, with H=1 input (i.e., the price)

re-

and G=5 outputs (i.e., the quality criteria), thus we set C H  c1 and CG  c2 ,..., c6  . As described in Section 3.1.1, the DEA and FDEA and the AHP and FAHP methods are to be preliminarily modified to respectively impose the ARs (see eq. (12)) and to convert in the Saaty’s scale the scores on price (see eq. (22a)) and on working hours (see eq. (22b)). TABLE IX. SCORES, FINAL RANKING, AND KPIS OF TENDER 2. EI score

rank

b1 b2 b3 b4 b5 b6 ALI% DIFF%(1,2)

85.49 1 71.48 6 79.26 4 80.58 3 85.10 2 75.63 5 66.67% -0.46%

(·102)

rank

DEA

score (·102)

rank

23.53 2 10.12 6 17.93 3 15.47 4 24.19 1 11.94 5 66.67% -2.73%

95.83 2 74.05 6 83.90 4 89.46 3 100.00 1 79.89 5 100.00% -4.17%

Jo

urn a

Bidders

AHP score

lP

Methods

FAHP

score (·102)

Rank

24.49 2 8.26 5 20.84 3 8.48 4 35.98 1 7.63 6 33.33% -31.93%

FDEA score (·102)

Average

% of rank rank occurrence

75.40 2 71.45 5 72.35 4 75.06 3 76.90 1 70.69 6 66.67% -1.95%

80% 60% 60% 60% 80% 60%

2 6 4 3 1 5

Figure 4 – FDEA possibility distributions of the fuzzy cross-efficiencies of bidders for Tender 2.

The final scores and ranking for each method are reported in Table IX, together with the ALI% and DIFF%(1,2) indices described in Section 3.1.1. All methods rank as first bidder b5, with the only exception of the EI method, according to which an inversion between the first and second position of the ranking is observed. In effect, the DIFF%(1,2) index confirms the poor discriminative power of this method. On the contrary, among the deterministic methods, as already highlighted, DEA provides the best discrimination in the awarding process, thanks to 23

Journal Pre-proof

pro of

the cross-efficiency evaluation, and shows the highest robustness in the ranking. Looking at the fuzzy methods, FAHP exhibits the highest ability to discriminate among bidders (much more than all the other methods), albeit with a lower robustness of the ranking. We also remark that, due to the exclusion of two bidders, a change in the final rankings provided by AHP and FAHP was observed with respect to the initially performed comparison with 8 bidders. Finally, we remark that the FDEA approach described in Section 2.2.3 allows obtaining possibility distributions of the bidders’ efficiencies, meaning that, in addition to the defuzzified score values in Table IX, interval solutions are also available, and thus it is possible to take into account potential errors in the judgements. In particular, Fig. 4 clearly shows that, in the considered tender, even in case of small changes in the experts’ evaluations, the final ranking would not change.

lP

re-

3.2.1. Overall comparison of the fuzzy MCDM approaches for PP This section summarizes the main findings obtained from the comparison of fuzzy methods. We remark that, since this phase of the research is still occurring, these are partial results, and a deep long-term analysis is still required. Table X reports, for the two fuzzy methods, the corresponding EI experts’ aggregated evaluation (in a 1-9 Saaty’s scale), with the same graphical representation adopted in Table VII. The final scores obtained by respectively applying the AHP and DEA methods are reported in the last columns of Table X, together with the average ranking. The last two rows of Table X also report the overall ALI% and DIFF%(1,2) indices of the methods, confirming that both AHP and DEA provide a robust ranking with a high discriminative power. Looking at the obtained rankings, we can conclude that the best fuzzy method for PP is FDEA. In fact, although the FAHP method has the ability to individually analyse each bidder on the basis of each criterion, its complexity in “manually” managing the fuzzy data actually strongly reduces its applicability to the PP field. TABLE X. EVALUATION OF FAHP AND FDEA APPROACHES. KPIs Param. Consist. (16%) (8%) Methods 6

8

8

●●●

●●●●

●●●●

6

FDEA

●●●

ALI% DIFF%(1,2)

3.3. Discussion

9

9

●●●●○

●●●●○

Ease Inf. Loss PP Compl. (18%) (14%) (40%) 6

7

6

●●●

●●●○

●●●

5

8

4

●●○

●●●●

●●

urn a

FAHP

Perpect. (4%)

AHP DEA average score score % of rank rank rank (·102) (·102) occurrence 0.29

2

0.57

2

100%

2

0.43

1

1.00

1

100%

1

100% -32.56%

100% -43.00%

Jo

Summing up, the motivation of this work is to properly support decisions in the PP purchasing problem, selecting which method best fits the PP sector and its requirements. In fact, PP requires that each tender is awarded to the best bidder, while ensuring that the bidding process and the contract awarding respect the mandatory principles of nondiscrimination, free competition, and transparency of the procedures. Since usually several decision makers are involved in this process and the evaluation of bidders is made under many different and often conflicting criteria, multi-criteria group decision making approaches are useful tools to support decision-maker(s) in this task. In this context, the paper contributes to the research in this field by reviewing some wellknown approaches for the classical multi-criteria decision making process. We select among them the most commonly utilized techniques from the different existing methodological classes, analyze their weaknesses and drawbacks, and their strengths and benefits, and identify, when necessary, how these methods can be modified to guarantee the compliance 24

Journal Pre-proof

urn a

lP

re-

pro of

with the strict and specific requirements of public tenders. In addition, a comparison of such techniques is performed through several real tenders so as to identify the most promising approach for the PP, also taking into account uncertain data and the presence of multiple decision makers. Furthermore, we also carried out some indication on when (i.e. in which kind of tender) each method is preferable. The practical relevance of this paper lies on the one hand in providing a comprehensive and straightforward insight for non-specialists, on the other hand in representing a useful guide for experts and workers in the PP field. In fact, while supporting the awarding decision making process, our investigation also helps tenderers in exploring how to improve their performance and influence the procurement office decisions. Summarizing the main findings of our research, we find out that the most promising methods for application to PP are the AHP and the DEA, showing respectively a more robust ranking and a good compromise between the advantages of the other techniques (as for the AHP) and the highest discrimination in the ranking and the highest transparency in the final ranking (as for the DEA). In particular, although DEA is the only technique that is able to carry out the final ranking by obtaining a single ratio, it does not fully respect some requirements of the PP office (mainly due to partial unsuitability of the Assurance Region constraints in properly setting the established weights and to the lack of information on the incidence of each criterion on the final score). On the contrary, although the AHP procedure is characterized by some subjectivity, it appears as the most suitable method for PP, as it fully improves the performance provided by the EI method, without strongly altering the PP procedures. More in detail, our study shows that DEA is to be preferred in the case of complex purchasing services with multiple conflicting criteria, since it allows comparing tenderers having considerably different and heterogeneous operating characteristics against multiple evaluating criteria only requiring the decision makers to express their judgment on each bidder and each criterion. Conversely, AHP best fits structured services having a complex hierarchy, but with a limited number of items at each level, due to the need for pairwise comparisons of alternatives. On the contrary, both PROMETHEE and MAUT show poor performance, and as such they can be applied only to simple and standard services with a small number of alternatives, due to the difficulties in defining suitable preference or utility functions in addition to the pairwise comparison of each couple of alternatives. In addition, the presence of multiple decision makers (which often occurs in the PP awarding process) makes it difficult to have a consensus on the preference function (in PROMETHEE) as well as on the utility function (in MAUT). Although the research is still ongoing and a long-term analysis is required to get more robust results, the overall findings confirm some well-known peculiarities of the considered techniques. 4. CONCLUSION

Jo

The Public Procurement (PP) purchasing problem is a particular supplier selection problem that, in addition to the typical objectives of the private sector (i.e., to acquire the right goods/services, from the right source, at the right price, at the highest service level, and taking into account norms requirements and environmental impact), also requires the strict compliance with the principles of non-discrimination, free competition, and transparency of the awarding procedures. Therefore, the PP problem is a complex Multi-Criteria Decision Making (MCDM) process, where, typically, several decision makers are involved. This paper adapts some well-known MCDM approaches for supplier selection to the PP sector and to the group decision making process to identify which one best fits this specific problem. Throughout a literature review and by analyzing several real public tenders at a European Institution (EI), we compare some well-known MCDM methods from different methodological classes, namely AHP (a pair-wise comparisons based method), 25

Journal Pre-proof

urn a

REFERENCES

lP

re-

pro of

PROMETHEE and MAUT (both outranking methods), and DEA (in the class of mathematical methods) with the currently adopted EI method (a typical Most Economically Advantageous Tender method). After modifying the considered methods to cope with the PP sector peculiarities and mandatory rules, by defining some suitable indices and taking into account the EI experts’ judgement, we find out that the most promising methods for application to PP are the novel variants of AHP and DEA. In addition, since nowadays uncertainty is present in almost any project, we deepen the analysis incorporating in the comparison also the fuzzy counterparts of the two most promising methods. A preliminary comparison of FDEA and FAHP shows that the former seems to be more suitable for a proper awarding process in public tenders. Although the analysis is still occurring to properly evaluate on a long-term period DEA, AHP, and their fuzzy versions, the procurement office of the EI has already started to adopt the AHP approach, selected because of its ease of formulation with respect to the DEA method. Summing up, this work is intended as a guideline aid to the public procurement, under the assumption that, as first, it is essential to perform a correct assessment of the weights on each criterion and of the experts’ judgements on tenderers, so as to reduce subjectivity and guarantee the mandatory principles of transparency, non-discrimination, equality, and proportionality in the awarding process. In addition, the research may also represent a useful tool for tenderers, which can take advantage of our work by performing what-if analyses to investigate how to improve their performances (e.g., where to invest or how to slightly modify their bids) so as to influence the awarding process. Finally, the practical relevance of this paper also lies in providing a comprehensive and straightforward insight on MCDM approaches for non-specialists, students, and scholars. The future research is mainly devoted to finding how to extend the MCDM approaches so as to cope with a correct and impartial assessment of the weights. To this aim it would be important to identify a list of criteria and corresponding weights to be associated, based on the type of tender to award. In addition, we plan to extend the comparison to other well-known techniques, among those able to incorporate in the analysis multiple criteria and multiple decision makers.

Jo

[1] Government at a Glance: Public procurement, (2017). https://stats.oecd.org/Index.aspx?QueryId=78413 (accessed December 4, 2019). [2] K.V. Thai, G. Piga, Advancing Public Procurement: Practices, innovation and knowledge sharing, PrAcademics Press, Boca Raton Fla., 2007. https://doi.org/10.1007/978-3-319-13434-5_1. [3] L. Knight, C. Harland, J. Telgen, K.V. Thai, G. Callender, K. McKen, Public Procurement: International cases and commentary, Routledge, London, U. K., 2012. https://doi.org/10.4324/9780203815250. [4] Directive 2014/24/EU, (n.d.). https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014L0024 (accessed December 4, 2019). [5] Directive 2014/25/EU, (n.d.). https://eur-lex.europa.eu/eli/dir/2014/25/oj (accessed December 4, 2019). [6] A. Ishizaka, P. Nemery, Multi-Criteria Decision Analysis: Methods and software, Wiley, 2013. https://doi.org/10.1002/9781118644898. [7] M. Velasquez, P.T. Hester, An analysis of Multi-Criteria Decision Making methods, Int. J. Oper. Res. 10 (2013) 56–66. [8] V.B. Vommi, S.R. Kakollu, A simple approach to Multiple Attribute Decision Making using loss functions, J. Ind. Eng. Int. 13 (2017) 107–116. https://doi.org/10.1007/s40092-016-0174-6. [9] J. Chai, E.W.T. Ngai, Decision-making techniques in supplier selection : Recent accomplishments and what lies ahead, Expert Syst. Appl. 140 (2020) 112903. https://doi.org/10.1016/j.eswa.2019.112903. [10] J. Geldermann, A. Schöbel, On the similarities of some multi criteria decision analysis methods, J. MultiCriteria Decis. Anal. 18 (2011) 219–230. https://doi.org/10.1002/mcda. [11] C. Kahraman, S.C. Onar, B. Oztaysi, Fuzzy Multicriteria Decision-Making: A literature review, Int. J. Comput. Int. Sys. 8 (2015) 637–666. https://doi.org/10.1080/18756891.2015.1046325. [12] M. Dotoli, N. Epicoco, Integrated network design of agile resource-efficient Supply Chains under

26

Journal Pre-proof

[18] [19] [20] [21] [22] [23]

[24]

[25]

[26]

[27] [28]

[29]

[30] [31] [32] [33] [34] [35] [36]

pro of

[17]

re-

[16]

lP

[15]

urn a

[14]

Jo

[13]

uncertainty, IEEE Trans. Syst. Man, Cybern. Syst. (2018) 1–15. In press. https://doi.org/10.1109/TSMC.2018.2854620. S.-L. Si, X.-Y. You, H.-C. Liu, P. Zhang, DEMATEL technique: A systematic review of the state-of-the-art literature on methodologies and applications, Math. Probl. Eng. 2018 (2018) 1–33. https://doi.org/10.1155/2018/3696457. E. León-Castro, E. Avilés-Ochoa, J.M. Merigó, A.M. Gil-Lafuente, Heavy moving averages and their application in econometric forecasting, Cybern. Syst. 49 (2018) 26–43. https://doi.org/10.1080/01969722.2017.1412883. T. Khodadadzadeh, S.J. Sadjadi, A state-of-art review on Supplier Selection Problem, Decis. Sci. Lett. 2 (2013) 59–70. https://doi.org/10.5267/j.dsl.2013.03.001. J. Chai, J.N.K. Liu, E.W.T. Ngai, Application of decision-making techniques in supplier selection: A systematic review of literature, Expert Syst. Appl. 40 (2013) 3872–3885. https://doi.org/10.1016/j.eswa.2012.12.040. S. Önüt, S.S. Kara, E. Işik, Long term supplier selection using a combined fuzzy MCDM approach: A case study for a telecommunication company, Expert Syst. Appl. 36 (2009) 3887–3895. https://doi.org/10.1016/j.eswa.2008.02.045. D. Dalalah, M. Hayajneh, F. Batieha, A fuzzy Multi-Criteria Decision Making model for supplier selection, Expert Syst. Appl. 38 (2011) 8384–8391. https://doi.org/10.1016/j.eswa.2011.01.031. M. Dotoli, M. Falagario, A hierarchical model for optimal supplier selection in multiple sourcing contexts, Int. J. Prod. Res. 50 (2012) 2953–2967. https://doi.org/10.1080/00207543.2011.578167. M. Abdollahi, M. Arvan, J. Razmi, An integrated approach for supplier portfolio selection: Lean or agile?, Expert Syst. Appl. 42 (2015) 679–690. https://doi.org/10.1016/j.eswa.2014.08.019. C. Parkan, M.L. Wu, Comparison of three modern multicriteria decision-making tools, Int. J. Syst. Sci. 31 (2000) 497–517. https://doi.org/10.1080/002077200291082. M. Wang, S.J. Lin, Y.C. Lo, The comparison between MAUT and PROMETHEE, in: IEEE Int. Conf. Ind. Eng. Eng. Manag., 2010: pp. 753–757. https://doi.org/10.1109/IEEM.2010.5675608. N. Caterino, I. Iervolino, G. Manfredi, E. Cosenza, Comparative analysis of Multi-Criteria DecisionMaking methods for seismic structural retrofitting, Comput. Civ. Infrastruct. Eng. 24 (2009) 432–445. https://doi.org/10.1111/j.1467-8667.2009.00599.x. B. Sen, P. Bhattacharjee, U.K. Mandal, A comparative study of some prominent Multi Criteria Decision Making methods for connecting rod material selection, Perspect. Sci. 8 (2016) 547–549. https://doi.org/10.1016/J.PISC.2016.06.016. E. Mulliner, N. Malys, V. Maliene, Comparative analysis of MCDM methods for the assessment of sustainable housing affordability, Omega. 59 (2016) 146–156. https://doi.org/10.1016/j.omega.2015.05.013. M.M.D. Widianta, T. Rizaldi, D.P.S. Setyohadi, H.Y. Riskiawan, Comparison of multi-criteria decision support methods (AHP, TOPSIS, SAW & PROMETHEE) for employee placement, J. Phys. Conf. Ser. 953 (2018) 1–6. https://doi.org/10.1088/1742-6596/953/1/012116. A. Van Weele, Purchasing and Supply Chain management: Analysis, strategy, planning and practice, Cengage Learning EMEA, 2010. http://books.google.com/books?hl=en&lr=&id=ZQr8T0tmH88C&pgis=1. A. Emrouznejad, M. Marra, The state of the art development of AHP (1979–2017): A literature review with a social network analysis, Int. J. Prod. Res. 55 (2017) 6653–6675. https://doi.org/10.1080/00207543.2017.1334976. G. Bruno, E. Esposito, A. Genovese, R. Passaro, AHP-based approaches for supplier evaluation: Problems and perspectives, J. Purch. Supply Manag. 18 (2012) 159–172. https://doi.org/10.1016/j.pursup.2012.05.001. V. Yadav, M.K. Sharma, Multi-criteria supplier selection model using the Analytic Hierarchy Process approach, J. Model. Manag. 11 (2016) 326–354. https://doi.org/10.1108/JM2-06-2014-0052. M. Dotoli, N. Epicoco, M. Falagario, A fuzzy technique for supply chain network design with quantity discounts, Int. J. Prod. Res. 55 (2017) 1862–1884. https://doi.org/10.1080/00207543.2016.1178408. W. Ho, X. Xu, P.K. Dey, Multi-criteria decision making approaches for supplier evaluation and selection: A literature review, Eur. J. Oper. Res. 202 (2010) 16–24. https://doi.org/10.1016/j.ejor.2009.05.009. Steering Committee for the Review of Provision Commonwealth/State Service, Data Envelopment Analysis: A technique for measuring the efficiency of government service delivery, Agps. (1997) 1–142. T.S. Sinaga, K. Siregar, Supplier selection based on the performance by using Promethee method, in: IOP Conf. Ser. Mater. Sci. Eng., 2017. https://doi.org/10.1088/1757-899X/180/1/012118. M. Segura, C. Maroto, A multiple criteria supplier segmentation using outranking and value function methods, Expert Syst. Appl. 69 (2017) 87–100. https://doi.org/10.1016/j.eswa.2016.10.031. M. Shaik, W. Abdul-Kader, Green supplier selection generic framework: A Multi-Attribute Utility Theory approach, Int. J. Sustain. Eng. 4 (2011) 37–56. https://doi.org/10.1080/19397038.2010.542836.

27

Journal Pre-proof

Jo

urn a

lP

re-

pro of

[37] T.L. Saaty, Decision making with the analytic hierarchy process, Int. J. Serv. Sci. 1 (2008) 83–98. https://doi.org/10.1504/IJSSCI.2008.017590. [38] R. Ramanathan, A note on the use of the Analytic Hierarchy Process for environmental impact assessment, J. Environ. Manage. 63 (2001) 27–35. https://doi.org/10.1006/jema.2001.0455. [39] C. Macharis, J. Springael, K. De Brucker, A. Verbeke, PROMETHEE and AHP: The design of operational synergies in multicriteria analysis - Strengthening PROMETHEE with ideas of AHP, Eur. J. Oper. Res. 153 (2004) 307–317. https://doi.org/10.1016/S0377-2217(03)00153-X. [40] I. Millet, W.C. Wedley, Modelling risk and uncertainty with the analytic hierarchy process, J. MultiCriteria Decis. Anal. 11 (2002) 97–107. https://doi.org/10.1002/mcda.319. [41] S.H. Huang, H. Keskar, Comprehensive and configurable metrics for supplier selection, Int. J. Prod. Econ. 105 (2007) 510–523. https://doi.org/10.1016/j.ijpe.2006.04.020. [42] J. Pérez, J.L. Jimeno, E. Mokotoff, Another potential shortcoming of AHP, Top. 14 (2006) 99–111. https://doi.org/10.1007/BF02579004. [43] E. Triantaphyllou, S.H. Mann, Using the Analytic Hierarchy Process for decision making in engineering applications: Some challenges, Int. J. Ind. Eng. Theory, Appl. Pract. 2 (1995) 35–44. [44] M. Behzadian, R.B. Kazemzadeh, A. Albadvi, M. Aghdasi, PROMETHEE: A comprehensive literature review on methodologies and applications, Eur. J. Oper. Res. 200 (2010) 198–215. https://doi.org/10.1016/j.ejor.2009.01.021. [45] J.S. Dyer, Multiattribute utility theory (MAUT), Int. Ser. Oper. Res. Manag. Sci. 233 (2016) 285–314. https://doi.org/10.1007/978-1-4939-3094-4_8. [46] J. Liesiö, A. Punkka, Baseline value specification and sensitivity analysis in multiattribute project portfolio selection, Eur. J. Oper. Res. 237 (2014) 946–956. https://doi.org/10.1016/j.ejor.2014.02.009. [47] A. Charnes, W.W. Cooper, E. Rhodes, Measuring the efficiency of decision making units, Eur. J. Oper. Res. 2 (1978) 429–444. https://doi.org/10.1016/0377-2217(78)90138-8. [48] W.W. Cooper, L.M. Seiford, K. Tone, Data Envelopment Analysis: A comprehensive text with models, applications, references and DEA-solver software, Springer US, 2007. https://doi.org/10.1007/978-0-38745283-8. [49] S.T. Liu, M. Chuang, Fuzzy efficiency measures in fuzzy DEA/AR with application to university libraries, Expert Syst. Appl. 36 (2009) 1105–1113. https://doi.org/10.1016/j.eswa.2007.10.013. [50] M. Dotoli, N. Epicoco, M. Falagario, F. Sciancalepore, A stochastic cross-efficiency Data Envelopment Analysis approach for supplier selection under uncertainty, Int. Trans. Oper. Res. 23 (2016) 725–748. https://doi.org/10.1111/itor.12155. [51] M. Dotoli, N. Epicoco, M. Falagario, F. Sciancalepore, A cross-efficiency fuzzy Data Envelopment Analysis technique for performance evaluation of Decision Making Units under uncertainty, Comput. Ind. Eng. 79 (2015) 103–114. https://doi.org/10.1016/j.cie.2014.10.026. [52] T.R. Sexton, R.H. Silkman, A.J. Hogan, Data Envelopment Analysis: Critique and extensions, New Dir. Progr. Eval. 1986 (1986) 73–105. https://doi.org/10.1002/ev.1441. [53] M. Falagario, F. Sciancalepore, N. Costantino, R. Pietroforte, Using a DEA-cross efficiency approach in Public Procurement tenders, Eur. J. Oper. Res. 218 (2012) 523–529. https://doi.org/10.1016/j.ejor.2011.10.031. [54] J. Doyle, R. Green, Efficiency and cross-efficiency in DEA derivations, meanings and uses, J. Oper. Res. Soc. 45 (1994) 567–578. https://doi.org/10.1057/jors.1994.84. [55] O.B. Olesen, N.C. Petersen, Stochastic Data Envelopment Analysis—A review, Eur. J. Oper. Res. 251 (2016) 2–21. https://doi.org/10.1016/j.ejor.2015.07.058. [56] A. Emrouznejad, M. Tavana, A. Hatami-Marbini, The state of the art in fuzzy Data Envelopment Analysis, Perform. Meas. with Fuzzy Data Envel. Anal. Stud. Fuzziness Soft Comput. 309 (2014) 1–45. https://doi.org/10.1007/978-3-642-41372-8-1. [57] H.-J. Zimmermann, Fuzzy set theory review 2010, Wiley Interdiscip. Rev. Comput. Stat. 2 (2010) 317– 332. https://doi.org/10.1002/wics.82. [58] A. Mardani, A. Jusoh, E.K. Zavadskas, Fuzzy Multiple Criteria Decision-Making techniques and applications - Two decades review from 1994 to 2014, Expert Syst. Appl. 42 (2015) 4126–4148. https://doi.org/10.1016/j.eswa.2015.01.003. [59] S. Kubler, J. Robert, W. Derigent, A. Voisin, Y. Le Traon, A state-of the-art survey & testbed of fuzzy AHP (FAHP) applications, Expert Syst. Appl. 65 (2016) 398–422. https://doi.org/10.1016/j.eswa.2016.08.064. [60] G. Kabir, M. Ahsan Akhtar Hasin, Comparative analysis of AHP and fuzzy AHP models for multicriteria inventory classification, Int. J. Fuzzy Log. Syst. 1 (2011) 1–16. [61] T.-C. Wang, Y.-H. Chen, Some issues on consistency of fuzzy Analytic Hierarchy Process, in: 2006 Int. Conf. Mach. Learn. Cybern., 2006. https://doi.org/10.1109/ICMLC.2006.259043. [62] E. Forman, K. Peniwati, Aggregating individual judgments and priorities with the Analytic Hierarchy

28

Journal Pre-proof

Jo

urn a

lP

re-

pro of

Process, Eur. J. Oper. Res. 108 (1998) 165–169. https://doi.org/10.1016/S0377-2217(97)00244-0. [63] J. Skorupski, Multi-criteria group decision making under uncertainty with application to air traffic safety, Expert Syst. Appl. 41 (2014) 7406–7414. https://doi.org/10.1016/j.eswa.2014.06.030.

29

*Highlights (for review)

Journal Pre-proof

HIGHLIGHTS  The paper proposes a comparison of Multi-Criteria Decision Making approaches.  We focus on the Public Procurement awarding process.

pro of

 This sector is characterized by strict requirements.

 Several real public tenders from a European Institution are evaluated.

Jo

urn a

lP

re-

 Models are compared both under deterministic and fuzzy settings.

*Declaration of Interest Statement

Journal Pre-proof

Prof. Ing. Mariagrazia Dotoli

pro of

Professor M. Köppen Editor in Chief Applied Soft Computing th

Bari, September 28 , 2019

Obj: Declaration of Interests Statement Dear Editor in Chief,

re-

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Sincerely,

lP

(signed on behalf of all the authors by the corresponding author) Prof. Mariagrazia Dotoli, Ph.D Associate Professor in Automation

Jo

urn a

Please send the correspondence to: Prof. Ing. Mariagrazia Dotoli Dipartimento di Elettrotecnica ed Elettronica Politecnico di Bari Via Re David 200 70125 Bari, Italy e-mail: [email protected] phone +39 080 5963667, fax +39 080 5963410

DEE - Politecnico di Bari, Via Re David, 200 - 70125 Bari e-mail: [email protected] Tel: +390805963667, +393358230784 (mob.), Fax: +390805963410

*Credit Author Statement

Journal Pre-proof

Prof. Ing. Mariagrazia Dotoli

pro of

Professor M. Köppen Editor in Chief Applied Soft Computing rd

Bari, December 23 , 2019

Obj: Credit author statement for manuscript ASOC-D-18-02589R3 Author Contributions to manuscript ASOC-D-18-02589R3: All the authors contributed equally to the manuscript, in particular as regards conceptualization, methodology, software, validation, writing - original draft, and writing - review and editing.

re-

Sincerely,

lP

Prof. Mariagrazia Dotoli, Ph.D Associate Professor in Automation

Jo

urn a

Please send the correspondence to: Prof. Ing. Mariagrazia Dotoli Dipartimento di Elettrotecnica ed Elettronica Politecnico di Bari Via Re David 200 70125 Bari, Italy e-mail: [email protected] phone +39 080 5963667, fax +39 080 5963410

DEE - Politecnico di Bari, Via Re David, 200 - 70125 Bari e-mail: [email protected] Tel: +390805963667, +393358230784 (mob.), Fax: +390805963410