A method for the evaluation of risk in IT projects

Accepted Manuscript

A method for the evaluation of risk in IT projects Antonio Rodr´ıguez , Francisco Ortega , Ramiro Concepcion ´ PII: DOI: Reference:

S0957-4174(15)00685-5 10.1016/j.eswa.2015.09.056 ESWA 10328

To appear in:

Expert Systems With Applications

Received date: Revised date: Accepted date:

4 August 2014 28 September 2015 29 September 2015

Please cite this article as: Antonio Rodr´ıguez , Francisco Ortega , Ramiro Concepcion ´ , A method for the evaluation of risk in IT projects, Expert Systems With Applications (2015), doi: 10.1016/j.eswa.2015.09.056

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

Highlights  A method for the evaluation of risk based on the combination of AHP (FAHP) and fuzzy inference system (FIS).  Several advantages over classic implementations.  A more flexible, intuitive, verifiable and easier way to implement risk evaluation incorporating graphical tools.  Detection and analysis of rank reversal.

ACCEPTED MANUSCRIPT

A method for the evaluation of risk in IT projects Antonio Rodríguez El Sabio, Villanueva de La Cañada, 28691 Madrid, Spain Corresponding author Tel +34913489146; +34686403654 E-mail address: [email protected]

Francisco Ortega

CR IP T

Departamento de Ingenierías TIC, Escuela Politécnica Superior, Universidad Alfonso X

AN US

Área de Proyectos de Ingeniería, Departamento de Explotación y Prospección de Minas, Universidad de Oviedo, Calle Independencia 13, 33004 Oviedo, Spain Tel +34985104272

M

E-mail address: [email protected]

Ramiro Concepción

ED

Área de Proyectos de Ingeniería, Departamento de Explotación y Prospección de Minas, Universidad de Oviedo, Calle Independencia 13, 33004 Oviedo, Spain

PT

Tel +34985105425

AC

CE

E-mail address: [email protected]

ACCEPTED MANUSCRIPT ABSTRACT Information technology projects are particularly prone to failure due to their specific characteristics, making risk management become one of the critical elements in IT projects management. That is why several authors have developed risk evaluation methods, some of them based on fuzzy logic. This article proposes a new risk

CR IP T

assessment method based in a combination of Fuzzy Analytic Hierarchy Process (FAHP) and Fuzzy Inference System (FIS). FIS is used for the integration of the groups of risk factors. These risk factors are the evaluation criteria of a modified FAHP which minimizes the disadvantages of the classic implementation of FAHP in order to obtain a

AN US

more intuitive and easily adjustable model for multicriteria decision analysis with a lower computational need. The proposed model takes into consideration the different levels of uncertainty, the interrelationship among groups of risk factors, and the possibility of adding or suppressing options without losing the consistency with The new method is especially suitable for the evaluation of

M

previous evaluations.

development projects in the area of IT in which multiple interrelated risk factors can be

ED

particularly uncertain and imprecise. To implement the evaluation method, a hierarchy of risk factors was implemented. A numerical example is presented with data from

PT

three actual cases of IT projects, showing the applicability of the new method, the suitability of the selected taxonomy, and the significance of a few risk factors. Several

CE

future lines of work are proposed.

AC

Keywords: AHP; FIS; Fuzzy logic; Risk assessment; Project evaluation 1.

Introduction

A project is a temporary organization to obtain a unique product (PMI, 2013) and risk management constitutes an integral part of the general project management (Cooper et al., 2005; Mignerat & Rivard, 2012). The definition of risk usually refers to uncertain events that may affect the project success, due to effects (Hillson, 2002) in cost, time, or in the quality of the project deliverables. The evaluation of the level of risk in a

ACCEPTED MANUSCRIPT project involves two aspects: the probability of materialization of the risk events and the expected magnitude of their effects (Haimes, 2004). Both aspects are affected by uncertainty, imprecision and subjectivity. Uncertainty, imprecision and subjectivity, are aggravated in Information Technology projects due to their specific characteristics (Fu, Li, & Chen, 2012; Gu et al., 2014;

CR IP T

Savolainen, Ahonen, & Richardson, 2012) with a considerable amount of interrelated risk factors (Coombs, 2015; Lehtinen, 2014) leading to a higher failure rate (Altahtooh & Emsley, 2014; Flyvbjerg & Budzier, 2011; Kawamura & Takano, 2014; Perkusich et al., 2015; Whitney & Daniels, 2013). To try to deal with these problems, several

AN US

methods have been proposed for the assessment of risk in IT projects, some of them based on fuzzy logic (Elzamly & Hussin, 2014; Goyal, Satapathy, & Rath, 2015; Samantra, Datta, & Mahapatra, 2014; Taylan, 2014). Fuzzy logic permits the use of linguistic variables and is especially suitable to deal with uncertainty and ambiguity.

M

Moreover, methods for the assessment of risk in IT projects that are based on AHP (Wu & Teng, 2010) can implement pairwise comparison for a more intuitive evaluation,

ED

and a hierarchy to deal with a certain amount of risk factors; while methods based on FIS (Pourdarab, Nosratabadi, & Nadali, 2011) incorporate expert knowledge and take

PT

into account the interrelationship among risk factors.

CE

Fuzzy logic (Zadeh, 1965) substitutes the classic crisp two-valued logic (true and false) with continuous graded membership functions that go from absolute true to absolute

AC

false. Fuzzy logic is useful for processing subjective evaluations and permits the implementation of mathematical models for the analysis of uncertain and imprecise situations (Wong & Lai, 2011). That is why it is suitable to deal with risk evaluation (Lin, Chen, & Peng, 2012; Liu et al., 2012; Wu et al., 2010; Huang, Zhao, & Tang, 2009). Analytic Hierarchy Process (AHP) (Saaty, 1980) is a methodology for multicriteria decision-making (MCDM) (Saaty, 2008) that builds a hierarchy of criteria to evaluate options. AHP calculates the criteria weights and performs the evaluation of options

ACCEPTED MANUSCRIPT through pair-wise comparisons. The pair-wise comparison is more intuitive and usually more coherent than the isolated assignation of values. The hierarchical scheme brings a more organized vision of the problem, providing the structure for analyzing and grouping decision criteria. AHP has been widely used for evaluation and decision making in different areas (Subramanian & Ramanathan, 2012) including risk

CR IP T

assessment (Dey, 2010). Fuzzy AHP (FAHP) (Van Laarhoven & Pedrycz, 1983) is a technique based on fuzzy logic and AHP which inherits the advantages of both (Ishizaka, 2014) and is often implemented with fuzzy triangular numbers (FTN) (Chang, 1996) as membership

AN US

functions. FAHP can be used for the evaluation and ranking of alternatives (Kahraman, Cebeci, & Ruan, 2004; Mikhailov & Tsvetinov, 2004; Rodríguez, Ortega, & Concepción, 2013; Sinuany-Stern, 1988). FAHP has the advantage of permitting the use of linguistic values which are suitable to deal with the imprecision and subjectivity of risk.

M

Fuzzy Inference Systems (FIS) (Mamdani, 1974) (also known as Fuzzy Rule Based Systems or FRBS) use inference rules to establish relationships between fuzzy

ED

variables to produce numeric output values from linguistic values associated to membership functions. FIS can be used for evaluation and classification purposes

PT

(Hudec & Vujošević, 2012).

CE

FIS has the advantage of the implementation of expert judgment through inference rules and the consideration of the interdependence among variables; FIS adjustment is

AC

simple and the inference mechanisms can be easily represented by surface graphs. Nevertheless, AHP (crisp AHP or FAHP) and FIS have some drawbacks. AHP has been criticized by many authors (Bana e Costa & Vansnick, 2008; Belton & Gear, 1983; Wang & Chin, 2009). The number of options to be evaluated with AHP will be limited by the human capability to perform simultaneous pair-wise comparisons (Junior, Osiro, & Carpinetti, 2013),

ACCEPTED MANUSCRIPT values assigned to pair comparisons can be affected by subjectivity (Saen, 2010) and need to be verified for consistency. AHP is a comparative method in which the entry of a new option will probably change the values previously assigned to others. AHP lack learning capabilities to introduce expert judgment (Castro-Schez et al., 2013).

CR IP T

Interdependence between factors is not considered in AHP. Analytical Network Process (ANP) (Saaty, 1996) is a more general form of AHP that takes into consideration the interaction between factors but ANP implementation is more complex (Abdolshah & Moradi, 2013; Hodgett, 2013; Mazurek & Kiszová, 2012) and it shares

AN US

other disadvantages present in AHP (Salo & Hämäläinen, 1997).

Wang, Luo, & Hua (2008) demonstrate that fuzzy AHP may drive to a wrong decision due to the calculation of weights that do not represent the relative importance of the evaluation criteria (Rodríguez, 2015). Fuzzy AHP assigns zero weight to some

M

evaluation criteria causing the waste of information.

ED

Ishizaka and Nguyen (2013) describe the lack of indications of how membership functions can be constructed, identifying 27 different representations of fuzzy

PT

membership functions, none of them justified. Rank reversal is the most debated problem of AHP (Ishizaka & Labib, 2009). It arises

CE

not only in AHP, but in other decision analysis methodologies (Wang & Luo, 2009), and

AC

consists in an inversion in the position in a ranking when an option is suppressed or a new one is added. Some modifications have been proposed to avoid these problems (Wang & Elhag, 2006; Wang & Chin, 2009; Wang & Chin, 2011). Some authors minimize the importance of rank reversal (Triantaphyllou & Lin, 1996; Zanakis et al., 1998) and others suggest that it is an inherent aspect of decision making and consider that, despite this and other problems, AHP and similar techniques should be considered

ACCEPTED MANUSCRIPT useful tools for decision making (Ishizaka & Labib, 2011; Kujawski, 2003; Tavana & Hatami-Marbini, 2011). After a very comprehensive and detailed review of the limitations of AHP, Ishizaka & Labib (2009) conclude that, despite still suffering from some theoretical disputes, AHP has reached a compromise solution between right modeling and usability that is the reason of its widespread use.

CR IP T

FIS may suffer of redundancy and inconsistency (Fantana et al., 1996; Štěpničkova, Štěpnička, & Dvořak, 2013) and has the disadvantage of being able to implement only a few evaluation criteria; otherwise, the number of inference rules may become unmanageable.

AN US

The contribution of the research described in this paper consists in the development of a new method for the quantitative evaluation of the overall level of risk in projects using risk factors as evaluation criteria. The proposed method is based on a combination of FAHP and FIS, benefiting from their advantages and minimizing their disadvantages. It

M

is based on a new approach to FAHP with classic pair-wise comparison for weight calculation and independent evaluation (Tüysüz & Kahraman, 2006) of risk factors. The

ED

highest level of the hierarchy is implemented through a Mamdani fuzzy inference

by FIS.

PT

system (FIS) (Mamdani, 1974). Other groups in the hierarchy may also be integrated

CE

The approach to FAHP proposed in this paper avoids the undesired assignation of null weights; justifies the construction of the membership function by implementing a

AC

measure of the evaluation uncertainty; suggest procedures for rank reversal verification; and presents a graphical method for AHP consistency assurance that simplifies the process. The autonomous evaluation (Tüysüz & Kahraman, 2006) permits the independent evaluation of projects in which the results obtained are not affected by the inclusion or exclusion of new options, permitting future comparisons and eliminating one cause of rank reversal. The use of FIS (Mamdani, 1974) facilitates the integration of expert knowledge and implements a more intuitive and adjustable

ACCEPTED MANUSCRIPT model that takes into consideration the interrelationship among factors. The use of surface graphs helps to detect possible inconsistencies in the FIS implementation. Furthermore, being used only for the integration of a few evaluation criteria, FIS implementation becomes simple and the probability of redundancy is low. 2. Related work

CR IP T

Several authors propose FIS for the implementation of decision making methods (Danisman, Bilasco, & Martinet, 2015; Hasan et al., 2015; Kumar et al., 2015; Nakashima-Paniagua, Doucette, & Moussa, 2014) some of them for the evaluation of risk (Camastra et al., 2015; Chandima Ratnayake, 2015; Paul, S.K. 2015). Other

AN US

authors propose decision making methods based on FAHP (Budak & Ustundag, 2015; Dong, Li, & Zhang, 2015; Jaiswal et al., 2015), including methods for risk evaluation (Mangla, Kumar, Barua, 2015; Nezarat, Sereshki, & Ataei, 2015; Ni et al., 2015) Some authors propose ways of combining AHP and FIS for the implementation of

M

decision making methods in general, or for the implementation of risk evaluation

ED

methods in particular.

There are several methods in which some aspects are evaluated with crisp or fuzzy

PT

AHP and others, in parallel, with FIS; both methodologies are applied separately (Bon & Utami, 2014; Kinlic, 2010; Makui & Nikkhah, 2011), so they cannot be adequately

CE

considered as combinations.

AC

Some authors use FIS as an evaluation method, and crisp AHP for the selection of evaluation criteria and the assignation of weights to them (Carreño, Cardona, & Barbat, 2012; Donevska et al., 2012; Nilashi et al., 2015). They incorporate the advantages of AHP only for the selection and weighting of the evaluation criteria but not to the evaluation itself. In an implementation for the assessment of risk, Zeng, An, & Smith (2007) use crisp AHP in part of the calculations of the weights and Sugeno FIS to obtain a unique value

ACCEPTED MANUSCRIPT for project risk from three fuzzy parameters; Sugeno type FIS (Takagi & Sugeno, 1985) is considered to be less intuitive and less suited to human input (Kaur, A., & Kaur, A. 2012) than Mamdani type (Mamdani, 1974). Instead of crisp AHP, other authors propose similar approaches using FAHP (Behret, Uçal, & Kahraman, 2012; Hidayati et al., 2013; Shakouri & Tavassoli, 2012; Kutlu,

CR IP T

Behret, & Kahraman, 2014; Singh et al., 2015; Verma & Chaudhri, 2014); in these cases, the benefits of fuzzy logic are extended to the selection and weighting of criteria but the evaluation itself is based solely on FIS, as above.

Li, Avanatti, & Ray (2012) present the opposite solution: they use crisp AHP as an

AN US

evaluation method and FIS for the assignation of weights to the criteria. This method brings the advantages of fuzzy logic to the criteria weighting but not to the evaluation itself.

In addition, several authors propose more complex approaches, combining FIS and

M

FAHP with a third method. Actually, in these approaches, FIS and FAHP are used only

ED

for the selection and weighting of the evaluation criteria or for the previous screening of options, in a similar way to the above methods, but using the third technique for the

PT

evaluation itself: CBR (Noori, 2015); DELPHI (Mina et al, 2014); Multi-objective mathematical programming (Azadnia, Saman, & Wong, 2015); TOPSIS (Asemi &

CE

Asemi, 2014; Wang et al., 2004); and CHOQUET (Kwon and Lee, 2014) that use Sugeno type FIS (Takagi & Sugeno, 1985) instead of Mamdani type.

AC

FIS can also be used in an initial stage to transform fuzzy values (assigned in the valuation of all or part of the criteria) into crisp values to feed a crisp AHP (Kassar, Kervella, & Pujolle, 2008; Zorriassatine & Bagherpour, 2009). These cases benefit from the use of fuzzy logic only in the valuation of the criteria (that is, in collecting data for evaluation), but they lose, at the same entry level, the benefits of pair-wise comparison. Yang, Khan, & Sadiq (2011), in a method concerning environmental risk, propose a similar approach using FAHP, instead of crisp AHP, extending the benefits of fuzzy

ACCEPTED MANUSCRIPT logic to the evaluation itself, while maintaining the inconveniences of the classic implementation of FAHP. Priadi, Tjahjono & Benabdelhafid (2012) present an interesting method for the assessment of risk. Crisp AHP is used to evaluate static and dynamic factors which are then processed through a FIS. This method does not benefit from fuzzy logic at the

CR IP T

data entry level. Shahraki et al. (2014), in another interesting approach, use FAHP to establish a hierarchy and assign weights to the criteria; and implements FIS to perform the calculations, reducing the computational requirements but losing the benefits of pair-

assessment of risk by Takács (2010). 3. Methodology

AN US

wise comparison in the evaluation itself. A similar approach is implemented for the

The proposed method uses risk factors that may negatively affect the project as

M

evaluation criteria to obtain a quantitative value of risk. The steps of the proposed

ED

method are:

PT

Step 1: The evaluation criteria ( Ci ) are identified and organized in a hierarchy. Step 2: A weight ( w i ) is assigned to every risk factor. Weights are obtained by FAHP

CE

through pair-wise comparison of the importance of the evaluation criteria in the project.

AC

A fuzzy triangular number is assigned to every comparison:

a~ij  lij , mij , u ij  ,

1  mij  9 9

(1)

For mij  1 :

d d   ,  1 m m ij ij  2 2 , lij   d  1, mij   1 2 

0d 8

(2)

ACCEPTED MANUSCRIPT d d  mij  2 , mij  2  9 , u ij   d  9, mij   9 2 

0d 8

(3)

1  1 1 1  a~ ji  ~   , , a ij  u ij mij lij 

a~ii  1,1,1

(5)

CR IP T

For i  j ,

(4)

In contrast to the classic FAHP implementation with triangular numbers, mij is not a

~ , and associated fixed value, related to a fixed width fuzzy triangular number a ij

AN US

through a table to a linguistic variable.

In the method that we propose here, mij is a flexible value which measures how important criterion Ci is in relation to criterion C j . An example of pair comparison is

M

the following: criterion Ci is considered to be mij times more important than criterion

ED

Cj.

When criterion i is considered to be less important than criterion j , m ij will be set to a

PT

value less than 1: 0  mij  1 .

CE

When both criteria are considered equally important, m ij will be set to a value equal to

AC

1: mij  1

The fuzzy triangular number associated to m ij has a flexible width d , a dispersion value that measures the uncertainty or lack of confidence in the value assigned to m ij .

The higher the level of confidence in the value assigned to m ij , the lower the value of

d.

ACCEPTED MANUSCRIPT In order to facilitate the consistency preservation, the dispersion value must be constant within the same group and level in the hierarchy; so that, there will be only one value of d for every comparison matrix. To ensure the consistency of the assigned values, a graphical procedure is proposed. An example of the application of this procedure is shown in Fig. 1, in which the

CR IP T

following conditions must be fulfilled:

mij  mi ( j 1)  m( j 1) j , i  1 ; j  i  2

1 mij

(7)

AN US

m ji 

(6)

All the comparisons between the N elements of the same group of factors are assembled in a comparison matrix:

a~12 a~

 a~1N   a~2 N       a~NN 

12

 ~ a

(8)

ED

N2

M

 a~11  a~ ~ A   21   ~ a N 1

The strict application of the proposed graphical procedure makes unnecessary any

PT

further consistency verification. Nevertheless, small inconsistencies may be permitted in which case it would be necessary the verification procedure usually applied in classic

CE

AHP analysis (Chan & Kumar, 2007; Kwong & Bai, 2003; Nurcahyo et al., 2003).

AC

~ for every criterion is obtained from comparison fuzzy values a~ : Fuzzy weights w ~ w a~ij  ~ i wj

(9)

n

~  w i

n

 lij j 1

n

n

 mij ,

n

u

j 1

n

,

n

j 1

n

ij

 u  m  l j 1 i 1

ij

j 1 i 1

ij

(10)

n

j 1 i 1

ij

ACCEPTED MANUSCRIPT From the comparison matrix, a unique weight value is calculated: N N  N   lij m  ij  uij 1  j 1 j 1 j 1  N N  N N wi   N N 3 u ij  mij  lij    1  1 j 1 i 1 j 1 i 1 j i 

      

(11)

CR IP T

wi are no longer triangular fuzzy numbers, but crisp values. Normalized weights N wi  are obtained using the following formula:

N wi  

wi

(12)

N

 wn

AN US

n 1

The obtained weights can be verified for rank reversal using tables or using a graphical scale, similar to the one proposed by Huszák & Imre (2010).

M

Step 3:

A value ( Fi ) is assigned to every criterion ( C i ) as a result of the evaluation of its

ED

performance in the project. Adapting the method proposed by Tüysüz & Kahraman (2006), the project is evaluated in relation to every risk factor of the lower level through

PT

pair-wise comparison of the suitability of three linguistic variables, H (High), M

CE

(Medium) and L (Low), to describe the performance of each factor in the project. An example of pair-wise comparison is the following: “HIGH is m times more suitable than

AC

LOW to describe the influence of criterion i ” As result of the pair-wise comparisons, fuzzy triangular numbers are obtained and arranged in a 3X3 matrix. The calculations are performed following the same procedure used for the calculation of weights in Step 2. Three crisp values ( FiL , FiM , FiH ) are obtained for every risk factor i . Step 4:

ACCEPTED MANUSCRIPT Weights are assigned to every linguistic variable ( W L , W M , W H ) to obtain a single value for every factor:

Fi  W L FiL  W M FiM  W H FiH

(13)

The maximum and minimum possible values for Fi ( max Fi  and min Fi  ) are used

N Fi  

CR IP T

to obtain the normalized value N Fi  :

Fi  min Fi  max Fi   min Fi 

(14)

Weighted normalized values are obtained for every group Cn of criteria (group of risk

AN US

factors):

X n   wi  N Fi  i

(15)

Where wi are the weights and N Fi  are the normalized values corresponding to

M

factors Ci that are included in group Cn .

ED

Step 5:

The highest level of the hierarchy is implemented through a Mamdani fuzzy inference

PT

system (FIS) (Fig. 2) in which the input variables X n are the values obtained in Step 4

CE

for every group Cn of criteria. For every inference rule r , using the input membership functions, the input crisp

AC

values X n are transformed in fuzzy values Arn . For every rule, the minimum of Arn values is applied to the corresponding output membership function to obtain an output fuzzy value Br . All the Br values are aggregated using their maximum values to obtain a unique fuzzy number B . Finally, a crisp number x c is obtained by the

application of the centroid function to B :

ACCEPTED MANUSCRIPT 

xc 

 xBx dx

 

(16)

 Bx dx



The output of the FIS y is normalized using its maximum and minimum possible

N xc  

xc  min  xc  max x c   min xc 

CR IP T

values max  x c  and min  x c  :

(17)

Not only the highest level but any other group in the hierarchy can also be integrated using a FIS. For the highest level of the hierarchy, the normalized output of the FIS is

PR  N  xc  .

AN US

the quantitative value of the project risk

Fig. 3 shows a simplified example of the implementation of the method, omitting normalizations and with only two groups with two criteria every one. 4. Case study

M

Following the steps of the proposed methodology, we implemented a procedure for the

ED

evaluation of three development projects (P1, P2 and P3). Risk was evaluated from a negative perspective: higher values are assigned to worse

PT

situations. Evaluation criteria (risk factors) were considered from a positive perspective: higher values were assigned to better performances. The importance of every risk

CE

factor was considered to remain the same for the three projects and therefore the

AC

criteria weights were calculated only once. Step 1:

We based the hierarchy of risk factors on the model proposed by Hu et al. (2012):

C1 Requirement complexity C1.1

Development cost and period

ACCEPTED MANUSCRIPT C1.2

Functional requisites and technology (function point; real time and security; technology complexity)

C1.3

Requirements stability

C1.4

Schedule and budget management

C 2.1

Level of IT application

C 2.2

Business process

C 2.3

Top management support

C 2.4

Client

(client

collaboration of client team)

C 3 Contractor risk Project management

C 3.1.1

support;

client

experience;

Project team (development team; number of team members;

M

C 3.1

department

AN US

contribution

CR IP T

C 2 Customer risk

Project manager

C 3.1.3

Industry experience

PT

C 3.1.2

Software engineering

AC

CE

C 3.2

ED

number of collaborators)

C 3.2.1

Requirement development and management

C 3.2.2

Development and testing

C 3.2.3

Engineering support

C 3.2.4

Plan and control

We implemented two FIS, one for the integration of C 3 from C 3.1 and C 3.2 and the other for the implementation of the highest level of the hierarchy from C1 , C 2 and C 3 . Step 2:

ACCEPTED MANUSCRIPT We calculated the weight of every risk factor of the lower level. Fig. 4 shows the pairwise comparisons between factors in group C1 . Tables 1 to 4 shows the results for the factors of the lower level. Step 3: We evaluated the performance of every factor of the lower level using the graphical

CR IP T

method.

Table 5 shows the evaluation of performance ( F1.1 ) of factor C1.1 for the three projects (P1, P2 and P3) in relation to the three linguistic values low (L), medium (M) and high

AN US

(H).

For convenience, all the triangular numbers used in the evaluation of performance of factors were assigned a width d  1 .

M

Step 4:

We assigned weights to calculate a unique value for every criterion C i :

ED

Fi  0.2 FiM  0.8 FiH ; W L  0

(18)

PT

In order to obtain normalized values, we calculated the maximum and minimum values for the comparison matrix, as shown in Table 6.

CE

Tables 7 to 9 show the normalized, weighted and aggregated values obtained for every

AC

project.

Step 5:

Two FIS were implemented: C3_FIS that integrates C 3 from C 3.1 and C 3.2 ; and

OUT_FIS that implements the highest level of the hierarchy to obtain a value of the risk in the project from C1 , C 2 and C 3 . Tables 10 and 11 show the fuzzy triangular numbers (FTN) that constitute the input and output membership functions for both FIS.

ACCEPTED MANUSCRIPT Table 12 shows the inference rules that implement C3_FIS. Table 13 shows the inference rules that implement OUT_FIS. In order to obtain normalized values, we calculated the maximum and minimum outputs that may produce both FIS: max x c   0.92 ; min  x c   0.08 . Table 14 shows the inputs and the output of C3_FIS.

CR IP T

Table 15 shows the inputs and the output of OUT_FIS. The normalized output of OUT_FIS N  x C  is the quantitative value obtained for the level of risk in the project

PR .

AN US

4.1 Analysis of results

Fig. 5 shows the variation of the values of the normalized weights N wi  for criteria in group C1 in relation to the width ( d ) of the fuzzy triangular numbers (FTN).

M

Fig 6 shows the graphical verification of possible rank reversals in the calculation of weights for criteria in group C1 . Tables 16 and 17 show the verification for the rest of

ED

criteria. No position interchange was detected. Figures 7 and 8 show the surface graphs of C3_FIS and OUT_FIS respectively. No

PT

significant inconsistency was detected.

CE

5. CONCLUSIONS

In comparison to previous related researches, the contribution of this work consists in

AC

the development of a new model which, based on a modification that simplifies the implementation of FAHP, uses FIS to integrate groups in the hierarchy in order to obtain a more intuitive model that, with lower calculation requirements, is able to preserve the consistency. The new model was developed taking into account the characteristics of the information technology projects that, with multiple interrelated risk factors that are especially affected by imprecision and uncertainty, make projects in the IT area to be particularly prone to failure. The new method, proposed in this work,

ACCEPTED MANUSCRIPT benefits from the combination of FIS and a modified FAHP, through the use of fuzzy logic and pair comparison to deal with imprecision and uncertainty; the implementation of a hierarchy, to deal with a considerable amount of risk factors; and the inclusion of expert knowledge, to consider the interrelationship among groups of risk factors; all these in order to obtain a tool that therefore is especially suitable for the evaluation of

CR IP T

development projects in the area of IT. The new method minimizes the disadvantages of classic implementations and incorporates a more intuitive model that not only facilitates risk assessment and its consistency, but provides, in each case, a better understanding of how project risk level is related to its risk factors.

AN US

Through the case study, this paper has shown how the new method has allowed to assess and rank the level of risk of development projects in the area of information technology in an intuitive way, while preserving consistency, with a low computational need. Furthermore, the results obtained with different inputs and the sensitivity of the

M

model to them are empirical evidences of the suitability of the selected taxonomy of risk factors and the possibilities of this taxonomy for its application in the evaluation of risk

ED

in IT projects using other assessment methods. Another practical aspect that we learned from this specific case study was the high relative significance of a few risk

PT

factors, two of them related to requirements (requirements development/management and requirements stability), another related to top management support and two other

CE

related to the project manager and the project team.

AC

In relation to the classic implementation of FAHP, the approach to FAHP proposed in this paper has the following advantages: The graphical procedure for pair-wise comparison permits an easier intuitive implementation of AHP that verifies the consistency, suppresses the necessity of additional mathematical checking and facilitates, if necessary, the iteration of pair-wise comparisons; the assignation of weights and the evaluation of criteria can be displayed at a glance which facilitates discussions in evaluation sessions. The use of numerical values (instead of linguistic

ACCEPTED MANUSCRIPT variables) to assess how many times the influence of a factor is more important than the influence of another (or how many times a linguistic value is more suitable than another to describe the performance of a criterion in a project), provides flexibility and facilitates the consistency in AHP. The introduction of adjustable widths in fuzzy triangular numbers permits to take into consideration the different levels of uncertainty

CR IP T

in the evaluations. The applied procedure prevents the unwanted assignation of null weights. The inclusion rank reversal analysis either by the graphical method or by the proposed tables increases the level of confidence in the obtained results, reinforces the better comprehension of the evaluation process and permits the detection of

AN US

inconsistencies that may advise the introduction of modifications to the implementation. The use of autonomous evaluation has the following advantages: It permits a comparison of new options maintaining the values obtained for previously evaluated options. It suppresses one cause of rank reversal.

M

The use of FIS to integrate groups in the hierarchy has the following advantages: The integration among criteria is more intuitive and easily adjusted. It has graphical results

ED

that may improve the comprehension of the behavior of the evaluation procedure. The judgments of experts can be incorporated and easily modified through the inference

PT

rules. It takes into consideration the interrelationships among criteria.

CE

Nevertheless, the proposed method has the following drawbacks: The graphical procedure for pair-wise comparison loses its advantages when the number of criteria

AC

increases within the same level and group. The advantages of the implementation of a FIS vanish when the number of inputs to the FIS is increased. The use of autonomous evaluation lacks one of the advantages of AHP (crisp or fuzzy) that is the direct comparison of the performance of a criterion in two different options. This research can open several future lines of work, some of them based on adaptations of the proposed model: We would investigate the implementation of the classic pairwise comparison in order to implement the comparison of the performance

ACCEPTED MANUSCRIPT of a criterion in two different options, that could be useful in comparing a close set of options that is not going to change. Another research is on the implementation of some kind of adjustable weighting procedure, like the variable weights analysis (VWA), for the dynamic modification of weights according to the significance of the evaluation criteria in order to increase the resolution of the assessment. The use of other type of

CR IP T

membership functions would also be investigated, like the trapezoidal fuzzy set that would facilitate the inclusion of different types of input variables as subsets of the trapezoidal function, like crisp values, intervals or the fuzzy triangular number itself. Other future lines of work would be focused on a more in-depth analysis of the new method and its practical implications: The proposed model can also implement a

AN US

method for the evaluation of the significance of risk factors by introducing historical data from post-mortem studies of former projects. On the other side, the new model has the potential of its application to other decision making problems, such as the supplier selection problem. Another research is on the applicability of the new model as

M

a dynamic tool for monitoring and control in the continuous evaluation of risk and its

ED

evolution over the lifecycle of a project. Moreover, a comparative study of the new

PT

method with other fuzzy multi-criteria decision methods should be undertaken.

ACKNOWLEDGMENTS

CE

The authors would like to thank the reviewers for the valuable comments and

AC

suggestions that helped in improving the quality of this work.

REFERENCES Altahtooh, U., & Emsley, M. W. (2014, January). Is a Challenged Project One of the Final Outcomes for an IT Project? In IEEE 47th Hawaii International Conference on System Sciences (HICSS), 4296-4304. Abdolshah, M., & Moradi, M. (2013). Fuzzy Quality Function Deployment: An Analytical Literature Review. Journal of Industrial Engineering, 2013.

ACCEPTED MANUSCRIPT Asemi, A. & Asemi, A. (2014). Intelligent MCDM method for supplier selection under fuzzy environment. International Journal of Information Science and Management (IJISM), 33-40. Azadnia, A.H., Saman, M.Z.M., Wong, K.Y. (2015). Sustainable supplier selection and order lot-sizing: An integrated multi-objective decision-making process. International

CR IP T

Journal of Production Research, 53 (2), 383-408. Bana e Costa, C. A., & Vansnick, J. C. (2008). A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research, 187 (3), 1422-1428.

AN US

Behret, H., Uçal, I., & Kahraman, C. (2012). Multicriteria evaluation of environmentally conscious manufacturers under fuzzy environment. Multiple-Valued Logic and Soft Computing, 18(5-6), 457-477.

Belton, V., & Gear, A. E. (1983). On a shortcoming of Saaty’s method of Analytic

M

Hierarchies. Omega, 11 (3), 228-230.

ED

Bon, A. T., & Utami, S. F. (2014). An analytical hierarchy process and fuzzy inference system tsukamoto for production planning: a review and conceptual research. The

PT

Business & Management Review, 5(3), 101.

CE

Budak, A., Ustundag, A. (2015). Fuzzy decision making model for selection of real time location systems. Applied Soft Computing Journal, 36, 177-184.

AC

Camastra, F., Ciaramella, A., Giovannelli, V., Lener, M., Rastelli, V., Staiano, A., Staiano, G., & Starace, A. (2015). A fuzzy decision system for genetically modified plant environmental risk assessment using Mamdani inference. Expert Systems with Applications, 42(3), 1710-1716. Carreño, M. L., Cardona, O. D., & Barbat, A. H. (2012). New methodology for urban seismic risk assessment from a holistic perspective. Bulletin of Earthquake Engineering, 10(2), 547-565.

ACCEPTED MANUSCRIPT Castro-Schez, J. J., Miguel, R., Herrera, V., & Albusac J. A. (2013). Supporting multicriteria decisions based on a hierarchical structure by taking advantage of acquired knowledge. Applied Soft Computing, 13(1), 509-526. Chan, F., & Kumar, N. (2007). Global supplier development considering risk factors using fuzzy extended AHP-based approach. Omega, 35 (4), 417-431.

CR IP T

Chang, D. Y. (1996). Applications of the extent analysis method on fuzzy AHP. European Journal of Operational Research, 95 (3), 649-655.

Chandima Ratnayake, R.M. (2015). Estimation of maximum in-service inspection intervals based on risk: A fuzzy logic based approach. International Journal of

AN US

Performability Engineering, 11 (1), 23-32.

Coombs, C. R. (2015). When planned IS/IT project benefits are not realized: a study of inhibitors and facilitators to benefits realization. International Journal of Project

M

Management, 33(2), 363-379.

Cooper, D. F., Grey, S., Raymond, G., & Walker, P. (2005). Project Risk Management

ED

Guidelines. Managing Risk in Large Projects and Complex Procurements. John Wiley & Sons, Ltd.

PT

Danisman, T., Bilasco, I. M., & Martinet, J. (2015). Boosting gender recognition

CE

performance with a fuzzy inference system. Expert Systems with Applications, 42(5), 2772-2784.

AC

Dey, P. K. (2010). Managing project risk using combined analytic hierarchy process and risk map. Applied Soft Computing, 10(4), 990-1000.

Donevska, K. R., Gorsevski, P. V., Jovanovski, M., & Peševski, I. (2012). Regional non-hazardous landfill site selection by integrating fuzzy logic, AHP and geographic information systems. Environmental Earth Sciences, 67(1), 121-131.

ACCEPTED MANUSCRIPT Dong, M., Li, S., Zhang, H. (2015). Approaches to group decision making with incomplete information based on power geometric operators and triangular fuzzy AHP. Expert Systems with Applications, 42 (21), 7846-7857. Elzamly, A., & Hussin, B. (2014). Managing software project risks (Planning Phase) with proposed fuzzy regression analysis techniques with fuzzy concepts. Int. J. Inform.

CR IP T

Comput. Sci, 3, 31-40. Fantana, N., Weisbrod, J., Brown, A., & Forschungszentrum, B. A. (1996). Detecting local inconsistency and incompleteness in fuzzy rule bases. In 4th European Congress on Fuzzy and Intelligent Technologies (EUFIT’96). 1, 656-660.

AN US

Flyvbjerg, B., & Budzier, A. (2011). Why your IT project may be riskier than you think. Harvard Business Review, 89(9), 601-603.

Fu, Y., Li, M., Chen, F., 2012. Impact propagation and risk assessment of requirement changes for software development projects based on design structure matrix.

M

International Journal of Project Management. 30 (3), 363–373.

ED

Goyal, M.V., Satapathy, S.M., Rath, S.K. (2015) Software project risk assessment based on cost drivers and Neuro-Fuzzy technique. International Conference on

PT

Computing, Communication and Automation, ICCCA 2015, art. no. 7148487, 823-827.

CE

Gu, V. C., Hoffman, J. J., Cao, Q., & Schniederjans, M. J. (2014). The effects of organizational culture and environmental pressures on IT project performance: A

AC

moderation perspective. International Journal of Project Management, 32(7), 11701181.

Haimes, Y. Y. (2004). Risk Modeling, Assessment, and Management. Second Edition. John Wiley & Sons, Inc., Hoboken, NJ., 21. Hasan, M. M., Shohag, M. A. S., Azeem, A., & Paul, S. K. (2015). Multiple criteria supplier selection: a fuzzy approach. International Journal of Logistics Systems and Management, 20(4), 429-446.

ACCEPTED MANUSCRIPT Hidayati, J., Sukardi, S., Suryani, A., Sugiharto, S., & Fauzi, A. M. (2013). Optimization of Business Partners Feasibility for Oil Palm Revitalization Using Fuzzy Approach. International Journal on Advanced Science, Engineering and Information Technology, 3(2), 29-35. Hillson, D. (2002). Extending the risk process to manage opportunities. International

CR IP T

Journal of Project Management, 20(3), 235-240. Hodgett, R. E. (2013). Multi-Criteria Decision-Making in Whole Process Design. PhD disertation.

School of Chemical Engineering and Advanced Materials Newcastle

University, January 2013.

AN US

Hu, Y., Mo, X., Zhang, X., Zeng, Y., Du, J., & Xie, K. (2012). Intelligent analysis model for outsourced software project risk using constraint-based bayesian network. Journal of Software, 7(2), 440-449.

Huang, T., Zhao, R., & Tang, W. (2009). Risk model with fuzzy random individual claim

M

amount. European Journal of Operational Research, 192 (3), 879-890.

ED

Hudec, M., & Vujošević, M. (2012). Integration of data selection and classification by fuzzy logic. Expert Systems with Applications, 39 (10), 8817-8823.

PT

Huszák, Á., & Imre, S. (2010). Eliminating Rank Reversal Phenomenon in GRA-based

CE

Network Selection Method. International Conference on Communications (ICC). 23-27. Cape Town, South Africa. IEEE, May 2010.

AC

Ishizaka, A., & Labib, A. (2009). Analytic hierarchy process and expert choice: Benefits and limitations. OR Insight, 22 (4), 201-220. Ishizaka, A., & Labib, A. (2011). Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38 (11), 14336–14345. Ishizaka, A., & Nguyen, N. H. (2013). Calibrated fuzzy AHP for current bank account selection. Expert Systems with Applications, 40(9), 3775-3783.

ACCEPTED MANUSCRIPT Ishizaka, A. (2014). Comparison of fuzzy logic, AHP, FAHP and hybrid fuzzy AHP for new supplier selection and its performance analysis. International Journal of Integrated Supply Management, 9 (1-2), 1-22. Jaiswal, R.K., Ghosh, N.C., Lohani, A.K., Thomas, T. (2015). Fuzzy AHP Based Multi Crteria Decision Support for Watershed Prioritization. Water Resources Management,

CR IP T

29 (12), 4205-4227. Junior, F. R. L., Osiro, L., & Carpinetti, L. C. R. (2013). A fuzzy inference and categorization approach for supplier selection using compensatory and noncompensatory decision rules. Applied Soft Computing. 13(10), 4133-4147.

AN US

Kahraman, C., Cebeci, U., & Ruan, D. (2004). Multi-attribute comparison of catering service companies using fuzzy AHP: The case of Turkey. International Journal of Production Economics, 87 (2), 171-184.

Kassar, M., Kervella, B., & Pujolle, G. (2008). An overview of vertical handover

M

decision strategies in heterogeneous wireless networks. Computer Communications,

ED

31(10), 2607-2620.

Kaur, A., & Kaur, A. (2012). Comparison of mamdani-type and sugeno-type fuzzy

PT

inference systems for air conditioning system. International journal of soft computing

CE

and engineering, 2(2), 2231-2307. Kawamura, T., & Takano, K. (2014). Factors Affecting Project Performance of IS

AC

Development: Evidence from Japanese IT Vendors. Journal of Information Processing, 22(4), 689-700. Kinlic, M. S. (2010). Evaluation of e-Government Website Projects in Fuzzy Environment. International Journal of Arts and Sciences. 3(9), 395-403. Kujawski, E. (2003). Multi-criteria decision analysis: Limitations, pitfalls, and practical difficulties. Lawrence Berkeley National Laboratory. eScholarship, University of California. (August 2014).

ACCEPTED MANUSCRIPT Kumar, K., Deep, S., Suthar, S., Dastidar, M.G., Sreekrishnan, T.R. (2015). Application of fuzzy inference system (FIS) coupled with Mamdani’s method in modelling and optimization of process parameters for biotreatment of real textile wastewater. Desalination and Water Treatment, 8 p. Article in Press. Kutlu, A.C., Behret, H., Kahraman, C. (2014). A fuzzy inference system for multiple

CR IP T

criteria job evaluation using fuzzy AHP. Journal of Multiple-Valued Logic and Soft Computing, 23 (1-2), 113-133.

Kwong, C. K., & Bai, H. (2003). Determining the importance weights for the customer requirements in QFD using a fuzzy AHP with an extent analysis approach. Iie

AN US

Transactions, 35 (7), 619-626.

Kwon, I. K., & Lee, S. Y. (2014). Choquet integral-based product satisfaction inference in consideration of subjective decision-making tendencies. Journal of Computer Virology and Hacking Techniques, 10(2), 71-80.

M

Lehtinen, T. O., Mäntylä, M. V., Vanhanen, J., Itkonen, J., & Lassenius, C. (2014).

ED

Perceived causes of software project failures–An analysis of their relationships. Information and Software Technology, 56(6), 623-643.

PT

Li, C., Avanatti, S. G., & Ray, T. (2012). Implementing analytical hierarchy process using fuzzy inference technique in route guidance system. The 2012 International

CE

Conference on Artificial Intelligence ICAI'12. Las Vegas, NV. July 2012

AC

Lin, J. W., Chen, C. W., & Peng, C. Y. (2012). Potential hazard analysis and risk assessment of debris flow by fuzzy modeling. Natural hazards, 64(1), 273-282. Liu, K. F. R., Ko, C. Y., Fan, C., & Chen, C. W., (2012). Combining risk assessment, life cycle assessment, and multi-criteria decision analysis to estimate environmental aspects in environmental management system. The International Journal of Life Cycle Assessment, 17(7), 845-862.

ACCEPTED MANUSCRIPT Makui, A., & Nikkhah, Z.

(2011). Designing fuzzy expert system for creating and

ranking of tourism scenarios using fuzzy AHP method. Management Science Letters, 1(1), 29-40. Mamdani, E.H. (1974). Application of fuzzy algorithms for control of simple dynamic plant. Electrical Engineers, Proceedings of the Institution of, 121 (12), 1585-1588.

CR IP T

Mangla, S.K., Kumar, P., Barua, M.K. (2015). Risk analysis in green supply chain using fuzzy AHP approach: A case study. Resources, Conservation and Recycling. Article in Press.

Mazurek, J., & Kiszová, Z. (2012). Modeling dependence and feedback in ANP with

AN US

fuzzy cognitive maps. In Proceedings of the 30th international conference Mathematical methods in Economics (pp. 558-563).

Mignerat, M., & Rivard, S. (2012). The institutionalization of information system project

M

management practices. Information and Organization. 22 (2), 125–153. Mikhailov, L., & Tsvetinov, P. (2004). Evaluation of services using a fuzzy analytic

ED

hierarchy process. Applied Soft Computing, 5 (1), 23-33. Mina, H., Mirabedini, S. N., Kian, H., & Ghaderi, S. F. (2014). A new two stage

PT

integrated approach for green supplier selection. Applied mathematics in Engineering,

CE

Management and Technology, (Feb. 2014), 1247-1264. Nakashima-Paniagua, T., Doucette, J., & Moussa, W. (2014). Fabrication process

AC

suitability ranking for micro-electro-mechanical systems using a fuzzy inference system. Expert Systems with Applications, 41(9), 4123-4138. Nezarat, H., Sereshki, F., Ataei, M. (2015). Ranking of geological risks in mechanized tunneling by using Fuzzy Analytical Hierarchy Process (FAHP). Tunnelling and Underground Space Technology, 50, art. no. 1898, 358-364. Ni, L., Jiang, J., Wang, Z., Yao, J., Song, Y., Yu, Y. (2015). The organic peroxides instability rating research based on adiabatic calorimetric approaches and fuzzy

ACCEPTED MANUSCRIPT analytic hierarchy process for inherent safety evaluation. Process Safety Progress. Article in Press. Nilashi, M., Zakaria, R., Ibrahim, O., Majid, M. Z. A., Zin, R. M., Chugtai, M. W., Abidin, N. I. Z., Sahamir, S. R., & Yakubu, D. A. (2015). A knowledge-based expert system for assessing the performance level of green buildings. Knowledge-Based Systems, 86,

CR IP T

194-209. Noori, B. (2015). Developing a CBR system for marketing mix planning and weighting method selection using fuzzy AHP. Applied Artificial Intelligence, 29 (1), pp. 1-32.

Nurcahyo, G. W., Shamsuddin, S. M., Alias, R. A., & Sap, M. N. M. (2003). Selection of

AN US

defuzzification method to obtain crisp value for representing uncertain data in a modified sweep algorithm. Journal of Computer Science & Technology. 3(2), 22-28. Paul, S.K. (2015). Supplier selection for managing supply risks in supply chain: a fuzzy approach. International Journal of Advanced Manufacturing Technology, 79 (1-4), 657-

M

664.

ED

Perkusich, M., Soares, G., Almeida, H., & Perkusich, A. (2015). A procedure to detect problems of processes in software development projects using Bayesian networks.

PT

Expert Systems with Applications, 42(1), 437-450.

CE

PMI (2013). A Guide to the Project Management Body of Knowledge (PMBOK Guide), Fifth Edition. Project Management Institute, Pennsylvania.

AC

Pourdarab, S., Nosratabadi, H.E., Nadali, A. (2011). Risk assessment of information technology projects using fuzzy expert system. Communications in Computer and Information Science, 166 CCIS (PART 1), 563-576. Priadi, A. A., Tjahjono, T., & Benabdelhafid, A. (2012). Assessing Safety of Ferry Routes by Ship Handling Model through AHP and Fuzzy Approach. Intelligent Information Management, 4(25), 277-283.

ACCEPTED MANUSCRIPT Rodríguez, A. (2015). A fuzzy method for the selection of customized equipment suppliers in the public sector. Artificial Intelligence Research 2015, 4(1), 36-44. Rodríguez, A., Ortega, F., & Concepción, R. (2013). A method for the selection of customized equipment suppliers. Expert Systems with Applications. 40 (4), 1170–1176. Saaty, T. L. (1980). The analytic hierarchy process: planning, priority setting, resource

CR IP T

allocation. McGraw-Hill, New York, NY. Saaty, T. L. (1996). Decision making with dependence and feedback: The analytic network process. Rws Publications. Pittsburgh, Pennsylvania.

Saaty, T. L. (2008). Decision making with the Analytic Hierarchy Process. International

AN US

Journal of Services Sciences, 1 (1), 83-98.

Saen, R. F. (2010). A new model for selecting third-party reverse logistics providers in the presence of multiple dual-role factors. The International Journal of Advanced

M

Manufacturing Technology, 46(1-4), 405-410.

Salo, A. A., & Hämäläinen, R.P. (1997). On the measurement of preferences in the

ED

analytic hierarchy process. Journal of Multi-Criteria Decision Analysis, 6(6), 309-319.

PT

Samantra, C., Datta, S., & Mahapatra, S. S. (2014). Risk assessment in IT outsourcing using fuzzy decision-making approach: An Indian perspective. Expert Systems with

CE

Applications, 41(8), 4010-4022. Savolainen, P., Ahonen, J.J., Richardson, I., 2012. Software development project

AC

success and failure from the supplier's perspective: A systematic literature review. International Journal of Project Management. 30 (4), 458-469. Shahraki, M.R., Forghani, S.F., Kalantari, S., Estanesti, M. (2014). Prioritizing suppliers of a petrochemical unit considering fuzzy approach (case study: Mobin petrochemical unit). Advances in Environmental Biology, 8 (7), 2062-2068.

ACCEPTED MANUSCRIPT Shakouri G. H., & Tavassoli N, Y. (2012). Implementation of a hybrid fuzzy system as a decision support process: A FAHP–FMCDM–FIS composition. Expert Systems with Applications, 39(3), 3682-3691. Singh, S., Olugu, E. U., Musa, S. N., & Mahat, A. B. (2015). Fuzzy-based sustainability evaluation method for manufacturing SMEs using balanced scorecard framework.

CR IP T

Journal of Intelligent Manufacturing, 1-18. Sinuany-Stern, Z. (1988). Ranking of sports teams via the AHP. Journal of the Operational Research Society, 661-667.

Štěpničkova, L., Štěpnička, M., & Dvořak, A. (2013). New results on redundancies of

and Technology (EUSFLAT 2013).

AN US

fuzzy/linguistic IF-THEN rules. 8th Conference of the European Society for Fuzzy Logic

Subramanian, N., & Ramanathan, R. (2012). A review of applications of Analytic

Economics. 138 (2), 215–241.

M

Hierarchy Process in operations management. International Journal of Production

ED

Takács, M. (2010). Multilevel Fuzzy Approach to the Risk and Disaster Management. Acta Polytechnica Hungarica, 7(4), 91-102.

PT

Takagi, T., & Sugeno, M. (1985). Fuzzy identification of systems and its applications to

132.

CE

modeling and control. IEEE Transactions on Systems, Man and Cybernetics. 15, 116–

AC

Tavana, M., & Hatami-Marbini, A. (2011). A group AHP-TOPSIS framework for human spaceflight mission planning at NASA. Expert Systems with Applications, 38 (11), 13588-13603. Taylan, O. (2014). IT project risk assessment of learning organizations by fuzzy set and systems. International Journal of Organizational Analysis, 22(2), 161-180.

ACCEPTED MANUSCRIPT Triantaphyllou, E., & Lin, C. T. (1996). Development and evaluation of five fuzzy multiattribute

decision-making

methods.

international

Journal

of

Approximate

reasoning, 14 (4), 281-310. Tüysüz, F., & Kahraman, C. (2006). Project risk evaluation using a fuzzy analytic hierarchy process: an application to information technology projects. International

CR IP T

Journal of Intelligent Systems, 21 (6), 559-584. Van Laarhoven, P. J. M., & Pedrycz, W., (1983). A fuzzy extension of Saaty's priority theory. Fuzzy sets and Systems, 11 (1), 199-227.

Verma, S. & Chaudhri, S. (2014). Integration of fuzzy reasoning approach (FRA) and

AN US

fuzzy analytic hierarchy process (FAHP) for risk assessment in mining industry. Journal of Industrial Engineering and Management, 7(5), 1347-1367.

Wang, Y., Elghoneimy, E., Gruver, W. A., Fleetwood, M., & Kotak, D. B. (2004). A fuzzy multiple decision support for jag selection. 2004 Annual Meeting of the North

M

American Fuzzy Information Processing Society (NAFIPS'04). 2, 717-722. IEEE, June

ED

2004.

Wang, Y. M., & Chin, K. S. (2009). A new data envelopment analysis method for

PT

priority determination and group decision making in the analytic hierarchy process.

CE

European Journal of Operational Research, 195 (1), 239-250. Wang, Y. M., & Chin, K. S. (2011). Fuzzy analytic hierarchy process: A logarithmic

AC

fuzzy preference programming methodology. International Journal of Approximate Reasoning, 52(4), 541-553. Wang, Y. M., & Elhag, T. M. S. (2006). An approach to avoiding rank reversal in AHP. Decision Support Systems, 42 (3), 1474–1480. Wang, Y. M., & Luo, Y. (2009). On rank reversal in decision analysis. Mathematical and Computer Modelling, 49 (5), 1221-1229.

ACCEPTED MANUSCRIPT Wang, Y. M., Luo, Y., & Hua, Z. (2008). On the extent analysis method for fuzzy AHP and its applications. European Journal of Operational Research, 186 (2), 735-747. Whitney, K. M., & Daniels, C. B. (2013). The Root Cause of Failure in Complex IT Projects: Complexity Itself. Procedia Computer Science, 20, 325-330. Wong, B. K., & Lai, V. S. (2011). A survey of the application of fuzzy set theory in

Production Economics. 129 (1), 157–168.

CR IP T

production and operations management: 1998–2009. International Journal of

Wu, D. D., Zhang, Y., Wu, D., & Olson, D. L. (2010). Fuzzy multi-objective programming for supplier selection and risk modeling: A possibility approach. European

AN US

Journal of Operational Research, 200 (3), 774-787.

Wu, H. & Teng, B. (2010). IT project risk assessment model based on fuzzy-AHP. 2nd International Conference on Information Engineering and Computer Science -

M

Proceedings, ICIECS 2010, art. no. 5677767

Yang, M., Khan, F. I., & Sadiq, R. (2011). Prioritization of environmental issues in

ED

offshore oil and gas operations: A hybrid approach using fuzzy inference system and

22-34.

PT

fuzzy analytic hierarchy process. Process Safety and Environmental Protection, 89(1),

CE

Zadeh, L. A. (1965). Fuzzy sets. Information and control, 8(3), 338-353. (August

AC

2014)

Zanakis, S. H., Solomon, A., Wishart, N., & Dublish, S. (1998). Multi-attribute decision making: A simulation comparison of select methods. European Journal of operational research, 107 (3), 507-529. Zeng, J., An, M., & Smith, N. J. (2007). Application of a fuzzy based decision making methodology to construction project risk assessment. International journal of project management, 25 (6), 589-600.

ACCEPTED MANUSCRIPT Zorriassatine, F., & Bagherpour, M. (2009). A new method for estimating project weight

AC

CE

PT

ED

M

AN US

CR IP T

values. Journal of Applied Sciences, 9, 917-923.

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

Fig. 1. Graphiical assignation of va alues to pa air-wise com mparisons

ED

M

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

Fig. 2. Fuzzy inference system (FIS)

AC

CE

PT

ED

M

AN US

CR IP T

ACCEPTED MANUSCRIPT

ACCEPTED MANUSCRIPT F Fig. 3. Simp plified exam mple of the im mplementattion of the proposed p m model omittin ng

CR IP T

n normalizatio on

AC

CE

PT

ED

M

AN US

Fig. 4. Graphiccal pair-wise e compariso ons between n evaluation n criteria in group C1

Fig. 5. Normalizzed weights s N wi  for criteria in group g s. width ( d ) of FTN. C1 vs

AC

CE

PT

ED

M

AN US

CR IP T

ACCEPTED MANUSCRIPT

Fig. 6. Verificatiion of rank reversal in tthe calculattion of weights of criterria in group C1

AC

CE

PT

ED

M

Fig g. 7. Surface e graph for C3_FIS

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

ACCEPTED MANUSCRIPT

Fig. 8. Surface graphs for O UT_FIS.

ACCEPTED MANUSCRIPT

Table 1. Pair-wise comparison matrix, weights and normalized weights for group C1 and d  2

N wi 

C1.2

C1.3

C1.4

wi

C1.1

(1, 1, 1)

(0.43, 0.75, 1)

(0.23, 0.3, 0.43)

(0.38, 0.6, 1)

0.1524

0.1320

C1.2

(1, 1.33, 2.33)

(1, 1, 1)

(0.29, 0.4, 0.67)

(0.44, 0.8, 1)

0.2137

0.1851

C1.3

(2.33, 3.33, 4.33)

(1.5, 2.5, 3.5)

(1, 1, 1)

(1, 2, 3)

0.5062

0.4385

C1.4

(1, 1.67, 2.67)

(1, 1.25, 2.25)

(0.33, 0.5, 1)

(1, 1, 1)

0.2821

0.2444

CR IP T

C1.1

Table 2. Pair-wise comparison matrix, weights and normalized weights for group C 2 and

d 2

N wi 

C 2.2

C 2.3

C 2.4

wi

C 2.1

(1, 1, 1)

(0.5, 2, 2)

(0.23, 0.3, 0.43)

(0.38, 0.6, 1)

0.1758pair-

0.1497

C 2.2

(0.5, 1, 2)

(1, 1, 1)

(0.23, 0.3, 0.43)

(0.38, 0.6, 1)

0.1758

0.1497

C 2.3

(2.33, 3.33, 4.33)

(2.33, 3.33, 4.33)

(1, 1, 1)

(1, 2, 3)

0.5328

0.4537

C 2.4

(1, 1.67, 2.67)

(1, 1.67, 2.67)

(0.33, 0.5, 1)

(1, 1, 1)

0.2899

0.2468

M

   

AN US

C 2.1

ED

Table 3. Pair-wise comparison matrix, weights and normalized weights for group C 3.1 and

C 3.1.1 (1, 1, 1)

C 3.1.2

(1, 1, 1)

C 3.1.3

(0.5, 0.5, 0.5)

AC

   

CE

C 3.1.1

N wi 

C 3.1.2

C 3.1.3

wi

(1, 1, 1)

(2, 2, 2)

0.4000

0.4000

(1, 1, 1)

(2, 2, 2)

0.4000

0.4000

(0.5, 0.5, 0.5)

(1, 1, 1)

0.2000

0.2000

PT

d 0

ACCEPTED MANUSCRIPT

Table 4. Pair-wise comparison matrix, weights and normalized weights for group C 3.2 and

d 2 C 3.2.2

C 3.2.3

C 3.2.4

wi

N wi 

C 3.2.1

(1, 1, 1)

(1, 2, 3)

(2, 3, 4)

(1, 1.5, 2.5)

0.4617

0.3922

C 3.2.2

(0.33, 0.5, 1)

(1, 1, 1)

(1, 1.5, 2.5)

(0.43, 0.75, 1)

0.2404

0.2043

C 3.2.3

(0.25, 0.33, 0.5)

(0.4, 0.67, 1)

(1, 1, 1)

(0.33, 0.5, 1)

0.1578

0.1341

C 3.2.4

(0.4, 0.67, 1)

(1, 1.33, 2.33)

(1, 2, 3)

(1, 1, 1)

CR IP T

C 3.2.1

    

0.3171

0.2694

Table 5. Evaluation of the performance of criteria C1.1 in every project as three values (

(1.5, 2, 2.5)

(4.1, 4.6, 5.1)

L=0.5804

M

(0.4, 0.5, 0.67)

(1, 1, 1)

(1.8, 2.3, 2.8)

M=0.2928

H

(0.2, 0.22, 0.24)

(0.36, 0.43, 0.56)

(1, 1, 1)

H=0.1268

L

(1, 1, 1)

(0.4, 0.5, 0.67)

(0.22, 0.25, 0.29)

L=0.1436

M

(1.5, 2, 2.5)

(1, 1, 1)

(0.4, 0.5, 0.67)

M=0.2874

H

(3.5, 4, 4.5)

(1.5, 2, 2.5)

(1, 1, 1)

H=0.5690

L

(1, 1, 1)

(0.2, 0.22, 0.25)

(0.11, 0.11, 0.12)

L=0.0687

M

(4, 4.5, 5)

(1, 1, 1)

(0.4, 0.5, 0.67)

M=0.3114

H

(8.5, 9, 9.5)

(1.5, 2, 2.5)

(1, 1, 1)

H=0.6198

CE

PT

P3

(1, 1, 1)

M

P2

L

ED

P1

AN US

F1.1L , F1.1M and F1.1H ) associated to the three linguistic variables and d  1 L M H F1.1

Table 6. Calculation of the maximum and minimum values for the comparison matrix for d  1 and Fi  0.2 FiM  0.8 FiH M

H

L

(1, 1, 1)

(0.67, 1, 1.5)

(0.11, 0.11, 0.12)

M

(0.67, 1, 1.5)

(1, 1, 1)

(0.11, 0.11, 0.12)

H

(8.5, 9, 9.5)

(8.5, 9, 9.5)

(1, 1, 1)

L

(1, 1, 1)

(8.5, 9, 9.5)

(8.5, 9, 9.5)

M

(0.11, 0.11, 0.12)

(1, 1, 1)

(0.67, 1, 1.5)

H

(0.11, 0.11, 0.12)

(0.67, 1, 1.5)

(1, 1, 1)

AC

L

Max

Min

max(F)= 0.6692

min(F)= 0.0948

ACCEPTED MANUSCRIPT Table 7. Evaluation of factors for project P1

C 3.1

C 3.2

C1.1

0.113462

0.0149816

C1.2 C1.3 C1.4

0.773811

0.14325138

0.7982552

0.34999942

0.0789083

0.01928356

C 2.1 C 2.2 C 2.3 C 2.4

0.8786049

0.13153986

0.8786049

0.13153986

0.7275532

0.33011421

0.7549489

0.18635086

C 3.1.1

0.3251249

0.13004997

C 3.1.2 C 3.1.3

0.0689188

0.0275675

0.7982552

0.15965104

C 3.2.1

0.3251249

0.12753005

C 3.2.2 C 3.2.3

0.3189317

0.06514964

0.6438414

0.08633386

C 3.2.4

0.1886192

0.05081104

Xn

X 1  0.5275

X 2  0.7795

CR IP T

C2

wi  N Fi 

X 3.1  0.3173

AN US

C1

N Fi 

X 3.2  0.3298

Table 8. Evaluation of factors for project P2

C1.1

0.7275532

C1.2 C1.3 C1.4

0.8786049

0.16265131

0.6100582

0.26748341

0.3251249

0.0794538

C 2.1

0.7275532

0.10892523

0.7275532

0.10892523

0.7982552

0.36219398

C 2.4

0.7982552

0.19704055

C 3.1.1 C 3.1.2 C 3.1.3

0.6438414

0.25753657

0.6438414

0.25753657

0.8786049

0.17572099

C 3.2.1 C 3.2.2 C 3.2.3

0.6638914

0.26041099

0.5492607

0.11219999

0.659985

0.08849858

C 3.2.4

0.3600826

0.09700059

C 2.2

C2

AC

C 3.1

C 3.2

Xn

0.09606661

ED

CE

C 2.3

PT

C1

wi  N Fi 

M

N Fi 

X 1  0.6057

X 2  0.7771

X 3.1  0.6908

X 3.2  0.5581

ACCEPTED MANUSCRIPT Table 9. Evaluation of factors for project P3

C 3.1

C1.1

0.8066532

0.10651102

C1.2 C1.3 C1.4

0.9773438

0.18093029

0.7982552

0.34999942

0.8742265

0.21364284

C 2.1 C 2.2 C 2.3 C 2.4

0.9497048

0.14218453

0.9497048

0.14218453

0.8124579

0.36863817

0.9773438

0.24124661

C 3.1.1

0.8710813

0.34843251

C 3.1.2

0.9773438 1

0.39093753 0.2

C 3.2.1 C 3.2.2 C 3.2.3

0.9089517

0.35653573

0.9089517

0.18567572

0.9497048

0.12734762

C 3.2.4

0.6908469

0.18610327

C 3.1.3

C 3.2

Xn

X 1  0.8511

X 2  0.8943

CR IP T

C2

wi  N Fi 

X 3.1  0.9394

AN US

C1

N Fi 

X 3.2  0.8557

Mnemonic

L

Linguistic value

Low

Medium

High

FTN

(0,0,0.5)

(0,0.5,1)

(0.5,1,1)

H

PT

ED

M

M

Table 10. Input membership functions for both FIS

CE

Table 11. Output membership functions for both FIS vL

L

M

H

vH

Linguistic value

Very Low

Low

Medium

High

Very High

FTN

(0,0,0.25)

(0,0.25,0.5)

(0.25, 0.5, 0.75)

(0.5, 0.75, 1)

(0.75, 1, 1)

AC

Mnemonic

ACCEPTED MANUSCRIPT Table 12. Inference rules for C3_FIS

xc

L

L

vL

L

M

L

L

H

L

M

L

M

M

M

M

M

H

M

H

L

H

H

M

vH

H

H

vH

Table 13. Inference rules for OUT_FIS

X1

X3

X2

xc

X

X

vH

X

X

L

vH

M

L

M

H

H

L

M

H

L

H

M

M

M

M

H

M

M

M

H

H

M

M

H

H

M

H

L

ED

M

M

M

PT

M

L

M

H

L

H

H

vL

M

M

H

AC

H

M

CE

H

M

L

CR IP T

X 3 .2

AN US

X 3.1

Table 14. Inputs and output of C3_FIS

X 3.1

X 3 .2

xC

N xc   X 3

P1

0.3173

0.3298

0.3830

0.3607

P2

0.6908

0.5581

0.6080

0.6286

P3

0.9394

0.8557

0.7790

0.8321

ACCEPTED MANUSCRIPT

Table 15. Inputs and output of OUT_FIS

X1

X3

X2

xC

N xc   R

0.5275

0.7795

0.3607

0.592

0.61

P2

0.6057

0.7771

0.6286

0.407

0.39

P3

0.8511

0.8943

0.8321

0.309

0.27

CR IP T

P1

Table 16. Rank reversal verification in the assignation of weights in C 2 and C 3 suppressing one criterion Original

w 2 .2

0.1497

0.1659

X

w 2 .3

0.4537

0.5421

w 2 .4

0.2468

w3.1.1

0.2771

Position

0.1986

AN US

0.1659

3

0.1986

3

0.5421

X

0.6028

1

0.2920

0.2920

0.4457

X

2

0.4000

X

0.6667

0.5000

1

w3.1.2

0.4000

0.6667

X

0.5000

1

w3.1.3

0.2000

0.3333

X

2

w3.2.1

0.3922

w3.2.2

0.2043

w3.2.3

0.1341

w3.2.4

0.2694

M

0.2771

AC

 

X

0.3333

ED

C 3.2

0.1497

X

0.5002

0.4598

0.5318

1

0.3297

X

0.2253

0.2908

3

0.2222

0.1716

X

0.1774

4

0.4481

0.3282

0.3149

X

2

PT

C 3.1

w2.1

CE

C2

wi suppressing one criterion

wi

ACCEPTED MANUSCRIPT

Table 17. Rank reversal verification in the assignation of weights in C 2 and C 3 suppressing two criteria Original

w 2 .1

0.1497

X

X

X

0.3696

0.2299

0.5000

3

w 2 .2

0.1497

X

0.2299

0.2299

X

X

0.5000

3

w 2 .3

0.4537

0.6556

X

0.7701

X

0.7701

X

1

w 2 .4

0.2468

0.3444

0.7701

X

0.6304

X

X

2

w3.2.1

0.3922

X

X

X

0.6159

0.7502

0.6556

1

w3.2.2

0.2043

X

0.4001

0.6159

X

X

0.3444

3

w3.2.3

0.1341

0.3444

X

0.3841

X

X

4

w3.2.4

0.2694

0.6556

0.5999

X

X

2

0.3841

AC

CE

PT

ED

M

AN US

C 3.2

Position

CR IP T

C2

wi suppressing two criteria

wi

0.2498 X