- Email: [email protected]
Quality vs risk: An investigation of their relationship in software development projects
JPMA-01592; No of Pages 10
Available online at www.sciencedirect.com
ScienceDirect International Journal of Project Management xx (2013) xxx – xxx www.elsevier.com/locate/ijproman
Quality vs risk: An investigation of their relationship in software development projects Lazaros Sarigiannidis a,⁎, Prodromos D. Chatzoglou b,1 a
b
Business Administration Dept., Kavala Institute of Technology, Agios Loukas, 65404 Kavala, Greece Production & Management Engineering Dept., Democritus University of Thrace, Library Building, Kimmeria, 67100 Xanthi, Greece Received 23 December 2012; received in revised form 3 October 2013; accepted 12 November 2013
Abstract Quality, risk and successful software development projects are three concepts which appear to be indisputably intertwined with one another. The purpose of the present study is to investigate the relationship between people quality, process quality and risk in the context of software development projects of Greek companies. Project team members with different characteristics were used as key respondents. The final sample consisted of 112 projects from 63 companies. Empirical data were analysed using the structural equation modelling technique. The main results indicate a negative effect of people quality on project risk level. On the contrary, process quality appears to have a slightly limited effect, defining only the risk level associated with the project team. The results contribute in the existing literature underlining the importance of quality on the reduction of the project risk level, thus, creating the necessary background for new similar research attempts in the future. © 2013 Elsevier Ltd. APM and IPMA. All rights reserved. Keywords: Software development; Project quality; Project risk; SEM
1. Introduction Between now and 2025, the ability of companies and their products, systems and services to compete, adapt and survive will depend to a great and continually increasing extent on the software they will be using (Boehm, 2006). This software could provide them with competitive differentiation and quick adaptation to competitive changes for today's products and services. However, software success depends on software quality (Gorla and Lin, 2010), effectiveness and completeness (Nienaber and Cloete, 2003). Software quality is frequently determined by the software development process quality (Schwalbe, 2000), which is defined through specially developed control metrics (Jorgensen, 1999; Sahraoui et al., 2010; Subramanian et al., 2007). ⁎ Corresponding author. Tel.: + 30 2513 006162. E-mail addresses: [email protected] (L. Sarigiannidis), [email protected] (P.D. Chatzoglou). 1 Tel./fax: +30 25410 79344.
Software development projects and the effects determining their outcomes have been the subjects of a continued research effort over the last thirty-five years (McLeod and MacDonell, 2011). Rothenberger et al. (2010) stated that software development process has no differences with other engineering disciplines: a managed approach is expected to lead to better quality of the product. Regardless of quality issues, software development projects are particularly difficult to be managed and this is because they incorporate a group of complex characteristics, which are not found in any other type of projects in the engineering or manufacturing field (areas which software development is mainly compared with). Human interaction, high level of complexity and product versatility are few of these characteristics (Jorgensen, 1999; Subramanian et al., 2007). Consequently, despite the fact that software has been effectively applied in a large range of areas, software development projects have a reputation for failure (Savolainen et al., 2012). Naturally, therefore, software development projects have been considered for many years as the ones with the highest risk.
0263-7863/$36.00 © 2013 Elsevier Ltd. APM and IPMA. All rights reserved. http://dx.doi.org/10.1016/j.ijproman.2013.11.001 Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
2
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
The overcoming of the chronic problems that are found in project development, like budget overruns, delays in its completion and weakness to manage and respond to user requirements (McLeod and MacDonell, 2011), are not just desired, but a basic priority for an economy as well (Yang et al., 2009). Subsequently, the role of risk management in software development is considered especially important. Not surprisingly researchers, as well as practitioners, have expressed their interest in managing risk in software development projects (Barki et al., 1993; Boehm, 1991; de Bakker et al., 2010; McLeod and MacDonell, 2011; Nakatsu and Iacovou, 2009). Risk management is a forecasting, complete, systematic and based on the archetypes approach (Hall, 1998), which attempts to standardise the relationships involving risk that threatens the (usually subjective) success of the development process (Lee et al., 2009), with the use of an applicable sum of principles and practices (Addison and Vallabh, 2002). The technological gap in Greece compared with other European countries (as measured by NRI) and the reluctance of several Greek companies-mainly SMEs to adopt new technologies, limit the potential market growth. Additionally, the software development industry in Greece does not seem to remain unaffected by the global economic crisis. Both companies and households are not willing to invest heavily in new technologies, before the end of the crisis. The impact on the software development market has been visible for about four years, as there is a direct dependence with the budgets of client companies. Thus, the current economic crisis in the EU has driven down budgets and formulates the growth rates at much lower levels. Furthermore, the highest percentage of family SMEs in the Greek economy, which does not have high requirements in software, and they are not aware of the prospects and the necessity of computer systems, lead to a restriction in the size of investment and the scope of new projects. The Greek software development projects seem to be very limited (in terms of budget, duration or employees) compared to those of the technologically developed countries (Greek National SME Observatory, 2011). Also the competition has been intensified with the entry of multinational companies in the Greek market in the last decade, which have higher funds, high costs for R & D and access to specialized workers, offering high quality products, and forming relatively quickly a wide deposit of important customers. Generally, the Greek software packages, although holding significant market shares, face stiff competition from the respective programs of foreign software houses. The latter have considerably strengthened their market shares, creating trends of market concentration, while several Greek SMEs, which constituted a large part of the industry, faced serious problems of competition. In order to create a more clear and integrated picture about the software development process in Greece in relation to other countries it can be used some well-known methodologies such as the capability maturity model integration (CMMI) which classifies software organizations into five levels based on the sophistication of their engineering and management practices (Na et al., 2007). Unfortunately Greek organizations are not listed in the aforementioned taxonomy and this kind of comparison can't be used.
The aim of this study is the creation of a new research framework, which will arise through a thorough examination of the international literature. This framework will also be tested, using empirical data, in order to measure the significance of the conceptual factors related to project quality and risk. 2. Literature review 2.1. Risk dimensions Barki et al. (1993) support that software project risks consist of interrelated dimensions and their measurement should not be done with the use of a one-dimensional scale, but, on the contrary, every dimension must be separately, theoretically and practically defined. Despite the importance of studying risk through its dimensions, only a few researches employing them have been conducted. McFarlan (1981) found three main risk dimensions of the software development process and, more precisely, the size, the technological experience and the structure of the project. He proposed that project managers should develop a complete and integrated risk profile for a software project. Later, Boehm (1991) proposed a risk management framework where he included the factors of risk estimation and control, while he also made a list of the ten most important risks, based on his personal professional experience. In spite of all these, Boehm's list lacks some theoretical foundation (Huang and Han, 2008) and also, due to complexity and other factors (invisibility, harmonisation, flexibility), which characterise software projects, it ceased to have any diachronic value as years pasted (since it was conducted in 1991) (Hughes and Cotterell, 2006). Barki et al. (1993) conducted an extensive review of studies (120 projects that had been carried out by 75 organizations) concerning software development risk, and proposed 35 parameters for assessing risks, which, in turn, were categorised in five dimensions: technological innovation, application size, expertise, application complexity and organizational environment. However, despite the fact that their study provided a quite useful and understandable tool for risk measuring, it also proved that the estimation scale was extremely complicated (Huang and Han, 2008; Wallace et al., 2004b). Further, Heemstra and Kusters (1996), based on earlier studies and their professional experience, composed a list of 36 software risks, which later were grouped into 9 categories. Moynihan (1997), in cooperation with 14 Irish software application design specialists, developed a group of 21 risk related points. Later, Longstaff et al. (2000) proposed a framework, named hierarchically holographic modelling (HHM), and distinguished seven dimensions of system integration that involved 32 types of risk. Also, Cule et al. (2000) identified 4 risk dimensions according to their source of origin (labour, customer, environment, individuality), which involved 55 software risks. Then, they proposed a primary risk management strategy for each dimension separately. Based on Boehm's (1991) risk list, Ropponen and Lyytinen (2000) developed a questionnaire which contained 6 risk dimensions for a research where 83 managers from 1100 software projects participated in. Sumner (2000), through structural
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
interviews, compared software project differences between MIS and ERP projects and proposed nine risks that are unique in ERP projects. Houston (2000), using as criteria the importance and frequency of appearance in the existing literature, presented a list of 29 software development project risks. In addition, Kliem (2001) developed a list of 38 BPR (business process reengineering) project risks, which were categorised in 4 main dimensions: people, management, business and techniques. Schmidt et al. (2001) also identified 53 risk variables, which were categorised in 14 factors and concluded that the cultural difference of the three countries (Finland, China and U.S.A.) their research was conducted in, could significantly affect the risk list, while only 11 of them had a cross-cultural application. Further, Addison (2003) used the Delphi technique to collect the views of 32 specialists and, then, presented a list of 28 risks for e-commerce projects. However, since the organizational or environmental financial and non-financial situations are constantly changing, it is certain that there will be corresponding and parallel changes in the risk items that are included on a checklist. So, it is necessary both the development of new instruments and/or the validation of the traditional instruments, allowing a cumulative background of research to be built within the project risk management framework. In the present study, the relatively recent approach of Wallace et al. (2004b) will be adopted (as for as risk dimensions is concerned). Their research has focused on the collection of questionnaires from 507 members of the Project Management Institute (PMI). Wallace et al. (2004b) proposed 27 software development risks, which were summarized into six dimensions (user, system requirements, project complexity, planning and control, team and organizational environment). These risk factors can be found in all types of IT projects (Nakatsu and Iacovou, 2009). 2.2. Project quality Although quality has been extensively studied in repetitive operations, it still remains under-researched in projects (Geraldi et al., 2011). Project quality is the factor that can significantly affect the possibility of risks appearing and the extent of these risks' consequences, during the development process of a software project (Ould, 1999). In the present study, the software project's total quality is measured using two sub factors, while the variables are also defined according to literature (Fenton et al., 2004, 2008). These two fundamental factors are: process quality (Hoffman, 2003) and people quality. Total product quality, although extensively studied in the international literature (Pressman, 2004), will not be considered in this study due to its complexity and the technical nature of its constituting characteristics. 2.3. The project quality — project risk relationship Despite the fact that quality in software development projects has been defined, analysed and theoretically supported over and over in the past in many surveys, none of them has examined the importance of its effect on project risk. In the present research, an
3
initial attempt to test this research framework, using variables proposed by Fenton et al. (2004), will be made. The main research question is: What is the magnitude of project's quality impact on project risk? 3. Research methodology 3.1. Population and sample characteristics The population of the study consists of the sum of Greek companies (software houses) that are involved in software development projects. A thorough analysis of various relevant databases (e.g. Greek ICT Market, 2009; S.Ε.P.Ε., 2010), revealed that the specific sector consists of 220 companies (study population). All these companies were informed (by post, phone and e-mails) for the existence of the present survey, and they were invited to participate. The aim was to persuade as many of these companies as possible to answer a specific questionnaire, which would allow us to base our analysis on a large sample and, therefore generalise the findings (external validity) to a larger number of similar cases. Each participating company had to specify one software development project which they were focusing on during the time the present study was carried out (sampling or statistical unit), as well as the number of people (research unit) involved in the development process, who would be requested to answer the questionnaire. From the 220 companies that were approached, only 72 responded positively, returning 124 completed questionnaires. However, twelve of these questionnaires, corresponding to nine companies, were discarded since they had several inappropriately answered questions. As a result, the final sample consists of 112 questionnaires (from 63 companies), with a response rate of 28.64%. Taking into consideration the difficulties that are presented in similar studies, where collecting primary data is an essential part (the refusal of many project managers to participate in similar researches claiming limited available time or the confidentiality policy of the company, the great geographical disperse of these companies) and the fact that one or more people from the biggest companies of this industry have finally participated, allow us to claim that the sample can be considered as extremely satisfactory, representative, as well as adequate (size wise). The majority of the participants were members of software development teams and more precisely, project managers (42%) and employees in the software development department (40.2%) (mainly programmers). The remaining 17.8% of the sample consisted of project team members who were performing additional or relative tasks (e.g. quality managers, software designers, auditors etc.). 3.2. Variable measurement — questionnaire design The collection of the necessary data from project managers or system analysts/programmers of each organisation, following similar, earlier conducted empirical studies (Barki et al., 1993; Han and Huang, 2007; Moynihan, 1997; Na et al., 2007; Nidumolu, 1995; Ropponen and Lyytinen, 2000; Schmidt et
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
4
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Table 1 Risk dimension measuring. Risk dimensions
Variables
References
User
5
Requirements Project complexity Planning and control Team
4 4 7 3
Organizational environment
4
McLeod and MacDonell (2011), Han and Huang (2007), Jiang and Klein (1999), Sakhtevil (2007), Schmidt et al. (2001), and Wallace et al. (2004b). Charette (1989), Curtis et al. (1988), Han and Huang (2007), Mizuno et al. (2000), and Wallace et al. (2004b). Barki et al. (1993), Han and Huang (2007), Kemerer and Sosa (1991), Nidumolu (1995), and Wallace et al. (2004b). Han and Huang (2007), Keider (1984), Schmidt et al. (2001), and Wallace et al. (2004b). Sakhtevil (2007), Abdel-Hamid (1989), Han and Huang (2007), Jiang et al. (2000), Mizuno et al. (2000), and Wallace et al. (2004b). Barki et al. (1993), Han and Huang (2007), Jarvenpaa and Ives (1991), Jones (1994), and Wallace et al. (2004b).
al., 2001; Wallace et al., 2004b), has been done with the use of a self-completed structured questionnaire. All the answers were collected electronically, since the respondents were allowed access to a specially designed webpage, where the electronic version of the questionnaire appeared. The reason for adopting this approach was primarily related to time, effort and economical issues, such as research cost reduction, speed of data collection, ease of coding, and quicker access to larger and geographically dispersed population groups (Dillman, 2000). The research instrument consists of questions (items) which have the appropriate psychometric abilities to describe various angles of different factors. The items that have been selected for the definition of the conceptual construct of project risk meet the following two criteria: 1) they have arisen from an extensive literature review, as well as discussions that were made with academics and other professionals, and 2) they have been tested for their validity in previous studies. On the contrary, the items used for measuring product quality are new and have arisen from literature. As it has been mentioned earlier, measuring project risk is based on the work of Wallace et al. (2004b), who have moved on to distinct 27 software development risks, grouped into six dimensions-conceptual factors (Table 1). However, contrary to Wallace et al. (2004b), who evaluated only the degree to which every risk characterises a project, the present study attempts to measure both the possibility of the appearance and the impact of risk on a project (Han and Huang, 2007), by defining its total level of risk. For the calculation of the total exposure to risk, the mathematical formula proposed by Cooper et al. (2005) was adopted. They substantiated the amount of disadvantages that the traditional way of measuring risk exposure faces (Boehm, 1991; Han and Huang, 2007; Heemstra and Kusters, 1996; Ward, 1999), as the product of possibility (P) and consistency (C) (RI = P ∗ C), stressing that the elements with high consistency but low possibility can produce low risk exposure factors and, as such, should not be considered important. In order to overcome the aforementioned problems, they calculated risk exposure as follows: RE = P + C − (P ∗ C). The participants were asked to evaluate the 27 proposed risks according to their possibility of appearance and impact (in terms of cost, schedule, technical performance and collaboration of the project team) on the most recent software development project of their company, in a scale of one (1) to ten (10), where 1 implied “rare” and 10 “almost certain” possibility of appearance, while, 1
implied “minimal or no effect” and 10 “great effect” as far as impact is concerned. Additionally, the total project quality has been measured with two basic factors, process quality and people quality (Table 2). Initially, people quality has been divided into its two basic sub-factors (management quality and staff quality). Management quality includes variables such as communications management adequacy, subcontract management adequacy, interaction management adequacy and internal management quality (AgenaRisk, 2005). On the other hand, staff quality refers to the quality of non-executive staff that is involved in a project. Variables like staff turnover, staff experience, staff motivation, staff training and programming language experience are used for measuring staff quality (Fenton et al., 2004). Moreover, process quality is another complex factor which is used to define project quality, by combining the specification clarity of a project and the development and testing process quality. Specification clarity is described using the variables of specification process quality and requirement difficulty. Development and testing process quality consists of variables concerning the regularity of reviews, the quality of documentation and the level of independent testing. The sum of quality measuring variables was evaluated in a scale of one (1) to five (5), where 1 implied that project quality was “very low”, while 5 that it was “very high”. 3.3. Questionnaire validity Before starting the distribution of the questionnaire, the necessary pre-testing took place, to verify its content validity (Zikmund, 2003). The level of understanding, “acceptance”, as well as interpretation of the questionnaire was measured at this experimental stage. Academics with research interests that are close to the object of the study were selected, as well as software development business executives that are located in Northern Greece. The pre-testing was necessary for avoiding inappropriate, partial, vague or double-meaning questions, determining the order Table 2 Project quality measuring. Project quality
Variables
References
People quality Process quality
10 6
AgenaRisk (2005), and Fenton et al. (2004) AgenaRisk (2005), Fenton et al. (2008), and Hoffman (2003)
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
5
Table 3 Exploratory factor analysis — project risk dimensions. Factor
Sub-factor
Items
Mean
Standard deviation
Loadings
ΚΜΟ Bartlett's test sig. TVΕ
Cronbach's alpha
Project risk dimensions
User
U1 U2 U3 U4 U5 D1 D2 D3 D4 PC1 PC2 PC3 PC4 DC1 DC2 DC3 DC4 DC5 DC6 DC7 T1 T2 T3 OE1 OE2 OE3 OE4
0.73 0.64 0.73 0.72 0.75 0.84 0.86 0.84 0.80 0.75 0.75 0.71 0.71 0.73 0.74 0.73 0.75 0.80 0.72 0.79 0.72 0.72 0.73 0.68 0.69 0.72 0.69
0.23 0.22 0.23 0.22 0.24 0.17 0.16 0.18 0.20 0.20 0.19 0.26 0.24 0.19 0.18 0.21 0.19 0.16 0.23 0.17 0.20 0.21 0.21 0.23 0.21 0.23 0.24
0.724 0.711 0.843 0.781 0.705 0.641 0.860 0.937 0.807 0.715 0.729 0.801 0.779 0.804 0.877 0.796 0.808 0.838 0.790 0.760 0.885 0.887 0.912 0.821 0.842 0.862 0.878
0.723 0.000 56.917
0.809
0.668 0.000 66.934
0.826
0.677 0.000 57.272
0.749
0.848 0.000 65.754
0.909
0.736 0.000 80.040 0.685 0.000 72.442
0.875
Requirements
Project complexity
Planning and control
Team
Organizational environment
of the questions in a way that no distortion tendencies are provoked, decreasing the size of the questionnaire to reduce respondents' lack of interest or frustration, and testing the adequacy and understanding of the introductory sections. Moreover, due to the fact that some of the variables that were used for measuring the factors and sub-factors are new or adopted from different studies, or even due to their validity and reliability in Greece having not been tested before, they might not have the necessary psychometric characteristics and, thus, probably negatively affect the reliability of the factors. Initially, the construct
0.873
validity of the variables was tested. More specifically principal component analysis (Varimax method) was applied for testing the unidimentionality of the factor elements, while for the estimation of their reliability the statistical metre used was Cronbach Alpha. The results (see Tables 3 and 4), allow us to claim that the used variables constitute solid and reliable structures capable of contributing to measuring the factor they belong to. Further, as it has been presented in Table 2, process quality is measured using six variables. However, due to low loadings (smaller than 0.5) two of them (PQ2 and PQ3) were eliminated.
Table 4 Exploratory factor analysis — project quality. Factor
Sub-factor
Items
Mean
Standard deviation
Loadings
ΚΜΟ Bartlett's Test Sig. TVΕ
Cronbach's alpha
People quality
Management quality
HRQ1 HRQ2 HRQ3 HRQ4 HRQ5 HRQ6 HRQ7 HRQ8 HRQ9 HRQ10 PQ1 PQ4 PQ5 PQ6
3.62 3.36 2.79 3.00 3.76 3.54 3.79 3.96 4.17 3.96 3.52 3.43 3.47 3.02
0.903 0.929 0.912 0.968 0.893 0.900 0.885 0.793 0.758 0.838 0.838 0.946 0.995 0.816
0.792 0.838 0.789 0.835 0.777 0.761 0.803 0.825 0.658 0.530 0.734 0.685 0.763 0.804
0.784 0.000 65.069
0.866
0.793 0.000 52.363
0.764
0.739 0.000 55.908
0.731
Staff quality
Process quality
–
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
6
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Table 5 First order confirmatory factor analysis. χ2/df
Factor User risks Requirements risks Complexity risks Planning & control risks Team risks Organizational environment risks Management quality Staff quality Process quality
3.548 3.078 0.223 4.148 0.000 0.541 4.453 0.727 1.008
Table 7 Hierarchical regression results. CFI
GFI
0.947 0.982 1.000 0.920 1.000 1.000 0.950 1.000 1.000
0.954 0.975 0.999 0.881 1.000 0.998 0.949 0.988 0.991
RMR 0.002 0.001 0.000 0.002 0.000 0.001 0.040 0.019 0.020
Composite reliability 0.80 0.84 0.73 0.91 0.88 0.90 0.85 0.78 0.74
The questions eliminated are related to the requirements management quality. After the completion of the exploratory factor analysis, confirmatory factor analysis (CFA) was also used in order to define the factors that will participate in the research model. The aim of this factor analysis was to test certain hypotheses which mainly derive from certain theories or are supported by valid, previous studies. From the results that are presented in Table 5, it can be stated that the previous tests for the examination of the data adjustment to the proposed model, have provided satisfactory results. After the completion of the first order confirmatory analysis, a second order CFA was also carried out. These factors were evaluated having as criteria the acceptable margins of the adjustment indices, the statistical significance of the causal relationships, as well as the suggestions of the modification index. Table 6 presents the results of this analysis (all the values of the indices examined are within acceptable levels).
Dependent variable
Project risk
Independent variable
β
t
Project quality R2/adjusted R2 F
− 0.293 0.086/0.077 10.294⁎⁎⁎
12.553⁎⁎⁎
Notes: *** Significant at the p b0.01 level.
the six dimensions of risk. On the contrary, process quality influences, in a negative and statistically significant manner, just the project team risk dimension. Both sub-factors of project quality have been found to have no statistically significant effect on the complexity dimension of the project risk (Fig. 1). 5. Summary and conclusions Based on an extensive literature review, a new conceptual model has been developed (with the parallel development of appropriate metrics for measuring the two conceptual constructs), evaluated and empirically tested. The present study is unique since none of the project risk management related researches that have taken place in Greece, have attempted to empirically test the validity and accuracy of the theoretical approaches used; they were rather limited to simple literature
4. Results 4.1. Structural model analysis First, the results of the evaluation of the relationship between project quality and risk, which was estimated using the hierarchical moderated regression analysis method are presented in Table 7. The results suggest that a (reasonably) negative (− 0.293), yet statistically significant, relationship exists between these two factors. To test the relationship of project quality and risk more extensively, project quality has been divided into its two main sub-factors, and project risk into its six dimensions. The findings suggest that there is a negative and statistically significant relationship between people quality and five out of
Table 6 Second order confirmatory factor analysis. Factor
χ2/df
CFI
GFI
RMR
Composite reliability
Project risk People quality Project quality
3.253 0.000 0.000
0.908 1.000 1.000
0.919 1.000 1.000
0.002 0.000 0.000
0.83 0.51 0.60
2 /df 0.575
CFI 1.000
GFI 0.994
RMSEA 0.000
RMR 0.002
Fig. 1. Research structural model.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
reviews or the proposition of risk eliminating tools (Igglesis, 2004; Kiountouzis, 2004; Kirytopoulos, 2006). Also, this study is the first to adopt the mathematical formula [RE = P + C −(P ∗ C)] of Cooper et al. (2005) for measuring risk exposure. Its empirical evaluation confirmed its validity and accuracy. Also, this research relies on data collected from many project team members with different characteristics (equal participation of project managers and other employees of the software development department), something that improves the estimation of the research parameters used and reduces the subjectivity of the conclusions drawn, since various perspectives of people with different levels of responsibility in the development process have been considered. Therefore, the present study is methodologically different (as far as the sample is concerned) compared to the majority of important previous studies, such as the ones of Boehm (1991) (project managers), Schmidt et al. (2001) (project directors), or Moynihan (1997) (application programmers). Additionally, it provides a clear answer to the question of Wallace et al. (2004b) whether the participation of people with different roles in the development process would result in reaching different conclusions. The answer derived from the results of this research does not seem to support directly their claim, since most of the results match those that have already been documented in literature. Moreover, great interest presents the measurement of project risk by using its specific six dimensions (measured with 27 variables). The dimension “requirements” is found to consist of the variables with the highest mean scores, thus indicating that according to the respondents, it is the dimension with the highest exposure to risk. On the contrary, “organizational environment” appears to be the least important risk factor (threat). Project team members consider that the most likely to appear risks are the ones emerging from incorrect, changing or vague requirements, which also have the highest negative effects on the project characteristics. On the other hand, Greek software development companies appear to operate in a stable organizational environment with small administrative changes during the implementation of the projects. These results are in accordance with the finding of Huang and Han (2008) and Han and Huang (2007), who presented these two dimensions as the most and least important, respectively. With regard to the mean scores of the variables concerning project quality, it is worth stating that they are quite similar, suggesting that project team members are overall highly satisfied with its quality. As far as the staff quality dimension is concerned, it is found that the mean score of the variables measuring it (staff general experience, experience with the programming language used, low rate of staff change) is very satisfactory (higher than the other ones). This shows that among the software development companies there is a widespread feeling of satisfaction with the relevant knowledge level and abilities of their members. All the above explain, to a certain degree, why the dimension “organizational environment” is considered as a low risk one.
7
Finally, the results of the structural model indicate that there is a statistically significant negative relationship between project quality and risk. Taking the analysis one step further, project quality was divided into its two main components, in order to measure their individual effect on the six project risk dimensions. It is found that people quality has a negative and statistically significant relationship with all dimensions of project risk (excluding the risks related to project complexity). This finding suggests that Greek software development companies pay attention to issues such as proper staffing, training and communication of their personnel. Quality features, like communication management quality, motives, staff change rate or staff education level, which concern the people employed at software development companies, appear to be directly related to specific risk dimensions such as planning and control and team and organizational environment of the project. This study empirically verifies the direct relationship that dictates the characteristics of these factors. However, the process quality characteristics appear not to significantly affect a project's exposure to risk. More precisely, it has been found that only the risk dimension that is related to project team is affected (negatively and statistically significantly) by process quality. The reason for this may be the relatively small size of the Greek software development companies, where processes, such as project inspections (and their frequency) and, basically, the level of independent tests, do not usually occur as organised, standardised and strategically oriented actions, which, through their careful modification or redirection, could reform those conditions and decrease projects' exposure to risks. Their exceptionally unique effect on the risk dimension of project team takes place mainly due to the special nature of the aforementioned risks. Even a non-systematic attempt to upgrade quality control and project inspection processes can provide an important solution to the problems that occur due to the insufficient training of the project team executives, by providing some guidelines for improving the quality of the project. The delicate balance that exists among these factors seems to significantly diminish during the examination of the remaining risk dimensions.
6. Managerial implications Many software development projects often fail due to lack of understanding of the risks involved. Specialists in this field claim that the risk related to software development projects have first, to be defined and then, be dealt with during the development process (Wallace et al., 2004a). This study presents a project risk model, as a second order factor model, which provides a useful framework consisting of six project risk dimensions, with each one of them being only a small part of the total “picture”. Risk management methods that focus on just one aspect of risk may even lead to more risky projects with great possibility of failure. For the above reasons, project managers will have to use similar tools in order to evaluate and manage risks, make the appropriate decision for avoiding or eliminating them and subsequently improving the possibilities for a successful project development.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
8
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
The results that derive from measuring the possibility of appearance and the impact of the 27 proposed risks and their subsequent classification, which is based on their level, provide the opportunity to professionals in the specific sector to implement and benefit from the techniques of the Pareto principle, which is widely accepted in the risk management field (Shtub et al., 2008). According to this rule, an individual can focus his/her attention on a small number of risks (the ones with the highest calculated level), in order to deal with approximately 80% of the total impact of these risks on the project (Shtub et al., 2008). Software development organizations can directly benefit from considering the findings of this study, by developing a risk profile for each one of their software development projects. The tool that has been developed in the present study could be used to create a risk framework for the distribution and evaluation of the risks involved in each project. This framework could be used in two ways. First, as a tool at many stages of the project life-cycle, that could monitor the changes of the project risk level, whenever they take place. And second, as an analysis tool for the already completed projects. The constant review of the results can be used to improve the management of future projects. Over time, the frameworks could provide guidelines informing project managers about high risk areas of similar projects that have been completed in the past. Thus, high risk projects could possibly be identified early in the project life-cycle and more appropriate decisions would be made about their continuation (or not). The risk frameworks could also be used in a way similar to the portfolio approach that McFarlan (1981) proposed. Having obtained the ability to better estimate the risk of a project, organizations can achieve the best possible results, through balancing the number of high risk projects that they take on, with a complementary amount of lower risk projects. Finally, the importance of the relationship between quality definitions (and especially the one of project people) and the level of project risk, should be stressed. The significant emerging role of quality characteristics, as a tool of moderating the level of risk, undoubtedly offers project managers the opportunity to manage their projects more effectively. Empirically tested practices of total quality management, appropriately adjusted to the requirements of every project, are likely to become a reliable and economically accessible ally over the successful confrontation of the occurring risks in software development projects. 7. Research limitations First of all, to test the validity of the results and the likelihood of generalising them, the collection of similar data from other countries, is considered essential. Another important issue involves the use of a structured questionnaire as the research instrument for collecting data. In spite of all the attention paid to its design and the pre-testing that followed, it is possible that some answers may have been affected from the format or size of the questionnaire. Ideally, structured or semi-structured interviews of all participants should have taken place to clarify some issues.
Appendix A. Questionnaire
Project risk User User1 User2 User3 User4 User5 Requirement Requirement1 Requirement2 Requirement3 Requirement4 Project complexity Project complexity1 Project complexity2 Project complexity3 Project complexity4 Planning&Control Planning&Control1 Planning&Control2 Planning&Control3 Planning&Control4 Planning&Control5 Planning&Control6 Planning&Control7 Team Team1 Team2 Team3 Organizational environment Organizational environment1 Organizational environment2 Organizational environment3 Organizational environment4
Users resistant to change Conflict between users Users with negative attitudes Users not committed to the project Lack of cooperation from users Continually changing system requirements Systems requirements not adequately identified Unclear system requirements Incorrect system requirements Project involved the use of new technology High level of technical complexity Immature technology Project involves use of technology that has not been used in prior projects Lack of an effective project management methodology Project progress not monitored closely enough Inadequate estimation of required resources Poor project planning Project milestones not clearly defined Inexperienced project manager Ineffective communication Inexperienced team members Inadequately trained development team members Team members lack specialized skills required by the project
Change in organizational management during the project Corporate politics with negative effect on project Unstable organizational environment Organisation undergoing restructuring during the project
People quality Management quality People quality1 People quality2
People quality3 People quality4 People quality5
Communications management quality Scale of distributed communications (geographical diversity, number of sites and team size) Scale of subcontracts Subcontract management quality Internal management quality
Staff quality People quality6 People quality7 People quality8 People quality9 People quality10
Staff motivation General level of staff training General level of staff experience Relevant program language experience Staff turnover
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx Appendix A (continued) Project risk Process quality Process quality1 Process quality4 Process quality5 Process quality6
Specification process quality Regularity of reviews Quality of documentation Level of independent testing
References Abdel-Hamid, T.K., 1989. A study of staff turnover, acquisition, and assimilation and their impact on software development cost and schedule. J. Manag. Inf. Syst. 6 (1), 21–39. Addison, T., 2003. E-commerce project development risks: evidence from a Delphi survey. Int. J. Inf. Manag. 23 (1), 25–40. Addison, T., Vallabh, S., 2002. Controlling software project risks — an empirical study of methods used by experienced project managers. Proceedings of SAICSIT 2002, pp. 128–140. AgenaRisk, 2005. Software Project Risk Model Manual, Bayesian Network and Simulation Software for Risk Analysis and Decision Support-AgenaRisk (Version 2.00). Barki, H., Rivard, S., Talbot, J., 1993. Toward an assessment of software development risk. J. Manag. Inf. Syst. 10 (2), 203–225. Boehm, B.W., 1991. Software risk management: principles and practices. IEEE Softw. 8 (1), 32–41. Boehm, B.W., 2006. Some future trends and implications for systems and software engineering processes. J. Syst. Eng. 9 (1), 1–19. Charette, R.N., 1989. Software Engineering Risk Analysis and Management. Intertext Publications, New York. Cooper, D.F., Grey, S., Raymond, G., Walker, P., 2005. Project risk management guidelines: Managing risk in large projects and complex procurements. John Wiley and Sons Ltd., England. Cule, P., Schmidt, R., Lyytinen, K., Keil, M., 2000. Strategies for heading off project failure. Inf. Syst. Manag. 17 (2), 65–73. Curtis, B., Ward, S., Chapman, C., 1988. Roles, responsibilities and risks in management contracting. Construction Industry Research and Information Association (CIRIA) Special Publication, 81. de Bakker, K., Boonstra, A., Wortmann, H., 2010. Does risk management contribute to IT project success? A meta-analysis of empirical evidence. Int. J. Proj. Manag. 28, 493–503. Dillman, D.A., 2000. Mail and Internet surveys — The Tailored Design Method. John Wiley & Sons, New York. Fenton, N., Marsh, W., Neil, M., Cates, P., Forey, S., Tailor, M., 2004. Making resource decisions for software projects. Proceedings of the 26th International Conference on Software Engineering. Fenton, N., Neil, M., Marsh, W., Hearty, P., Radliński, L., Krause, P., 2008. On the effectiveness of early life cycle defect prediction with Bayesian Nets. Empirical Software Engineering (Published Online). Geraldi, J., Kutsch, E., Turner, N., 2011. Towards a conceptualization of quality in information technology projects. Int. J. Proj. Manag. 29, 557–567. Gorla, N., Lin, S.C., 2010. Determinants of software quality: a survey of information systems project managers. Inf. Softw. Technol. 52, 602–610. Greek ICT Market, 2009. Annual Guide of the Greek IT and Communications, Athens. Greek National SME Observatory, 2011. Services sector: software development applications. Recovered from: http://www.acsmi.gr/Portals/0/ anaptiksilogismikou.pdf. Hall, E., 1998. Managing Risk: Methods for Software System Development. Addison-Wesley, New York. Han, W.M., Huang, S.J., 2007. An empirical analysis of risk components and performance on software projects. J. Syst. Softw. 80 (1), 42–50. Heemstra, F.J., Kusters, R.J., 1996. Dealing with risk: a practical approach. J. Inf. Technol. 11 (4), 333–346. Hoffman, G., 2003. Integrating PSP and CMMI level 5. STC 2003 Proceedings.
9
Houston, D., 2000. A Software Project Simulation Model for Risk Management. PhD Dissertation Arizona State University, Tempe, AZ. Huang, S.-J., Han, W.-M., 2008. Exploring the relationship between software project duration and risk exposure: a cluster analysis. J. Inf. Manag. Sci. 45, 175–182. Hughes, B., Cotterell, M., 2006. Software Project Management, Fourth edition. McGraw-Hill. Igglesis, V., 2004. Software Risk Management, Special Subjects of Software Technology. PC & IT Engineering Department, University of Patra. Jarvenpaa, S.L., Ives, B., 1991. Executive involvement and participation in the management of information technology. MIS Q. 15 (2), 205–277. Jiang, J., Klein, G., 1999. Risks to different aspects of system success. Inf. Manag. 36 (5), 264–272. Jiang, J.J., Klein, G., Means, T., 2000. Project risk impact on software development team performance. Proj. Manag. J. 31 (4), 19–26. Jones, C., 1994. Assessment and Control of Software Risks. PTR Prentice-Hall, Englewood Cliffs, NJ. Jorgensen, M., 1999. Software quality measurement. Adv. Eng. Softw. 30, 907–912. Keider, S.P., 1984. Why systems development projects fail. J. Inf. Syst. Manag. 1 (3), 33–38. Kemerer, C.F., Sosa, G.L., 1991. Systems development risks in strategic information systems. Inf. Softw. Technol. 33 (3), 212–223. Kiountouzis, Ε., 2004. IT Project Management. Stamoulis Publications, Athens. Kirytopoulos, Κ., 2006. Project Risk Management Manual. Klidarithmos Publications, Athens. Kliem, R., 2001. Risk management for business process reengineering projects. Inf. Syst. Manag. 17 (4), 71–73. Lee, E., Park, Y., Shin, J.G., 2009. Large engineering project risk management using a Bayesian belief network. Expert Syst. Appl. 36, 5880–5887. Longstaff, T.A., Chittister, C., Pethia, R., Haimes, Y.Y., 2000. Are we forgetting the risks of information technology? IEEE Comput. 33 (12), 43–51. McFarlan, F.W., 1981. Portfolio approach to information systems. Harv. Bus. Rev. 59 (5), 142–150. McLeod, L., MacDonell, S.G., 2011. Factors that affect software systems development project outcomes: a survey of research. ACM Comput. Surv. 43 (4), 24–55. Mizuno, O., Kikuno, T., Takagi, Y., Sakamoto, K., 2000. Characterization of risky projects based on project managers' evaluation. International Conference on Software Engineering, pp. 387–395. Moynihan, T., 1997. How experienced project managers assess risk. IEEE Softw. 14 (3), 35–41. Na, K.-S., Simpson, J.T., Li, X., Singh, T., Kim, K.-Y., 2007. Software development risk and project performance measurement: evidence in Korea. J. Syst. Softw. 80, 596–605. Nakatsu, R.T., Iacovou, C.L., 2009. A comparative study of important risk factors involved in offshore and domestic outsourcing of software development projects: a two-panel Delphi study. Inf. Manag. 46, 57–68. Nidumolu, S.R., 1995. The effect of coordination and uncertainty on software project performance: residual performance risk as an intervening variable. Inf. Syst. Res. 6 (3), 191–219. Nienaber, R., Cloete, E., 2003. A software agent framework for the support of software project management. Proceedings of SAICSIT 2003, pp. 16–23. Ould, M., 1999. Managing Software Quality and Business Risk. Wiley, Chichester. Pressman, R., 2004. Software Engineering: A Practitioner's Approach, 6th edition. McGraw-Hill. Ropponen, J., Lyytinen, K., 2000. Components of software development risk: how to address them? A project manager survey. IEEE Trans. Softw. Eng. 26 (2), 98–112. Rothenberger, M.A., Kao, Y.-C., Van Wassenhove, L.N., 2010. Total quality in software development: an empirical study of quality drivers and benefits in Indian software projects. Inf. Manag. 47, 372–379. S.Ε.P.Ε., 2010. IT and Communications Business Association of Greece: Member-Companies. Recovered from: www.sepe.gr/gr/Members. Sahraoui, S., Briand, L.C., Guéhéneuc, Y.-G., Beaurepaire, O., 2010. Investigating the impact of a measurement program on software quality. Inf. Softw. Technol. 52, 923–933.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
10
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Sakhtevil, S., 2007. Managing risks in offshore systems development. Communications 50 (4), 69–75. Savolainen, P., Ahonen, J.J., Richardson, I., 2012. Software development project success and failure from the supplier's perspective: a systematic literature review. Int. J. Proj. Manag. 30, 458–469. Schmidt, R., Lyytinen, K., Keil, M., Cule, P., 2001. Identifying software project risks: an international Delphi study. J. Manag. Inf. Syst. 17 (4), 5–36. Schwalbe, K., 2000. Information Technology Project Management. Thompson learning. Shtub, A., Bard, J.F., Globerson, S., 2008. Project Management, Processes, Methodologies, and Economics, 2nd ed. Epikentro Publications, Athens. Subramanian, G.H., Jiang, J.J., Klein, G., 2007. Software quality and IS project performance improvements from software development process maturity and IS implementation strategies. J. Syst. Softw. 80, 616–627.
Sumner, M., 2000. Risk factors in enterprise wide/ERP projects. J. Inf. Technol. 15 (4), 317–327. Wallace, L., Keil, M., Rai, A., 2004a. Understanding software project risk: a cluster analysis. J. Inf. Manag. 42 (1), 115–125. Wallace, L., Keil, M., Rai, A., 2004b. How software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. Decis. Sci. 35 (2), 289–321. Ward, S., 1999. Assessing and managing important risks. Int. J. Proj. Manag. 17 (6), 331–336. Yang, C.L., Huang, R.H., Ho, M.T., 2009. Multi-criteria evaluation model for a software development project. International Conference on Industrial Engineering and Engineering Management, Proceedings of the 2009 IEEE IEEM, pp. 1840–1844. Zikmund, W.G., 2003. Business Research Methods, 7th edition. Thomson/ South-Western, Mason.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
Available online at www.sciencedirect.com
ScienceDirect International Journal of Project Management xx (2013) xxx – xxx www.elsevier.com/locate/ijproman
Quality vs risk: An investigation of their relationship in software development projects Lazaros Sarigiannidis a,⁎, Prodromos D. Chatzoglou b,1 a
b
Business Administration Dept., Kavala Institute of Technology, Agios Loukas, 65404 Kavala, Greece Production & Management Engineering Dept., Democritus University of Thrace, Library Building, Kimmeria, 67100 Xanthi, Greece Received 23 December 2012; received in revised form 3 October 2013; accepted 12 November 2013
Abstract Quality, risk and successful software development projects are three concepts which appear to be indisputably intertwined with one another. The purpose of the present study is to investigate the relationship between people quality, process quality and risk in the context of software development projects of Greek companies. Project team members with different characteristics were used as key respondents. The final sample consisted of 112 projects from 63 companies. Empirical data were analysed using the structural equation modelling technique. The main results indicate a negative effect of people quality on project risk level. On the contrary, process quality appears to have a slightly limited effect, defining only the risk level associated with the project team. The results contribute in the existing literature underlining the importance of quality on the reduction of the project risk level, thus, creating the necessary background for new similar research attempts in the future. © 2013 Elsevier Ltd. APM and IPMA. All rights reserved. Keywords: Software development; Project quality; Project risk; SEM
1. Introduction Between now and 2025, the ability of companies and their products, systems and services to compete, adapt and survive will depend to a great and continually increasing extent on the software they will be using (Boehm, 2006). This software could provide them with competitive differentiation and quick adaptation to competitive changes for today's products and services. However, software success depends on software quality (Gorla and Lin, 2010), effectiveness and completeness (Nienaber and Cloete, 2003). Software quality is frequently determined by the software development process quality (Schwalbe, 2000), which is defined through specially developed control metrics (Jorgensen, 1999; Sahraoui et al., 2010; Subramanian et al., 2007). ⁎ Corresponding author. Tel.: + 30 2513 006162. E-mail addresses: [email protected] (L. Sarigiannidis), [email protected] (P.D. Chatzoglou). 1 Tel./fax: +30 25410 79344.
Software development projects and the effects determining their outcomes have been the subjects of a continued research effort over the last thirty-five years (McLeod and MacDonell, 2011). Rothenberger et al. (2010) stated that software development process has no differences with other engineering disciplines: a managed approach is expected to lead to better quality of the product. Regardless of quality issues, software development projects are particularly difficult to be managed and this is because they incorporate a group of complex characteristics, which are not found in any other type of projects in the engineering or manufacturing field (areas which software development is mainly compared with). Human interaction, high level of complexity and product versatility are few of these characteristics (Jorgensen, 1999; Subramanian et al., 2007). Consequently, despite the fact that software has been effectively applied in a large range of areas, software development projects have a reputation for failure (Savolainen et al., 2012). Naturally, therefore, software development projects have been considered for many years as the ones with the highest risk.
0263-7863/$36.00 © 2013 Elsevier Ltd. APM and IPMA. All rights reserved. http://dx.doi.org/10.1016/j.ijproman.2013.11.001 Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
2
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
The overcoming of the chronic problems that are found in project development, like budget overruns, delays in its completion and weakness to manage and respond to user requirements (McLeod and MacDonell, 2011), are not just desired, but a basic priority for an economy as well (Yang et al., 2009). Subsequently, the role of risk management in software development is considered especially important. Not surprisingly researchers, as well as practitioners, have expressed their interest in managing risk in software development projects (Barki et al., 1993; Boehm, 1991; de Bakker et al., 2010; McLeod and MacDonell, 2011; Nakatsu and Iacovou, 2009). Risk management is a forecasting, complete, systematic and based on the archetypes approach (Hall, 1998), which attempts to standardise the relationships involving risk that threatens the (usually subjective) success of the development process (Lee et al., 2009), with the use of an applicable sum of principles and practices (Addison and Vallabh, 2002). The technological gap in Greece compared with other European countries (as measured by NRI) and the reluctance of several Greek companies-mainly SMEs to adopt new technologies, limit the potential market growth. Additionally, the software development industry in Greece does not seem to remain unaffected by the global economic crisis. Both companies and households are not willing to invest heavily in new technologies, before the end of the crisis. The impact on the software development market has been visible for about four years, as there is a direct dependence with the budgets of client companies. Thus, the current economic crisis in the EU has driven down budgets and formulates the growth rates at much lower levels. Furthermore, the highest percentage of family SMEs in the Greek economy, which does not have high requirements in software, and they are not aware of the prospects and the necessity of computer systems, lead to a restriction in the size of investment and the scope of new projects. The Greek software development projects seem to be very limited (in terms of budget, duration or employees) compared to those of the technologically developed countries (Greek National SME Observatory, 2011). Also the competition has been intensified with the entry of multinational companies in the Greek market in the last decade, which have higher funds, high costs for R & D and access to specialized workers, offering high quality products, and forming relatively quickly a wide deposit of important customers. Generally, the Greek software packages, although holding significant market shares, face stiff competition from the respective programs of foreign software houses. The latter have considerably strengthened their market shares, creating trends of market concentration, while several Greek SMEs, which constituted a large part of the industry, faced serious problems of competition. In order to create a more clear and integrated picture about the software development process in Greece in relation to other countries it can be used some well-known methodologies such as the capability maturity model integration (CMMI) which classifies software organizations into five levels based on the sophistication of their engineering and management practices (Na et al., 2007). Unfortunately Greek organizations are not listed in the aforementioned taxonomy and this kind of comparison can't be used.
The aim of this study is the creation of a new research framework, which will arise through a thorough examination of the international literature. This framework will also be tested, using empirical data, in order to measure the significance of the conceptual factors related to project quality and risk. 2. Literature review 2.1. Risk dimensions Barki et al. (1993) support that software project risks consist of interrelated dimensions and their measurement should not be done with the use of a one-dimensional scale, but, on the contrary, every dimension must be separately, theoretically and practically defined. Despite the importance of studying risk through its dimensions, only a few researches employing them have been conducted. McFarlan (1981) found three main risk dimensions of the software development process and, more precisely, the size, the technological experience and the structure of the project. He proposed that project managers should develop a complete and integrated risk profile for a software project. Later, Boehm (1991) proposed a risk management framework where he included the factors of risk estimation and control, while he also made a list of the ten most important risks, based on his personal professional experience. In spite of all these, Boehm's list lacks some theoretical foundation (Huang and Han, 2008) and also, due to complexity and other factors (invisibility, harmonisation, flexibility), which characterise software projects, it ceased to have any diachronic value as years pasted (since it was conducted in 1991) (Hughes and Cotterell, 2006). Barki et al. (1993) conducted an extensive review of studies (120 projects that had been carried out by 75 organizations) concerning software development risk, and proposed 35 parameters for assessing risks, which, in turn, were categorised in five dimensions: technological innovation, application size, expertise, application complexity and organizational environment. However, despite the fact that their study provided a quite useful and understandable tool for risk measuring, it also proved that the estimation scale was extremely complicated (Huang and Han, 2008; Wallace et al., 2004b). Further, Heemstra and Kusters (1996), based on earlier studies and their professional experience, composed a list of 36 software risks, which later were grouped into 9 categories. Moynihan (1997), in cooperation with 14 Irish software application design specialists, developed a group of 21 risk related points. Later, Longstaff et al. (2000) proposed a framework, named hierarchically holographic modelling (HHM), and distinguished seven dimensions of system integration that involved 32 types of risk. Also, Cule et al. (2000) identified 4 risk dimensions according to their source of origin (labour, customer, environment, individuality), which involved 55 software risks. Then, they proposed a primary risk management strategy for each dimension separately. Based on Boehm's (1991) risk list, Ropponen and Lyytinen (2000) developed a questionnaire which contained 6 risk dimensions for a research where 83 managers from 1100 software projects participated in. Sumner (2000), through structural
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
interviews, compared software project differences between MIS and ERP projects and proposed nine risks that are unique in ERP projects. Houston (2000), using as criteria the importance and frequency of appearance in the existing literature, presented a list of 29 software development project risks. In addition, Kliem (2001) developed a list of 38 BPR (business process reengineering) project risks, which were categorised in 4 main dimensions: people, management, business and techniques. Schmidt et al. (2001) also identified 53 risk variables, which were categorised in 14 factors and concluded that the cultural difference of the three countries (Finland, China and U.S.A.) their research was conducted in, could significantly affect the risk list, while only 11 of them had a cross-cultural application. Further, Addison (2003) used the Delphi technique to collect the views of 32 specialists and, then, presented a list of 28 risks for e-commerce projects. However, since the organizational or environmental financial and non-financial situations are constantly changing, it is certain that there will be corresponding and parallel changes in the risk items that are included on a checklist. So, it is necessary both the development of new instruments and/or the validation of the traditional instruments, allowing a cumulative background of research to be built within the project risk management framework. In the present study, the relatively recent approach of Wallace et al. (2004b) will be adopted (as for as risk dimensions is concerned). Their research has focused on the collection of questionnaires from 507 members of the Project Management Institute (PMI). Wallace et al. (2004b) proposed 27 software development risks, which were summarized into six dimensions (user, system requirements, project complexity, planning and control, team and organizational environment). These risk factors can be found in all types of IT projects (Nakatsu and Iacovou, 2009). 2.2. Project quality Although quality has been extensively studied in repetitive operations, it still remains under-researched in projects (Geraldi et al., 2011). Project quality is the factor that can significantly affect the possibility of risks appearing and the extent of these risks' consequences, during the development process of a software project (Ould, 1999). In the present study, the software project's total quality is measured using two sub factors, while the variables are also defined according to literature (Fenton et al., 2004, 2008). These two fundamental factors are: process quality (Hoffman, 2003) and people quality. Total product quality, although extensively studied in the international literature (Pressman, 2004), will not be considered in this study due to its complexity and the technical nature of its constituting characteristics. 2.3. The project quality — project risk relationship Despite the fact that quality in software development projects has been defined, analysed and theoretically supported over and over in the past in many surveys, none of them has examined the importance of its effect on project risk. In the present research, an
3
initial attempt to test this research framework, using variables proposed by Fenton et al. (2004), will be made. The main research question is: What is the magnitude of project's quality impact on project risk? 3. Research methodology 3.1. Population and sample characteristics The population of the study consists of the sum of Greek companies (software houses) that are involved in software development projects. A thorough analysis of various relevant databases (e.g. Greek ICT Market, 2009; S.Ε.P.Ε., 2010), revealed that the specific sector consists of 220 companies (study population). All these companies were informed (by post, phone and e-mails) for the existence of the present survey, and they were invited to participate. The aim was to persuade as many of these companies as possible to answer a specific questionnaire, which would allow us to base our analysis on a large sample and, therefore generalise the findings (external validity) to a larger number of similar cases. Each participating company had to specify one software development project which they were focusing on during the time the present study was carried out (sampling or statistical unit), as well as the number of people (research unit) involved in the development process, who would be requested to answer the questionnaire. From the 220 companies that were approached, only 72 responded positively, returning 124 completed questionnaires. However, twelve of these questionnaires, corresponding to nine companies, were discarded since they had several inappropriately answered questions. As a result, the final sample consists of 112 questionnaires (from 63 companies), with a response rate of 28.64%. Taking into consideration the difficulties that are presented in similar studies, where collecting primary data is an essential part (the refusal of many project managers to participate in similar researches claiming limited available time or the confidentiality policy of the company, the great geographical disperse of these companies) and the fact that one or more people from the biggest companies of this industry have finally participated, allow us to claim that the sample can be considered as extremely satisfactory, representative, as well as adequate (size wise). The majority of the participants were members of software development teams and more precisely, project managers (42%) and employees in the software development department (40.2%) (mainly programmers). The remaining 17.8% of the sample consisted of project team members who were performing additional or relative tasks (e.g. quality managers, software designers, auditors etc.). 3.2. Variable measurement — questionnaire design The collection of the necessary data from project managers or system analysts/programmers of each organisation, following similar, earlier conducted empirical studies (Barki et al., 1993; Han and Huang, 2007; Moynihan, 1997; Na et al., 2007; Nidumolu, 1995; Ropponen and Lyytinen, 2000; Schmidt et
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
4
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Table 1 Risk dimension measuring. Risk dimensions
Variables
References
User
5
Requirements Project complexity Planning and control Team
4 4 7 3
Organizational environment
4
McLeod and MacDonell (2011), Han and Huang (2007), Jiang and Klein (1999), Sakhtevil (2007), Schmidt et al. (2001), and Wallace et al. (2004b). Charette (1989), Curtis et al. (1988), Han and Huang (2007), Mizuno et al. (2000), and Wallace et al. (2004b). Barki et al. (1993), Han and Huang (2007), Kemerer and Sosa (1991), Nidumolu (1995), and Wallace et al. (2004b). Han and Huang (2007), Keider (1984), Schmidt et al. (2001), and Wallace et al. (2004b). Sakhtevil (2007), Abdel-Hamid (1989), Han and Huang (2007), Jiang et al. (2000), Mizuno et al. (2000), and Wallace et al. (2004b). Barki et al. (1993), Han and Huang (2007), Jarvenpaa and Ives (1991), Jones (1994), and Wallace et al. (2004b).
al., 2001; Wallace et al., 2004b), has been done with the use of a self-completed structured questionnaire. All the answers were collected electronically, since the respondents were allowed access to a specially designed webpage, where the electronic version of the questionnaire appeared. The reason for adopting this approach was primarily related to time, effort and economical issues, such as research cost reduction, speed of data collection, ease of coding, and quicker access to larger and geographically dispersed population groups (Dillman, 2000). The research instrument consists of questions (items) which have the appropriate psychometric abilities to describe various angles of different factors. The items that have been selected for the definition of the conceptual construct of project risk meet the following two criteria: 1) they have arisen from an extensive literature review, as well as discussions that were made with academics and other professionals, and 2) they have been tested for their validity in previous studies. On the contrary, the items used for measuring product quality are new and have arisen from literature. As it has been mentioned earlier, measuring project risk is based on the work of Wallace et al. (2004b), who have moved on to distinct 27 software development risks, grouped into six dimensions-conceptual factors (Table 1). However, contrary to Wallace et al. (2004b), who evaluated only the degree to which every risk characterises a project, the present study attempts to measure both the possibility of the appearance and the impact of risk on a project (Han and Huang, 2007), by defining its total level of risk. For the calculation of the total exposure to risk, the mathematical formula proposed by Cooper et al. (2005) was adopted. They substantiated the amount of disadvantages that the traditional way of measuring risk exposure faces (Boehm, 1991; Han and Huang, 2007; Heemstra and Kusters, 1996; Ward, 1999), as the product of possibility (P) and consistency (C) (RI = P ∗ C), stressing that the elements with high consistency but low possibility can produce low risk exposure factors and, as such, should not be considered important. In order to overcome the aforementioned problems, they calculated risk exposure as follows: RE = P + C − (P ∗ C). The participants were asked to evaluate the 27 proposed risks according to their possibility of appearance and impact (in terms of cost, schedule, technical performance and collaboration of the project team) on the most recent software development project of their company, in a scale of one (1) to ten (10), where 1 implied “rare” and 10 “almost certain” possibility of appearance, while, 1
implied “minimal or no effect” and 10 “great effect” as far as impact is concerned. Additionally, the total project quality has been measured with two basic factors, process quality and people quality (Table 2). Initially, people quality has been divided into its two basic sub-factors (management quality and staff quality). Management quality includes variables such as communications management adequacy, subcontract management adequacy, interaction management adequacy and internal management quality (AgenaRisk, 2005). On the other hand, staff quality refers to the quality of non-executive staff that is involved in a project. Variables like staff turnover, staff experience, staff motivation, staff training and programming language experience are used for measuring staff quality (Fenton et al., 2004). Moreover, process quality is another complex factor which is used to define project quality, by combining the specification clarity of a project and the development and testing process quality. Specification clarity is described using the variables of specification process quality and requirement difficulty. Development and testing process quality consists of variables concerning the regularity of reviews, the quality of documentation and the level of independent testing. The sum of quality measuring variables was evaluated in a scale of one (1) to five (5), where 1 implied that project quality was “very low”, while 5 that it was “very high”. 3.3. Questionnaire validity Before starting the distribution of the questionnaire, the necessary pre-testing took place, to verify its content validity (Zikmund, 2003). The level of understanding, “acceptance”, as well as interpretation of the questionnaire was measured at this experimental stage. Academics with research interests that are close to the object of the study were selected, as well as software development business executives that are located in Northern Greece. The pre-testing was necessary for avoiding inappropriate, partial, vague or double-meaning questions, determining the order Table 2 Project quality measuring. Project quality
Variables
References
People quality Process quality
10 6
AgenaRisk (2005), and Fenton et al. (2004) AgenaRisk (2005), Fenton et al. (2008), and Hoffman (2003)
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
5
Table 3 Exploratory factor analysis — project risk dimensions. Factor
Sub-factor
Items
Mean
Standard deviation
Loadings
ΚΜΟ Bartlett's test sig. TVΕ
Cronbach's alpha
Project risk dimensions
User
U1 U2 U3 U4 U5 D1 D2 D3 D4 PC1 PC2 PC3 PC4 DC1 DC2 DC3 DC4 DC5 DC6 DC7 T1 T2 T3 OE1 OE2 OE3 OE4
0.73 0.64 0.73 0.72 0.75 0.84 0.86 0.84 0.80 0.75 0.75 0.71 0.71 0.73 0.74 0.73 0.75 0.80 0.72 0.79 0.72 0.72 0.73 0.68 0.69 0.72 0.69
0.23 0.22 0.23 0.22 0.24 0.17 0.16 0.18 0.20 0.20 0.19 0.26 0.24 0.19 0.18 0.21 0.19 0.16 0.23 0.17 0.20 0.21 0.21 0.23 0.21 0.23 0.24
0.724 0.711 0.843 0.781 0.705 0.641 0.860 0.937 0.807 0.715 0.729 0.801 0.779 0.804 0.877 0.796 0.808 0.838 0.790 0.760 0.885 0.887 0.912 0.821 0.842 0.862 0.878
0.723 0.000 56.917
0.809
0.668 0.000 66.934
0.826
0.677 0.000 57.272
0.749
0.848 0.000 65.754
0.909
0.736 0.000 80.040 0.685 0.000 72.442
0.875
Requirements
Project complexity
Planning and control
Team
Organizational environment
of the questions in a way that no distortion tendencies are provoked, decreasing the size of the questionnaire to reduce respondents' lack of interest or frustration, and testing the adequacy and understanding of the introductory sections. Moreover, due to the fact that some of the variables that were used for measuring the factors and sub-factors are new or adopted from different studies, or even due to their validity and reliability in Greece having not been tested before, they might not have the necessary psychometric characteristics and, thus, probably negatively affect the reliability of the factors. Initially, the construct
0.873
validity of the variables was tested. More specifically principal component analysis (Varimax method) was applied for testing the unidimentionality of the factor elements, while for the estimation of their reliability the statistical metre used was Cronbach Alpha. The results (see Tables 3 and 4), allow us to claim that the used variables constitute solid and reliable structures capable of contributing to measuring the factor they belong to. Further, as it has been presented in Table 2, process quality is measured using six variables. However, due to low loadings (smaller than 0.5) two of them (PQ2 and PQ3) were eliminated.
Table 4 Exploratory factor analysis — project quality. Factor
Sub-factor
Items
Mean
Standard deviation
Loadings
ΚΜΟ Bartlett's Test Sig. TVΕ
Cronbach's alpha
People quality
Management quality
HRQ1 HRQ2 HRQ3 HRQ4 HRQ5 HRQ6 HRQ7 HRQ8 HRQ9 HRQ10 PQ1 PQ4 PQ5 PQ6
3.62 3.36 2.79 3.00 3.76 3.54 3.79 3.96 4.17 3.96 3.52 3.43 3.47 3.02
0.903 0.929 0.912 0.968 0.893 0.900 0.885 0.793 0.758 0.838 0.838 0.946 0.995 0.816
0.792 0.838 0.789 0.835 0.777 0.761 0.803 0.825 0.658 0.530 0.734 0.685 0.763 0.804
0.784 0.000 65.069
0.866
0.793 0.000 52.363
0.764
0.739 0.000 55.908
0.731
Staff quality
Process quality
–
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
6
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Table 5 First order confirmatory factor analysis. χ2/df
Factor User risks Requirements risks Complexity risks Planning & control risks Team risks Organizational environment risks Management quality Staff quality Process quality
3.548 3.078 0.223 4.148 0.000 0.541 4.453 0.727 1.008
Table 7 Hierarchical regression results. CFI
GFI
0.947 0.982 1.000 0.920 1.000 1.000 0.950 1.000 1.000
0.954 0.975 0.999 0.881 1.000 0.998 0.949 0.988 0.991
RMR 0.002 0.001 0.000 0.002 0.000 0.001 0.040 0.019 0.020
Composite reliability 0.80 0.84 0.73 0.91 0.88 0.90 0.85 0.78 0.74
The questions eliminated are related to the requirements management quality. After the completion of the exploratory factor analysis, confirmatory factor analysis (CFA) was also used in order to define the factors that will participate in the research model. The aim of this factor analysis was to test certain hypotheses which mainly derive from certain theories or are supported by valid, previous studies. From the results that are presented in Table 5, it can be stated that the previous tests for the examination of the data adjustment to the proposed model, have provided satisfactory results. After the completion of the first order confirmatory analysis, a second order CFA was also carried out. These factors were evaluated having as criteria the acceptable margins of the adjustment indices, the statistical significance of the causal relationships, as well as the suggestions of the modification index. Table 6 presents the results of this analysis (all the values of the indices examined are within acceptable levels).
Dependent variable
Project risk
Independent variable
β
t
Project quality R2/adjusted R2 F
− 0.293 0.086/0.077 10.294⁎⁎⁎
12.553⁎⁎⁎
Notes: *** Significant at the p b0.01 level.
the six dimensions of risk. On the contrary, process quality influences, in a negative and statistically significant manner, just the project team risk dimension. Both sub-factors of project quality have been found to have no statistically significant effect on the complexity dimension of the project risk (Fig. 1). 5. Summary and conclusions Based on an extensive literature review, a new conceptual model has been developed (with the parallel development of appropriate metrics for measuring the two conceptual constructs), evaluated and empirically tested. The present study is unique since none of the project risk management related researches that have taken place in Greece, have attempted to empirically test the validity and accuracy of the theoretical approaches used; they were rather limited to simple literature
4. Results 4.1. Structural model analysis First, the results of the evaluation of the relationship between project quality and risk, which was estimated using the hierarchical moderated regression analysis method are presented in Table 7. The results suggest that a (reasonably) negative (− 0.293), yet statistically significant, relationship exists between these two factors. To test the relationship of project quality and risk more extensively, project quality has been divided into its two main sub-factors, and project risk into its six dimensions. The findings suggest that there is a negative and statistically significant relationship between people quality and five out of
Table 6 Second order confirmatory factor analysis. Factor
χ2/df
CFI
GFI
RMR
Composite reliability
Project risk People quality Project quality
3.253 0.000 0.000
0.908 1.000 1.000
0.919 1.000 1.000
0.002 0.000 0.000
0.83 0.51 0.60
2 /df 0.575
CFI 1.000
GFI 0.994
RMSEA 0.000
RMR 0.002
Fig. 1. Research structural model.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
reviews or the proposition of risk eliminating tools (Igglesis, 2004; Kiountouzis, 2004; Kirytopoulos, 2006). Also, this study is the first to adopt the mathematical formula [RE = P + C −(P ∗ C)] of Cooper et al. (2005) for measuring risk exposure. Its empirical evaluation confirmed its validity and accuracy. Also, this research relies on data collected from many project team members with different characteristics (equal participation of project managers and other employees of the software development department), something that improves the estimation of the research parameters used and reduces the subjectivity of the conclusions drawn, since various perspectives of people with different levels of responsibility in the development process have been considered. Therefore, the present study is methodologically different (as far as the sample is concerned) compared to the majority of important previous studies, such as the ones of Boehm (1991) (project managers), Schmidt et al. (2001) (project directors), or Moynihan (1997) (application programmers). Additionally, it provides a clear answer to the question of Wallace et al. (2004b) whether the participation of people with different roles in the development process would result in reaching different conclusions. The answer derived from the results of this research does not seem to support directly their claim, since most of the results match those that have already been documented in literature. Moreover, great interest presents the measurement of project risk by using its specific six dimensions (measured with 27 variables). The dimension “requirements” is found to consist of the variables with the highest mean scores, thus indicating that according to the respondents, it is the dimension with the highest exposure to risk. On the contrary, “organizational environment” appears to be the least important risk factor (threat). Project team members consider that the most likely to appear risks are the ones emerging from incorrect, changing or vague requirements, which also have the highest negative effects on the project characteristics. On the other hand, Greek software development companies appear to operate in a stable organizational environment with small administrative changes during the implementation of the projects. These results are in accordance with the finding of Huang and Han (2008) and Han and Huang (2007), who presented these two dimensions as the most and least important, respectively. With regard to the mean scores of the variables concerning project quality, it is worth stating that they are quite similar, suggesting that project team members are overall highly satisfied with its quality. As far as the staff quality dimension is concerned, it is found that the mean score of the variables measuring it (staff general experience, experience with the programming language used, low rate of staff change) is very satisfactory (higher than the other ones). This shows that among the software development companies there is a widespread feeling of satisfaction with the relevant knowledge level and abilities of their members. All the above explain, to a certain degree, why the dimension “organizational environment” is considered as a low risk one.
7
Finally, the results of the structural model indicate that there is a statistically significant negative relationship between project quality and risk. Taking the analysis one step further, project quality was divided into its two main components, in order to measure their individual effect on the six project risk dimensions. It is found that people quality has a negative and statistically significant relationship with all dimensions of project risk (excluding the risks related to project complexity). This finding suggests that Greek software development companies pay attention to issues such as proper staffing, training and communication of their personnel. Quality features, like communication management quality, motives, staff change rate or staff education level, which concern the people employed at software development companies, appear to be directly related to specific risk dimensions such as planning and control and team and organizational environment of the project. This study empirically verifies the direct relationship that dictates the characteristics of these factors. However, the process quality characteristics appear not to significantly affect a project's exposure to risk. More precisely, it has been found that only the risk dimension that is related to project team is affected (negatively and statistically significantly) by process quality. The reason for this may be the relatively small size of the Greek software development companies, where processes, such as project inspections (and their frequency) and, basically, the level of independent tests, do not usually occur as organised, standardised and strategically oriented actions, which, through their careful modification or redirection, could reform those conditions and decrease projects' exposure to risks. Their exceptionally unique effect on the risk dimension of project team takes place mainly due to the special nature of the aforementioned risks. Even a non-systematic attempt to upgrade quality control and project inspection processes can provide an important solution to the problems that occur due to the insufficient training of the project team executives, by providing some guidelines for improving the quality of the project. The delicate balance that exists among these factors seems to significantly diminish during the examination of the remaining risk dimensions.
6. Managerial implications Many software development projects often fail due to lack of understanding of the risks involved. Specialists in this field claim that the risk related to software development projects have first, to be defined and then, be dealt with during the development process (Wallace et al., 2004a). This study presents a project risk model, as a second order factor model, which provides a useful framework consisting of six project risk dimensions, with each one of them being only a small part of the total “picture”. Risk management methods that focus on just one aspect of risk may even lead to more risky projects with great possibility of failure. For the above reasons, project managers will have to use similar tools in order to evaluate and manage risks, make the appropriate decision for avoiding or eliminating them and subsequently improving the possibilities for a successful project development.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
8
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
The results that derive from measuring the possibility of appearance and the impact of the 27 proposed risks and their subsequent classification, which is based on their level, provide the opportunity to professionals in the specific sector to implement and benefit from the techniques of the Pareto principle, which is widely accepted in the risk management field (Shtub et al., 2008). According to this rule, an individual can focus his/her attention on a small number of risks (the ones with the highest calculated level), in order to deal with approximately 80% of the total impact of these risks on the project (Shtub et al., 2008). Software development organizations can directly benefit from considering the findings of this study, by developing a risk profile for each one of their software development projects. The tool that has been developed in the present study could be used to create a risk framework for the distribution and evaluation of the risks involved in each project. This framework could be used in two ways. First, as a tool at many stages of the project life-cycle, that could monitor the changes of the project risk level, whenever they take place. And second, as an analysis tool for the already completed projects. The constant review of the results can be used to improve the management of future projects. Over time, the frameworks could provide guidelines informing project managers about high risk areas of similar projects that have been completed in the past. Thus, high risk projects could possibly be identified early in the project life-cycle and more appropriate decisions would be made about their continuation (or not). The risk frameworks could also be used in a way similar to the portfolio approach that McFarlan (1981) proposed. Having obtained the ability to better estimate the risk of a project, organizations can achieve the best possible results, through balancing the number of high risk projects that they take on, with a complementary amount of lower risk projects. Finally, the importance of the relationship between quality definitions (and especially the one of project people) and the level of project risk, should be stressed. The significant emerging role of quality characteristics, as a tool of moderating the level of risk, undoubtedly offers project managers the opportunity to manage their projects more effectively. Empirically tested practices of total quality management, appropriately adjusted to the requirements of every project, are likely to become a reliable and economically accessible ally over the successful confrontation of the occurring risks in software development projects. 7. Research limitations First of all, to test the validity of the results and the likelihood of generalising them, the collection of similar data from other countries, is considered essential. Another important issue involves the use of a structured questionnaire as the research instrument for collecting data. In spite of all the attention paid to its design and the pre-testing that followed, it is possible that some answers may have been affected from the format or size of the questionnaire. Ideally, structured or semi-structured interviews of all participants should have taken place to clarify some issues.
Appendix A. Questionnaire
Project risk User User1 User2 User3 User4 User5 Requirement Requirement1 Requirement2 Requirement3 Requirement4 Project complexity Project complexity1 Project complexity2 Project complexity3 Project complexity4 Planning&Control Planning&Control1 Planning&Control2 Planning&Control3 Planning&Control4 Planning&Control5 Planning&Control6 Planning&Control7 Team Team1 Team2 Team3 Organizational environment Organizational environment1 Organizational environment2 Organizational environment3 Organizational environment4
Users resistant to change Conflict between users Users with negative attitudes Users not committed to the project Lack of cooperation from users Continually changing system requirements Systems requirements not adequately identified Unclear system requirements Incorrect system requirements Project involved the use of new technology High level of technical complexity Immature technology Project involves use of technology that has not been used in prior projects Lack of an effective project management methodology Project progress not monitored closely enough Inadequate estimation of required resources Poor project planning Project milestones not clearly defined Inexperienced project manager Ineffective communication Inexperienced team members Inadequately trained development team members Team members lack specialized skills required by the project
Change in organizational management during the project Corporate politics with negative effect on project Unstable organizational environment Organisation undergoing restructuring during the project
People quality Management quality People quality1 People quality2
People quality3 People quality4 People quality5
Communications management quality Scale of distributed communications (geographical diversity, number of sites and team size) Scale of subcontracts Subcontract management quality Internal management quality
Staff quality People quality6 People quality7 People quality8 People quality9 People quality10
Staff motivation General level of staff training General level of staff experience Relevant program language experience Staff turnover
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx Appendix A (continued) Project risk Process quality Process quality1 Process quality4 Process quality5 Process quality6
Specification process quality Regularity of reviews Quality of documentation Level of independent testing
References Abdel-Hamid, T.K., 1989. A study of staff turnover, acquisition, and assimilation and their impact on software development cost and schedule. J. Manag. Inf. Syst. 6 (1), 21–39. Addison, T., 2003. E-commerce project development risks: evidence from a Delphi survey. Int. J. Inf. Manag. 23 (1), 25–40. Addison, T., Vallabh, S., 2002. Controlling software project risks — an empirical study of methods used by experienced project managers. Proceedings of SAICSIT 2002, pp. 128–140. AgenaRisk, 2005. Software Project Risk Model Manual, Bayesian Network and Simulation Software for Risk Analysis and Decision Support-AgenaRisk (Version 2.00). Barki, H., Rivard, S., Talbot, J., 1993. Toward an assessment of software development risk. J. Manag. Inf. Syst. 10 (2), 203–225. Boehm, B.W., 1991. Software risk management: principles and practices. IEEE Softw. 8 (1), 32–41. Boehm, B.W., 2006. Some future trends and implications for systems and software engineering processes. J. Syst. Eng. 9 (1), 1–19. Charette, R.N., 1989. Software Engineering Risk Analysis and Management. Intertext Publications, New York. Cooper, D.F., Grey, S., Raymond, G., Walker, P., 2005. Project risk management guidelines: Managing risk in large projects and complex procurements. John Wiley and Sons Ltd., England. Cule, P., Schmidt, R., Lyytinen, K., Keil, M., 2000. Strategies for heading off project failure. Inf. Syst. Manag. 17 (2), 65–73. Curtis, B., Ward, S., Chapman, C., 1988. Roles, responsibilities and risks in management contracting. Construction Industry Research and Information Association (CIRIA) Special Publication, 81. de Bakker, K., Boonstra, A., Wortmann, H., 2010. Does risk management contribute to IT project success? A meta-analysis of empirical evidence. Int. J. Proj. Manag. 28, 493–503. Dillman, D.A., 2000. Mail and Internet surveys — The Tailored Design Method. John Wiley & Sons, New York. Fenton, N., Marsh, W., Neil, M., Cates, P., Forey, S., Tailor, M., 2004. Making resource decisions for software projects. Proceedings of the 26th International Conference on Software Engineering. Fenton, N., Neil, M., Marsh, W., Hearty, P., Radliński, L., Krause, P., 2008. On the effectiveness of early life cycle defect prediction with Bayesian Nets. Empirical Software Engineering (Published Online). Geraldi, J., Kutsch, E., Turner, N., 2011. Towards a conceptualization of quality in information technology projects. Int. J. Proj. Manag. 29, 557–567. Gorla, N., Lin, S.C., 2010. Determinants of software quality: a survey of information systems project managers. Inf. Softw. Technol. 52, 602–610. Greek ICT Market, 2009. Annual Guide of the Greek IT and Communications, Athens. Greek National SME Observatory, 2011. Services sector: software development applications. Recovered from: http://www.acsmi.gr/Portals/0/ anaptiksilogismikou.pdf. Hall, E., 1998. Managing Risk: Methods for Software System Development. Addison-Wesley, New York. Han, W.M., Huang, S.J., 2007. An empirical analysis of risk components and performance on software projects. J. Syst. Softw. 80 (1), 42–50. Heemstra, F.J., Kusters, R.J., 1996. Dealing with risk: a practical approach. J. Inf. Technol. 11 (4), 333–346. Hoffman, G., 2003. Integrating PSP and CMMI level 5. STC 2003 Proceedings.
9
Houston, D., 2000. A Software Project Simulation Model for Risk Management. PhD Dissertation Arizona State University, Tempe, AZ. Huang, S.-J., Han, W.-M., 2008. Exploring the relationship between software project duration and risk exposure: a cluster analysis. J. Inf. Manag. Sci. 45, 175–182. Hughes, B., Cotterell, M., 2006. Software Project Management, Fourth edition. McGraw-Hill. Igglesis, V., 2004. Software Risk Management, Special Subjects of Software Technology. PC & IT Engineering Department, University of Patra. Jarvenpaa, S.L., Ives, B., 1991. Executive involvement and participation in the management of information technology. MIS Q. 15 (2), 205–277. Jiang, J., Klein, G., 1999. Risks to different aspects of system success. Inf. Manag. 36 (5), 264–272. Jiang, J.J., Klein, G., Means, T., 2000. Project risk impact on software development team performance. Proj. Manag. J. 31 (4), 19–26. Jones, C., 1994. Assessment and Control of Software Risks. PTR Prentice-Hall, Englewood Cliffs, NJ. Jorgensen, M., 1999. Software quality measurement. Adv. Eng. Softw. 30, 907–912. Keider, S.P., 1984. Why systems development projects fail. J. Inf. Syst. Manag. 1 (3), 33–38. Kemerer, C.F., Sosa, G.L., 1991. Systems development risks in strategic information systems. Inf. Softw. Technol. 33 (3), 212–223. Kiountouzis, Ε., 2004. IT Project Management. Stamoulis Publications, Athens. Kirytopoulos, Κ., 2006. Project Risk Management Manual. Klidarithmos Publications, Athens. Kliem, R., 2001. Risk management for business process reengineering projects. Inf. Syst. Manag. 17 (4), 71–73. Lee, E., Park, Y., Shin, J.G., 2009. Large engineering project risk management using a Bayesian belief network. Expert Syst. Appl. 36, 5880–5887. Longstaff, T.A., Chittister, C., Pethia, R., Haimes, Y.Y., 2000. Are we forgetting the risks of information technology? IEEE Comput. 33 (12), 43–51. McFarlan, F.W., 1981. Portfolio approach to information systems. Harv. Bus. Rev. 59 (5), 142–150. McLeod, L., MacDonell, S.G., 2011. Factors that affect software systems development project outcomes: a survey of research. ACM Comput. Surv. 43 (4), 24–55. Mizuno, O., Kikuno, T., Takagi, Y., Sakamoto, K., 2000. Characterization of risky projects based on project managers' evaluation. International Conference on Software Engineering, pp. 387–395. Moynihan, T., 1997. How experienced project managers assess risk. IEEE Softw. 14 (3), 35–41. Na, K.-S., Simpson, J.T., Li, X., Singh, T., Kim, K.-Y., 2007. Software development risk and project performance measurement: evidence in Korea. J. Syst. Softw. 80, 596–605. Nakatsu, R.T., Iacovou, C.L., 2009. A comparative study of important risk factors involved in offshore and domestic outsourcing of software development projects: a two-panel Delphi study. Inf. Manag. 46, 57–68. Nidumolu, S.R., 1995. The effect of coordination and uncertainty on software project performance: residual performance risk as an intervening variable. Inf. Syst. Res. 6 (3), 191–219. Nienaber, R., Cloete, E., 2003. A software agent framework for the support of software project management. Proceedings of SAICSIT 2003, pp. 16–23. Ould, M., 1999. Managing Software Quality and Business Risk. Wiley, Chichester. Pressman, R., 2004. Software Engineering: A Practitioner's Approach, 6th edition. McGraw-Hill. Ropponen, J., Lyytinen, K., 2000. Components of software development risk: how to address them? A project manager survey. IEEE Trans. Softw. Eng. 26 (2), 98–112. Rothenberger, M.A., Kao, Y.-C., Van Wassenhove, L.N., 2010. Total quality in software development: an empirical study of quality drivers and benefits in Indian software projects. Inf. Manag. 47, 372–379. S.Ε.P.Ε., 2010. IT and Communications Business Association of Greece: Member-Companies. Recovered from: www.sepe.gr/gr/Members. Sahraoui, S., Briand, L.C., Guéhéneuc, Y.-G., Beaurepaire, O., 2010. Investigating the impact of a measurement program on software quality. Inf. Softw. Technol. 52, 923–933.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001
10
L. Sarigiannidis, P.D. Chatzoglou / International Journal of Project Management xx (2013) xxx–xxx
Sakhtevil, S., 2007. Managing risks in offshore systems development. Communications 50 (4), 69–75. Savolainen, P., Ahonen, J.J., Richardson, I., 2012. Software development project success and failure from the supplier's perspective: a systematic literature review. Int. J. Proj. Manag. 30, 458–469. Schmidt, R., Lyytinen, K., Keil, M., Cule, P., 2001. Identifying software project risks: an international Delphi study. J. Manag. Inf. Syst. 17 (4), 5–36. Schwalbe, K., 2000. Information Technology Project Management. Thompson learning. Shtub, A., Bard, J.F., Globerson, S., 2008. Project Management, Processes, Methodologies, and Economics, 2nd ed. Epikentro Publications, Athens. Subramanian, G.H., Jiang, J.J., Klein, G., 2007. Software quality and IS project performance improvements from software development process maturity and IS implementation strategies. J. Syst. Softw. 80, 616–627.
Sumner, M., 2000. Risk factors in enterprise wide/ERP projects. J. Inf. Technol. 15 (4), 317–327. Wallace, L., Keil, M., Rai, A., 2004a. Understanding software project risk: a cluster analysis. J. Inf. Manag. 42 (1), 115–125. Wallace, L., Keil, M., Rai, A., 2004b. How software project risk affects project performance: an investigation of the dimensions of risk and an exploratory model. Decis. Sci. 35 (2), 289–321. Ward, S., 1999. Assessing and managing important risks. Int. J. Proj. Manag. 17 (6), 331–336. Yang, C.L., Huang, R.H., Ho, M.T., 2009. Multi-criteria evaluation model for a software development project. International Conference on Industrial Engineering and Engineering Management, Proceedings of the 2009 IEEE IEEM, pp. 1840–1844. Zikmund, W.G., 2003. Business Research Methods, 7th edition. Thomson/ South-Western, Mason.
Please cite this article as: L. Sarigiannidis, P.D. Chatzoglou, 2013. Quality vs risk: An investigation of their relationship in software development projects, Int. J. Proj. Manag. http://dx.doi.org/10.1016/j.ijproman.2013.11.001