Software process improvement success factors for small and medium Web companies: A qualitative study

Information and Software Technology 54 (2012) 479–500

Contents lists available at SciVerse ScienceDirect

Information and Software Technology journal homepage: www.elsevier.com/locate/infsof

Software process improvement success factors for small and medium Web companies: A qualitative study Muhammad Sulayman a,⇑, Cathy Urquhart b, Emilia Mendes c, Stefan Seidel d a

The University of Auckland, New Zealand Manchester Metropolitan University, UK c Zayed University, United Arab Emirates d University of Liechtenstein, Liechtenstein b

a r t i c l e

i n f o

Article history: Received 30 April 2011 Received in revised form 13 December 2011 Accepted 26 December 2011 Available online 11 January 2012 Keywords: Software process improvement Success factors Grounded theory Small and medium Web companies

a b s t r a c t Context: The context of this research is software process improvement (SPI) in small and medium Web companies. Objective: The primary objective of this paper is to identify software process improvement (SPI) success factors for small and medium Web companies. Method: To achieve this goal, we conducted semi-structured, open-ended interviews with 21 participants representing 11 different companies in Pakistan, and analyzed the data qualitatively using the Glaserian strand of grounded theory research procedures. The key steps of these procedures that were employed in this research included open coding, focused coding, theoretical coding, theoretical sampling, constant comparison, and scaling up. Results: An initial framework of key SPI success factors for small and medium Web companies was proposed, which can be of use for small and medium Web companies engaged in SPI. The paper also differentiates between small and medium Web companies and analyzes crucial SPI requirements for companies operating in the Web development domain. Conclusion: The results of this work, in particular the use of qualitative techniques – allowed us to obtain rich insight into SPI success factors for small and medium Web companies. Future work comprises the validation of the SPI success factors with small and medium Web companies. Ó 2012 Elsevier B.V. All rights reserved.

1. Introduction Software processes play an important role in helping project teams in software development organizations, and in addition motivate the use of similar and sound practices [1]. Formal processes emphasize the explicit command-and-control side of the organization due to their concrete nature, while informal team practices emphasize the mutual adjustment and explorations needed to successfully accomplish the software project and associated process tasks [2,3]. Almost all modern software organizations operate in a competitive market, under tight time and cost constraints [4]. As an answer to their needs, organizations have started to undertake software process improvement (SPI) initiatives ([5] for an overview ⇑ Corresponding author. Address: The University of Auckland, City Campus: Level 3, Room 596, Science Centre Building 303, 38 Princes St., Auckland 1010, New Zealand. Tel: +64 210 253 2034; fax: +64 9 373 7453. E-mail addresses: [email protected] (M. Sulayman), C.Urquhart@ mmu.ac.uk (C. Urquhart), [email protected] (E. Mendes), [email protected] (S. Seidel). 0950-5849/$ - see front matter Ó 2012 Elsevier B.V. All rights reserved. doi:10.1016/j.infsof.2011.12.007

of different approaches) aimed at increasing the maturity and quality of their software processes [6]. Investment in process improvement has had various business benefits e.g., improved product quality, reduced time to market, better productivity [6], increased organizational flexibility, and customer satisfaction [7–9]. Development processes have been found to be related to the quality of the products delivered [10,11]. Therefore, it is important to investigate the factors that enable the adoption of new processes and improve existing processes. In this regard, various researchers have investigated SPI success factors [12–15] and people issues that inherently play a major role in the adoption of new processes or improvement of existing processes by software organizations [8]. In the current era, small and medium software development companies represent a large percentage out of the total number of software companies; according to a recent survey 99.2% of the world’s software companies are small and medium in context [16]. It has also been observed that in recent years small and medium-sized software development companies have emerged very swiftly and many of them are working in the domain of Web development [17], which are termed Web companies [18]. Within the context of this work Web companies are defined as companies that

480

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

are exclusively involved in Web software development related services. SPI is a repetitive activity and irrespective of the SPI approach adopted; it requires certain time, resources, steps, and iterations for its effective and successful implementation [13]. SPI success factors, also termed as critical success factors, are those factors that are considered determinants of the success of a SPI program within an organization (small, medium or large) [12,14]. SPI models used for large organizations are not considered suitable for small and medium-size organizations due to their complex nature and expensive costs [15]. The need for a different approach to SPI for small and medium size organizations is also supported by corporate SPI giants such as ISO and CMMI, which have formulated focus groups to cater specifically to the SPI needs of small and medium software companies [19,20]. In addition, a number of researchers have proposed their own SPI frameworks for small and medium software organizations [21,22]. However, very few studies have specifically focused on SPI for ‘Web’ companies, despite Web development being an industry with a growth rate of 20% each year and yielding very high revenues, e.g. 95 billion US$ in 2004, which is almost three times the aerospace industry [23]. Web development differs from traditional software development in a number of ways such as the use of different development methodologies; different nature of projects; and different quality, testing and design requirements [24,25]. The differences between traditional software and Web applications are detailed in Table 1 [18]. There are numerous definitions for small and medium enterprises. According to the European Union classification [26], small companies comprise up to 20 employees, with a turnover of up to two million euros, whereas medium companies have 21–250 employees, and a turnover of less than fifty million euros [26]. Small and medium software companies are defined as those where staff numbers fall in the range of 20–100 [27–30]. Web companies are companies engaged in pure Web development including Web hypermedia applications, Web software applications, and Web applications [18 Chap. 1,31]. To date, there are no standard definitions of small and medium Web companies. Based on our findings

from the systematic literature review (see Section 2.2) and replication study (see Section 2.3) we have characterized small Web companies as companies that have from 3 to 20 employees, working on small Web projects with tight budget constraints and strict deadlines [32,33]; medium Web companies have from 20 to 50 employees who work on Web projects lasting for 6 months on average [32,33]. In order to deal with the specific needs of Web companies and software companies that develop Web applications [34], a new research field, ‘Web Engineering’ [35] was created. Web Engineering is defined as ‘the use of scientific, engineering, and management principles and systematic approaches with the aim of successfully developing, deploying and maintaining high quality Web-based systems and applications’ [35]. Web engineering recommends the use of Web agile process models [36–38], such as adoptions of Rapid Application Development (RAD), customized Scrum for Web projects [39–41], Fusebox Lifecycle Process [42], evolving Web specifications [43], extreme programming extensions for Web [44], collaboration in Web processes [45], implying that the Web development methodologies are also different from traditional software methodologies [37,46–49]. Web application developers are also different in attitude and approach from the traditional software developers by focusing strongly on hypermedia context and continuous evolutionary approaches [35]. Some examples of Web development methods that differ from traditional software methods, include the Object Oriented Hypermedia Design Method (OOHDM), Simple Web Method (SWM), Object Oriented Web Solution (OOWS), and UML-based Web Engineering (UWE) and WebML as an extension of Unified Modeling Language (UML) [25,50–55]. Web Requirements Engineering is also moving from task-orientation towards goal-orientation [56] based on Navigational Development Techniques (NDT) [57]. Researchers are also focusing on devising special project management initiatives like Web Integrated Project Support Environment (WIPSE), Action minutes and Project administration Web Site (PAWS) for Web development companies [58,59]. In addition, testing of Web applications is also different from that of traditional systems and revolves around

Table 1 Differences between traditional and Web applications [18 Chap. 3]. Web-based approach

Traditional approach Integration of distinct components (e.g. COTS, databases, graphical images), monolithic singleplatform applications

Approach to quality delivered Development process drivers Availability of the application

Integration of numerous distinct components (e.g. fine-grained, interpreted scripting languages, COTS, multimedia files, HTML/SGML/XML files, databases, graphical images), distributed, cross-platform applications, and structuring of content using navigational structures with hyperlinks Variety of Java solutions (Java servlets, Enterprise JavaBeans, applets, and JavaServer Pages), HTML, JavaScript, XML, UML, databases, third-party components and middleware, etc. Quality is considered as of higher priority than time to market Reliability, usability, and security Throughout the whole year (24/7/365)

Customers (stakeholders)

Wide range of groups, known and unknown, residing locally or overseas

Update rate (maintenance cycles) People involved in development

Frequently without specific releases, maintenance cycles of days or even hours Web designers and programmers, graphic designers, librarians, database designers, project managers, network security experts, usability experts, artists, writers Two-tier to n-tier clients and servers with different network settings and bandwidth, sometimes unknown Software engineering, hypermedia engineering, requirements engineering, usability engineering, information engineering, graphics design, and network management Content can be easily copied and distributed without permission or acknowledgment of copyright and intellectual property rights. Applications should take into account all groups of users including those handicapped Structured and unstructured content, use of hyperlinks to build navigational structures

Application characteristics

Primary technologies used

Architecture and network Disciplines involved

Legal, social, and ethical issues

Information structuring and design

Object-oriented methods, generators, and languages, relational databases, and CASE tools Time to market takes priority over quality Time to market Except for a few application domains, no need for availability 24/7/365 Generally groups confined within the boundaries of departments, divisions, or organizations Specific releases, maintenance cycles ranging from a week to several years IT professionals with knowledge of programming, database design, and project management One to two-tier architecture, network settings, and bandwidth are likely to be known in advance Software engineering, requirements engineering, and usability engineering Content can also be copied infringing privacy, copyright, and IP issues, albeit to a smaller extent Structured content, seldom use of hyperlinks

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

different quality dimensions like load, stress, and volume [60]. Rich Internet Applications (RIA), for example, leverages the above specified Web Engineering practices [61]. The fact that the engineering of Web applications differs from the engineering of software applications and that the SPI of large organizations is different from SPI of small and medium organizations [15,21,22] provided the primary motivation for this research. The contribution of this research is twofold: first it adds to the body of knowledge in Web engineering by identifying key SPI success factors important for small and medium Web companies, thus enabling them to also achieve SPI benefits such as organizational excellence, overall improvement and competitive advantage [12– 14]; second, it demonstrates that qualitative methodology, particularly drawing on principles leveraged from grounded theory, that can be usefully employed in the domain of software engineering to identify SPI success factors for software organizations. The data used in this research was gathered from interviews with practitioners of 11 Pakistani companies. Pakistan was chosen because it has a vibrant and growing Web development industry with annual IT exports of US$1.4 billion and has a large number of contributing small and medium Web development companies which predominantly operate in the outsourcing market [62,63]. The Web companies that participated in this research represented an excellent ‘purposive sample’ [64] because data was gathered from only those companies which were either small or medium in size and were exclusively involved in Web/Web applications development. We believe that Web companies comprising the sample used in this research are also characteristic of Web companies with similar characteristics operating either in other countries or in outsourcing markets; therefore the results can be applied outside the scope of only Pakistani Web companies. Given the motivation of our research, the research questions addressed are the following:  What are the major Web development process improvement success factors for small and medium Web companies and how are the identified factors are interrelated?  How can an inductive and qualitative approach to data analysis be leveraged to gain a rich understanding of Web development process improvement factors for small and medium Web companies?  What are the specific SPI requirements of small and medium Web companies? The remainder of the paper is organized as follows. Section 2 discusses the related work, which is used as motivation to our research. Section 3 explains the methodology employed to identify SPI success factors, followed by a detailed explanation of the research procedures in Section 4. Section 5 presents our findings, and Section 6 presents the resultant empirically developed framework. In Section 7, we discuss the framework and in Section 8 we present our conclusions and comments on future work.

2. Related work 2.1. SPI success factors in software companies SPI is an established research area and numerous studies have contributed towards its different aspects [12,14,28,65–69]. SPI success factors are the factors that influence SPI initiatives in software companies [13,14,67,70–73]. SPI success factors play a major role in implementation of successful processes resulting in quality software, lesser development time and enhanced productivity [74]. Numerous studies have listed different sets of SPI success factors according to the size of software development companies

481

[13,67,68,72] or environmental/cultural conditions [41] or geographical locations [71,75,76]. There are certain existing studies that have summarized existing studies on SPI success factors [72,73,77]. Habib [77] has identified critical SPI success factors for software companies from previous literature without considering type, size or environmental/cultural conditions. Listed factors from 17 studies include ‘senior management support’, ‘staff involvement’, ‘experience of staff’, ‘training’, ‘allocation of resources’, ‘communication’, ‘SPI goals’, ‘organization culture and politics’, ‘visibility of process and their success’, ‘process champions’, ‘reviews’, ‘clear vision’, ‘tools’, ‘reward schemes’, and ‘process ownership’ [77]. In another recent study [41], Nayoung has identified ‘build product map and process customization’, ‘relieve developers from customer support tasks’, ‘implement a process as early as possible’, ‘reflect on and improve the process continuously’, ‘keep system documentation consistent with the system state’, ‘avoid uneven knowledge distribution’, ‘implement a communication pattern’, and ‘plan for growth’ as key lessons learnt for the growth of business with respect to processes [41]. Pino et al. [72] conducted a systematic review on SPI in small and medium software companies, where data on 45 existing studies stating was synthesized, showing as key SPI success factors the following: ‘use of existing SPI models’, ‘establish software processes’, ‘prioritize the SPI efforts by linking with business goals’, ‘evaluation of SPI programmes with metrics’, ‘use knowledge management for SPI’, ‘improve the relationship and cooperation with clients’, ‘lead process improvement experiment’, ‘SPI consultancy’, ‘systematic process improvement approach – iterative’, ‘short SPI cycles’, ‘look for external financial aid’, ‘joining together with other companies so as to both finance and share specialized ‘resources’, ‘track SPI by frequent assessment of processes’, ‘SPI awareness’, ‘tailoring of process measurements’, ‘establishment of infrastructure for communication support’ and ‘managerial commitment and support’ [72]. Khan et al. [73] have conducted a systematic review of SPI success factors in off-shore development outsourcing software vendors, where data was synthesized from 122 studies. The critical success factors identified were: ‘cost-saving’, ‘skilled human resource’, ‘appropriate infrastructure’, ‘quality of products and services’, ‘efficient outsourcing relationships management’, ‘organization’s track record of successful projects’, ‘efficient project management’, ‘efficient contract management’, ‘SPI certification’, ‘knowledge of the client’s language and Culture’, ‘timely delivery of the product’, ‘knowledge exchange’, ‘data protection laws’, ‘financial stability’, ‘company Size (large and medium)’, ‘risk sharing’, ‘pilot project performance’, ‘vendor’s responsiveness’, ‘political stability’, ‘overseas Offices’, ‘soft deliverable’, and ‘industry–university linkage’. 2.2. Systematic literature review (SR) of SPI in small and medium Web companies Given the motivation of this research, our first step was to carry out a systematic literature review (SR) on SPI for small and medium Web companies’, which took place using two iterations [33,78]. The difference between our SR and the other studies conducted is that our SR has specifically focused on SPI for small and medium Web companies. The two SR iterations led to the selection of eight studies that were truly related to our domain of interest [30,76,79–84]. Seven studies proposed generic SPI models claiming to be valid for small and medium Web companies as well [30,79–84] (IMPACT [84], PRISMS [83], Project Postmortem [82], PmCOMPETISOFT Framework [30,81], XPMM Model [80], the Open Group Architectural Framework (TOGAF) [79]). However, none of the studies have

482

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

suggested a specific model for Web SPI. We believe that the nature of Web projects developed by small and medium companies in our SR may have had an influence on the choice of using generic SPI models e.g., the Web companies that develop Web applications (e.g., e-commerce applications, Internet banking, etc.) may fit well with a conventional SPI framework, as opposed to those Web companies that develop hypertext rich applications. Irrespective of the assumption, the fact remains that a specific model of Web SPI was not found in the literature. The eight study selected by our SR was the only one proposing a set of techniques/practices believed to be useful for SPI success in Web companies [76]. Four of the studies selected in our SR have indicated that most Web companies are relatively low budgeted [30,81,83,84]. Moreover, two studies have indicated that their strategies are intensive and they demand immediate results within limited time [83,84]. Corporate SPI giants like SEI and ISO are also formulating focus groups to make their models more practicable and feasible for smaller sized companies [19,20]; however their explicit focus so far includes software companies, not Web companies. Four selected studies also suggested integrating and adopting existing SPI models to Web companies [79,82–84]. However, the integration and adaption phenomena have not been completely narrated and the studies do not indicate how to tailor new Web standards and procedures along with the existing SPI models. No specific SPI model or technique applicable to Web companies are identified in these studies and thus suggests a possible research gap related to SPI frameworks for Web companies that consider the constraints and challenges specific to Web companies. The need for new development models that deal specifically with Web projects has also been evidenced in recent research; e.g., new size measures for Web cost estimation [85], UML based Web engineering [50], OOHDM for Hypermedia Web applications, and navigational design techniques (NDT) for Web [57], thus suggesting that Web projects and companies have special requirements. The SPI success factors identified from our SR were the following: ‘Automated tools support for process improvement’ [30,79– 84], ‘use of measurements and metrics’ [30,76,79–81,83], ‘breakdown of SPI steps and adopting iterative approach’ [79–82,84], ‘project monitoring through reviews’ [79,80,82,83], ‘compliance with Existing Standards i.e. CMMI, IDEAL, ISO, etc.’ [79,82–84], ‘client Satisfaction’, ‘development Team Satisfaction’, ‘feedback from discussion’ [82–84], ‘company’s support to SPI’, ‘definition of SPI objectives’, ‘employee participation in SPI’ [30,81], ‘employee trainings’ [79,80], ‘prioritization of processes’ [30], ‘periodic assessment of processes’ [80], ‘documentation of processes’ [79], ‘alignment of SPI with business goals’ [83] and ‘achieving SPI goals and benefits’ [30,76,79–84].

Given that we were also interested in investigating SPI success factors for SM Web companies we believe that the first step would be to replicate that study in order to assess whether the factors identified in [28] would also be applicable within our context – SMs Web companies. In summary, our replication study [32] aimed at investigating whether the theoretical model of SPI success factors proposed by Dyba [13], which was related to small and large software companies, would also be applicable to small and medium Web companies. Dyba’s theoretical model consists of six independent variables – ‘business orientation’, ‘involved leadership’, ‘employee participation’, ‘concern for measurement’, ‘exploitation of existing knowledge’, and ‘exploration of new knowledge’ [13]. The results of the replication study [32] indicated low values of regression coefficients, which were not the same as those shown in [13]. The low regression scores also indicated a possibility that there were some other critical SPI success factors for small and medium Web companies that were not part of the theoretical model proposed in [28]. This was a reasonable assumption given that the scope and size of the companies investigated in both studies differed. This further strengthened our argument that SPI success factors for small and medium Web companies are different from those useful to software companies. In addition, the instrument proposed in [28] was developed in 2000 when the Web Engineering discipline was at its embryonic stages [35,49]. This also motivated our work into identifying SPI success factors specific to small and medium Web companies. 2.4. Research gap In summary, to the best of our knowledge, to date no specific framework or model of SPI success factors and their measurements specifically exist for small and medium Web companies. As discussed above, the replicated study [32] performed aimed at checking whether the framework proposed in [28], which was specific to small and large software companies, was also applicable to small and medium Web companies. However, results showed that this was not exactly the case. The replicated study [32], suggests that the SPI factors identified by Dyba [13] do not completely apply to small and medium web companies. The different context and nature of Web projects and therefore, Web companies, makes it an interesting case to investigate as to how SPI should be tailored for them and what factors can be influential regarding their initiatives for SPI success. Hence, we chose to explore this avenue of research further and in this study we identify success factors, their relationships and measurements for the SPI activities within the domain of the small and medium Web companies. The current study uses grounded theory based qualitative data analysis to better understand the results of the replicated study, to develop an alternative set of factors, and to extend that set into an initial exploratory framework.

2.3. Replication study of a theoretical model of SPI success factors 3. Research methodology One of the main goals of the SR we carried out was to gather evidence on the state of the art relating to SPI success factors for small and medium (SM) Web companies. No evidence was found on this aspect specifically; however, during the search phase for primary studies, one of the studies we came across investigated SPI success factors within a broader context – that of small and large companies (software and Web companies) [28]. Further research revealed other related publications of the same author that elaborated their work further [13,86]. This study was the only one retrieved during the search phase that presented a theoretical model of SPI success factors. Out of the companies surveyed in [13,28,86], a very small sample comprised Web development companies, but no explicit success factors or measurements of the SPI success for small and medium Web companies were identified.

The research methodology employed herein is qualitative in nature. Qualitative research methods in the field of empirical software engineering have gained popularity because of the ‘rich diverse data,’ ‘triangulation,’ and ‘greater interpretive results’ [87] and are considered to be very useful for studies where there is an involvement of social or soft factors in empirical software engineering and information systems [87–90]. Specifically, this study applies a case study approach [91] and draws on the principles from grounded theory [92]. Case study research is particularly suitable in order to investigate a phenomenon within the boundaries of its natural context [91], and is used in almost all branches of science and engineering, including empirical software engineering [93].

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

For the purpose of analyzing the data, this study makes use of grounded theory procedures as proposed by Glaser and Strauss [92]. Grounded theory is an advanced form of inductive research, where the theory is built from data, concepts are linked to each other, and then the emergent theory is engaged with an existing theory [75,90,94–96]. It is ideal for use in situations where no previous theory exists [97], also the case for the domain under this investigation; its procedures are fit to investigate social and cultural issues and the methodology’s inductive nature makes it ideal for the discovery of new factors and the elaboration of existing SPI factors in small and medium Web companies. Grounded theory was first introduced in 1967 by Glaser and Strauss [92]. Later, however, Glaser and Strauss published differing versions of grounded theory i.e., Glaserian and Straussion [98,99]. The major differences between two versions are on coding techniques, families and paradigms [94]. This research leveraged and adapted the Glaserian procedures for coding because they are considered to be simpler to use, closer to the original version, and to be applied more flexibly [100]. While it is one of the primary features of grounded theory, this study does not aim at generating a grounded theory, but to develop an initial framework of SPI success factors. This is in line with the well-accepted, quite flexible usage of grounded theory in the information systems discipline, where often researchers make selective use of grounded theory techniques in a variety of qualitative designs [101] without generating grounded theory [102,103]. In a recent book on the method, Corbin and Strauss argue that researchers can leverage grounded theory procedures without actually setting out to develop a grounded theory, as long as they explicitly state they are doing so [104]. Specifically, in this study, phase we first identified the substantive area to be investigated, SPI success factors for small and medium Web companies. The next phase was gathering ‘slices of data’ as suggested by Glaser and Strauss [92] in order to generate conceptual categories from the transcribed open ended interviews. Following this, and based on the coding procedures proposed by Glaser, categories, and properties of categories, were identified, and ‘relationships’ were formulated which link different categories together [92]. ‘Theoretical sampling’ – a technique where the analyst decides on analytical grounds where to sample from next – was applied until the categories were saturated [100] and no new categories were found within the data. Finally, the categories were grouped and abstracted into larger categories [92] so as to develop an integrated framework of SPI success factors. 3.1. Data collection There are numerous small and medium Web companies operating in Pakistan, mainly because of the large growth of the global outsourcing business in the region. These companies operate in the international marketplace, and therefore are inclined towards excellence in processes and increases in quality for the purposes of international competition. SPI for them represents a way to compete and gain more business. As such, Pakistan appeared to be an excellent environment to investigate how SPI operates in Web development businesses. The data used in our investigation was gathered from the professionals of Pakistani Web companies by conducting open ended interviews. The contacted small and medium Web development companies had already participated in a previously conducted replication study [32]. Initially for this research, CEOs and managers of 18 small and medium Pakistani Web development companies were contacted to seek their consent for participation. The management of 11 companies agreed to participate, and open ended interviews with 21 professionals, working within these 11 companies, were conducted after obtaining their willingness to participate in this

483

research. The employee size, which is the general criteria used to determine size of a company [27,28,105], varied from 14 to 96. All the small and medium Web companies that participated in this research were engaged in pure Web applications development. Table 2 illustrates the characteristics of interviewees and companies which participated in this research. The number of interviewees from each participating company is presented in Appendix A. Note that prior to interviewing practitioners, ethics approval was obtained from the Human Ethics committee of the University of Auckland, New Zealand (Ref. No. 2009/240). Some of the respondents were interviewed twice or thrice after the main interview for the purpose of ‘theoretical sampling’ [100]. While theoretical sampling is defined in grounded theory as deciding on analytic grounds where to sample from next, in order to expand and densify the theory, it also allows the researcher to revisit respondents for the purpose of checking on relevance. As [106] states ‘relevance can . . . be checked and elaborated on by asking top informants to appraise and give more data on categories proving to become core to the analysis’. So while no new respondents were sampled using theoretical sampling, we did use the technique to help us saturate categories. Collecting rich data allowed us to gain an in-depth understanding of the subject matter. The interviews were conducted in the native language of the professionals, which was ‘Urdu’, so that the interviewees could express themselves with ease. The interviews were audio-taped, translated to English, and transcribed. The translation and transcription were done with great care so as to preserve the context of the dialogs. For testing purposes, two volunteers were hired. They re-translated a sample of the transcribed interviews into ‘Urdu’ language to ensure that the process was performed correctly. Having verbatim transcriptions of the interviews allowed for a detailed analysis and cross-checking among researchers. We used the qualitative data analysis tool NVivo for data storage, retrieval, organization, and analysis, thus ensuring that all researchers had access to both data and conceptualization. 4. Data analysis This section explains the major steps of the data collection and analysis using grounded theory procedures. The process is briefly elaborated in Fig. 1. As mentioned in Section 3.1, the data gathering phase comprised open ended semi-structured interviews, which were conducted with the practitioners of small and medium Web companies. The interviews were later translated and transcribed. Our data analysis steps contained open, focused coding and theoretical coding phases leveraged from the principles of grounded theory as suggested by Glaser and Charmaz [92,100,107]. The coding techniques were used to generate codes from the data slices which resulted from identifying categories, their key properties and existing relationships among the categories, as shown in Fig. 1. Memos were created throughout the analysis phases which supported the identification of key properties and formulation of meaningful relationships among different categories. It is important to note that these stages are not necessarily discrete, and that some reflexivity and iteration occurs between the stages. For instance, the focused coding stage, where more significant codes from the open coding phase are identified and coded for, may necessitate a return to that open coding stage as codes are further considered. Similarly, the theoretical coding stage, where relationships between categories are considered, may mean that the meaning and naming of categories are also reconsidered. 4.1. Open coding Open coding was applied to the transcribed interviews to identify categories, sub categories, and their properties. To do so, all

484

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500 Table 2 Respondents and companies’ characteristics. Respondents characteristics Work experience

Mean

Std. deviation

Average number of years in Web company Average number of years in Web development

3.96 7.07

2.82 1.86

Highest education achieved

Frequency

Percentage

Bachelor’s degree Master’s degree

7 14

33.3 66.6

Job title

Frequency

Percentage

Web project manager/development manager Quality assurance professional/team leader Software process improvement professional Web developer

6 6 5 4

28.6 28.6 23.8 19.1

Total number of interviewees

21

100

Web companies characteristics Age

Frequency

Percentage

Less than 5 years Between 5 and 10 years Between 10 and 15 years 15 years and above Total number of companies participated

2 5 3 1 11

30 55 10 5 100

Project duration

Mean

Std. deviation

Number of Weeks

26.1

24.6

Number of Web development staff

Frequency

Percentage

Less than 20 More than 20 and less than 100

6 5

54.5 45.5

Category of applications

Frequency

Percentage

Standard applications i.e., Web product software Tailor made Web solutions for external customers Tailor made Web solutions for internal company customers i.e., in-house development

10 9 5

90.1 81.2 45.5

Table 3 An example of identifying open codes. Excerpt: ‘‘According to my opinion, the overall (objective of SPI)1 is to (reduce cost)2 and (process improvement directly relates to the cost)3. On one hand (you incur cost by introducing a process)4 yet on the other hand you can (save cost by using that same process in the development)2. This (cost benefit analysis)4 should be evaluated very carefully.’’ [Interviewee 7] Open codes: SPI objectives1, cost reduction by SPI2, cost for SPI3, cost–benefit analysis4

Fig. 1. Steps of data collection and analysis – leveraged from the principles of grounded theory.

interviews were coded line-by-line in order to identify open codes [92,100,108]. The coding was performed using NVivo version 8. During the open coding phase we assigned codes to words or relevant group of words that were pertinent to the context of our research. An example of this process is given in Table 3 where an excerpt from an interview is provided and identified words/groups

of words are underlined. Assigned open codes and associated words/groups of words were underlined for easier identification. Important codes emerged from the data analysis and were labeled appropriately. Codes were compared to each other both within and across interviews – this is an important element of the grounded theory procedure where ‘constant comparison’ is performed by continually comparing occurrence incidents of categories to see if they fit and are workable [99]. Table 3 gives an example of open coding. The coding process for the first interview took 40–45 h to do; however, the amount of time applying open coding shortened as more interviews were processed. Finally, memos, suggested by Glaser to be the core element of grounded theory [108], were used with codes to recall the context of open codes identified during the coding phase. 4.2. Focused coding In the classic version of Glaserian grounded theory, selective coding follows open coding and is about deciding upon and coding around a core category, but this was not our aim – we simply

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

wanted to be able to organize and group our open codes for SPI factors. As Glaser says, at this stage, the other variables are not lost. Some grouping is necessary to get to a reasonable level of abstraction, as open coding is typically carried out at the word or sentence level [92,108,109]. Thus we decided to adopt Charmaz’s [107] ‘focused coding’ stage where the larger and most significant codes in open coding were used to code the data. Relevant open codes were grouped in the form of categories, which are concepts at a more abstract level [94]. Some of the open codes take the form of categories or sub-categories while other become properties of these categories. We identified 18 categories into which a large number of sub-categories and properties were incorporated. This was performed by employing ‘Constant comparison’, where each instance is compared to other instances of the same category to see if it is representative of that category. We believe that 18 is not a small number of categories as these encompass their respective sub-categories and properties. Table 4 gives an example of the focused code of ‘Automated Tools Support’ and the open codes that comprise this focused code. 4.3. Theoretical coding In the Glaserian strand of grounded theory, theoretical coding is used to determine the relationships among categories [92,99]. Theoretical coding provides the criteria to analyze and develop relationships between categories [99,106]. This type of coding knits the pieces of data to conceptualize the relationships. Theoretical coding is performed after the completion of selective coding which helps in developing a coherent theoretical scheme. Memos created during open coding phase were found beneficial while formulating relationships among the identified categories. Glaser [110] suggested 23+ ‘coding families’ which can guide the relating of categories. We preferred to look to the explanations given by our respondents as a guide to relationships between categories. An example of theoretical coding is given in Table 5 which shows that within the given text ‘Driven By’ is the key relationship identified between the open codes ‘SPI Success’ and ‘Company vision’. The adopted approach for theoretical coding was flexible and it helped relate the categories that led to the theoretical model explained further in Section 5. 4.4. Theoretical sampling and constant comparison As indicated previously, two basic principles of grounded theory, namely constant comparative analysis and theoretical sampling were applied in this research. Constant comparison of data refers to the comparison of different incidents or ‘slices of data’ [92] in order to develop problem Table 4 Automated tools support category and relevant concepts. Automated tools support

Customized tools for SPI, trainings for automated tools, eliminate manual laborious work, gap fillers in processes, less error prone, improved reporting features, manage complex processes with ease, implement SPI measurements and data collection, make processes streamlined

Table 5 An Example of Identifying Relationships among Categories. Excerpt: ‘‘SPI is driven by the corporate strategy and vision of the company and this is one of the actual success factors of SPI.’’ [Interviewee 11] Categories: SPI Success, Company Vision Relationship: Driven By

485

space and combining the data to conceptualize the categories. Constant comparison is performed during all the three aforementioned coding phases, until the ‘saturation’ of categories is reached. This was done by initially comparing data with data and later data with theory until no new categories, sub categories, or relationships were found and saturation was reached. According to Glaser, theoretical sampling is used to obtain new sample data and compare that with existing cases [92]. Also, theoretical sampling helps avoid the application of pre-convinced notions of the researcher on the theory as the process of data collection and analysis is driven by the emergent theory [108]. In the present study, this was done by interviewing a subset of interviewees repeatedly, based upon emergent categories and relationships. ‘Theoretical sampling’ was performed as a parallel step where new samples of data were acquired based on already coded data. The new data was ‘fit’ and ‘relevant’ as suggested by Glaser [106], thus adding necessary rigor and confirming the existing datum. This research gained the two advantages of theoretical sampling, as stated by Glaser [106]. First, the sample could be enlarged, thus supporting the full development of categories. Second, theoretical sampling allowed the researcher to gain deep insight into the respondent’s different opinions, thus allowing analysis of data with regards to both similarities and differences. One example of how theoretical sampling was applied in this research is the analytical concept of ‘Automated tools support’ which was initially related to ‘SPI success’. Upon further exploration, it was also found to be related to ‘SPI measurements’.

5. Findings This section presents the findings of this research i.e., an initial exploratory framework of SPI success factors for small and medium Web companies and the specific requirements for SPI in small and medium Web companies. The findings were reached with the help of the steps outlined in Section 4. 5.1. An initial exploratory framework of SPI success factors for small and medium Web companies This section details the selective categories of SPI success factors for small and medium Web companies. The identified categories and the resulting initial exploratory framework of SPI success factors constitute the main contributions of this research. The description for each of the categories also includes the respective sub categories, properties and important relationships. The data source of each category is the interviews conducted for this research and thus the categories represent the true perspective of a set of practitioners based on their experiences of working with small and medium Web companies and on projects specific to Web development. The chain of evidence is provided for each category in the form of quotes from interviews. The categories of SPI success factors discussed below are specific to small and medium Web companies. Appendix B of this paper contains a table mentioning the number of interviewees who stated or commented on the categories or its factors/aspects to provide a brief overview of the categories importance. 5.1.1. SPI success As this research is a qualitative investigation of SPI success factors for small and medium Web companies, SPI success emerged as the core category of this research. As such, the core category has the analytical power to form a coherent theoretical scheme, holding together the other categories that emerge from the research [111].

486

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

The successful implementation of SPI, as suggested by a few interviewees, may result in achieving overall company success and longer term growth. This is evidenced by the following quotes from the interviews: ‘‘SPI provides a set of guidelines to develop quality software and for that proper steps should be defined. This way the overall goals of SPI & company are met through proper implementation of SPI and so longer term targets can be achieved.’’ [Interviewee 7] ‘‘I think that SPI is very important for a company to succeed and it a way to perform projects and related tasks properly with proper procedures in place.’’ [Interviewee 11] ‘‘A mature process always results in a mature product or service. Eventually this leads to the success of the company in terms of monetary and environmental/cultural benefits.’’ [Interviewee 14] The data also suggests that for successful SPI implementation it is also important to highlight the benefits of SPI in the Web companies: ‘‘Efficiency ratios tell us whether SPI is successfully working or not. The results without SPI and with SPI should be compared as practically highlighting benefits of SPI is very important to prove in the company that it is beneficial.’’ [Interviewee 19]

Box. 1 Based on our qualitative investigation, we define SPI success in the context of small and medium Web companies as ‘guidelines and principles to develop quality software with proper processes in place, resulting in organizational excellence’.

5.1.2. SPI goals and benefits All the interviewees suggested that successful implementation of SPI results in achieving SPI benefits and SPI goals and vice versa. Successful SPI helps in achieving standardization in the company and results in the development of consistent products and services. For example one of the interviewees stated: ‘‘As the processes are properly defined, therefore the switching will be very seamless. SPI results in adaption of a consistent approach among different teams.’’[Interviewee 6] Similarly, better project planning and execution to produce better quality software was deemed important by the interviewees. They said that early identification of bugs, cost reduction in terms of dollars, and increased employees’ productivity are goals and benefits of SPI which cannot be ignored. ‘‘With SPI quality of software is improved. When the quality will be improved there would be fewer bugs in the project. Generally, maintenance costs are highest in software development life cycle (SDLC) but with good SPI costs can be substantially reduced. There would be less rework required and it will result in saving time as well as cost.’’ [Interviewee 2] Interviewees suggested that focused SPI results in mature and repeatable processes, which reduces the failure rate of projects and enhances the traceability within Web products and projects. The interviewees said that SPI results in overall organizational growth and client satisfaction. Also, SPI helps a small or medium Web company to establish better interaction with other companies. An interviewee stated: ‘‘SPI is basically a complete programme. It helps you in fixing your problems, meeting your business objectives, defining the direction

of a company in pursuit of quality, reducing cost, and gaining new customers.’’ [Interviewee 17] ‘‘The main effect of adapting SPI is that our software is more quality assured and they are well appraised in the market and reported number of bugs become less and software reliability increases’’ [Interviewee 10] ‘‘Process improvement is at both individual as well as team level in our company. Process improvement helps us in increasing the project success rate and the waste of resources has been minimized.’’ [Interviewee 2] ‘‘As we follow SPI, every phase in SDLC has become very specialized and our Web development is quite systematic.’’ [Interviewee 11] With SPI, resource utilization increases and more reliable software with a smaller failure rate is produced, as suggested by the interviewees. They mentioned that SPI facilitates Web development by using a systematic and methodological approach which helps improve every activity of SDLC with streamlined management and development tasks. With mature and reusable processes in practice as a result of successful SPI implementation, employee turnover is also comparatively smaller in Web companies. ‘‘As smooth processes are running in our company for years, people are very comfortable in executing the development of products and services.’’ [Interviewee 11] SPI has both long term and short term benefits for small and medium Web companies as suggested by most interviewees; however, interviewees also mentioned that SPI has more long term benefits as compared to short term benefits. ‘‘We have both long and short term benefits of successful SPI. Short term complex projects, even if they are of small duration, can be handled very well. Long term benefits of SPI give us a reason to implement better traceability among numerous Web development phases.’’ [Interviewee 16] According to the interviewees, longer term benefits of SPI also result in improved products and services quality for Web companies and help in achieving competitive advantage. The interviewees suggest that proper processes help in implementing the longer term vision of higher management in the company as operational improvement by identifying and rectifying the problem areas occur and confidence of stakeholders on the company increases. The other benefits mentioned by participants are improved communication, better estimates, proper logging and reporting, fewer mistakes made by employees, and overall reduction in the volume of work. ‘‘Longer term SPI benefits are more as compared to the shorter term benefits. Maintenance, system support and client satisfaction would be better and will result in overall longer term growth of the company. Revenues will increase and company reputation will be built better.’’ [Interviewee 10]

Box. 2 Based on our qualitative investigation, we define SPI goals and benefits in the context of SPI success as ‘the extent to which small and medium Web companies define overall, long term and short term goals and benefits to be obtained with the help of SPI programmes’.

5.1.3. Automated tools support According to numerous participants, automated tools accelerate the implementation of SPI and help reduce development time. One of the interviewees said:

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

‘‘There should be more automated tools which should support and accelerate SPI.’’ [Interviewee 8] However, it was suggested that automated tools should be assessed carefully before adoption and specialized training for these tools should be provided. A few interviewees advocated the development of customized tools for SPI according to the specific needs of their own Web companies. It was also mentioned that automated tools fill the gaps and leaks in the processes by providing proper guidelines as well as by eliminating laborious manual work and extensive documentation. Processes become more streamlined with simpler implementation and chances for the occurrence of errors are reduced in cases of complex processes. Numerous interviewees stated that automated tools reduce the burden of SPI tasks on the development staff. The following quote is a good example of interviewees’ opinions: ‘‘Automated tools can make implementation of SPI much easier, e.g., for project management processes we use MS-Project which makes life much easier for project managers. However, we do not have automated tools and customized automated tools for all processes. Well, SPI can be very well implemented if customized automated tools are used from the beginning of processes’ implementation.’’ [Interviewee 3] It was also mentioned that automated tools provide good support for pre and post project analysis by collecting useful data and can be very useful for SPI measurements as a respondent stated: ‘‘Automated tools support facilitates the measurement of the processes and other related activities. We use automated tools for testing, release management; change management processes and these tools have facilitated implementation of processes with support for relevant measurements’’ [Interviewee 14] It was stressed by interviewees that automated tools support is mandatory for the success of SPI in small and medium Web companies. Box. 3 Based on our qualitative investigation, we define automated tools in the context of SPI success for our domain as ‘the extent to which automated tools accelerate SPI and facilitate collection of SPI related measurements and data collection in a comprehensive way’.

5.1.4. Client support Interviewees stated that client support is important as clients provide valuable feedback which can be useful in improving software processes. It was also said by the interviewees that there are certain risks associated with client involvement as the project costs increase with SPI in some cases. For example one of the interviewees said: ‘‘Initially as companies improve themselves with SPI, they miss the client side perspective of SPI. Client feedback is also important for SPI as they are important stakeholders. Companies are afraid to take client feedback as they feel they get exposed in front of clients and they never want to lose their clients. I understand this risk factor is present but clients always provide very practical feedback which facilitates improvement of processes in our domain of work.’’ [Interviewee 7] ‘‘We can engineer our processes based on the client feedback. Client satisfaction is one measure of SPI success.’’ [Interviewee 1]

487

It was observed from the interviews that client feedback can be more useful in the scenario of Pakistani Web companies where onshore–offshore development model is in practice. Interviewees said that companies in Pakistan are motivated to align their processes with the processes of their overseas partners and can tailor SPI according to the client needs. This results in gaining more clients and retaining existing clients, as an interviewee commented: ‘‘The clients can see the progress by employing SPI. This increases their trust and they give us more and more work and thus the company grows. This is both in our and clients’ benefit.’’ [Interviewee 12]

Box. 4 Based on our qualitative investigation, client support in the context of SPI success is defined as ‘the extent to which client support facilitates SPI by bearing some SPI cost and providing constructive feedback that results in the process improvement by taking it to the next level’.

5.1.5. Communication The interviewees mentioned that communication among departments, employees, and teams is vital for SPI and a key success factor of SPI in small and medium Web companies: ‘‘SPI is all about coordination and communication. Without that you can never manage SPI programs.’’ [Interviewee 21] ‘‘SPI and well defined processes help in proper documentation and co-ordination among project teams.’’ [Interviewee 11] It was stated by the interviewees that communication should be clear, logged, and recorded as it helps in documenting and defining new processes as well as improving the existing ones. ‘‘Sometimes it is verbal and sometimes it is through emails. Proper and recorded communication among different managers and departments is very important. Flaws in communication can have very drastic effect on SPI.’’ [Interviewee 7] Interviewees stressed that a good communication structure facilitates the best way of performing activities within the processes. They also said that poor communication is harmful and may result in SPI failure, as an interviewee stated: ‘‘If communication gap exists between employees and managers, then SPI can backfire too.’’ [Interviewee 9]

Box. 5 Based on our qualitative investigation, we define communication in the context of SPI success as ‘the extent to which different departments, teams and employees effectively communicate to achieve SPI success and improve existing processes’.

5.1.6. Company vision According to the interviewees, linking the company vision with SPI programmes is mandatory and an essential driver for successful SPI in a small or medium Web company. SPI alignment with company’s goals and corporate strategy as a major success factor was

488

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

narrated by almost all the interviewees, e.g., two of the managers stated:

‘‘Success in SPI is only visible when it is translated into money.’’ [Interviewee 2]

‘‘SPI is driven by the corporate strategy and vision of the company and this is one of the actual success factors of SPI.’’ [Interviewee 7]

Therefore, cost–benefit analysis plays a major role in the success of SPI programmes in small and medium Web companies for which the dollar value of SPI is very important.

‘‘SPI deals with maturity and maturity can only be achieved if a company knows its vision and business requirements. SPI should be mapped with business requirements and that is how it helps in achieving the long term goals of a company.’’ [Interviewee 11] It was also mentioned by some interviewed managers that exactly replicating another company’s SPI initiatives can be dangerous and the linkage of SPI with the business goals of the company is a must for SPI success. It was also stated by the interviewees that a company needs to have positive thinking about SPI, it must develop an SPI culture in the company, and the SPI objectives must be in line with the company’s objectives. ‘‘Overall SPI is based on the vision and requirements of the company and on how they perceive the concept of SPI.’’ [Interviewee 2]

Box. 6 Based on our qualitative investigation, we define company vision in the context of SPI success as ‘the extent to which SPI’s objectives are aligned with the needs and corporate strategy of the small and medium Web companies’.

5.1.7. Cost–benefit analysis All interviewees said that the eventual goal of every small or medium Web company is to cut costs and increase overall benefits. It was mentioned by the interviewees that SPI ventures help a great deal in cutting development and management costs and thus SPI is seen as a major benefit. With regards to cost benefit analysis two interviewees stated: ‘‘Eventual goal of every company is to save money and that is why many companies have linked SPI to save costs. Cost is the primary reason for SPI implementation.’’ [Interviewee 18] ‘‘I would again stress that I would only follow any SPI standard provided that leads me to reduced costs, increased profits, as well as improved quality.’’ [Interviewee 19] For small projects sometimes SPI costs are more than the cost of overall project, therefore, some companies are reluctant to adopt SPI. Also, SPI initiatives can be beneficial after a certain time and cannot be very advantageous in reducing costs instantly as two interviewees said: ‘‘Definitely, SPI involves cost. Quality is not for free. In order to buy standards, to implement processes, and for professional trainings you need to bear the cost.’’ [Interviewee 10] ‘‘Smaller level organizations cannot afford to have SPI due to the expensive cost. The documentation requirements of SPI are also very high and it cannot be performed with small projects.’’ [Interviewee 7] As said by the interviewees, initially companies incur more costs in implementing standards, hiring specialized SPI staff and for extensive documentation requirements. However, it was also mentioned by them that in the long run, the costs are cut down by major percentage with SPI and higher levels of quality are achieved. The managers are very keen on the cost–benefit analysis which can be noticed from the quote below:

Box. 7 Based on our qualitative investigation, we define cost– benefit analysis in the context of SPI success as ‘the extent to which small and medium Web companies need to maintain the balance between investment in SPI initiatives and overall development costs’.

5.1.8. Employee support Employee support is seen as an essential and vital factor of SPI success by many interviewees as employees are the actual practitioners of SPI in a company. Interviewees said that involvement of every employee is necessary for SPI to succeed in all phases of SDLC and for this employees must believe in the idea of SPI. It was also mentioned that during the initial stages of SPI, employees bear the SPI cost by spending additional time on SPI tasks. ‘‘I think commitment towards SPI programme is very important. It is very important and highly demands contributions from employees both at individual and team levels as well as from the management.’’ [Interviewee 10] ‘‘Management invests the money and technical staff invests their time. Both of them play major roles in successful implementation of SPI.’’ [Interviewee 11] A manager said that employees always learn SPI best with experience when they apply it to the projects in accordance with the company’s quality procedures. A member of a development team narrated that employees can participate in SPI in a better way when employees are not over-burdened with SPI tasks, the SPI targets are explained to them clearly, reasonable time is allocated for following processes, they understand the importance of SPI, and measures are taken to increase their involvement in SPI tasks. The interviewees also suggested that imparting SPI knowledge from one employee to another and their collaboration and team work is necessary for SPI success. An SPI practitioner stated that: ‘‘The willingness of employees is a big factor which improves SPI for small and medium Web companies.’’ [Interviewee 20] When new processes are introduced in a Web company, employees always show a certain level of resistance in adopting these processes as mentioned by many interviewees. The stated reasons include huge documentation requirements, lack of vision for long term SPI benefits, additional SPI tasks and stubborn thinking to adopt change, as one manager said: ‘‘Web project teams and their employees resist change and do not like to adapt anything new, they oppose SPI.’’ [Interviewee 9] However, it was also mentioned that necessary steps must be taken to motivate employees as they are the actual implementers of SPI in the company. ‘‘Employees can be told that by adopting SPI they can improve themselves and identify their problems, being SPI practitioners SPI is in their own benefit, task allocation can become very systematic, and they can get exact credit of the work they do if proper processes are in place.’’ [Interviewee 4]

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

489

It was said by the interviewees that employees must be encouraged to practice SPI and should never be forced to sacrifice processes when deadlines are near for a project as practiced by some Web development companies. Similarly, the feedback of employees about processes was suggested to be vital in improving existing processes within a company as well as defining SPI targets.

Some interviewees suggested that their direct involvement should be compulsory. A few managers also said that the higher management of a company needs to have a vision to provide initial required monetary support to SPI initiatives in order to reap longer term benefits, and that management must be able to link SPI with the long term vision of the company:

‘‘Based on employee feedback new processes can be implemented and existing processes can be modified.’’ [Interviewee 6]

‘‘SPI programme starts as a result of higher management initiative and it is linked with their longer term vision for the organization.’’ [Interviewee 1] ‘‘Actually it is the top level management of the organization which gets processes implemented and provides financial support to the SPI initiatives.’’ [Interviewee 7]

Box. 8 Based on our qualitative investigation, we define employees support in the context of SPI success as ‘the extent to which employees of the small and medium companies own, believe, learn, practice, adopt, and trust SPI initiatives’.

5.1.9. Gradual approach An emphasis by the interviewees was put on the gradual implementation of SPI in small and medium Web companies by giving it proper implementation time and it was suggested to be essential for SPI success. ‘‘I always say that SPI in Web companies should be gradual and it should be given its proper implementation time.’’ [Interviewee 17] It was suggested by the interviewees that the new processes should be built on top of existing processes, rather than totally adopting new processes and jumping directly on SPI targets by missing processes’ details. A process for the implementing processes was stressed. It was suggested by a few managers that the focus should be put on piloting an approach on a sample project before adopting new processes company wide. The results of piloting the processes should be analyzed before making a decision. ‘‘You should gradually start SPI to engrain it in other projects undertaken by the company.’’ [Interviewee 21]

Box. 9 Based on our qualitative investigation, we define gradual approach in context of SPI success as ‘the extent to which small and medium Web companies adapt SPI by building upon existing processes and taking a step by step approach towards processes improvement’.

5.1.10. Higher management support Higher management of a company plays a very important role in SPI success of a small and medium Web company as stressed by all interviewees. Higher management according to the interviewees includes company owners, directors, and senior managers. The interviewees stated that higher management should be strong believers in SPI and must have direct involvement in SPI initiatives to ensure continuous processes improvement in the company. ‘‘Leadership of a company determines the future direction for the company. They need to identify how to participate and which steps can lead them to their vision.’’ [Interviewee 11] According to all interviewees, higher management provides financial support to the SPI programme and helps to create SPI culture within the company by providing proper guidelines for SPI.

Many SPI managers suggested that the belief of senior management in SPI ensures continuous processes improvement of a company. Similarly, development of consistent SPI thinking in the company and continuous monitoring of SPI progress are important tasks which should be performed by the senior management. Some interviewees said that senior level managers think of SPI standards as a marketing trick and they take it as a logo, this should be avoided. ‘‘Generally if we consider small and medium Web companies, they do not take SPI seriously. Many companies say that it is just a label and we are still completing our projects. Management of many companies also considers SPI as a label which is a wrong approach.’’ [Interviewee 8] It was also stated that managers should first support SPI, then give the SPI team autonomy. ‘‘Initially the top management should be convinced about SPI themselves. Then they should hire and trust the SPI team and should give them the authority.’’ [Interviewee 12] According to the interviewees, higher management itself needs to have a firm belief in SPI and the produced measurements in order to be able to convince the clients about the benefits of the processes adopted by their company, formulate proper decisions, and implement SPI with billability. ‘‘Management’s role is to change the billing methodology and whenever they bill the client they should tell the client that the previous work was on an ad-hoc basis and the new work is process oriented and would result in a mature and better quality product and is therefore, the reason for an increase in the bill.’’ [Interviewee 20]

Box. 10 Based on our qualitative investigation, we define higher management support as ‘the extent to which higher management of a small and medium Web companies gets involved with SPI by financing, supporting, understanding, and trusting SPI’.

5.1.11. SPI consultancy SPI consultants contribute towards SPI success by guiding companies in adopting new processes with the help of their experience with past companies, as an interviewee stated: ‘‘SPI consultants have a very major role to play in getting SPI successfully implemented in a Web company as they have years of experience getting SPI implemented in various companies.’’ [Interviewee 2] Interviewees stressed that SPI consultants always adopt an incremental approach and also help companies in adopting SPI

490

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

certifications of commercial SPI models. SPI consultants act as guides and tell companies about their nonconformance with the requirements of SPI standards, which results in improvement of processes within the company, as a manager said: ‘‘Technical evaluations by consultants for certification and recertification are also identifying factors for the success of SPI program.’’ [Interviewee 20] SPI consultants also facilitate in the creation of new processes and new process templates. However, it was also suggested in the interviews that SPI consultants must understand the maturity, vision, and direction of the company before putting up their valuable suggestions. ‘‘Input of SPI consultants is also very important for successful SPI but it can be dodgy at times as consultants do not understand the exact focus of the company. Consultants many times recommend very lengthy processes which cannot be translated into very specific company needs’’. [Interviewee 14]

Box. 11 Based on our qualitative investigation, we define SPI consultancy in context of SPI success as ‘the extent to which SPI consultants, based on their past experience, help small and medium Web companies in adopting formal SPI standards while staying aligned with the Web company’s vision’.

5.1.12. SPI implementer’s role The interviewees mentioned that SPI implementers have a challenging role as they are responsible for the development of new reusable processes and improvement of the existing ones according to the company needs. Therefore, they perform gap analysis for the new processes according to the needs of the Web company, as one of the interviewees said: ‘‘The SPI working groups are responsible for defining the processes, training staff on these processes and conducting gap analysis for those processes.’’ [Interviewee 8] Generally, it is the responsibility of the SPI implementers to ensure the smooth flow of processes in the Web company, as one manager stated: ‘‘I also believe that supervisory level or people who take care of SPI implementation are very important. They are the people who design reusable processes, get the SPI tasks implemented and act as a bridge between the senior management and the employees.’’ [Interviewee 13]

Three of the interviewed practitioners said that SPI implementers provide all projects’ teams with SPI resources including a document template for each process and they further create checklists for each process’ acceptance criteria including dates, logs, previous bugs, and versions. An SPI manager said: ‘‘For our company’s SPI we have made an acceptance criteria for each process which is implemented with the help of a comprehensive checklist.’’ [Interviewee 10] Two interviewees stated that SPI implementers play a critical role as they perform the important task of mentoring employees towards SPI. They actually trigger employee support towards SPI, as an interviewee said: ‘‘SPI group, which has good knowledge of processes, is always required for successful SPI implementation. This is the group that other employees should look up to for further knowledge and clarification of processes.’’ [Interviewee 16] They also perform the required R&D, learn from the peers in other companies, and take input from academia and researchers about the new processes as one of the SPI managers said: ‘‘We take input from academia and researchers many times and we definitely study new standards and new research papers on the topic.’’ [Interviewee 10]

Box. 12 Based on our qualitative investigation, we define SPI implementer’s role in context of SPI as ‘the extent to which SPI implementer plans for implementing new processes, improving existing ones depending upon the needs of the Web company and communicates SPI strategies to both higher management and company employees’.

5.1.13. SPI measurements Measurements of SPI programme are very important as suggested by most of the interviewees. They mentioned that the results of SPI cannot be seen without the measurements as they help in identifying potential areas for improvement. They help in determining trends of SPI in Web companies. ‘‘All companies believe in measurements as results of SPI cannot be seen unless proper measurements are taken.’’ [Interviewee 2] According to the interviewees SPI measurements can be both quantitative and qualitative. They help in periodic benchmarking of company’s performance to handle bigger, complex and diverse projects. A quality manager stated:

According to some interviewees, SPI practitioners convince senior management about the importance of SPI and communicate key benefits in the company; i.e., both senior management and employees. It was also stated in the interviews that SPI implementers are responsible for the creation of an SPI culture in the company and provide the required support to the development staff. They define SPI plans, establish process groups, and develop professional and reusable resources considering all risks, constraints, and their mitigations.

Interviewees said that SPI measurements help in reviewing processes across teams and at different SDLC phases, thus providing true picture of SPI in the company. Similarly, project level measurements are used for postmortem analysis, trend analysis, and comparison of results among projects that have similar nature.

‘‘SPI in our company has been implemented through a special SPI team.’’ [Interviewee 12] ‘‘If SPI project is started in a company and there is no SPI team, it can never be successful.’’ [Interviewee 10]

‘‘Data from previous projects, when there was limited or no SPI, is also stored and compared with data from the projects with proper SPI to see the difference in performance and demonstrate improvement.’’ [Interviewee 17]

‘‘The definition of SPI measurements and their monitoring, both are very important as it is a well-known saying that you cannot control what you cannot measure.’’ [Interviewee 7]

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

‘‘Most of our SPI measurements are project based measures. We analyze the execution pattern of different projects. Then we relate these patterns with the targets of organization and also relate these with long term goals of the company. Overall trend analysis is performed and necessary steps are taken accordingly.’’ [Interviewee 18] Interviewees said that SPI measurements are used for measuring bidirectional traceability, bugs and defects, and cost and schedule slippage; directly and indirectly measuring the revenue; and monitoring employee productivity. Comparative studies of different projects can always support measurements in the best possible way. Different measurements for comparisons include line of code measures, project manager’s time, developer’s time, cost etc. [Interviewee 4] ‘‘Actually all goals and vision of company are linked with the SPI programme and we check the alignment with the help of SPI measurements. Mainly we check by taking measurements of quality, time and other cost related factors the differences between actual and expected values.’’ [Interviewee 14]

491

‘‘According to the needs of the company, we modify the existing standards and get numerous activities enforced into the processes based on our expert opinion because we need processes which can be adopted in every project.’’ [Interviewee 5] It was also specifically mentioned in the interviews that it is difficult to practice lengthy processes on smaller scale Web projects and that this may result in failure of SPI in the company. ‘‘For example a small Web project is just one month long, if all SPI templates will be applied on that project the project will absolutely fail.’’ [Interviewee 12]

Box. 15 Based on our qualitative investigation, we define tailoring of processes in context of SPI success as ‘the extent to which small and medium Web companies modify the processes according to the needs of their projects/products and implement a flexible approach towards SPI implementation’.

Box. 13 Based on our qualitative investigation, we define SPI measurements in the context of SPI success as ‘the extent to which small and medium Web companies collect, store and analyze process related data to perform trend analysis of processes in all SDLC phases and gain process maturity by periodic benchmarking’.

5.1.14. SPI supportive policies Supportive policies by government and corporate bodies can further promote SPI in small and medium Web companies as suggested by some interviewees. In recent years, the Government of Pakistan has provided support for SPI initiatives as mentioned by some interviewees. A few of the interviewees were also of the view that companies should collaborate with each other by creating SPI forums to share SPI data, experiences, and best practices. ‘‘Companies should share their processes related data so that other companies can also learn from their experiences.’’ [Interviewee 17] ‘‘In some cases Pakistani government has also funded some SPI projects but they are for limited time.’’ [Interviewee 15]

Box. 14 Based on our qualitative investigation, we define SPI supportive polices in context of SPI success as ‘the extent to which government, corporate IT bodies and small and medium Web companies support better implementation of SPI initiatives’.

5.1.15. Tailoring of processes The need for tailoring of processes according to company and project size and need was stressed by many interviewees. It was mentioned that tailoring of processes helps in making processes flexible for better adoption and thus results in SPI success of small and medium Web companies. A SPI practitioner said:

5.1.16. Applying existing knowledge about SPI Web development companies and their staff always rely on their existing SPI knowledge, and this was mentioned as an important success factor for SPI. Different sources identified for previous knowledge include academic degrees and knowledge from the employees’ previous companies as well as from the past projects in the current company. ‘‘We learn the new processes with experience that we get from different projects and then apply these in new projects.’’ [Interviewee 11]

Box. 16 Based on our qualitative investigation, we define applying existing knowledge about SPI in the context of SPI success as ‘the extent to which already obtained knowledge from academia, other companies, past and current projects is applied to attain maturity of processes in small and medium Web companies’.

5.1.17. SPI awareness SPI awareness programmes in a company result in the success of SPI as stated by an interviewee. It was considered important to create awareness about SPI within the Web development sector: ‘‘In Pakistani Web industry including our company we need to create awareness about SPI and explain its benefits.’’ [Interviewee 5] A lack of awareness resulting in a lack of knowledge about SPI was also reported by the interviewees. Many small and medium Web companies cannot implement SPI due to a lack of the required knowledge base, as was stated by some interviewees. ‘‘For small and medium Web companies neither top management nor employees have significant knowledge about software process improvement which has resulted in either no SPI or an SPI failure.’’ [Interviewee 3] There are many ways and means for increasing SPI awareness within the company, and SPI trainings were mentioned as the most important mechanism.

492

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

‘‘Training is also an important factor for SPI success to create SPI awareness. These include external as well as internal trainings.’’ [Interviewee 8] There are two main types of SPI trainings – internal and external. In case of internal training, senior managers and SPI team train employees about new or existing processes prevailing in the company, whereas external trainings are obtained from specialized SPI trainers/companies as said by the involved interviewees. ‘‘By internal trainings we impart knowledge into our junior staff from our senior staff.’’ [Interviewee 2] ‘‘We also enhance skills of our employees with external trainings offered by professional training institutes’’. [Interviewee 4] It was mentioned by the interviewees that SPI trainings can be more useful if they are aligned with the SPI needs of the company. ‘‘Staff training is important as this helps them understand the meaning of SPI standards which are to be implemented within the company and how these standards will enable them to help achieve company’s goals.’’ [Interviewee 17]

Box. 17 Based on our qualitative investigation, we define SPI awareness programmes in the context of SPI success as ‘the extent where initiatives like SPI trainings and certifications provide support to the human resources of the small and medium Web companies resulting in proper SPI implementation.’

5.1.18. Company success SPI success results in overall company success as mentioned by a large number of interviewees. For example, one of the interviewees said: ‘‘In order to grow and succeed companies try to meet internal and external objectives. The internal objectives are to improve the productivity of the company and make processes stable as well as repeatable. Second thing is customer satisfaction. Customer satisfaction boils down to cost and quality. So, what is the cost at which you are achieving quality?’’ [Interviewee 1] With SPI, small and medium companies gain increased stability. It is not just an increase in the return on investment, but also an increase in the level of customer satisfaction which, in turn, results in gaining new customers while retaining existing ones as suggested by the interviewees. ‘‘My personal experience says that SPI not only streamlines the development activities but also increases the productivity and stability of the company.’’ [Interviewee 3] The other stated benefits include, overall development efforts reduction, overall performance increase, employees’ satisfaction, company growth, and improved organizational culture. With SPI success, overall company performance increases and companies can handle bigger and more complex projects as they gain competitive advantage. ‘‘If we are meeting our targets and if we are growing upwards in terms of project completion and fewer bugs then yes, SPI is successfully implemented. Similarly, gaining new clients and retaining previous clients is also an indicator of company success.’’ [Interviewee 8]

‘‘Having proper thinking by the management that company will grow as a result of investment in SPI which will change the league of the company to gain bigger businesses is vital.’’ [Interviewee 6] ‘‘The worth of company in the market increases with SPI and company achieves stability.’’ [Interviewee 14]

Box. 18 Based on our qualitative investigation, we define company success in the context of SPI success as ‘the extent to which small and medium companies gain operational excellence, stability, and overall improved quality; earn better profits; and have satisfied employees and customers’.

5.2. Specific SPI requirements for Web development The motivation of this research was the unique characteristics and nature of Web applications and their development processes, which make the requirements for SPI unique for Web companies. Due to this reason, during the interviews the specific SPI requirements for Web development were also discussed. Interviewees elaborated the differences of Web development and traditional software development based on their industrial experience. ‘‘We will have to investigate all the attributes of Web projects which differentiate them from other projects and we should define new processes according to these differentiating attributes.’’ [Interviewee 12] ‘‘Implementation of SPI in small and medium Web companies is not the same as that of large scale companies because of their different nature.’’ [Interviewee 8] ‘‘We should have different SPI for Web companies. We cannot follow traditional processes straight away. The processes should be tailored according to the requirements of projects and company. Web applications are totally different from desktop applications. The basic activities are same but implementation is very different as I have mentioned tailoring. The processes should be flexible so as to be able to accommodate change.’’ [Interviewee 19] ‘‘As in design of complex Web applications we make sequence and class diagrams but for simple Web sites these are not applicable. As we do not have top down architecture and these Web sites are always evolving, traditional SPI is not applicable for simple Web development.’’ [Interviewee 9] The interviewees acknowledged that SPI is a new concept for Web companies and that they need more mature processes. ‘‘The concept of SPI in small and medium Web companies is almost not present.’’ [Interviewee 1] ‘‘SPI can reap benefits when applied on bigger companies. For small and medium Web companies, following exact standards is difficult due to smaller nature of projects and also because specialized processes are needed.’’ [Interviewee 16] The interviewees mentioned that Web projects are always evolving with changing demands and more customized needs as compared to traditional projects. They also stressed that Web projects are more agile and have a global market base. Interviewees also mentioned that Web development is a very rapidly growing area with diverse client types. ‘‘Multiple processes should run in parallel for Web projects as more focus is on agile techniques in case of Web projects.’’ [Interviewee 11]

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

‘‘Web development specifically is agile based and has continuously changing demands both due to the global nature of Web projects and continuously evolving and improving development and hosting technologies. It is very important to have useful processes and their continuous improvement in order to keep things on track.’’ [Interviewee 2] ‘‘Web development itself is a very vast and powerful area. There are a number of possibilities in it because it is based on a global market and therefore, you need global access to all the resources. Specifically, in case of sub-contracting we need to deal with diverse companies and different sets of clients. Diversified nature of applications makes it more challenging.’’ [Interviewee 3] Many interviewees also suggested that there should be some specific processes to cater for the needs of small and medium Web companies. For example one of the interviewees said: ‘‘Right now we do not have any specific SPI models for Web but Web development is different from traditional development. Web development processes are somewhat different from traditional development processes. The life cycle of projects is different. The development models are different hence, processes are tailored and they are different too.’’ [Interviewee 9] ‘‘The measurements for Web are different. Measurements of Web projects deal more with the aesthetics and look and feel of the applications. In addition, performance related measures are also very important.’’ [Interviewee 11] ‘‘We mainly follow agile development model for Web, so processes should also be formulated based on agility rather than having them based on waterfall model.’’ [Interviewee 7] It was suggested that there should be smaller iterations of processes and not all SPI standards should be enforced on them. SPI for Web should strengthen intra-team and inter-team processes. Similarly, based on the differences of Web development and traditional software development, interviewees suggested development of specialized processes for Web security, Web testing, and stronger yet interlinked processes for communication. ‘‘Network and performance related quality standards are very important for Web, so processes to tackle these requirements need to be introduced. Testing strategies are also different. Stress, volume, and load testing processes are different. As the number of users is unknown for Web therefore, there are performance and security issues which should be tackled with specific processes which currently do not exist in general SPI models.’’ [Interviewee 12] ‘‘We may need some new processes for Web companies like a very strong process for communication; and inter team interaction should be strengthened as well as cross-team processes must be improved.’’ [Interviewee 10] The interviewees also criticized existing SPI models and mentioned that they are not an exact fit for the Web domain as they are not aligned with the specialized requirements of Web development. ‘‘I think yes, SPI for Web is to be different. We will have to investigate all the attributes of Web projects which differentiate them from the other types of software projects and we should define new processes based on these differentiating attributes rather than blindly applying existing SPI standards.’’ [Interviewee 6] The interviewees stressed the need to separate SPI requirements for Web projects and mentioned the need for more interlinked processes of smaller duration. They suggested the need of more user centered processes with support to handle agility and multiple processes running in parallel. Interviewees also said that Web applications are more multimedia focused, user centered, and use a layered approach to development.

493

‘‘Many of the processes in Web development are interlinked and mixed so that they are achieved at the similar time in small phases rather than having specialized resources allocated to them which makes traditional SPI not very feasible in the case of Web.’’ [Interviewee 18] ‘‘Web applications are more user-centered; therefore, standards should be based on more user interaction and specialized towards user interface and multimedia nature.’’ [Interviewee 13] Interviewees also mentioned that for small projects current SPI standards cannot be practiced and result in an overhead. However, they are somewhat applicable to large Web applications. It was stated that processes vary from project to project in the domain of Web and very flexible SPI strategies are required. The processes for Web development must be light and flexible with the facility to add and leave out the processes as per project needs. Tailoring of SPI according to the needs of the Web company and project at hand was highlighted by numerous interviewees: ‘‘We should have different SPI for Web. We cannot follow processes straight away. They should be tailored according to the requirements of the projects and the company. Web applications are totally different from desktop applications. The basic activities are same but implementation is different as I have mentioned tailoring of processes. The processes should be flexible so that they must be able to accommodate change.’’ [Interviewee 13] ‘‘Yes, tailoring of traditional SPI for Web projects is required. It is always required to check which processes can be followed and which cannot be followed and it varies from project to project because of varying and diversified nature of Web projects.’’ [Interviewee 16] In short, the interviewees stressed that flexible, specialized, light, and easily implementable processes that are specialized for the need of Web domain are required for small and medium Web companies. ‘‘There is a need of SPI in small and medium Web companies but unlike traditional SPI standards; processes should be light and easily implementable.’’ [Interviewee 3]

5.3. Differences between small and medium Web companies One aim of this study was to specifically define, differentiate and identify key traits of small and medium Web companies. This section separately defines ‘small’ and ‘medium’ Web companies and elaborates the differences among these based on the interviewees’ perspectives. 5.3.1. Small Web company When asked about the description of a small Web company in Pakistan, most of the interviewees were of the view that small Web companies have 15–20 employees which work on smaller projects that on average span 1–2 months and deviate little from the requirements. However, one interviewee also suggested that small Web companies may have up to 50 employees. ‘‘With respect to Pakistan, I would say between 15 and 20 would be a small Web company.’’ [Interviewee 3] ‘‘The companies that do not have more than 20 employees are considered to be small. Normally these also do not have very big projects.’’ [Interviewee 19] It was also mentioned by the interviewees that generally there is only one development team in small companies which work on multiple projects that have lesser documentation requirements.

494

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

With regards to SPI, the interviewees mentioned that generally it is not present in these companies due to expensive costs and extensive documentation requirements. 5.3.2. Medium Web company Upon asking about the description of medium Web company in Pakistan, most of the interviewees were of the view that medium Web companies have 20–100 employees which work on projects spanning 4–6 months on average and deal with bigger client companies. Medium companies can have more than one development teams working on different projects. However, two interviewees suggested that medium Web companies may have up to 150 employees and the projects may last up to 1 year. ‘‘Generally around 50 people form a medium level Web company.’’ [Interviewee 17] ‘‘Medium Web companies can have more than 20 employees and project size also increases.’’ [Interviewee 8] It was mentioned by most of the interviewees that medium companies work on bigger projects and have multiple stakeholders involved in the projects and hence work on a bigger scope. These are more mature than small Web companies, have an increased need for processes, have a high number of employees, and require increased maturity. 5.3.3. Comparison between small and medium Web companies While discussing with the interviewees the differences between small and medium Web companies, all interviewees mentioned that it is hard to differentiate between the two as no strict rules exist that differentiate them. However, they mentioned that there is a ‘complex factor’ which can tell whether the Web company is small or medium. The factors include number of employees, number of projects, scale of projects, locations of the company, generated revenue, maturity level and number of clients. ‘‘It is a complex factor as I have said the number of employees, number of projects and project duration would determine the size of a company. I cannot give you exact figures as the factor is rather complex and difficult to measure.’’ [Interviewee 20] ‘‘I think multiple factors can determine this. Number of employees, number and size of projects determine the size of company.’’ [Interviewee 4] ‘‘Small companies may also have large projects. So the differentiation is very hard and we have no set rules for that.’’ [Interviewee 10] The interviewees also mentioned that the scale of projects is determined by the project budget, project complexity and diversity, project duration and number of system users. Medium companies were said to be more process focused. It was also narrated that different process templates are used in small and medium Web companies as the processes of small Web companies are less detailed as compared to medium Web companies. 6. An initial exploratory framework of SPI success factors for small and medium Web companies The product of our study, the initial exploratory framework of SPI success factors for small and medium Web companies is depicted by Fig. 2. The presented framework gives the core categories of SPI success factors, detailed in Section 5.1, for small and medium Web companies and the relationships among them. The categories presented in the exploratory framework were obtained by performing ‘open’ and ‘focused’ coding techniques subsequently as mentioned in Sections 4.1 and 4.2 respectively. Key Relationships were identified among the obtained categories during ‘theoretical’ coding phase as discussed in Section 4.3. Inte-

grative ‘memos’ were also useful throughout the coding process which resulted in development of the presented exploratory framework. The obtained relationships knit the data together and the relationships conceptualized the underlying data patterns for a set of empirical indicators [106] which are selective categories in our case. A brief description of the categories and relationships are presented in Table 6. Our findings, integrated into the framework given above, extends the findings reported in the existing literature (our systematic review, replication study and broader literature on the topic) by providing specific categories of SPI success factors for small and medium Web companies but also the relationships amongst these categories. Table 7 shows the set of SPI success factors for small and medium Web companies which are identified by this research and their linkage to the literature. There are differences between given category set of SPI success factors for small and medium Web development companies reported herein and the SPI factors reported in the existing literature i.e., the provided categories are distilled categories from the broader literature according to the domain under investigation. Interviewees suggested that shorter development life cycles in web development, need for security, and that not all SPI standards are appropriate for web development need to be taken into account. Also, the measures need to be different for the domain of small and medium Web companies.

7. Discussion This study has investigated SPI success factors for small and medium Web companies by employing qualitative research methodology. There are four main contributions of this study: (1) it gives an exploratory framework of SPI success factors for small and medium Web companies, (2) it gives the viewpoint of SPI for small and medium Web companies from the practitioners’ perspective, (3) it also makes a methodological contribution by demonstrating how grounded theory techniques can be leveraged in the discipline of software and Web engineering to give greater insights into the current state of practice, and (4) helps in obtaining a practitioners’ perspective to identify characteristics of small and medium Web companies. It should also be noted that SPI success factors that apply to small, medium and large software companies have been reported in the existing literature (Table 7). We identified specific SPI success factors for the domain under investigation i.e., small and medium Web companies. It is interesting to speculate why this was the case. First, it was clear from the interviews that the practitioners understood the specific requirements of small and medium Web companies, hence the special SPI requirements for these companies. Second, the companies involved saw a distinct benefit in applying SPI, and it should be noted that, in Pakistan, there is government support for practices such as SPI which makes SPI adoption beneficial at a lesser expense for these companies. What is helpful about the factors identified in this study is that they give real insight into how SPI might be managed in a company. We would contend that these factors would probably hold true for other domains, represent a support of these factors on this specific domain under investigation and contribute towards the development of theory. We have also brought these factors together to give a much richer and multi-faceted definition of SPI success in small and medium Web development companies. The framework proposed in this study extends the findings reported in the existing literature by identifying specific SPI success factors for small and medium Web companies. The SPI success factors identified in this study were integrated into eighteen categories. The core category is that of SPI success.

495

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

SPI Supportive Policies

Benefits and Goals of

Client Support Results in

SPI

Tailoring of Processes Company Success Facilitates

Harness

Applying existing knowledge about SPI

SPI Implementer’s Role Results in achieving

SPI Awareness

Results in

Results in Triggers Is essential for SPI Measurements Quantify

Automated Tools

Employee Support

SPI Success

Facilitates

Contributes

Accelerates

towards

Support

Introduces Is necessary for

Is essential for Gradual Approach

Triggers

Drives Facilitates

Higher Management Support

Communication

Company Vision

Provides

Contributes towards Drives

Is based on

SPI Consultancy Cost Benefit Analysis

Fig. 2. Initial exploratory framework of SPI success factors for small and medium Web companies.

Another important contribution of this study is that it presents tailored sub-categories or SPI success factors under each main category for the investigated companies. It was mentioned by almost all participants that ‘SPI success’ results in overall ‘company success’ and that there are many ‘short term as well long term benefits of achieving SPI success’. It was found in this study that cost–benefit analysis plays a major role in the success of SPI program in small and medium Web companies in Pakistan and for these companies the monetary value of SPI is of high importance. ‘Employee participation ’, ‘SPI implementer’s role and ‘higher management support’ were specifically mentioned as the most important SPI success factors by the interviewees. The developed theoretical model also elaborated ‘SPI awareness program’, ‘automated tools support’, ‘client support’, and ‘communication’ as important SPI success factors. ‘Linking of SPI strategy with company’s vision’ is also a key success factor and the use of ‘SPI measurements’ for improvement were also considered critical. Similarly, ‘SPI consultancy’, ‘gradual approach’, ‘tailoring of processes’, and ‘applying existing SPI knowledge’ were found to be key SPI success factors for small and medium Web companies in Pakistan. It is also worth mentioning that SPI has associated drawbacks in the investigated domain. This was also reflected by the interviewees. The drawbacks mentioned by the interviewees include ‘expensive cost of SPI’, ‘huge implementation time requirement’, ‘an increase in documentation requirements’, ‘increased project

cost’, and ‘shift in company culture that results in employees as well as clients’ resistance to SPI’. However, a consensus was found among all the interviewees who corroborated that the benefits of SPI are more than its drawbacks and therefore is a must for small and medium Web companies. We contend that the thorough and rigorous nature of grounded theory based techniques strengthened this research area by providing valuable insights into the data and practice of Web development companies. The techniques helped us to identify key SPI success factors for small and medium Web companies and gain useful insights around those factors. The applied process of coding was extensive and iterative. Different types of coding techniques like open, focused, and theoretical coding techniques helped in developing richer insights into the data. We would further suggest that our study, with its rich insights into practice, illustrates how useful grounded theory techniques can be in unexplored areas. The theory building capabilities of grounded theory allow us not only to build a model, but also elaborate on relationships between factors. We hope that other researchers can test and further elaborate on the model put forward here. 7.1. Threats to validity There are two major categories of threats to the validity of qualitative research, namely researcher bias which is caused by

496

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

Table 6 Categories and relationships. Category

Relationship

Category

Description

SPI success

results in

SPI goals and benefits

results in

Company success SPI success

Automated tools support Automated tools support Client Support

accelerate

SPI success

facilitate

SPI measurements SPI success

Client support

results in

Communication

facilitates

Company success SPI success

This relationship explains the outcome of successful SPI initiatives which is growth and development of the company and overall company success Success of SPI results in achieving long and short term SPI goals and vice versa i.e., standardization, consistency, etc. Automated tools accelerate the implementation of SPI and save up on implementation and development time Automated tools facilitate SPI by collection and analysis of processes related data which can be helpful for SPI measurements Client support is important as clients provide valuable feedback which can be useful in improving software processes further and can help tailor processes according to the client needs Client support helps improve processes and achieve overall company success

Company vision Cost–benefit analysis Employees support

drives drives contributes towards

SPI success SPI success SPI success

Gradual approach

is essential for

SPI success

Higher management support Higher management support Higher management support Higher management support Higher management support

is necessary for

SPI success

triggers

SPI consultancy

contribute towards

Employee support Cost–benefit analysis Company vision SPI implementer’s role SPI success

SPI implementer’s role

is necessary for

SPI success

SPI implementer’s role

triggers

SPI measurements

quantify

Employees support SPI success

SPI supportive polices

harness

SPI success

Tailoring of processes

results in

SPI success

Applying existing knowledge about SPI SPI awareness programmes

results in

SPI success

result in

SPI success

facilitate

is based on provides introduces

Clear communication among departments, employees, and teams is vital for SPI success in small and medium Web companies Alignment of company vision with SPI programmes is an essential driver for SPI to be successful Proper cost–benefit analysis between SPI initiatives and product/project costs drives SPI success Employee support is seen as an essential and vital contributor of SPI success as employees are the actual practitioners of SPI in small and medium Web companies Gradual implementation of SPI by slowly improving existing processes and introducing new processes is essential for SPI success Higher management should be strong believer of SPI and must have direct involvement in SPI initiatives to ensure continuous SPI Higher management support towards SPI triggers employees support by appreciating SPI and introducing it among employees Higher management support towards SPI is based on cost–benefit analysis and support increases if they understand the long term benefits of SPI Higher management is responsible for setting company vision and corporate strategy Higher management support towards SPI results in introducing SPI implementer’s role in the company responsible for the implementation of SPI in the company SPI consultants contribute towards SPI success by guiding companies in adapting new processes and improving existing processes with the help of their experience with working in other companies in the past SPI implementers are important for SPI success as they are responsible for ensuring the smooth flow of processes, development of new reusable processes, and improvement of the existing ones SPI implementers trigger employees support as they are SPI communicators and provide the required support to employees Results of SPI cannot be seen without SPI measurements as they help in determining SPI trends and areas that need improvement SPI supportive policies by government, companies and corporate bodies can promote SPI in small and medium Web companies Flexible processes that can be tailored according to the needs of small and medium Web companies result in successful SPI implementation Attained knowledge from academia, past companies and previous projects results in SPI success

SPI awareness programmes including internal and external trainings result in improved processes and overall SPI success

the researcher’s preconceptions about the research topic – in this case SPI in small and medium Web companies – and reactivity i.e., the researcher’s influence on the setting and respondents [114]. Following threats to the validity of this research were identified which were aimed to minimize in line with the strategies proposed by Maxwell [114]:  The research participants are all from a single country. As cultural values and corporate cultures are very likely to vary around the globe, we believe that the factors that make SPI successful may vary from one country to another, hence limiting the external validity and making the proposal of a general set of SPI success factors more challenging. Future studies on the same topic using data from other parts of the globe can strengthen findings of this study and can also support the proposal of a generalized set of SPI success factors for small and medium Web companies.  The data gathered in this research was through open-ended interviews which in our opinion may sometimes result in imposing the researcher’s ideas on the interviewees. However, this threat

was addressed by the researchers by taking special care during the interviews so as to refrain from imposing any ideas and giving any hints on their pre-conceived notions if any, and by using a neutral tone thereby, maintaining the validity of data for this research. There is still a possibility of unintentional subtle clues from researcher’s tone or body language which could have, to a very small degree, impacted the research findings.  Our preconceptions about SPI in small and medium Web companies at the outset of this research may also be identified as potential researcher bias during the coding process. However, we took care not to impose concepts on the data, but rather to let the concepts emerge from the data and not to ’force’ the data [99], and then integrate our emergent concepts with the literature. We were of course aware of the relevant SPI literature and the SPI success factors in small and medium Web companies as a result of the conducted SR and replication study, but preferred to approach our coding processes with what [115] calls an ‘an open mind, as opposed to an empty head’. Therefore, we believe that the research is devoid of this bias as much as qualitative research can possibly be.

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

497

Table 7 Theoretical integration of SPI success factors for small and medium Web companies. SPI success factor

Relates to factors from systematic review

Relates to factors from replication study

Relates to factors from broader literature

SPI goals and benefits

–

Setting SPI goals and associated benefits i.e., project and process benefits [21,66,73,77,112]

Automated tools support Client support

Increase in productivity, reduced time for development, client and development team satisfaction, operational excellence [30,33,77,78,80–82] Automated tools [30,76,78,79,81,83,84] Client satisfaction [80–82]

– –

Communication

–

–

Company vision

-

Business orientation [13,32]

Cost–benefit analysis Employees support

Better return on investment [30,77,78,81,82] Employee participation in SPI [30,77]

Tools and technology for SPI [67,113] Improve the relationship and cooperation with clients [72] Establishment of infrastructure for communication support [12,70,72,113] Prioritize the SPI efforts by linking with business goals [72], organizational culture [12,68,73] Cost-saving [73] Staff involvement [12,67,73]

Gradual approach

Periodic assessment and prioritization of processes, breakdown of SPI tasks and adopt iterative approach [30,33,77,79,80,82] Company support to SPI [30,77,78]

Higher management support SPI consultancy SPI implementer’s role SPI measurements

Employee participation [13,32] –

SPI supportive polices

–

Leadership involvement [13,32] – – Concern for measurement [13,32] –

Tailoring of processes

–

–

Applying existing knowledge about SPI SPI awareness

–

Exploitation of existing knowledge [13,32] Exploration of new knowledge [13,32]

– – Statistical analysis/use of metrics [30,33,78,79,81]

Employee trainings [78–80]

 The complete coding phase of this research was mainly carried out by the first author. Therefore, one could say that there were chances of bias in interpretation and thus coding of the interviews. However, a subset of interviews was coded by the second and fourth authors and a volunteer, and the resultant codes were compared. This comparison revealed no significant differences. Additionally, we also had respondents systematically validate both the collected data and the results of the research which allowed us to avoid misinterpretations and also identify our own biases. We thus believe that the bias in coding is as minimum as possible, even if not completely absent.  The study only considered the recurring relationships among categories which were found relevant in the context of this research. The possibility of different researchers identifying different relationships based on their perceptions and/or interpretations cannot be ignored. We revisited the data multiple times and so, to the best of our knowledge, no important patterns were missed out but possibilities of slight interpretation variations by different researchers cannot be ignored.  The issue of respondent bias was minimized by constant comparisons which allowed us to include different cases in the study and compare them for their similarities and differences. This also allowed broadening of the analytical base for the identification of concepts and relationships. However, there can be a possibility that an intentional negative response of an interviewee due to certain environmental conditions (mood, reference, overstatement bias, etc.) could have affected the findings of this research.  The likelihood of possible systematic biases was also minimized, if not completely removed, by triangulation across different methods of data collection as well as different researchers. This also led to a strong substantiation of the emergent concepts.

Track SPI by frequent assessment of processes [72] Managerial commitment and support [12,14,66,72,73,77] SPI consultancy [21,72] Skilled SPI resources [12,21,73] Evaluation of SPI programmes with metrics [12,70,72,73] Join together with other companies so as to both finance and share specialized resources [72,73] Systematic process improvement approach – iterative [72], build product map and process customization [12,66,68,70,73] Use of existing SPI models and knowledge [72] SPI awareness [72], training and mentoring [12,73]

 The issues related with incomplete and/or inaccurate hearing of the interviews while translating and transcribing can also not be ruled out. However, these were minimized by spending a generous amount of time on translating and transcribing, making extensive use of pause and play features of the digital media players, and taking volunteer services for retranslating a subset of transcribed interviews into the language in which the interviews were originally conducted. Still there can be a slight possibility that the viewpoints of interviewees may not be exactly captured because of language and expression limitations. 8. Conclusion This research investigates key success factors of SPI in the domain of small and medium Web companies using a qualitative research approach leveraging the principles of grounded theory. For the purpose of this research, 21 interviews were conducted with professionals from 11 Pakistani small and medium Web companies. The interviews were open-ended and conducted in the interviewees’ native language and were then translated and transcribed. Open, focused, and theoretical coding were performed on the transcribed interviews which resulted in the identification of important categories, sub-categories, and relationships. The core categories and their relationship were integrated into an exploratory framework of SPI success factors for small and medium Web companies. The major contribution of this study is, therefore, the investigation of SPI success factors and the resulting framework of SPI success factors for the said companies. This is also a novel contribution as it is the first time that this methodology has been applied on the mentioned domain of small and medium Web companies, purely investigating SPI success factors. The future work entails the validation of this framework on a larger sample of small and medium Web companies and to per-

498

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

form further quantitative investigations. This research also presents an opportunity for other researchers to further explore this subject area and to perform further theoretical integration. Appendix A This appendix contains a table giving the number of interviewees from each participating company. Company Company Company Company Company Company Company Company Company Company Company Company

Number of interviewees 1 2 3 4 5 6 7 8 9 10 11

1 2 1 3 1 2 2 1 2 4 2

Appendix B This appendix contains a table giving the number of interviewees who stated or commented on the categories or its factors/aspects to provide a brief overview of the categories importance. Category

Number of interviewees stated or commented on the category

SPI success Automated tools support Client support Communication Company vision Cost–benefit analysis Employees support Gradual approach Higher management support SPI consultancy SPI implementer’s role SPI measurements SPI supportive polices Tailoring of processes Applying existing knowledge about SPI SPI awareness programmes Benefits and goals of SPI Company success

21 9 5 7 8 16 18 13 21 11 16 14 6 15 14 17 21 21

References [1] R. Glass, Software Creativity, Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1994. [2] L. Harjumaa, I. Tervonen, P. Vuorio, Improving software inspection process with patterns, in: Proceedings of the Fourth International Conference on Quality Software, 2004 (QSIC 2004), 2004, pp. 118–125. [3] M. Lepasaar, T. Makinen, Integrating software process assessment models using a process meta model, in: International Engineering Management Conference (IEMC), IEEE, vol. 221, 2002, pp. 224–229. [4] G. Cugola, C. Ghezzi, Software processes: a retrospective and a path to the future, Software Process: Improvement and Practice 4 (1998) 101–123. [5] H.E. Thomson, P. Mayhew, Approaches to software process improvement, Software Process: Improvement and Practice 3 (1997) 3–17.

[6] S. Zahran, Software Process Improvement: Practical Guidelines for Business Success, Addison-Wesley Reading, MA, 1998. [7] W. Florac, R. Park, A. Carleton, Practical Software Measurement: Measuring for Process Management and Improvement, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, 1997. [8] P. Abrahamsson, Rethinking the concept of commitment in software process improvement, Scandinavian Journal of Information Systems 13 (2001) 69–98. [9] R. van Solingen, Measuring the ROI of software process improvement, Software, IEEE 21 (2004) 32–38. [10] J.P. Kuilboer, N. Ashrafi, Software process and product improvement: an empirical assessment, Information and Software Technology 42 (2000) 27–34. [11] M. Tortorella, G. Visaggio, Empirical investigation of innovation diffusion in a software process, International Journal of Software Engineering and Knowledge Engineering 9 (1999) 595–621. [12] M. Niazi, D. Wilson, D. Zowghi, Critical success factors for software process improvement implementation: an empirical study, Software Process: Improvement and Practice 11 (2006) 193–211. [13] T. Dyba, An empirical investigation of the key factors for success in software process improvement, IEEE Transactions on Software Engineering 31 (2005) 410–424. [14] D. Wilson, T. Hall, N. Baddoo, A framework for evaluation and prediction of software process improvement success, Journal of Systems and Software 59 (2001) 135–142. [15] S. Alexandre, A. Renault, N. Habra, OWPL: a gradual approach for software process improvement in SMEs, in: 32nd EUROMICRO Conference on Software Engineering and Advanced Applications, 2006 (SEAA ’06), 2006, pp. 328–335. [16] M. Fayad, M. Laitinen, R. Ward, Thinking objectively: software engineering in the small, Communications of the ACM 43 (2000) 118. [17] P. Allen, M. Ramachandran, H. Abushama, PRISMS: An Approach to Software Process Improvement for Small to Medium Enterprises, 2003. [18] E. Mendes, N. Mosley, Web Engineering, Springer-Verlag, New York Inc, 2006. [19] SEI, Improving Processes in Small Settings (IPSS Project), 2006. . [20] ISO, ISO/IEC JTC1/SC7 Working Group 24, 2007. . [21] G. Santos, M. Montoni, J. Vasconcellos, S. Figueiredo, R. Cabral, C. Cerdeiral, A.E. Katsurayama, P. Lupo, D. Zanetti, A.R. Rocha, Implementing software process improvement initiatives in small and medium-size enterprises in Brazil, in: 6th International Conference on the Quality of Information and Communications Technology, 2007 (QUATIC 2007), 2007, pp. 187–198. [22] T. Grechenig, W. Zuser, Creating Organic Software Maturity Attitudes (COSMA) Selected Principles and Activities for Software Maturity in Small and Medium Software Enterprises, 2004. [23] AIA, Aerospace Industries Association, 2010. . [24] M. Taylor, J. McWilliam, H. Forsyth, S. Wade, Methodologies and website development: a survey of practice, Information and Software Technology 44 (2002) 381–391. [25] P. Fraternali, P. Paolini, Model-driven development of Web applications: the AutoWeb system, ACM Transactions on Information Systems (TOIS) 18 (2000) 382. [26] E. Union, The New SME Definition, 2005. . [27] A. Cater-Steel, M. Toleman, T. Rout, Process improvement for small firms: an evaluation of the RAPID assessment-based method, Information and Software Technology 48 (2006) 323–334. [28] T. Dyba, Factors of Software Process Improvement Success in Small and Large Organizations: An Empirical Study in the Scandinavian Context, ACM, 2003, pp. 148–157. [29] N. Habra, S. Alexandre, J.-M. Desharnais, C.Y. Laporte, A. Renault, Initiating software process improvement in very small enterprises: experience with a light assessment tool, Information and Software Technology 50 (2008) 763– 771. [30] F.J. Pino, O. Pedreira, F. García, M.R. Luaces, M. Piattini, Using Scrum to guide the execution of software process improvement in small organizations, Journal of Systems and Software 83 (2010) 1662–1677. [31] S. Christodoulou, P. Zafiris, T. Papatheodorou, WWW2000: The Developer’s View and a Practitioner’s Approach to Web Engineering, 2000, pp. 75–92. [32] M. Sulayman, E. Mendes, Quantitative assessments of key success factors in software process improvement for small and medium web companies, in: Proceedings of the 2010 ACM Symposium on Applied Computing, ACM, Sierre, Switzerland, 2010, pp. 2319–2323. [33] M. Sulayman, E. Mendes, A systematic literature review of software process improvement in small and medium web companies, Advances in Software Engineering, Springer CCIS 59 (2009) 1–8. [34] Y. Deshpande, S. Hansen, Web engineering: creating a discipline among disciplines, Multimedia, IEEE 8 (2001) 82–87. [35] Y. Deshpande, S. Hansen, Web engineering: creating a discipline among disciplines, IEEE Multimedia 8 (2001) 82–87. [36] A. McDonald, R. Welland, Agile Web Engineering (AWE) Process, Department of Computing Science Technical Report TR-2001-98, University of Glasgow, Scotland, 2 December, 2001. [37] A. McDonald, R. Welland, Agile web engineering (AWE) process: multidisciplinary stakeholders and team communication, in: Proceedings of

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

[38]

[39]

[40] [41]

[42] [43] [44] [45] [46] [47] [48] [49] [50] [51]

[52]

[53] [54] [55]

[56]

[57] [58] [59] [60] [61]

[62] [63] [64]

[65]

[66]

[67]

[68]

[69] [70]

[71]

the 2003 International Conference on Web engineering, Springer-Verlag, Oviedo, Spain, 2003, pp. 515–518. A. McDonald, R. Welland, Agile Web engineering (AWE) process: perceptions within a fortune 500 financial services company, Journal of Web Engineering 4 (2005) 283. S. Vasudevan, D. Wilemon, Rapid application development: major issues and lessons learned, in: Innovation in Technology Management – The Key to Global Leadership, PICMET ‘97: Portland International Conference on Management and Technology, 1997, pp. 484–492. L. Rising, N.S. Janoff, The Scrum software development process for small teams, Software, IEEE 17 (2000) 26–32. H. Nayoung, Customization of Scrum methodology for outsourced Ecommerce projects, in: Y. Junbeom, C. Sungdeok (Eds.), Asia Pacific Software Engineering Conference, IEEE, Sydney, Australia, 2010, pp. 310–315. H. Helms, J. Quarto-vonTivadar, Discovering Fusebox 3 with Coldfusion, Techspedition, 2002. D. Lowe, J. Eklund, Client needs and the design process in web projects, Journal of Web Engineering 1 (2002) 23–36. D. Wallace, I. Raggett, J. Aufgang, Extreme Programming for Web Projects, Addison-Wesley Professional, 2003. J.R. Burdman, Collaborative Web Development: Strategies and Best Practices for Web Teams, Addison-Wesley, 1999. R. Ahmad, Z. Li, F. Azam, Web engineering: a new emerging discipline, in: IEEE Symposium on Emerging Technologies, IEEE, 2005, pp. 445–450. A. Ginige, S. Murugesan, Web engineering: an introduction, IEEE Multimedia 8 (2001) 14–18. R. Pressman, Software Engineering: A Practitioner’s Approach, sixth ed., McGraw Hill, 2006. Y. Deshpande, M. Gaedke, Web Engineering: Developing Successful Web Applications in a Systematic Way, 2005, pp. 10–14. D. Schwabe, G. Rossi, An object oriented approach to Web-based applications design, Theory and Practice of Object Systems 4 (1998) 207–225. G. Griffiths, B.D. Hebbron, M.A. Lockyer, B.J. Oates, A simple method and tool for web engineering, in: Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, ACM, Ischia, Italy, 2002, pp. 755–762. A.F.P. de Carvalho, J.C.A. Silva, Extending UWE to improve Web navigation project – a case study, in: IEEE International Conference on Systems, Man and Cybernetics, vol. 2603, 2005, pp. 2608–2613. G. ObjectManagement, UML 2.0 Superstructure Specification, 2005. J. Gómez, C. Cachero, OO-H method: extending UML to model web interfaces, Information Modeling for Internet Applications (2003) 144–173. M. Lockyer, G. Griffiths, B. Hebbron, B. Oates, A teaching method and tool for Web engineering, in: International Conference on Learning Advanced Technologies, IEEE, 2003, pp. 284–285. D. Bolchini, J. Mylopoulos, From task-oriented to goal-oriented Web requirements analysis, in: Proceedings of the Fourth International Conference on Web Information Systems Engineering, 2003 (WISE 2003), 2003, pp. 166–175. M. Jose Escalona, G. Aragon, NDT. A model-driven approach for web requirements, IEEE Transactions on Software Engineering 34 (2008) 377–390. G. Griffiths, CASE in the third generation, Software Engineering Journal 9 (1994) 159–166. A. Jones, M. Birtle, An individual assessment technique for group projects in software engineering’, Software Engineering Journal 4 (1989) 226–232. H. Nguyen, Web application testing beyond tactics, in: Sixth International Workshop on Web Site Evolution, IEEE, 2004, pp. 83–90. J. Preciado, M. Linaje, S. Comai, F. Sanchez-Figueroa, Designing rich internet applications with web engineering methodologies, in: 9th IEEE International Workshop on Web Site Evolution, WSE 2007, 2006, pp. 23–30. Pasha, Pakistan Software Houses Association (P@SHA), 2010. . Pseb, Pakistan Software Export Board (PSEB), 2010. . C.E. Lunneborg, Convenience Sample, Blackwell Publishing, Blackwell Reference Online, 2007. . D.R. Goldenson, After the Appraisal: A Systematic Survey of Process Improvement, Its Benefits, and Factors that Influence Success, in: DTIC Document, 1995. K. El-Emam, D. Goldenson, J. McCurley, J. Herbsleb, Modelling the likelihood of software process improvement: an exploratory study, Empirical Software Engineering 6 (2001) 207–229. A. Rainer, T. Hall, Key success factors for implementing software process improvement: a maturity-based analysis, Journal of Systems and Software 62 (2002) 71–84. M. Niazi, M.A. Babar, N.M. Katugampola, Demotivators of software process improvement: an empirical investigation, Software Process 13 (2008) 249–264. B. McFeeley, IDEALSM: A User’s Guide for Software Process Improvement, Handbook, CMU/SEI-96-HB-001, Pittsburgh, PA, 1996. N. Nikitina, M. Kajko-Mattsson, Impact of growing business on software processes, Systems, Software and Services Process Improvement (2010) 189–200. M. Montoni, A. Rocha, A methodology for identifying critical success factors that influence software process improvement initiatives: an application in

[72]

[73]

[74] [75]

[76]

[77]

[78]

[79] [80]

[81] [82] [83]

[84]

[85] [86] [87] [88] [89] [90] [91] [92] [93]

[94]

[95]

[96]

[97]

[98] [99] [100]

[101]

499

the Brazilian software industry, Software Process Improvement (2007) 175–186. F.J. Pino, F. García, M. Piattini, Software process improvement in small and medium software enterprises: a systematic review, Software Quality Journal 16 (2008) 237–261. S.U. Khan, M. Niazi, R. Ahmad, in: Critical Success Factors for Offshore Software Development Outsourcing Vendors: A Systematic Literature Review, IEEE, 2009, pp. 207–216. M.C. Paulk, B. Curtis, M.B. Chrissis, C.V. Weber, Capability maturity model, version 1.1, Software, IEEE 10 (1993) 18–27. G. Coleman, R. O’Connor, Using grounded theory to understand software process improvement: a study of Irish software product companies, Information and Software Technology 49 (2007) 654–667. A.E. Sheikh, H. Tarawneh, A survey of web engineering practice in small Jordanian web development firms, in: Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ACM, Dubrovnik, Croatia, 2007, pp. 481–490. Z. Habib, The Critical Success Factors in Implementation of Software Process Improvement Efforts: CSFs, Motivators & Obstacles, Rapport Nr.: Report/ Department of Applied Information Technology 2009: 056, 2009. M. Sulayman, E. Mendes, An extended systematic literature review of software process improvement in small and medium web companies, in: Empirical Assessments in Software Engineering (EASE 2011), BCS, Durham, UK, 2011. B. Day, S.C. Ke-Zun, L. Lovelock, C. Lutteroth, Climbing the Ladder: CMMI Level 3, Enterprise Information Systems, vol. 13, Taylor & Francis, 2010. H. Altarawneh, A.E. Shiekh, A theoretical agile process framework for web applications development in small software firms, in: Proceedings of the 2008 Sixth International Conference on Software Engineering Research, Management and Applications, IEEE Computer Society, 2008, pp. 125–132. F.J. Pino, J.A.H. Alegría, J.C. Vidal, F. García, M. Piattini, A Process for Driving Process Improvement in VSEs, Vancouver, BC, 2009, pp. 342–353. R. Naidu, Software Process Improvement of Small and Medium Organizations, Computer Science, The University of Auckland, Auckland, New Zealand, 2003. P. Allen, M. Ramachandran, H. Abushama, PRISMS: an approach to software process improvement for small to medium enterprises, in: H. Lin, H.D. Ehrich (Eds.) Proceedings. Third International Conference on Quality Software. Dallas, TX, 2003. L. Scott, R. Jeffery, L. Carvalho, J. D’Ambra, P. Rutherford, Practical software process improvement – the IMPACT project, in: D.D. Grant, L. Sterling (Eds.) Proceedings 2001 Australian Software Engineering Conference. Canberra, ACT, Australia. Instn. Eng., Australia (IEAust). Software Eng. Rea. Consultative Council (SERCC) of the Australian Comput. Soc. (ACS), 27–28 August, 2001. E. Mendes, N. Mosley, S. Counsell, A replicated assessment of the use of adaptation rules to improve Web cost estimation, in: ISESE ’03, 2003. T. Dyba, An instrument for measuring the key factors of success in software process improvement, Empirical Software Engineering 5 (2000) 357–390. C.B. Seaman, Qualitative methods in empirical studies of software engineering, IEEE Transactions on Software Engineering 25 (1999) 557–572. R. Galliers, M.L. Markus, Exploring Information Systems Research Approaches: Readings and Reflections, Routledge, 2006. A.S. Lee, J. Liebenau, Information Systems and Qualitative Research, Chapman & Hall, Ltd., 1997. S. Pace, A grounded theory of the flow experiences of Web users, International Journal of Human–Computer Studies 60 (2004) 327–363. R.K. Yin, Case Study Research: Design and Methods, Sage Publications, Inc, 2009. B.G. Glaser, A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research, AldineTransaction, 1967. P. Runeson, M. Höst, Guidelines for conducting and reporting case study research in software engineering, Empirical Software Engineering 14 (2009) 131–164. C. Urquhart, An encounter with grounded theory: tackling the practical and philosophical issues, Qualitative Research in IS: Issues and Trends (2001) 104–140. C. Urquhart, Strategies for conversation and systems analysis in requirements gathering: a qualitative view of analyst–client communication, The Qualitative Report 4 (2000) 1–12. C. Urquhart, Exploring Analyst–Client Communication: Using Grounded Theory Techniques to Investigate Interaction in Informal Requirements Gathering, Information Systems and Qualitative Research. Chapman and Hall, London, 1997, pp. 149–181. W.J. Orlikowski, CASE tools as organizational change: investigating incremental and radical changes in systems development, MIS Quarterly (1993) 309–340. A.L. Strauss, J. Corbin, Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Sage, 1990. B.G. Glaser, Basics of Grounded Theory Analysis: Emergence vs. Forcing, Sociology Press, 1992. C. Urquhart, H. Lehmann, M.D. Myers, Putting the ‘theory’ back into grounded theory: guidelines for grounded theory studies in information systems, Information Systems Journal 20 (2010) 357–381. D.J. Pauleen, An inductively derived model of leader-initiated relationship building with virtual team members, Journal of Management Information Systems 20 (2003) 227–256.

500

M. Sulayman et al. / Information and Software Technology 54 (2012) 479–500

[102] B. Webb, B. Mallon, A method to bridge the gap between breadth and depth in IS narrative analysis, Journal of the Association for Information Systems 8 (2007) 24. [103] D. Strong, O. Volkoff, Understanding organization-enterprise system fit: a path to theorizing the information technology artifact, Management Information Systems Quarterly 34 (2010) 731–756. [104] J.M. Corbin, A.L. Strauss, Basics of Qualitative Research, Sage Publications, Inc, 2008. [105] A. Cater-Steel, Low-Rigour, Rapid Software Process Assessments for Small Software Development Firms, 2004, pp. 368–377. [106] B.G. Glaser, Theoretical Sensitivity: Advances in the Methodology of Grounded Theory, Sociology Press, 1978. [107] K. Charmaz, Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis, Sage Publications Ltd, 2006. [108] B.G. Glaser, Doing Grounded Theory: Issues and Discussions, Sociology Press, Mill Valley, CA, 1998.

[109] B.G. Glaser, The Grounded Theory Perspective: Conceptualization Contrasted with Description, Sociology Press, Mill Valley, CA, USA, 2001. [110] B.G. Glaser, The Grounded Theory Perspective III: Theoretical Coding, Sociology Press, 2005. [111] A.L. Strauss, J.M. Corbin, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage Publications, Inc, 1998. [112] F. Pino, F. García, M. Piattini, Software process improvement in small and medium software enterprises: a systematic review, Software Quality Journal 16 (2008) 237–261. [113] F. Guerrero, Y. Eterovic, Adopting the SW-CMM in a Small IT Organization, Software, IEEE 21 (2004) 29–35. [114] J.A. Maxwell, Qualitative Research Design: An Interactive Approach, Sage Publications, Inc, 2005. [115] G. Walsham, Interpretive case studies in IS research: nature and method, European Journal of Information Systems 4 (1995) 74–81.