Transitioning to agile software development: Lessons learned from a government-contracted program

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

Contents lists available at ScienceDirect

Journal of High Technology Management Research journal homepage: www.elsevier.com/locate/hitech

Transitioning to agile software development: Lessons learned from a government-contracted program ⁎

Peerasit Patanakul , Reneé Rufo-McCarron The Pennsylvania State University, The Behrend College, Sam and Irene Black School of Business, USA

A R T IC LE I N F O

ABS TRA CT

Keywords: Agile project management SCRUM Software development Lessons learned Agile implementation

With a need to implement an agile software development methodology in government programs, this study was conducted to explore how organizations transitioned from a traditional waterfall software development method to agile methods. To accomplish this study, a case study research was utilized. A multimillion-dollar government-contracted program in the US was studied. The research findings reveal key challenges faced by organizations during the transition and the lessons learned that can be applied to future programs. Propositions for future research are also presented.

1. Introduction As projects increasingly become an integral part of an organization as a means to execute organization's business strategies and to improve organization's efficiency, a need to improve the success of projects is of the priority of both researchers and practitioners. Within Information Technology (IT) project environments, many studies have been conducted to identify factors contributing to project success. New methods and approaches for software development projects have been proposed. Among them is a development process that revolves around multiple development iterations through the development cycle that can facilitate the continuous interactions with the customers and address uncertainties. This iterative process is typically a part of agile methodologies that have been increasingly implemented by many organizations to replace the traditional, front-end planning process (Jorgensen, 2016; Nurdiani, Borstler, & Fricker, 2016; Serrador & Pinto, 2015). Several well utilized agile methodologies include Extreme Programming (XP), Dynamic Systems Development Method (DSDM), Feature Driven Development (FDD), Scrum, Lean software development, Agile Unified Process (AUP), and Crystal (Braude & Bernstein, 2011). Typically, an agile methodology focuses on lightweight processes employing short iterative cycles. It also requires actively and continuously involved users both in establishing, prioritizing, and verifying project goals and requirements and providing feedback to progressive prototypes as the project moves thorough its life cycle (Boehm & Turner, 2005; Patanakul, Henry, & Leach, 2016; Serrador & Pinto, 2015). With these differences from a traditional method, transitioning to an agile methodology can be very challenging in many organizations (Boehm & Turner, 2005; Dikert, Paasivaara, & Lassenius, 2016; Ghobadi & Mathiassen, 2015). Problems that have been identified include the integration with the standard development and business processes established in the organization, the attitudes and resistance to change of the people within the organization, and the implementation of self-organized collocated teams (Boehm & Turner, 2005; Heeager, 2012; Jalali & Wohlin, 2012). Literature also suggests that the implementation of agile practices on small, standalone projects is less burdensome. More challenges have been found when scaling up and integrating them into traditional, top-down systems development organizations (Boehm & Turner, 2005; Dikert et al., 2016).

⁎

Corresponding author. E-mail address: [email protected] (P. Patanakul).

https://doi.org/10.1016/j.hitech.2018.10.002

1047-8310/ © 2018 Elsevier Inc. All rights reserved.

Please cite this article as: Patanakul, P., Journal of High Technology Management Research, https://doi.org/10.1016/j.hitech.2018.10.002

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

With an increasing need of transitioning to an agile methodology and the challenges associated with it, the objective of this study is to explore how organizations transitioned from a traditional waterfall software development method to agile software development methods. In particular, we would like to explore the key challenges faced by organizations during the transition and the lessons learned that can be beneficial for future endeavors. To accomplish this objective, a case study methodology was employed. We studied the transition of a multimillion-dollar government-contracted program in the United States. The results of this study contribute to the literature and practice in many ways. First, this study focuses on the transition period. It highlights the dynamics of transitioning from one development methodology, which the team and the customer had the high degree of knowledge and practice, to other development methodologies that they were not familiar with. Second, it focuses on the transition of a large-scale software development program. While the knowledge on the implementation of agile methodologies is rich for small-scale programs, not much has been researched on transitioning to agile methodologies in large-scale programs and as indicated above that many challenges exist when scaling up agile methods (Dikert et al., 2016). Third, unlike other studies that were conducted in private sectors, this study focuses on a government-contracted program. Studying the transition in this context is significant since the research outcomes highlight the dynamics and interactions between the project team, representing both private and public sectors from different organizational backgrounds. The research outcomes also capture the challenges in transition when the project team members from different sectors had various levels of knowledge, understanding, and familiarity with agile methodologies. In the next section, we present the literature review followed by our research method. Then, research findings and discussion are presented. Contributions and managerial implications are discussed toward the end of the paper. 2. Literature review 2.1. What is an agile methodology? Agile is a form of project planning and execution. It is widely used in software development as an alternative to traditional software development approaches. While traditional approaches typically emphasize the sequential process, starting from requirements gathering, planning, designing, code development, testing, and implementation; agile methodologies emphasize an iterative workflow and incremental delivery of software products in short iterations (Patanakul et al., 2016). Agile is a part of an evolutionary model for software development that was developed to address the uncertainties in requirements and software architects (Fairley, 2009). Extreme Programming (XP), Dynamic Systems Development Method (DSDM), Feature Driven Development (FDD), Scrum, Lean software development, Rational Unified Process (RUP), and Crystal are among the agile methodologies widely used in practice (Braude & Bernstein, 2011; Wysocki, 2006). XP, for example, was created to effectively address projects that involve requirement changes. Instead of assembling large, separately developed modules, XP relies on continual integration. XP projects are divided into iterations, approximately one to three weeks long to produce software that is fully tested. These iterations represent the development, integration, and thorough testing of a small amount of new capability (Braude & Bernstein, 2011). Similar to XP, the main features of DSDM include user involvement, iterative and incremental development, increased delivery frequency, integrated delivery frequency, integrated test at each phase, and fulfilling requirements. FDD is also an iterative and incremental software development process focusing on features (client-valued functionality). Scrum is another agile methodology that has been widely used. The Scrum framework emphasizes empirical feedback, team selfmanagement, and striving to build properly tested product increments within short iterations (Schwaber, 2004). In Scrum, the software developers are referred to as the Team, which typically includes 6–10 developers. The customer or the user representative is termed the Product Owner. Scrum Master typically has a similar role as a team leader or a project manager of a small team. Within Scrum framework, a project is broken down into self-organizing teams (scrums). Each team focuses on a self-contained area of work (Schwaber, 2004). To develop software, a list of customer wants and needs (user stories) is created, referred to as a product backlog. Scrum uses fixed-length iterations, called Sprints, which are typically 2–4 weeks long. While in a sprint, scrum teams are responsible for taking on a set of features from the backlog and developing a deployable product that is properly tested (Cohn, 2010; Patanakul et al., 2016). The team is given full authority of how to successfully complete the sprint (Cohn, 2010). An approximately 15-minute daily scrum meeting is organized to assess the status of the sprint, report on problems, and identify future tasks. At the end of a sprint, a customer demonstration is conducted. The leftover features and new tasks are gathered, a new backlog is created, and a new sprint begins (Patanakul et al., 2016; Wysocki, 2006). This approach in software development allows the teams to develop a subset of highvalue features as early as possible to incorporate early feedback from the customers (Schwaber, 2004). Similar to Scrum, Kanban is an agile methodology that promotes continuous collaboration, learning, and improvement by utilizing self-managed teams. Kanban relies on three basic principles that evolve around visualizing current workflow, limiting the amount of work in progress, and managing lead time (Cottmeyer & Stevens, 2015). While Scrum utilizes, e.g., predetermined roles and time-boxed sprints, Kanban does not have prescribed roles and focuses on continuous delivery. Changes can be made at any time. Scrum is more appropriate when works can be prioritized in batches. Kanban is suitable in an environment when task priority changes frequently (Versionone, 2016). 2.2. Implementing agile methodologies While an agile methodology has many benefits leading to project success (Jorgensen, 2016; Serrador & Pinto, 2015), many challenges exist when implementing agile (GAO, 2012). Among them are the challenges with respect to personnel and processes that include issues such as changing mindset of personnel, unclear roles and responsibilities, knowledge sharing, and the integration with 2

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

the existing business and development processes (Boehm & Turner, 2005; Ghobadi & Mathiassen, 2015; Heeager, 2012; Rola, Kuchta, & Kopczyk, 2016). Literature suggests that while many organizations were successfully piloting an agile methodology, implementing it can be difficult. With the paradigm shift of empowering and engaging individuals in an agile method, challenges related to personnel do exist. Changing the mindset of project stakeholders to work in an agile environment is among the top challenges listed by many researchers. Many problems are related to management attitude when losing control over the team. Also, it is challenging to persuade the autonomous teams to focus on the broader goals of the organization in addition to the team goals (Dikert et al., 2016). The logistical and working space issues when needing a collocated team can become problematic in many organization (Jalali & Wohlin, 2012; Rola et al., 2016). Resistance to change to a new way of doing things is also common. In many cases, stakeholders may have wrong perception of agile, which may cause from lacking of sufficient knowledge and communication (Dikert et al., 2016; Heeager, 2012). Engaging the customers or the users in agile can be difficult. Lacking of committed customers or users makes it difficult for requirement management process (Heeager, 2012). Roles of product owners and middle managers is often unclear in many organizations when implementing agile. Researchers also indicated the existence of the challenges related to knowledge sharing (Ghobadi & Mathiassen, 2015). Different from a traditional method, where knowledge sharing tends to occur through documentation, the agile methods facilitate knowledge sharing through informal communications among team members (Heeager, 2012). With the reliance on team members, barrier toward knowledge sharing can be in the forms of e.g., team diversity, communication, and project setting (Ghobadi & Mathiassen, 2015). The implementation of a communication and knowledge sharing strategy is necessary (Heeager, 2012). In addition to the personnel related issues, challenges exist within the development processes as agile practices differ from the traditional approach (Boehm & Turner, 2005; Konnola et al., 2016). For example, in many organizations, implementing an agile requirement process can be problematic as agile requirements tend to be primarily functional and reasonably informal. Requirements are gathered throughout the iterative life cycle in an agile process, whereas in a traditional method the requirements are gathered and specified in the beginning of the project. Additionally, requirement handling in an iterative manner can be challenging when the development process is milestone-driven (Heeager, 2012). Literature suggests that challenges related to testing also exist. Agile software methodology suggests that test cases are written prior to the development of software and that all parts of an increment are tested within the iteration. The execution of this time-boxed unit testing can be difficult due to the problems related to the coordination between the existing quality assurance teams and the development teams (Dikert et al., 2016), causing the testing to be postponed (Heeager, 2012). Implementing an agile method can also cause conflicts with the business processes. Traditional contracting processes may not support an agile method as contracts and payments tend to be based on milestones. Many researchers suggested that agile methodologies should be tailored to accommodate existing business and development processes and vice versa (Ben-David, Gelbard, & Milstein, 2012; Heeager, 2012; Konnola et al., 2016). Process integration is necessary (Jahr, 2014). Literature also suggests that it is more simplistic to implement agile methodologies on small, standalone projects than scaling up and integrating them into traditional, top-down systems development organizations (Dikert et al., 2016). The Scaled Agile Framework (SAFe) was introduced as a programming knowledge base that can be used to facilitate the development of complex enterprise-level software systems (Heusser, 2015). Not much research has been done to explore the benefits of this enterprise-level approach. 3. Research method The main objective of this research is to explore how an organization transitioned from a traditional software development method to an agile software development method. In particular, we would like to explore the key challenges faced by the organization during the transition and the lessons learned that can be beneficial for future endeavors. In this study, our focus was on a government-contracted program that transitioned from a waterfall approach to Scrum and Kanban approaches. Case study research was utilized as a methodology for this study (Eisenhardt, 1989). 3.1. Case background To fulfill the research objective, we performed a study on an ISA program. ISA was a software development, maintenance, and enhancement program, consisting of multiple subprojects. According to the government contract, agile project management must be implemented in the ISA program. This program was selected because of the accessibility of the information and its fit with the context of this research. The following is the background information of the ISA program. In late 2009, a major US based Aerospace and Defense Company (referred to as Alpha) won an approximate $400 million contract with a US Government Agency (referred to as The Agency) to develop, maintain and enhance projects under one of The Agency's main responsibilities, acquisition services. During the five-year contract term, the majority of the software development, maintenance, and enhancement projects were executed under the Software Development Life Cycle (SDLC) waterfall approach. In 2014, Alpha recompeted to continue the contract, referred to as the ISA program. To leverage competitive advantage and respond to the initiative set forth by The Agency, Alpha was committed to transition all projects from the SDLC approach to agile methodologies. On the 2009 contract, most projects would have releases mainly on a quarterly basis. While many of these releases were quite successful, problems and issues existed. For instance, The Agency would not have the opportunity to see the desired product until user acceptance testing (UAT), which was just prior to deployment. UAT usually brought a handful of comments, concerns, change requests, etc. Sometimes UAT feedback would delay the deployment. Additionally, many of The Agency's customers (other 3

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

government units or agencies) would bring scope changes and requirements changes late in the SDLC, which would greatly impact the schedule. A lot of the earlier work would need to be reworked because of these changes and the release date and budget would then be at risk. Given the various problems and issues, Alpha began working with The Agency to discuss transitioning projects to agile methodologies under the 2009 contract. Both parties decided to run a project using agile such that the project would involve incremental software development where cross-functional teams would collaborate and self-organize. Each project team – or scrum team – would have a Product Owner (members from The Agency) and development team and a scrum master (members from Alpha). This setup would allow for a lot more interaction and collaboration between Alpha and The Agency. The intent of transitioning to agile is to provide The Agency with more hands on access to the projects, working very closely with Alpha project teams. The Agency can provide daily feedback to the development team, rather than waiting until UAT. Additionally, short sprints mean that requirements or scope change will not set a project back by months. Instead, the change can be worked, and implemented, two weeks later in the next sprint. Alpha was awarded the approximate $400 million ISA contract. With this contract, all of the projects must be transitioned to agile methodologies by the end of the 2015 calendar year. An agile adoption metric was used by the ISA program to track whether or not the project teams have either already transitioned to agile methodology or have a plan in place to do so. The program plans to implement an additional agile maturity metric to track how well each project is executing agile. Prior to using agile, project teams were typically led by a project manager and team leads (Requirements Lead, Test Lead, etc.). Teams were used to following a detailed project schedule that outlined waterfall tasks. Team members would meet frequently with their project manager and/or team leads to discuss the status of the current tasks and learn about the future tasks assigned to them. These tasks would include project-related tasks, in addition to functional process required tasks, e.g., documentation and gate reviews. In the agile setting, team members are assigned tasks during the planning meeting, and then expected to execute these tasks accordingly by the completion of the sprint. There is a daily scrum call in which each team member is expected to report status and any issues/blockers. If a team member needs assistance completing a task, he/she is expected to communicate as such. The scrum master, product owners, and other team members do not walk through each task to find out the associated status and ensure that tasks are on schedule. It is expected that each individual team member is responsible for taking ownership of each task and working with respective team members should they need to coordinate/collaborate/need assistance. In July 2015 when the study was conducted, about 70% of the ISA program has implemented agile methodologies – a mix of Scrum and Kanban approaches. 3.2. Information gathering and analysis Information was collected through interviews and observation. In this study, 12 members of the ISA program were interviewed (See Table 1). To maintain confidentiality, we are not able to provide specific information that could potentially link to the participants. In general, these respondents were selected for their experience with agile implementation in ISA program. They also represented members of project management and functional departments with different levels of project management experience (median = 8 years), the experience with Alpha (median = 11 years), and the industry experience (median = 17 years). Seven of the participants hold project manager, program manager, or manager of program management position. Three participants were program management staff. One participant was a functional manager and one participant was a project team member. This helped us have a balance of perspectives. The respondents were asked the same set of questions concerning agile transition and the challenges they faced during the transition. The respondents were also asked to share any ideas to improve the implementation of agile approach in ISA program. Some of the guiding questions are: 1) how would you define agile? Do you think that the ISA program is fully executing to your definition of agile? Why or why not? 2) Do you think The Agency and Alpha have the same understanding of agile? Why or why not? 3) What are some advantages/disadvantages of transitioning to agile? 4) What do you think is the biggest challenge the ISA program has faced with transitioning to agile? How do you think this can be addressed? 5) Do you have any general thoughts/comments regarding implementing agile on the ISA program that you would like to share? The interviews were transcribed. Notes were written for each interview. In addition to the interview, a research team member had a chance to observe the transition and review some Table 1 Case participants. Participants

Position

A B C D E F G H I J K L

Management: Functional Staff: Program management Management: Program management Management: Project management Management: Project management Staff: Program management Management: Project management Member: Functional team Management: Program management Management: Project management Management: Project management Staff: Program management

4

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

documentation. After information gathering, a case analysis was performed to identify the critical challenges during the transition to agile methodology. Particularly, the within-case and cross case analyses were performed. The chains of evidence were developed. Literature was reviewed to validate the findings and propositions were developed. 4. Findings and discussion 4.1. The understanding of agile software development The research participants responded with very differently worded definitions of agile, yet they all held the same common theme or idea. Many described agile using words like “methodology,” “framework,” “iterative,” “rapid,” “flexible,” and “time-boxed.” When asked if the ISA program executes to his/her definition of agile, responses were again very similar. They believe that the program has most certainly started utilizing agile methodologies but there is a lot of room for growth and improvement. One of the respondents mentioned: “a variation of maturity of understanding across individuals that create differences of opinion on the feasibility and value of agile. Some of The Agency and Alpha individuals have an expert level of understanding of agile. Others are still novices and have only a rudimentary understanding of agile.” The respondents see value in executing agile methodologies on the ISA program. Responses included ability to change, flexibility, customer engagement, velocity, immediate customer feedback, team members knowing and understanding their tasks/priorities, faster delivery time, more cohesive relationships between the product owners and project team, more buy-in from the customer, increased quality of the product being delivered. 4.2. Challenges during transitioning to agile Many challenges during transitioning to agile were found in this study. Resistance to change is the issue that was repeated by many respondents, who felt that change management was lacking on both Alpha and The Agency. Another common issue that arose is lacking of Product Owner commitment. Many of the respondents also felt that more training is needed across the program as agile implementation continues to mature. The following are the critical challenges identified in this study (see Table 2 for the summary). 4.2.1. Change management One of the challenges we found from the interviews is the resistance to change. Many of the program's employees and customers were adverse to change, and there was resistance to fully adapting and implementing agile, which led to various problems. One of the project managers noted that resistance to change is “the biggest challenge on the program because team members are used to a specific Table 2 Themes and issues from the case study. Challenges Change management

Accessibility to expert and training

Product owner commitment

Documentation

Integration with standard processes, tools, and techniques

Implementing test automation

Findings and recommendations

Indicated by

- Resistance to change: Team members are used to a specific mindset in software development - Need to utilize a mature change management process that involves both the contractors and the customers (government personnel) when implementing agile methodology - Need a designated “champion” to push agile forward successfully - Stakeholders lack of knowledge on agile methodology - Need proper trainings and education for both contractors and customers. Such training should be done in a coordinated and collaborated manner to ensure compliment knowledge, skill, including a better understanding of different roles and responsibilities - Need mentors across the program both for the contractors and customers. Need to provide accessibility to agile experts - Lack of ownership/commitment from the customer - Need to promote an understanding of the product owner role and responsibility, including its significance to agile software development - Need to engage the product owner early in the transition process - Misconception that documentation/deliverables are not required, leading to lack of documentation - Need proper integration of project and project management documents with agile process - Poor integration with the standard and required processes, including tools and techniques in the areas of e.g., requirements tracking and project monitoring and control - Need to establish a tracking system to link agile deliverables and baseline requirements - Need to establish a tracing mechanism to apply Earned Value Management to agile - Lack of an understanding on the benefits of automation in agile software development - Establish a need for automation and ensure the understanding of its benefits among stakeholders.

5

B, E, F, I

A, B, C, E, F, H, I, J

C, F, G, I, J, K

A, D, E, H

A, B, E, G, H, I, J, L

A, C, H, I

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

mindset.” There is the mindset of working on a requirement, and completing it, rather than having a requirement changed the next day and immediately working on that change. He also noted the mindset surrounding testers. Under a waterfall approach, testers would conduct system testing and there would be a ramp up and a ramp down for them. However, under agile, “the team is slowly engaging testers in development activities, and vice versa. Developers are gradually being brought into the end of the sprint cycle to help the testers. The team is merging and there are a lot of team members who are having a hard time adjusting to these types of changes.” Another project team member noted that, it was not only Alpha's personnel, but also the staff of The Agency who were resistant to change. “Some customers are afraid that they will lose visibility into their projects even though the projects that have already adopted agile on this program have proven how much involvement and visibility the customer has.” These findings support what suggested in the literature (Dikert et al., 2016). By nature, many organizational members do not want to change the way they operate as they prefer not to encounter both foreseeable and unforeseeable uncertainties. Several destructive behaviors can be the outcome of resistance to change. Agile introduces many methods and techniques that organizational members may not be used to. For example, agile focuses on people aiming at empowering individuals by providing ownership and flexibility, in coupling with supporting reasonable goals and shorter feedback cycles. A self-organizing team is common for agile. This may create conflicts both in terms of the use and control of resources from management perspective and personal conflicts as some of the organizational members may not be ready for such empowerment (Boehm & Turner, 2005; Dikert et al., 2016). Overall, the research participants understood the situation and suggested some recommendations. One of the participants stated “becoming 100% agile requires a complete turnaround for just about everyone on the program, aside from those who may have previous experience… the program needs a Champion to help push the program forward successfully given the many changes involved with this agile implementation.” One of the managers mentioned that the implementation of agile requires a mature change management process from both Alpha and the Agency. He stated, “Change initiatives could be defined across several parallel streams including method selection and unification, training for beginners and experienced practitioners, mentoring and coaching for the teams, and automation support.” This finding supports the literature. Effective change management is necessary for implementing any new initiative in an organization. Similar to implementing new initiatives, transitioning from a traditional software development method to agile requires change management. To address this resistance to change, literature suggests that management should set forth an environment that values collaboration, teamwork, and trust; autonomy is engaged; team members can grow; and there are opportunities for mentoring and learning (Crowder & Friess, 2015; Suscheck & Ford, 2008). From the findings and the literature, it should be concluded that “the transition to agile software development should be taken as a change initiative to be able to address the resistance to change both from the contractors and from the customers.” Particularly, “the utilization of mature change management process that involves both the contractors and the customers impacts the successful transition to agile software development” and “a designated champion is necessary for successful agile transition.” See Fig. 1 for an illustration. 4.2.2. Accessibility to training and agile experts Introducing a different approach to software development requires significant knowledge and experience. Agile advocates collaborative development and is dependent on knowledge sharing. Different methods, including tools and techniques, are used to ensure the effective collaboration and the successful development of the product (Cohn, 2010; Rubin, 2012). Even though these methods, including tools and techniques, look simplistic, effective utilization requires training and education (Boehm & Turner, 2005; Dikert et al., 2016; Patanakul et al., 2016). From the case we studied, many participants were aware that until a team has proper agile training, the team will not succeed in utilizing agile methodology. The interviewees generally indicated the lack of knowledge and understanding of the agile approach. An interviewee mentioned that Alpha has experienced project managers who have worked in the traditional waterfall setting for many years. “If you hand those managers projects and tell them to run a scrum, they won't know what they're doing, nor will the customer know what they're doing, and so the managers will be set up for failure.” The interviewees acknowledged the existence of a large learning curve across the program from both sides— Alpha and The Agency. Accessibility to agile experts, including training and education were needed. Particularly, the case participants mentioned the need of mentors across the program and the ability to reach out to agile experts. Having the experts on the program can be beneficial. This includes the certified Scrum Masters, which the program is lacking. It is typical that a Scrum Master is responsible for removing obstacles for the development team to be able to deliver the products. A Scrum Master also helps enforce the rules to ensure that the Scrum process is used as intended (Schwaber, 2004). In addition to the accessibility to the experts, many of the interviewees

Contractor involvement

Customer involvement

Change management Successful transition to Agile

Role of Champion

Fig. 1. Role of change management in a successful transition to agile software development. 6

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

Contractor involvement Coordinated and collaborated training

Customer involvement

Successful transition to Agile

Accessibility to experts

Fig. 2. Role of training and agile expert in a successful transition to agile software development.

mentioned the need of training and education. One of the functional team members indicated that the ISA team is currently trained but not everyone is fully trained yet. An ISA project manager stated, “the program needs training and practice to optimize the application of agile and large scale initiative integration.” Along the same line, another interviewee mentioned, “there is room for opportunities for additional training or ‘brownbag’ sessions to help everyone get a little more comfortable.” Several interviewees indicated that the training must be conducted for the members of ISA program from both Alpha and The Agency. One of the interviewee mentioned, “They [The Agency] started taking training that we [Alpha] told them to take.” He continued to say that as [Alpha] continues to learn, so does [The Agency], as they are able to learn from [Alpha]. Another interviewee indicated, “Until a team has training on agile and the right methodology…agile cannot do well. The program has a lot of project managers who are not experienced in agile. Having them run a scrum with a business line who also isn't experienced in agile will produce negative results.” Even though these findings are not new and much has been discussed in the literature, the findings reveal that the challenge associated with insufficient knowledge still exist during an agile transition. It is an important issue that management should pay attention to. In essence, the findings highlight what suggested in the literature that providing training on agile methods is necessary for the successful transition (GAO, 2012). Literature suggests that in addition, the principle of learning by doing by providing coaching to the team should be utilized when the team practices the agile method (Dikert et al., 2016). The findings of this study indicate that the trainings are needed for both the contractors and the customers (government personnel) and imply that those trainings should not be done in isolation. This leads to the conclusion (see Fig. 2) that “coordinated and collaborated trainings should be conducted both to the contractors and the customers such that both parties have compliment knowledge, skill, and a better understanding of different roles and responsibilities.” In addition, “the accessibility to agile experts are necessary for the successful transitioning to agile project management in a large-scale government-contracted program.” 4.2.3. Commitment of product owner In agile method, the product owner represents the customer of the project and has a significant role to the success of a software development project (Dikert et al., 2016). Differing from the traditional waterfall method, the product owner, who is accountable for ensuring the value of software product, is highly engaged in agile software development. Based on the product requirements, the product owner typically provides the customer-centric items (referred to as user stories) to the development team. The product owner also prioritizes the items and is the one who adds additional items to the product backlog. Software products will be moved to production only after the product owner accepts the output from the development team (Cohn, 2010; Patanakul et al., 2016; Schwaber, 2004). Similar to the literature, we found in our study that the successful implementation of agile in the ISA program requires the commitment of the product owners from The Agency. One of the ISA program managers stated, “They [the product owners] are responsible for managing the backlog and prioritizing, which takes more time and requires more real time decisions.” However, it was found that not all of the representatives from The Agency were ready and willing to take on the responsibilities of a product owner. Two interviewees mentioned that in the previous 2009 contract, a project was issued a stop work because the end customer was not involved in the initial project team. Instead of The Agency's personnel, the product owner who was assigned to the project team was a contractor and therefore was not authorized to make decisions. While this could be a lesson learned, the issue of project owner's commitment still surfaced in the ISA contract. As for the ISA contract, several interviewees both from functional and project teams mentioned that getting the commitment of the product owners was rather challenging. One of the interviewees stated, “Agile has no value if you do not have the right product owner…there is a challenge getting someone from [The Agency] to participate to ensure we produce exactly what the customer wants. If you want agile, the customer needs to make sure that the product owner is in place – this is critical.” However, two of Alpha's project managers agreed that while some customers were immersed with agile and starting to understand and realize the benefits of the approach, some customers were still on the periphery and sat back, rather than taking ownership as the product owners and assisting in determining priorities. One project manager stated “there are product owners who want nothing to do with writing user stories and working processes on their side, which is an intricate part…they don't want to take the time to create a backlog…it's hard to adapt to change on both the customer and internal sides.” An interviewee also stated that it was rather challenging to get The Agency to assign product owners. In many cases, the project managers assumed the responsibilities of product owners. However, the project managers who worked on different projects did not own the products. Along the same line, another interviewee stated, “To date the program has only a few business owners willing to take up the responsibilities of the product owner role. Hence, either an IT manager must act as the surrogate product owner, or the development team must assume some of the prioritization and scope decisions for iteration sprint planning.” 7

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

Early engagement of product owner

Commitment of product owner

Successful transition to Agile

Fig. 3. Role of product owner in a successful transition to agile software development.

To be successful in the implementation of the agile approach, the ISA program needed to address the lack of product owner commitment. One of the interviewees mentioned that the program should “get [The Agency] engaged, get them trained and get them to buy in to the methodology before starting any project. As agile teams mature on the program, the customer will begin to see the value of agile. The teams will be able to incorporate customer feedback much quicker than traditional waterfall methods. Agile also allows more room for creativity, quicker turnarounds, and rapid value delivered to the customer. The project teams will need to communicate to the customer in a way that the customer will understand. Training and educating the customer will be critical to getting them engaged and committed to the new processes.” Our findings augment what has been suggested in the literature. Many authors suggested that the introduction of agile does not only affect the project team involved but it also affects the stakeholders, particularly, the customers who need to play significant different roles (GAO, 2012; Heeager, 2012). Agile method requires significant customer input, interaction, and feedback. Particular attention must be put on process matching and customer education for a smooth transition (Boehm & Turner, 2005; Patanakul et al., 2016). Management must assign the product owner with the right knowledge and competences to the team (Heeager, 2012). Extending the literature, the findings of this study highlight the early engagement of the product owners that can potentially help increase the level of commitment of the product owners to the successful agile transition. With this, we propose: “the early engagement of the product owner in the agile transition increases the level of commitment of the product owner and the more the commitment the product owner provides, the more successful transition becomes,” see Fig. 3.

4.2.4. Documentation Previously on the 2009 contract, there was a formal and structured process in place in which each project would need to be classified in order to determine which deliverables were required contractually. Some of the interviewees noted that they felt the ISA program and the transition to agile have moved projects away from the project classification. One raised the following concerns: “Are we all still sending all of the deliverables that are required? Who is checking and how do we know if we're compliant? Schedules are very high level. [I'm] afraid that projects may be missing deliverables.” It is true that agile methods are designed to use minimum documentation in order to facilitate flexibility and responsiveness to changing conditions (Serrador & Pinto, 2015). However, a common misconception for agile is that documentation is not required. Many of the interviewees acknowledged this misconception and agreed that documentation was lacking across the program, even though it was required contractually. A functional team member recognized that “documentation tends to get left behind and that people on the program believe that agile means no documentation, which is completely false.” Several interviewees echoed this sentiment. A project manager stated, “Minimal documentation doesn't mean not doing documentation. [The team] should be able to reference something.” Another interviewee acknowledged the need for documentation on agile projects. He said that the program needs to work to “mature our approach,” and “the User Guide should be updated to document enhancements; the program should focus on how to improve in this area.” While recognizing the need of documentation, the challenge with scheduling documentation and deliverables within an agile schedule still existed and was mentioned by several interviewees. One of the project managers stated, “It is hard to know the process for how to incorporate deliverables and documentation into agile schedules.” The workaround was to place the deliverables and documentation at the end of the schedule/project as part of the final deployment. However, this workaround did not indicate when team members should be working on their documentation within sprints from the schedule standpoint. Another project manager also noted that placing documentation and deliverables at the end of the sprint means that the Technical Publications team is getting handed documents at the last minute for review, placing a burden on that team. Another challenge mentioned was that government stakeholders may expect documentation related to technical reviews, such as the Preliminary Design Review (PDR) and Critical Design Review (CDR). Similar to what is found in the literature, documentation created under agile typically doesn't support the documentation expected at these reviews (GAO, 2012). As stated in the literature, the expectations, supporting documentation, and acceptance criteria need to be established in advance of these reviews with the stakeholder (Broadus, 2013). Perhaps, a rule of thumb is to only create documentation that is requested by the stakeholder. Teams can include documentation updates/completion as a task in the user story. Other ways that teams can incorporate documentation into the agile process is to include the documentation completion as part of the Definition of Done for each user story and allow documentation sign-off at the end of release (Burba, 2015). This recommendation goes along the same line as stated in the TechFAR Handbook for Procuring Digital Services Using Agile Processes (OMB, 2014). Another recommendation is to require design documentation, at a minimum, such as architectural specifications that include key design principles and serve as a medium between system stakeholders and the design team. The team should update and maintain this documentation in proportion to the design's stability, meaning, the more stable the design, the more adequate the documentation should be (Selic, 2009). Government stakeholders tend to rely heavily on documents, reports, metrics, milestone reviews, etc. Since agile documentation generally does not 8

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

Integraon of required project and project management documents with Agile

Successful transion to Agile

Fig. 4. Role of documentation in a successful transition to agile software development.

support these types of expectations, it is critical to determine what needs to be documented, captured, and tailored during the proposal and negotiation processes (Broadus, 2013). Tailoring is necessary (Konnola et al., 2016). Based on the finding and the literature, we can conclude that “the successful transition to agile software development depends on the integration of the documents representing project and project management deliverables with the agile process,” see Fig. 4. 4.2.5. Integration with standard processes, tools, and techniques While there was also a clear process for what was required in terms of gate reviews, quality assurance, etc. in the 2009 contract, with the ISA, a project manager acknowledged the possibility that some of these processes are “easily forgotten or mulled over to fit within the time-box period” for agile projects. One of the functional project team members stated that “agile is not a process to circumvent the important steps of the SDLC. When transitioning from the traditional life cycle to agile, some team members think that means there is no longer a need for formal reviews and such.” The interviewees agreed that having processes and deliverables is necessary. The challenge was integrating agile with existing standard processes, tools, techniques, and deliverables required by the customers. One of the challenges that an interviewee mentioned was around the integration of baseline requirements and testing. This challenge has been acknowledged in the literature (Dikert et al., 2016; GAO, 2012; Heeager, 2012). A functional team member indicated that 80% of the ISA program has thousands of baseline requirements that were mapped to test. Transitioning these baseline requirements to an agile tracking tool was very difficult and challenging. There was a gap with linking agile user stories back to requirements, determining where to document defects, etc. The interviewee stated, “When documenting user stories in agile projects, it's incredibly challenging to track the baseline.” When using the Waterfall approach, “should the customer reach out and ask for the current baseline, it would be easy to provide them the requirements. However, with user stories in agile projects, it's difficult to gather all user stories that have potentially changed in many ways over time. Other challenges faced are that customers are still requesting a Requirements Traceability Matrix (RTM), which is difficult to produce since the agile project teams are no longer tracking requirements in the way they were tracked and documented when using the waterfall approach. Additionally, a test repository was previously linked to all requirements.” The interviewee indicated that “teams are trying to resolve how to streamline and automate between the multiple tools, and are trying to work it internally first before involving the customer.” Along this line, another project team member mentioned that the testers need to begin adapting to agile by becoming deeply involved with the day-to-day tasks of the development team. He stated, “Agile teams are shifting to test automation and continuous integration practices in order to keep up with the constantly evolving agile project speed. The ISA program is already looking to transition the program to automated testing, which will greatly support the overall transition to agile methodology.” Another challenge was the integration of the agile approach with the monitoring and controlling processes, especially the integration with Earned Value Management (EVM). For the ISA program, reporting EVM is a contractual requirement for projects bearing significant cost, risk, visibility, etc. According to most of the interviewees, closing the gap between working agile projects and reporting EVM is a challenge. One of the project managers stated that “Earned Value [EVM] and agile have opposing objectives and these need to be balanced and reconciled. Agile holds staff levels constant at the Sprint team level and allows scope to vary from sprint to sprint. EVM assumes scope is a constant and staff levels are allowed to vary. New units of measure such as Epics, Features and Stories need a historical basis to be useful for estimation. The program has no history so estimation is still an inchoate discipline.” Furthermore, he noted the contractual constrains of not being able to remove EVM from the program reporting; therefore, project teams will need to learn and adapt in order to apply EVM to agile. Other interviewees similarly mentioned that it is necessary to improve the EVM reporting and find a way to tie agile to EVM. One project manager mentioned an attempt to track earned value on agile projects but recognized that more work is needed. One of the interviewees mentioned that projects will report earned value “based on the features and the time frame planned.” The ISA program has adapted an approach that utilizes the breakout of the stories and associated story points planned for a specific feature based work package. This breakout is the basis for calculating Budgeted Cost of Work Performed (BCWP) on the work package. Actual Percent Complete (APC) is the ratio between the total story points completed and the total story points planned. This value is then applied to the work package Budget at Completion (BAC) to determine the earned value. This approach is similar to what proposed in the literature, see e.g., (Sulaiman, Barton, & Blackburn, 2006). Overall, the interviewees agreed on the challenges relating to the integration of the agile approach with the standard processes, tools, and techniques that were required by the contract; they acknowledged that the ISA program was on its way to working out the challenges. They recognized that agile excels in a collaborative setting. It is designed to exemplify each team member's unique strengths, capitalizing on each individual, allowing them to grow and succeed. Processes should be tailored and adapted to the strengths of each team member, and to the project team as a whole. Agile principles take into account that people are more important than constraining processes, and it attempts to filter out as much unnecessary work as possible (Papatheocharous & Andreou, 2014). One of the project managers recommended the ISA program to look into the principles of agile and assess each one for the future of the program. One interviewee also mentioned that the ISA program “is also working to mature to the point where Release Train Engineers can be assigned to each division.” Such a role is important to the program. One of the team members stated, “Because many of our 9

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

Requirement tracking system

Successful transion to Agile

Report mechanism involving Earned Value Management

Fig. 5. Role of requirement tracking system and reporting mechanism in a successful transition to agile software development.

initiatives cross applications and cross portfolios, agile methods need supplemental processes to ensure requirements synthesis, systems engineering, communication and reporting of status and integration is efficient and effective across Scrum Teams.” This type of role could help streamline processes across the different divisions so that each agile project team is consistent in both processes and communication. Based on the research finding and the literature, we propose, “The integration of the agile project management approach, with the standard project management processes required as part of the government contract, is necessary for the successful implementation of agile.” Particularly, “the establishment of a tracking system to link agile deliverables and baseline requirements is significant to the successful transition” and “the establishment of a reporting mechanism in order to apply Earned Value Management to agile is significant to the successful transition to agile methods,” see Fig. 5.

4.2.6. Implementing automation Given the initiative to implement agile across the program, the leadership team supported an initiative to engage and utilize automation to help support the agile effort. We heard from the interviews that while some of the program team members recognized the benefits of automation, some of the members found it rather challenging to implement. One of the interviewees stated, “When you introduce automation as a framework, along with agile, you can significantly improve the delivery of IT capabilities.” He noted how one of the divisions was able to change from quarterly to monthly releases because of agile and automation working together and mentioned that agile and automation go “hand in hand.” The project teams can do one without the other, but they go very well together at a lower cost. As beneficial and cost savings as agile and automation can be together, there is still a large learning curve to adapting automation on the program. One of the team members felt that there are too many tools on the program to be efficient. “We need to reduce the number of disparate or overlapping tools and we need to integrate a few tools well to reach full efficiency.” Additionally, automation tasks require hours that need to be included as part of the estimated hours planned at the beginning of each sprint. However, the functional team members believe that scrum team members and customers have yet to understand the need and benefit of automation and then try to remove the automation-related tasks from sprints. The ISA program is moving forward with automation despite many challenges. According to the TechFAR Handbook (OMB, 2014), “automated testing is encouraged where the tester can write and run automated tests that are supplemented with exploratory testing…The contractor should be responsible for delivering automated tests, as well as functional code.” To ensure success, the TechFAR Handbook (OMB, 2014) recommends changing “the focus from document compliance to process quality evaluation. Automated test software can pick up defects so vulnerabilities are patched in development.” One of the managers mentioned that the ISA program should “leverage continuous integration to ensure a high standard of sustainable quality and speed of delivery.” Test automation will help shorten the release cycle by speeding up the regression test before each release, which helps the project remain cost effective (Dikert et al., 2016). The manager also recommended changing the way the program conducts development by automating testing, automating peer reviews, and automating test case development. There is a tool that automates the test case development from the requirements that builds into executable test cases. A test driven design takes the user story and puts it into a “process-able” language. The components would be parsed by a computer program to turn into a test case in the exact way the user story is written. According to our case informants and the literature, automation assists in “shortening the release cycle while maintaining product quality requires discipline” and “these types of tools can help the program claim success.” Based on the information above, we suggest, “the successful implementation of automation can significantly contribute to the success in agile implementation and development success,” see Fig. 6.

Successful transition to Agile

Implementation of test automation

Fig. 6. Role of test automation in a successful transition to agile software development. 10

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

5. Contributions and limitations This study was conducted to explore the key challenges and lessons learned an organization encountered during the transition from a traditional software development method to an agile software development method. The focus of this study is on a context of a large-scale government contracted program. Despite several limitations of this study, the research findings contribute to both literature and practice in many ways, listed below. This study highlights many challenges and lessons learned during the transition that can be grouped in six categories. They are the challenges and lessons learned regarding 1) managing changes, 2) accessibility to experts and training, 3) commitment of product owners, 4) documentation, 5) integration with standard processes, tools, and techniques, and 6) implementing test automation. These six categories of challenges are both support and extend the existing literature, e.g., (GAO, 2012; Ghobadi & Mathiassen, 2015; Heeager, 2012; Konnola et al., 2016), to the context of government-contracted programs that has not been much researched. Particularly, the findings extend the work of Dikert et al. (2016), conducted based on the literature review, by proving empirical evidences from a government contracted program. As a big picture, understanding these challenges should inspire researchers to further conduct studies in this context to promote a better understanding of these challenges and to develop an approach or process for seamless transition from traditional to agile software development methods. Several propositions for potential future research were highlighted and summarized in Figs. 1 to 6. Some of these propositions may be perceived as being straightforward, they however shed light on the important issues in the context of a government-contracted program, which team members come from different organizations of different background and organizational culture and have different mindset and levels of knowledge. These propositions should be further developed into research hypotheses to empirically explore the relationship between the key challenges and the successful transition. As showed in Figs. 1 to 6, some of these propositions can be developed into research hypotheses that explore the direct, mediation, and moderation effects of many critical factors on the successful transition to agile software development. In essence, the propositions highlight the importance of change management and the coordinated and collaborated trainings that involve both the contractors and the customers; the role of a designated champion during the transition; the early engagement of the customers (the product owner); the integration of required project and project management documents with agile process; the importance of the requirement tracking system and reporting mechanism involving Earned Value management; and the implementation of automation to facilitate testing. In addition to large-sample studies to investigate the relationship between the key challenges and the successful transition, future research can be conducted to explore the elements of each proposition specifically. The study highlights the need of mature change management process and the designated champion. It is worth investigation a change management process that can effectively implement changes both to the contractors and the customer organizations. It is important that such a process should implement changes in a coordinated way to promote seamless transition. Since the transition occurs both in the contractors' and customers' organizations, the identification of who the designated champion should be, including the characteristics, roles, and responsibilities of the champion should be further investigated. The study focusing on identifying the training necessary for successful transition is also worth pursuing. This includes the approach used, the frequency, and the contents. Our study highlights the coordinated and collaborated training for both the contractors and the customers such that they have compliment knowledge, skill, and a better understanding of different roles and responsibilities. Extending the works of e.g., Dikert et al. (2016) and Heeager (2012), studies should be commenced to investigate the training contents that are suitable for different parties that have different levels of knowledge and degrees of engagement in software development programs. Using the work of Ghobadi and Mathiassen (2015) as a foundation, future research should also be conducted to explore the knowledge sharing among different parties, including the role and impact of agile experts. This study highlights the early engagement of the product owner (government personnel) in the agile transition. As indicated, it is not only the contractors who have significant roles and responsibilities during the transition, the early involvement of the product owners also helps increase the level of commitment of the product owners later on in the process. Since not much research has been conducted in this area, this proposition indicates an opportunity to further study an approach for effectively engaging the product owners early on. The level of engagement necessary at any given time for successful transition should also be researched. The finding of this study also highlights the need for the integration between agile approach and standard processes, tools, and techniques, including documentation and deliverables that have been established in an organization and especially as part of the contract obligation. Future studies in this area is worth pursuing and will extend the work of Heeager (2012), Jahr (2014), and Konnola et al. (2016). The further development and validation of an approach, including tools and techniques, for utilizing Earned Value Management in an agile environment is needed. Additionally, the finding of this study indicates an important role of automation in agile software development. An approach for effectively introducing automation during the transition should be studied. A comparative study on the impact of automation in an agile environment should provide significant contribution to both literature and practices. As for practitioners, the information from the case study should provide some insights into the transition to agile methods based on a real-life context. Several stories and examples can be drawn from the case study to raise an awareness of what a transition entails. In addition to being aware of six categories of challenges indicated in this study, practitioners can benefit from the lessons learned or recommendations found in this study that can be grouped into three themes— transition process, personnel, and development processes. To effectively transition from a traditional software development approach to an agile methodology in a government-contracted program, it is important that such a transition is implemented as a change initiative both in the contractors and the government organizations. To address an issue such as resistant to change, the transition should follow a mature change management process. Having a dedicated champion is also necessary. In terms of personnel, proper training and education are necessary 11

Journal of High Technology Management Research xxx (xxxx) xxx–xxx

P. Patanakul, R. Rufo-McCarron

for both contractors and government personnel. Even though different levels of trainings may be necessary depending on the level of involvement of the stakeholders, those trainings must be coordinated and should be done in a collaborative manner. Mentoring systems should also be established. Agile experts should be identified and provide the accessibility to the program teams. It is necessary that the product owners are identified and engage in the transition early on. They must understand their important roles and responsibilities in software development. Proper training and education, as indicated above, should help establish the understanding of roles and responsibilities, and in turn help secure the commitment of the product owners. As for the development process, the focus should be on the integration between agile approach with the standard processes, tools, and techniques, including documentation and deliverables that are necessary as parts of the contractual agreement. Even though documentation is not a focus of agile, proper integration of project and project management documents and deliverables with the agile process is necessary. As indicated by the research participants, among other processes, it is important to establish a requirement tracking system to associate agile deliverables with the baseline requirements. A monitoring system to keep track of the development progress to be able to apply the information to Earned Value Management should also be established. It is important to note that in order to maintain the level of agility, the standard requirements for documentation and deliverables, including the use of certain processes and techniques, may have to be adjusted. Despites its contributions to literature and practices, this study has some limitations. It was conducted within a specific context— a government-contracted program. Even though it is important to put a specific research focus on this type of software development programs because of the increase in transition to agile methodologies, the findings of this study may be specific to this context. In addition, one specific program was studied. Even though a case study research was utilized and the information from multiple respondents with different roles, responsibilities, and managerial levels in the program was studied, such information could be program specific. To alleviate the limitations mentioned above, literature was used for triangulation and validation of the research findings. In any case, practitioners should use a contingency approach when utilizing the research findings. In other words, practitioners are advised to adapt the research findings contingently to their organization's context. References Ben-David, A., Gelbard, R., & Milstein, I. (2012). Supplier ranking by multi-alternative proposal analysis for agile projects. International Journal of Project Management, 30, 723–730. Boehm, B., & Turner, R. (2005). Management challenges to implementing agile process in traditional development organizations. IEEE Software, 30–39 September/ October. Braude, E. J., & Bernstein, M. E. (2011). Software engineering modern approaches (Second ed.). Hoboken NJ: Wiley. Broadus, W. (2013). The challenges of being agile in DOD. Defense AT&L, 42, 4–6. Burba, D. (2015). When agile meets auditor, PM network. 60–67. Cohn, M. (2010). Succedding with agile: Software development using scrum. Pearson Education. Cottmeyer, M., & Stevens, D. (2015). Kanban for agile teams. Whitepaper ed, Versionone.com1–14. Crowder, J. A., & Friess, S. (2015). Agile project management: Managing for success. Switzerland: Springer. Dikert, K., Paasivaara, M., & Lassenius, C. (2016). Challenges and success factors for large-scale agile transformations: A systematic literature review. Journal of Systems and Software, 119, 87–108. Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14, 532–550. Fairley, R. E. (2009). Managing and leading software projects Wiley, Hoboken NJ. GAO (2012). Effective practices and federal challenges in applying agile methods, software development. Washington DC: United States Governement Accountability Office. Ghobadi, S., & Mathiassen, L. (2015). Perceived barriers to effective knowledge sharing in agile software teams. Information Systems Journal, 26, 95–125. Heeager, L. T. (2012). Introducing agile practices in a documentation-driven software development practice: A case study. Journal of Information Technology Case and Application Research, 14, 3–24. Heusser, M. (2015). Introducing the scaled agile framework. CIO. Jahr, M. (2014). A hybrid approach to quantitative software project scheduling within agile framework. Project Management Journal, 45, 35–45. Jalali, S., & Wohlin, C. (2012). Global software engineering and agile practices: A systematic review. Software: Evolution and Process, 24, 643–659. Jorgensen, M. (2016). A survey on the characteristics of projects with success in delivering client benefits. Information and Software Technology, 78, 83–94. Konnola, K., Suomi, S., Makila, T., Jokela, T., Rantala, V., & Lehtonen, T. (2016). Agile methods in embedded system development: Multi-case study of three industrial cases. Journal of Systems and Software, 118, 134–150. Nurdiani, I., Borstler, J., & Fricker, S. A. (2016). The impacts of agile and lean practices on project constranits: A tertiary study. Journal of Systems and Software, 119, 162–183. OMB (2014). The TechFAR handbook for procuring digital services using agile processes. Office of Management and Budget. Papatheocharous, E., & Andreou, A. S. (2014). Empirical evidence and state of practice of software agile teams. Journal of Software: Evolution and Process, 26, 855–866. Patanakul, P., Henry, J., & Leach, J. A. (2016). Agile project execution. In R. J. Martinelli, & D. Z. Milosevic (Eds.). Project management toolbox (pp. 301–322). Hoboken, NJ: John Wiley & Sons. Rola, P., Kuchta, D., & Kopczyk, D. (2016). Conceptual model of working space for agile (Scrum) project team. Journal of Systems and Software, 118, 49–63. Rubin, K. S. (2012). Essential scrum: A practical guide to the most popular agile process. Pearson education. Ann Arbor: MI. Schwaber, K. (2004). Agile project management with scrum. (Microsoft). Selic, B. (2009). Agile documentation, anyone? IEEE Software, 26, 11–12. Serrador, P., & Pinto, J. K. (2015). Does agile work? A quantitative analysis of agile project success. International Journal of Project Management, 33, 1040–1051. Sulaiman, T., Barton, B., & Blackburn, T. (2006). AgileEVM-earned value management in Scrum projects. In J. Chao, M. Cohn, F. Maurer, H. Sharp, & J. Shore (Eds.). Agile 2006 conference. IEEE (pp. 10). Minneapolis: Minnesota (July). In Agile Conference, 2006 (pp. 10-pp). IEEE. Suscheck, C. A., & Ford, R. (2008). Jazz improvisation as a learning metaphor for the scrum software development methodology. Software Process: Improvement and Practice, 13, 439–450. Versionone (2016). What is Kanban? Kanban Software Tool. Wysocki, R. K. (2006). Effective softwar project management. Hoboken NJ: Wiley.

12