- Email: [email protected]
A Bayesian networks-based approach to assess and improve the teamwork quality of agile teams
Accepted Manuscript
A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams Arthur Freire, Mirko Perkusich, Renata Saraiva, Hyggo Almeida, Angelo Perkusich PII: DOI: Reference:
S0950-5849(17)30020-4 10.1016/j.infsof.2018.04.004 INFSOF 5980
To appear in:
Information and Software Technology
Received date: Revised date: Accepted date:
11 January 2017 8 March 2018 13 April 2018
Please cite this article as: Arthur Freire, Mirko Perkusich, Renata Saraiva, Hyggo Almeida, Angelo Perkusich, A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams, Information and Software Technology (2018), doi: 10.1016/j.infsof.2018.04.004
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
highlights • We present a Bayesian network-based model to assess the quality of agile
CR IP T
teamwork; • A procedure to use and adapt the model is presented;
• The model practical utility and the procedure were evaluated with a case study executed in three Scrum teams;
• The usage of both model and procedure assists on the continuous improve-
AC
CE
PT
ED
M
AN US
ment of the teamwork quality with positive cost-benefit.
1
ACCEPTED MANUSCRIPT
A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams
CR IP T
Arthur Freirea , Mirko Perkusichb , Renata Saraivaa , Hyggo Almeidaa , Angelo Perkusichc a Department
AN US
of Computing and Systems, Federal University of Campina Grande, Campina Grande, PB, Brazil b Federal Institute of Paraiba, Monteiro, PB, Brazil c Department of Electrical Engineering, Federal University of Campina Grande, Campina Grande, PB, Brazil
Abstract
CONTEXT: According to the agile principles and values, as well as recent research articles, teamwork factors are critical to achieve success in agile projects. However, teamwork does not automatically arise. There are some existing instruments with the purpose of assessing the teamwork quality based on
M
Structural Equation Modeling (i.e., empirically derived) and Radar Plots, but they may not be useful in a concrete situation because these techniques are not
ED
advised for prediction and diagnosis purposes. OBJECTIVE: Analytically derive a Bayesian network model based on a literature review and a practitioner’s knowledge; and to assess its practical
PT
utility through a case study.
METHOD: To build the model, we executed a top-down approach using data
CE
collected through a literature review and a domain practitioner. We assessed the model with a case study executed in three Scrum teams. RESULTS: Given the context of the case study, the model assists agile teams
AC
on assessing teamwork quality and identifying improvement opportunities, is easy to learn, and the cost-benefit for using it with the proposed procedure is Email addresses: [email protected] (Arthur Freire), [email protected] (Mirko Perkusich), [email protected] (Renata Saraiva), [email protected] (Hyggo Almeida), [email protected] (Angelo Perkusich)
Preprint submitted to Information and Software Technology
April 13, 2018
ACCEPTED MANUSCRIPT
positive. CONCLUSION: We concluded that we achieved promising results with the presented solution. However, it needs more evaluation and validation to gener-
CR IP T
alize the obtained results. Keywords: Teamwork, Agile Teams, Bayesian Network, Assessment, Measurement, Improvement
1. Introduction
Agile software development consists of a lightweight approach to deliver
AN US
software continuously and has recently become popular in the industry. It is an
alternative approach to plan-driven development, in which the project plan and 5
system architecture accommodate foreseeable change with the goal of achieving predictability, stability and high assurance. Conversely, agile methods are reactive with the goal of achieving rapid value and responsiveness to change.
or environment changes [1].
M
Therefore, they are a better fit in contexts with rapid marketplace, technology
To achieve their goals, agile methods consider individuals and interactions as
10
ED
more important than processes and tools [2]. This is reflected on its principles: six out of the twelve principles are related to the individuals involved in the
PT
product development. These principles focus on factors such as team motivation, communication, collaboration and team management. This is also reflected by 15
popular agile practices such as Daily Stand-up Meetings [3], Pair Programming
CE
[4] and Planning Poker [5]. According to the Agile Manifesto [2], the best architecture, requirements and
design emerge from self-organized teams. For this purpose, it is necessary that
AC
team members collaborate to embrace the concept of whole-team responsibility
20
and commitment. Katzenbach & Smith [6] define team as a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually accountable.
3
ACCEPTED MANUSCRIPT
Several researchers [7, 8, 9, 10, 11] show that teamwork factors are critical to 25
achieve success in agile projects. On the other hand, good quality teamwork does not automatically arise [12]. In this paper, the concept of teamwork is defined
CR IP T
as: the capability of team members to work in a synchronous and cohesive way, keeping the team’s goals as priority, without unwanted interference of an outside agent (e.g. client or upper management) on the executed tasks.
In the literature, there are instruments with the purpose of assessing team-
30
work quality (TWQ) [13, 14, 15]. Amengual et al. [13] focused on ISO/IEC
15504 (i.e., SPICE); Hoegl & Gemuenden [14], on teams working on innovative
AN US
projects; and Moe et al. [15], on agile teams. However, there is still a need
for more research efforts be made to validate the agile TWQ construct and to 35
advance the measurement of team performance [16].
In this paper, we complement the current state-of-art by: (i) presenting a list of agile teamwork key factors identified on a literature review; (ii) constructing an analytically derived Bayesian networks based on the literature review and a
40
M
practitioners knowledge; and (iii) performing a case study to assess the practical utility of the proposed solution.
ED
We used Bayesian networks to model the causal relationships between the key factors of teamwork in agile teams and to handle uncertainty in the measurements and relationships [17]. The context of our problem is subject to
45
PT
randomness (i.e., uncertain future outcome) and incompleteness, which results in considering techniques for uncertainty reasoning such as fuzzy logic, neural
CE
networks, Bayesian networks, Dempster-Shafer, possibility theory and Kalman filters. The lack of available data invalidates the usage of a data-based approach such as Kalman filters, neural networks or any statistical model, but motivates
AC
the usage of knowledge-based approaches. More specifically, in our context, we
50
need to handle causal reasoning, adaptability, clarity of inference and support decision-making. According to Verbert et al. [18], Bayesian networks is the most suitable approach for this purpose. Researchers have applied this technique in software engineering for several purposes such as process improvement [19], productivity prediction of eXtreme 4
ACCEPTED MANUSCRIPT
55
programming (XP) teams productivity [20], risk management [21] and product quality prediction [22, 23]. A first version of the model was is presented in [24]. In [24], the model is
CR IP T
based primarily on factors extracted from two papers and it was not evaluated in real projects. Now, we increment our research by extracting the teamwork 60
key factors from a literature review, and evaluating the model on a case study with three software development teams.
The research method applied is composed of three main activities: (i) per-
form a literature review to identify the factors that influence the quality of agile
65
AN US
teamwork; (ii) construct the Bayesian network model and define the procedure
to use it; and (iii) perform a case study to assess the practical utility of the model and evaluate the proposed procedure for using the model. To build the Bayesian network, we executed a top-down approach to build the model according to knowledge extracted from both a practitioner and the literature. The practitioner has over five years of experience with Scrum projects, holds an MBA on project management and certifications from Scrum Alliance1 .
M
70
To use the model, we propose to apply a procedure that consists of four stages.
ED
This procedure is based on the procedure presented by Perkusich et al. [19]. However, we performed some adaptations on it to fit the context of our problem. We evaluated both model and procedure with a case study applied to three Scrum-based software development teams at Virtus 2 . For each team, we col-
PT
75
lected data from its leader and documentation. The goal of the study was to
CE
evaluate the practical utility of the proposed solution. According to the case study results, the model assists agile teams on assessing
teamwork quality and identifying improvement opportunities, is easy to learn, and the cost-benefit of using it with the presented procedure is positive. Given
AC
80
the case study results, we concluded that we achieved promising results with the proposed solution, but it needs more evaluation and validation. 1 www.scrumalliance.org 2 http://www.virtus.ufcg.edu.br
5
ACCEPTED MANUSCRIPT
In what follows, in Section 2, we present an overview on Bayesian networks and justify its use; in Section 3, we present background on agile teamwork; 85
in Section 4, we present results of a literature review to identify factors to
CR IP T
assess the teamwork quality of agile teams; in Section 5, we present the details regarding the model construction; in Section 6, we present the procedure to use the proposed model; in Section 7, we present the results of the case study; finally, in Section 8, we present our conclusions including our work’s current 90
limitations and future works.
AN US
2. Bayesian Networks Overview
Bayesian networks belong to the family of probabilistic graph models and are used to represent knowledge about an uncertain domain [17]. A Bayesian network, B, is a directed acyclic graph that represents a joint probability dis95
tribution over a set of random variables V [25]. The network is defined by the
M
pair B = {G, Θ}. G is the directed acyclic graph in which the nodes X1 , . . . , Xn represent random variables and the arcs represent the direct dependencies between these variables. Θ represents the set of the probability functions. This set
100
ED
contains the parameter θxi |πi = PB (xi |πi ) for each xi in Xi conditioned by πi , the set of the parameters of Xi in G. Equation 1 presents the joint distribution
PT
defined by B over V .
CE
PB (X1 , . . . , Xn ) =
n Y
i=1
PB (xi |πi ) =
n Y
i=1
θXi |πi
(1)
We present an example of a Bayesian network in Figure 1. Circles represent
the nodes and arrows represent the arcs. The probability functions are usually
AC
represented as node probability tables (NPT). Even though the arcs represent
105
the casual connection’s direction between the variables, the information can propagate in any direction [26]. In many cases, there is not enough data to define the NPT and it is necessary
to collect them from domain experts. To reduce this effort, Fenton et al. [27] presented the concept of ranked nodes, which are based on the doubly truncated 6
ACCEPTED MANUSCRIPT
X2
X1 T
F
0.7
0.3
T
F
0.2
0.8
CR IP T
MATURE TESTING PROCESS
AUTOMATED TESTS
X3
X2
F
F
F
T
T
F
T
T
T
F
0.1
0.9
0.7
0.3
0.4
0.6
0.9
0.1
AN US
STABLE RELEASE VERSION
X1
Figure 1: A Bayesian network example [24], used by permission.
110
Normal distribution (TNormal) limited in the [0, 1] region. This distribution is
M
based on four parameters: µ, mean (i.e., central tendency); σ 2 , variance (i.e., confidence in the results); a, lower bound (i.e., 0); and, b, upper bound (i.e.,
ED
1). This distribution enables us to model a variety of shapes (i.e., relationships) such as a uniform distribution, achieved when σ 2 → ∞, and highly skewed
distributions, achieved when σ 2 → 0. We show TNormal examples in Figure 2.
AC
CE
PT
115
µ = 0.5 σ2=1 a=0 b=1
µ = 0.5 σ 2 = 0.01 a=0 b=1
µ = 0.2 σ 2 = 0.1 a=0 b=1
Figure 2: TNormal examples.
In this approach, µ is defined by a weighted function of the parent nodes.
7
ACCEPTED MANUSCRIPT
There are four weighted functions: weighted mean (WMEAN), weighted minimum (WMIN), weighted maximum (WMAX) and the mixture of WMIN and WMAX (MIXMINMAX). According to the authors, these functions are enough to represent the types of relationship necessary for defining NPT. In Figure 3,
CR IP T
120
we show examples of NPT calculated with these functions.
In the examples presented in Figure 3, WMEAN and MIXMINMAX have the same values. The difference between the two functions is that WMEAN
calculates the weighted mean of the parent nodes values (i.e., the weights are 125
set for the parent nodes) and MIXMINMAX mixes the minimum and maximum
AN US
functions (i.e., the weights are set for the functions).
WMIN
M
WMEAN
MIXMINMAX
AC
CE
PT
ED
WMAX
Figure 3: Weighted functions examples.
To define which function should be used, the model developer must perform
”what if” analysis with the expert by defining questions and collecting answers to define a truth table [27]. The model developer must analyze the answers and
130
define the most appropriate function. The variance is defined empirically, and 8
ACCEPTED MANUSCRIPT
it should reflect the confidence of the domain expert in the results [27]. We show an example of questions and answers in Table 1, in which the node C has two parents, A and B. In the example shown, the most appropriate function
135
CR IP T
is WMIN. We asked the domain expert to, for the questions asked, define the expected probabilities for each state, based on the parent nodes states. Given
this, since Teamwork depends on Cohesion and Team Autonomy, we present the calibration results for Teamwork in Table 4. Table 1: Calibration example.
B
C
Very High
Very High
Very High
Very Low
Very Low
Very Low
Very Low
Very High
Low
Very High
Very Low
Low
M
3. Background on teamwork
AN US
A
3.1. Teamwork in agile software development
ED
Even though so much of some agile methodologies assume that collaborative
140
behavior will happen, so the teamwork quality can be increased, as stated in [12],
PT
teamwork does not automatically arise. Because of the different characteristics between agile and plan-driven development, teamwork in agile contexts requires much more than each team member doing its part of the work, and making sure everything piece of the puzzle will be integrated to work together as whole.
CE
145
Teamwork in agile requires a bigger sense of collective ownership, in which
every member can replace another without major issues, and shared leadership
AC
really exists. Moreover, the agile practices provide opportunities for team members to collaborate with each member and increase communication. As stated
150
in Section 1, in this paper, we defined agile teamwork as: the capability of team members to work in a synchronous and cohesive way, keeping the team’s goals as priority, without unwanted interference of an outside agent (e.g. client or
9
ACCEPTED MANUSCRIPT
upper management) on the executed tasks. In this Section, we give an overview of teamwork key factors, challenges, and importance. Chow & Cao [7] analyzed data collected through a survey from 109 agile
155
CR IP T
projects from 25 countries around the world and identified Team Capability as a factor that impacts the time and cost of agile projects. These finding agree
with the fifth principle of the Agile Manifesto [2], which is to build projects around motivated individuals. However, Stankovic et al. [28] showed data 160
invalidating the conclusions of Chow & Cao [7], but their study is limited to former Yugoslavia IT companies.
AN US
In a case study in one company, Moe et al. [8] show that problems with team orientation, team leadership and coordination in addition to highly specialized skills and corresponding division of work are important barriers for achieving 165
team effectiveness. Fontana et al. [10] present results of an exploratory study in which they concluded that, in agile context, maturity means fostering more subjective capabilities such as collaboration, communication, commitment, care,
M
sharing and self-organization. Williams et al. [11] presented a tool to measure the agility of companies and consider teamwork factors (e.g., team composition) as one of the dimensions to be evaluated.
ED
170
In Lindsjørn et al. [16], the authors present result of applying Structural Equation Modeling (SEM) to evaluate the results of a survey with 477 respon-
PT
dents from 26 companies. They concluded that TWQ significantly affects team performance, when rated by team members and leaders, and the team members’ success. SEM is a widely used method for quantifying and evaluating an as-
CE
175
sumed causal process. Therefore, its primary goal is to evaluate if a theoretical network is a reasonable representation of the process that generated the study
AC
data [29]. Since this study’s goal is to assess agile teamwork quality and assist the team on predicting and diagnosing issues (i.e, risks) within the team, we
180
used Bayesian networks. In the literature, there are studies proposing the usage of frameworks based
on assessments to assist on the adoption (i.e., usage) of agile methods such as [30] and [31]. Qumer & Henderson-Sellers [30] presented a framework to 10
ACCEPTED MANUSCRIPT
support the evaluation, adoption and improvement of agile methods to guide the 185
behavior of self-organizing and empowered agile teams. Gandomani & Nafchi [31] presented a framework based on PDCA (Plan-Do-Check-Act cycle) to adopt
CR IP T
agile methods. According to the authors, assessments are important during the adoption (i.e., transition) process, because it helps to find the challenges of agile teams and monitor the processes and progresses of the transition. However, none 190
of these solutions present an instrument to assess teamwork quality.
According to Salo & Abrahamsson [32], since the focus of agile methods is different than plan-driven development focus, there is a need of new software
AN US
process improvement (SPI) (i.e., initiatives that can be used in software organizations to mature their operations [33]) mechanisms for them. In addition, 195
after executing an action research in two companies, Ringstad et al. [9] concluded that the use of diagnosis and action planning to improve teamwork in agile projects improves shared leadership, team orientation and learning. Therefore, there is a need for instruments to assess agile teamwork. These
200
M
instruments can be used to identify improvement opportunities and support the agile principle “at regular intervals, the team reflects on how to become
ED
more effective, then tune and adjusts its behavior accordingly” [2]. The results of the instruments would guide the improvement of agile teams and would be especially beneficial for immature teams (i.e., organizations). Moreover, since
205
PT
these instruments help on the TWQ improvement, they can be considered new SPI mechanisms for agile methods.
CE
3.2. Existing instruments to measure the TWQ of software teams Amengual et al. [13] state that software engineering is currently paying
AC
special attention to cooperative and human aspects of software development. Given this statement, the authors present a SPICE-based Teamwork Assessment
210
Model for software teams. This assessment model is based upon a Teamwork Reference Model and a Measurement Framework presented, respectively, in [34] and [35]. According to the authors, in the context of software development, before publishing their work, there did not exist a common framework that 11
ACCEPTED MANUSCRIPT
could be used as reference for teamwork assessment in software teams. So, 215
the authors expected their model to be used as a reference framework for the development of new software tools for the simulation of the behavior of software
CR IP T
teams. However, their model does not relate the teamwork key factors to output the current teamwork status. The users are required to respond to questions regarding the teamwork key factors and analyze their own answers. In addition 220
to that, since this model does not focus on agile teams, it does not address important agile teamwork factors such as Self-Organizing and Redundancy.
Hoegl & Gemuenden [14] present a comprehensive concept of the collabora-
AN US
tion in teams called Teamwork Quality (TWQ). This construct has six facets
(i.e., Communication, Coordination, Balance of Member Contributions, Mutual 225
Support, Effort, and Cohesion). Based on these facets and on data collected in this study, the authors propose a way for measuring the TWQ through a linear regression model, where the high order factor (i.e., TWQ) is the dependent variable, and the construct facets are the independent variable. According
230
M
to the authors, the TWQ construct provides a comprehensive measure of the collaborative team-task process focusing on the quality of interactions rather
ED
than on activities in teams. In addition to that, to avoid bias, the data was collected from multiple team members. However, in the context of this paper, this instrument is limited because it only measures the teams’ degree of Collab-
235
PT
oration. So, it does not address some important agile teamwork factors such as Self-Organizing and Team Autonomy.
CE
Moe et al. [15] propose an instrument that addresses key concerns and characteristics of agile teamwork and presents them along five dimensions: Shared
Leadership, Team Orientation, Redundancy, Learning, and Autonomy. The in-
AC
strument outputs a radar plot of the status of the teamwork. An expert group
240
to whom the model was presented, composed of 35 people, found the model useful for understanding team problems such as the team agreeing on using test-driven development, but not applying it in practice; the team having no control over the members assigned to the team; and management not trusting the team. According to the authors, the instrument gives researches and de12
ACCEPTED MANUSCRIPT
245
velopers a common language for discussing teamwork. To assess the teamwork current status, it is necessary to answer a set of questions for each dimension, and, based on these answers, assign a score on a scale from 0 to 10 for the
CR IP T
dimension. In our prior work [24], we have proposed a Bayesian networks-based model 250
to assess agile teams’ teamwork quality. Since most of the model nodes are subjective, we used Bayesian networks given its capability to handle uncertainty [17]. Although our prior work has yielded several promising results, there are
AN US
some limitations with respect to the objective of our current work:
• It was built only based on the factors presented in [14, 15], and evolved with the help of practitioners. So, important agile teamwork key factors
255
are missing;
• The probability functions of the model were defined using only the one of
M
the four possible functions presented in [27]; • It was not evaluated in real projects; • It lacks a process to assist on its usage.
ED
260
In this paper, we seek to eliminate these threats by proposing a new model,
PT
presented in Section 5, and a procedure to apply it to agile projects, presented in Section 6. Furthermore, we applied both model and procedure to real projects
CE
through a case study as presented in Section 7.
265
4. Literature Review
AC
To perform the literature review, we executed a process based on the Guide-
lines for performing Systematic Literature Reviews in Software Engineering [36]. The main goal of this review is to identify the as many key factors that influence the quality of the Teamwork in agile teams as possible. Due to this fact, we
270
perceived it to be unnecessary to assess the quality of the papers. Moreover,
13
ACCEPTED MANUSCRIPT
the selection and extraction steps were performed by only one person, but the aggregation of the results was performed by three people. This process was composed of four major steps: (i) planning; (ii) search
275
CR IP T
strategy definition; (iii) study selection; and (iv) study extraction. In step (i), we defined the following research question: “What key factors influence the quality
of the Teamwork in agile teams?”. Then, we defined the inclusion/exclusion criteria as well as the study selection and extraction strategy. In step (ii), we
defined the search engines to perform the searching for papers, the search string, and extracted the .bib files to import all papers information to the StArt tool3 .
In steps (iii) and (iv), we applied the inclusion/exclusion criteria according to the search strategy defined in (i).
AN US
280
For brevity, in this paper, we briefly present details regarding the literature review process. It was based on automatic search on digital libraries, namely: ACM, IEEEXplore, Scopus, ScienceDirect and Google Scholar. We defined a 285
search string based on the keywords agile, teamwork, team and factor, and the
M
considered period of publication was from 2009 to 2015. To select the studies, we analyzed the title, abstract and keywords and included the ones that were in
ED
the scope of agile teamwork. Afterwards, for the remaining studies, we analyzed the entire text and selected papers that presented, at least, one factor to assess 290
agile teamwork or identified any relationship between teamwork factors. At the
PT
end of the process, we extracted 20 agile teamwork key factors from 15 studies.
CE
We present the extracted factors in Table 2.
AC
Factor
Communication
Table 2: Agile Teamwork Key Factors. Definition Information sharing across team members.
Reference [8] [14] [37] [38] [39] [40] [41] [42]
3 http://lapes.dc.ufscar.br/tools/start_tool
14
ACCEPTED MANUSCRIPT
Coordination
Refers to the tasks execution by team mem-
[8] [14] [37] [39] [41]
bers in a synchronized and integrated man-
Cohesion
CR IP T
ner.
Interpersonal attraction between the team
[14] [37] [39]
members, their commitment to the team
tasks, to the team itself, and group pride spirit. Trust
The vulnerability from one part to another
[37] [39] [40] [43]
AN US
based on the expectation that the other will execute a given action, regardless the monitoring or controlling capacity. Collaboration
Refers to the commitment that the team
[14] [37] [40] [42]
members have with each other to achieve the common goals.
The team members share the same values
M
Value Diversity
[37]
ED
and goals. Shared Leadership
The authority on the decision making pro-
[41] [44]
cess is shared.
CE
PT
Team Orientation
Redundancy
AC
Team Autonomy
Team Learning
[8] [9] [15] [39]
Refers to the respect that the team members have with each other
[8] [9] [15] [38] [41] [44] [45]
Team members capacity to replace each other on the tasks execution without the
[8] [15] [38] [44]
need of training. Refers to the influence external agents have
[9] [15] [41]
on the team tasks execution. The team’s capability to identify changes on its environment and adjust its strategies as necessary.
15
[9] [15] [39] [41] [44] [45]
ACCEPTED MANUSCRIPT
Monitoring
Team synchronization regarding the tasks
[8] [41] [44]
and barriers.
Feedback
CR IP T
Refers to the act of providing, forwarding, and receiving information about the team members’ performance.
Set of experiences, comprehensions, and
Culture
[8] [41]
[38]
shared meanings among the team members. Team members personalities.
Team Distribution
The team’s physical distribution.
Team Size
The quantity of members of a team.
AN US
Personality
[38] [39] [46] [46] [46]
The team members’ capacity to contribute Balance of Mem-
with all their necessary knowledge for the
bers Contribution
development of the tasks.
[14]
Workload sharing and team tasks
M
prioritization according to the others in Effort
relation to other obligations are indicators
[14]
ED
of the team members’ effort to execute the common tasks. Team members’ motivation to execute the
[47]
team tasks and work together.
CE
PT
Motivation
AC
5. Proposed Model
295
In this paper, we propose a Bayesian networks-based model to assess the
teamwork quality of agile teams. To build the model, we used the factors presented in Table 2 as a foundation. However, since there are many different agile methods, it is not possible to build a model for all cases due to the differences between these methods and the context of the projects. Therefore, the model
16
ACCEPTED MANUSCRIPT
300
presented in this paper is generic, focused on collocated teams. Moreover, the proposed model is adaptable to fit a given team’s context (e.g., a geographically distributed team or a team that uses Pair Programming), as presented
CR IP T
in Section 6.2. On the other hand, we attempted to minimize the adaptation effort.
We decided to use Bayesian networks due its capability to handle uncertainty
305
[17], formally represent knowledge and the easiness to model and quantify the relationships between the key factors that have impact on the teamwork quality.
Furthermore, it has adequate tool support. We used AgenaRisk 4 , due to its
AN US
support to ranked nodes.
The Bayesian network construction can be divided in two steps: the directed
310
acyclic graph (DAG) construction and the probability functions definition. Each node of the DAG represents a TWQ key factor and there is an edge between two nodes whenever they relate to each other. For each edge, the influenced TWQ key factor on the relationship is the edge’s endpoint. Furthermore, each key factor has possible states and each one of them has an associated probabil-
M
315
ity. Given that, as presented in [48], each key factor represents a set of tuples
ED
N = {(s1 , p1 ), . . . , (s|N | , p|N | )}, where si is a possible state and pi is the associated probability. Also, we have the set of key factors F = {N1 , . . . , N|F | }. Furthermore, we have the set of edges R = {(Nj , Nk ) | Nj ⊂ F ∧ Nk ⊂ F }, where Nj is the initial point and Nk is the endpoint of the edge. Therefore, the
PT
320
problem is to find all elements of the sets F and R. To find all elements of the
CE
set F , we need to identify all key factors Na , and for each key factor Na , find all the possible states si and associated probability pi , where a ≤ |F | and i ≤ |Na |.
Finally, to find all elements of the set R, we need to identify all fj and fk where
fj and fk ∈ F .
AC
325
According to Perkusich et al. [48], a probability function can be represented
by P r = {A|B}, where A is a dependent variable and B is a set of parent nodes.
Thus, the set P = {P ri , . . . , P r|P | } represents all the probability functions for 4 http://www.agena.co.uk/
17
ACCEPTED MANUSCRIPT
the Bayesian network. So, the problem is to find all the elements of the set 330
P . To find them, for each P ri where 1 ≤ i ≤ |P |, we need to quantify the relationship between A and Bj , where 1 ≤ j ≤ |B| and Bj ∈ B.
CR IP T
We interviewed an agile practitioner to help on the process of building the model. Since the construction process was divided, we interviewed the practitioner according to the two phases of this process: an interview to build the 335
DAG, in which the practitioner was responsible for deriving some relationships between the factors; and other one to define the probability functions. Be-
sides the knowledge on agile methods, the practitioner also has knowledge on
AN US
Bayesian networks, which was a key factor to achieve success in building the model.
The practitioner is a certified Scrum Master and Product Owner (PO) by
340
the Scrum Alliance and has a MBA degree in project management. Moreover, he has over five years working as team leader in agile software development
5.1. DAG Construction
M
projects.
To define the DAG, we broke down the problem into two sub-problems:
ED
345
identify the elements of F and R, and identify the elements of N . On the first sub-problem we are only interested on TWQ factors and their relationship. On
PT
the second sub-problem, we identify the possible states and their probabilities for each node on the DAG. We used a top-down approach to, along with the practitioner, decompose
350
CE
the top-level node, Teamwork, into factors that we judged to be observable. To decompose the nodes, we used reasoning and data extracted from the literature
AC
review.
355
According to Mullen & Copper [49], Cohesion refers to the interpersonal
attraction of team members, their commitment to the team task, and grouppride spirit. Other researchers [14, 37, 39] claim that Cohesion is a key factor
to assess Teamwork. Furthermore, Team Autonomy is an important factor to guarantee that the team has an efficient Teamwork [9, 15, 41]. Moe et al. [15] 18
ACCEPTED MANUSCRIPT
describe it as the team’s ability to regulate their boundary conditions, which 360
refers to the external influences of management and other individuals on the activities of the team. Therefore, we can say that if a team is cohesive and
dependent on Cohesion and Team Autonomy.
CR IP T
has autonomy, the TWQ is high. Given this, we decided to make Teamwork
The Agile Manifesto [2] claims that the best solutions come from self-organized 365
teams. Therefore, we considered that, to be cohesive, agile teams must be self-organized. In addition to that, it is necessary that team members collab-
orate to embrace the concept of whole-team responsibility and commitment
Organizing and Collaboration.
AN US
[14, 37, 40, 42]. Therefore, we defined that Cohesion is dependent on Self-
Team Learning, Shared Leadership, and Redundancy are important char-
370
acteristics for self-organized teams [15]. The authors describe them, respectively, as the ability to identify the changes in the team environment and adjust the strategies as needed; the sharing of decision authority and leader-
375
M
ship; and the team members’ capacity to perform (parts of) each others jobs and substitute each other as circumstances demand. Other studies suggest
ED
that these factors need to be addressed for achieving high quality Teamwork [8, 9, 15, 39, 41, 44, 45]. Other works [8, 44, 38] present the concept of Team expertise, the sum of the knowledge of the individuals composing the team, as
380
PT
important for self-organized teams. Given their similarity, we merged the concepts Redundancy and Team expertise into the factor Expertise. Given this, we
CE
added Team Learning, Shared Leadership, and Expertise to the model as parent nodes of Self-Organizing. For a team to collaborate, it must be coordinated. Hoegl & Gemuenden
AC
[14] state that Coordination refers to team members executing their activities
385
in a timely and integrated manner. It is described as a key-factor for Teamwork by several studies [8, 14, 37, 39, 41]. On the other hand, only coordination is not enough. Team Orientation refers to the mutual respect among the team members as well as prioritization of team goals over individual goals [50]. Several researchers [8, 9, 15, 41, 44, 45, 38] identified it as an important indicator of 19
ACCEPTED MANUSCRIPT
390
team collaboration. Given this, we added Coordination and Team Orientation as parents of Collaboration. For a team to be oriented is a result of mixing the personalities of the team
CR IP T
members [39, 38, 46]. Furthermore, the Expertise of the team members with Redundancy, increases the mutual respect among the team members and, con395
sequently, the quality of the Team Orientation. Therefore, we added Personal Attributes and Expertise as parents of Team Orientation.
For a team to coordinate its task, it must communicate. Since agile is based
on tacit knowledge sharing [1], Communication is a must factor to assess agile
400
AN US
teams [8, 14, 37, 39, 41, 40, 42, 38]. Furthermore, Daily Meetings play an important role on synchronizing the team members’ tasks, as well as removing impediments and mitigating risks. Given the definition of Coordination we
believe that these two factors improve the Coordination of the team. So, we introduced both factors in the model as parent nodes of Coordination. The communication of the team depends on the communication channels and team distribution. Furthermore, according to Bustamante [51], the ideal
M
405
agile team is co-located and communicates face-to-face daily. So, the Team
ED
Distribution and the Means of Communication adopted by the team members are factors that influence the quality of the Communication. Therefore, we added these factor to the model and made them parent nodes of Communication. In Daily Meetings, agile teams synchronize their tasks and problems [8, 41,
PT
410
44]. However, it is important that all team members are present on the Daily
CE
Meetings, so they can be aware of what is the current status. Therefore, we added Monitoring and All Members Present as parent nodes of Daily Meetings
AC
in the model. Finally, we present the resulting DAG in Figure 4.
20
ACCEPTED MANUSCRIPT
Teamwork
Team Autonomy
Collaboration
Self-Organizing
Team Orientation
Coordination
Team Distribution
Means of Communication
Personal Attributes
Monitoring
Expertise
Shared Leadership
Team Learning
AN US
Daily Meetings
Communication
CR IP T
Cohesion
All Members Present
M
Figure 4: Proposed model.
For the second sub-problem (i.e., identify the elements of N ), we defined that
415
ED
each node is composed of a 5-point Likert scale (e.g., Very Low, Low, Medium, High, and Very High). Each state corresponds to the quality level of the given
PT
node (i.e., factor). To avoid misunderstanding with the scale, for each node, we defined the expected observation for the extreme states (i.e., Very Low and Very High) in Table 3.
AC
CE
420
Table 3: DAG nodes states definition.
Node
Teamwork
State Very
The team works cohesively and there is no ex-
High
ternal agent affecting the work.
Very
The team does not work cohesively and there
Low
is an external agent affecting the work.
21
ACCEPTED MANUSCRIPT
Node Team Autonomy
State Very
There is no external agent interfering on how
High
the team execute its tasks. The external agent
CR IP T
collaborate with them to define what will be done and only when adequate.
Cohesion
Very
There is an external agent interfering on how
Low
the team execute its tasks.
Very
The team works cohesively and synchronously,
High
prioritizing the team goals, and self-organize
Very Low
AN US
efficiently.
The team does not work cohesively and synchronously, does not prioritize the team goals, and does not self-organize efficiently.
Collaboration
Very High
There is a high degree of collaboration in the team for achieving success on the project de-
M
velopment.
There is no collaboration in the team for
Low
achieving success on the project development.
Very
The team is capable to self-organize effi-
High
ciently in order to face challenges and complex
ED
Very
CE
PT
Self-Organizing
AC
Coordination
Team Orientation
changes.
Very
The team is not capable to self-organize in or-
Low
der to face challenges and complex changes.
Very
The team execute its tasks in a synchronous
High
and integrated manner.
Very
The team does not execute its tasks in a syn-
Low
chronous and integrated manner.
Very
The team members trust each other and feel
High
motivated to work together for achieving the team goals.
22
ACCEPTED MANUSCRIPT
Node Very
The team members do not trust each other and
Low
do not feel motivated to work together.
Very
The communication is effective.
CR IP T
Communication
State
High Very
The communication is not effective.
Low Very
It was possible to gather all obstacles, find a
High
way out of them, and synchronize the team.
Very
It was not possible to gather all obstacles, find
Low Team Distribution
Very High Very
a way out of them, and synchronize the team. All members share the same work place.
None of the members share the same work place.
M
Low
AN US
Daily Meetings
The team members communicate face-to-face
Very
The team members do not communicate face-
Low
to-face.
Very
The team members expose their obstacles and
High
progress regarding their tasks in a clear and
AC
CE
PT
Monitoring
ED
Very Means of Communication High
All Members Present
whenever possible.
objective way. Very
The team members do not expose their ob-
Low
stacles and progress regarding their tasks in a clear and objective way. Instead, they enjoy the opportunity to justify themselves about their decisions [3].
Very
All members attended to the daily meetings.
High
23
ACCEPTED MANUSCRIPT
Node Very
There was no daily meeting in which all mem-
Low
bers attended.
Very
The mixture of personalities contributes for a
High
good relationship of the team members.
Very
The mixture of personalities does not con-
Low
tribute for a good relationship of the team members.
Very
The team members have the necessary knowl-
High
edge for developing the tasks with redundancy.
Very Low
AN US
Expertise
CR IP T
Personal Attributes
State
The team members do not have the necessary knowledge for developing the tasks with redundancy.
Shared Leadership
Very High
shared.
The decision authority and leadership is not
M
Very
The decision authority and leadership is
shared.
Very
The team adapts itself to changes in the
High
team environment and adjust the strategies as
ED
Low
needed.
Very
The team does not adapt itself to changes in
Low
the team environment and adjust the strategies as needed.
AC
CE
PT
Team Learning
5.2. Probability Tables Definition To define the NPT, since we only used ranked nodes, we applied a modified
version of the method presented in [27], which we present an overview in Section
425
2. For each dependent node in the DAG, we defined a set of questions for the domain practitioner. For each question, the practitioner replied with a 24
ACCEPTED MANUSCRIPT
probability for each state of the child node. Afterwards, we analyzed the data collected and calibrated the NPT of the nodes (i.e., defined σ 2 and µ). For brevity, in this paper, we only show the process to define one NPT: Teamwork. We created a webpage5 to present the data collected to define all the NPTs.
CR IP T
430
In Table 4, we show the data collected for Teamwork. By analyzing Table 4,
we concluded that if any of the attributes value is low, Teamwork quality tends to be low. Therefore, we decided to use WMIN function with different weights
for each attribute. Empirically, we chose the weights to be 5 and 1, for Cohesion 435
and Team Autonomy, respectively, and σ 2 = 0.0005, because we observed that
AN US
WMIN tends to be a minimum function reflecting the practitioner’s confidence in the results with this configuration.
Table 4: Data collected to calibrate the NPT of Teamwork.
Teamwork
Team Autonomy
Very Low
Low
Medium
High
Very High
Very High
Very Low
0
0
0
0.8
0.2
Very Low
Very High
0.2
0.8
0
0
0
Very High
Medium
0
0
0
0.2
0.8
Medium
Very High
0
0
0.8
0.2
0
ED
M
Cohesion
PT
6. Procedure to use the Model One of the goals of this paper is to present a procedure to use the model presented in Section 5 with the purpose of assisting on the improvement of the
CE
440
TWQ of agile teams. Perkusich et al. [19] proposed a procedure for detecting problems of processes in software development projects based on Bayesian net-
AC
works. Given that both works are based on Bayesian networks and continuous improvement, we adapt the given procedure to our context.
445
The procedure presented by Perkusich et al. [19], shown in Figure 5, contains 5 https://sites.google.com/site/istbnteamwork/
25
ACCEPTED MANUSCRIPT
five steps. We apply all the steps, but adapt their goals to fit the purposes of
M
AN US
CR IP T
our work. More specifically, we adapted steps I, II and IV.
Figure 5: Perkusich et al. proposed procedure for using the model.
ED
6.1. Step I - Model Construction
The goal of this step is to have a Bayesian network that represents the 450
teamwork key factors, as well as the relationships between them, given the
PT
context of the team in which the procedure will be applied. In Section 5, we presented details regarding how we executed this step. However, in this paper
CE
we propose a generic model that represents the essence of agile teams. Since the difference of agile teams is only on some practices adoption and the geographical
455
distribution, there is no need to perform this step because the model presented
AC
in Section 5 is already considered its output. 6.2. Step II - Model Evaluation This is the initial stage of the cyclic part of the procedure. After constructing
the model, the next step is to define how long the cycles will last (e.g., for 460
iterative processes, a procedure’s cycle could last one iteration), and when each 26
ACCEPTED MANUSCRIPT
stage will be executed. Then, it is necessary to evaluate the model to check if it still suits the project’s current context (i.e., DAG and probability functions). Since the only factors that may differ from one agile team to another are the
465
CR IP T
team members distribution and the adopted agile practices, only the parent nodes of Coordination should be modified to make the model fit the team’s context.
We present two examples on how to modify the model according to a give team as follows: 1. Distributed Team
In distributed teams, it is not possible to consider that the main means
AN US
470
of communication will be a face-to-face conversation. Thereby, it would be possible to replace the parent nodes of Communication (i.e., Team Distribution and Means of Communication) for Frequency and Efficiency. These factors would need to be periodically observed with a certain caution because they are more subjective than Team Distribution and Means of Communication.
M
475
Usually, in distributed teams, the members use email groups and online in-
ED
stant messaging and video conference applications to centralize the team’s communication. Therefore, it would be necessary to observe the Frequency and the Efficiency of the Communication on these applications. By doing
PT
480
so, the resulting modified model would be as represented in Figure 6.
2. Extreme Programming (XP) Teams
CE
In XP teams, it is usually adopted the practices of Pair Programming and Code Review. So, in XP teams that adopt these practices, it would be
AC
485
necessary to add nodes corresponding to these factors to the model. It could be done by creating the node Practices and adding it as child of Daily Meetings, and parent of Coordination. Then, the nodes Pair Programming and Code Review would also be added as parent nodes of Practices. By doing so, the model would represent an XP team that adopts, besides
490
Daily Meetings, PP and Code Review. It would also be possible to add
27
ACCEPTED MANUSCRIPT
Coordination
Daily Meetings
Efficiency
Frequency
AN US
Communication
CR IP T
...
Monitoring
All Members Present
M
Figure 6: DAG example for a distributed team.
parent nodes to PP and Code Review in order to decompose them into
ED
less subjective factors.
Thereby, we present the modified DAG for an XP team as described above
495
PT
in Figure 7.
6.3. Step III - Model Data Input At this stage, the user inputs data (i.e., evidences) to the model. Ideally,
CE
all of the input nodes should have evidences. For the input nodes that data could not be measured from, and therefore, evidences could not be set, the
AC
uncertainty should be the same for all possible states (i.e., a uniform distribu-
500
tion in the interval [0, 1]). After inputting all evidences into the model, the outputs should be calculated using a Bayesian networks specific tool such as
28
ACCEPTED MANUSCRIPT
...
CR IP T
Coordination Practices
Communication Corde Review
Pair Programming
Team Distribution
Means of Communication
AN US
... ... ... ...
Monitoring
Daily Meetings
All Members Present
M
Figure 7: DAG example for an XP team.
GeNIe6 , AgenaRisk7 or Netica8 . For this research, we used AgenaRisk because
ED
it implements the algorithm of ranked nodes, which was adopted in the model construction. The outputs are data with probability values for each state of 505
each node that represent the current quality status of each factor.
PT
This stages outputs are that the calculated results for each node of the model, except for the input nodes. We propose that the outputs of this stage are presented in an iteration retrospective, so all members can be aware of the
CE
team’s problems.
6.4. Step IV - Model Output’s Analysis
AC
510
At this stage, the user should analyze the model’s calculated outputs to
detect problems impacting the TWQ. The goal of this stage is to assess the 6 http://genie.sis.pitt.edu/ 7 http://www.agenarisk.com 8 http://www.norsys.com/
29
ACCEPTED MANUSCRIPT
current TWQ and develop a plan to apply corrective and preventive actions to improve the projects chances of success. There is also the possibility to use 515
sensitivity analysis to prioritize the problems resolution.
CR IP T
Since the factors that influence the TWQ are subjective, there might be some difficulties on the interpretation of the outputs. However, it is known that the TWQ has positive impact on the team’s performance. Yet, the team’s performance, not only depends on the TWQ, but also on other factors such 520
as planning, product complexity, usage agile software engineering techniques, resources, programming languages, and tools [52]. Given this, to analyze the
mance, as an indicator of it.
AN US
model outputs, we propose the users to compare them to the team’s perfor-
The output of this stage is a plan with the corrective and preventive actions 525
identified during it. We propose to use this plan as basis for the planning of new iterations.
M
6.5. Step V - Corrective and Preventive Actions
During this stage, the plan defined in Step IV is executed and managed by
530
ED
the project manager or another project member. At the end of this stage, it is necessary to verify if the actions that were committed to were executed and
PT
analyze their consequences.
7. Case Study
CE
Case study is a research methodology suitable for studying contemporary phenomena in its natural context [53]. To evaluate the proposed procedure, we conducted a case study at Virtus 9 . Virtus is located at the Federal University
of Campina Grande, and we chose it due to its academia-industry relations.
AC
535
Due to availability, we chose three agile software development teams as units
of analysis. The projects in which these teams were working on are executed in partnership with multinational technology companies to develop software 9 http://www.virtus.ufcg.edu.br
30
ACCEPTED MANUSCRIPT
540
products. These projects are managed with Scrum 10 . The duration of the case study was forty-five days (i.e., three iterations). The goal of the case study is to evaluate the practical utility of the proposed
CR IP T
solution to assist on the assessment and improvement of the TWQ of agile teams. With this purpose, we defined the following research questions:
RQ1 : Are the model calculated results acceptable by the subjects at
545
Virtus?
RQ2 : Does the model assist on the detection of opportunities for improv-
AN US
ing the TWQ of an agile team at Virtus?
RQ3 : Is it easy to implement and apply the procedure in Scrum-based teams routine at Virtus?
550
RQ4 : Is the cost-benefit of applying the procedure positive at Virtus?
7.1.1. Units of Analysis
M
7.1. Case Study Design
555
ED
We executed this case study with three agile software development teams. Each unit of analysis was an agile software development team executing projects at Virtus. These units of analysis were nominated Team A, Team B, and Team
PT
C.
The Team A was working on an app for Desktop/Tablet x(86). Team B’s goal was to develop a tool to monitor and control equity security assets using Android. Team C’s goal was to develop a web app that needs to be integrated
CE 560
to other hardware and software components being developed at Virtus and by
AC
their client.
In the context of all teams, the teamwork quality is assessed informally.
Therefore, they do not use any instrument or defined process with the purpose
565
of assessing the teamwork quality. Conversely, they rely on the experience of 10 http://www.scrumguides.org/scrum-guide.html
31
ACCEPTED MANUSCRIPT
the Scrum Masters and daily interaction between the team members and Daily Scrum meetings to assess if there are opportunities to improve teamwork quality. Furthermore, as described by Scrum, at the end of the iterations (i.e., sprints),
570
CR IP T
the team meets to discuss how to improve the way they work during the Sprint Retrospective meeting. In Table 5, we show descriptive data of the units of analysis. Table 5: Units of Analysis Information.
Team
A
B
C
4
6
5
2.5
2.0
2.0
1.0
1.0
2.0
AN US
Characteristic Team size
Mean experience, in years, working on software development projects
Mean experience, in years, working in agile teams
M
7.1.2. Subjects - Who uses the model and the procedure?
For each unit of analysis (i.e., team), the subjects are Scrum Masters. At
575
ED
Virtus, the Scrum Masters execute tasks related to the process and team leadership (i.e., management), design and implementation. Therefore, in practice, one person shares the roles of Scrum Master and technical leader. In Table 6,
PT
we present the profile of the subjects. Table 6: Subjects Profile.
Subject 2
3
Experience, in years, working on software development projects
5.0
10.0
10.0
Experience, in years, leading software development teams
0.5
3.0
2.0
Experience, in years, using metrics for support the decision making process
1.5
2.0
6.0
Experience, in years, using agile methods
5.0
2.0
7.0
CE
1
AC
Characteristic
7.1.3. Methods The data collection is a necessary step to answer the research questions in
580
a case study. According to Lethbridge et al. [54], there are three different 32
ACCEPTED MANUSCRIPT
categories of data collection methods: direct (e.g., interviews), indirect (e.g., questionnaires), and independent (e.g., documentation analysis). In this case study, we collected the data through all methods presented by
585
CR IP T
Lethbridge et al. [54]. The input data for the model was collected through interviews and the performances of the teams on each sprint were calculated through documentation analysis. The research questions were answered by col-
lecting data through questionnaires and analyzing the calculations of the model. The questions to input data to the model and to answer the research questions are available on a website11 .
AN US
Each question to feed the model was created to define the status of the given
590
factor and could be answered in a five-point Likert scale. For example, for the node Expertise, we created the following question: “Do all the team members have the enough knowledge to complete all stories with intersection (i.e., ability to replace each other with minimum effort)?”. The possible answers for this question are given below, as well as the corresponding state of the given node according to each answer:
ED
False → Very Low
M
595
More False than True → Low
PT
Neither False nor True → Medium More True than False → High
600
CE
True → Very High
To collect data to directly answer the research questions, we created a ques-
AC
tionnaire composed of four Likert scale questions with five points, one for each research question. In addition to that, we added three open-ended questions,
605
each one of them related to the first three research questions, in which the subjects could comment and add more information regarding their Likert scale 11 https://sites.google.com/site/istbnteamwork/
33
ACCEPTED MANUSCRIPT
answers. However, the open-ended question for RQ1 was asked at the end of each sprint, which allowed us to collect more information regarding the acceptance of the model calculated results, as well as to help on the analysis process. We analyzed the open-ended data using thematic analysis [55].
CR IP T
610
The possible answers for each Likert scale question were the same as the ones to feed model, as described before. 7.1.4. Case Study Procedure
We used AgenaRisk to execute the Bayesian networks calculations because it implements the algorithm of ranked nodes, which we applied to our model. Due
AN US
615
to licensing limitations, the models were built and executed on the computer of one of the researchers. For each unit of analysis, we executed the procedure presented in Figure 5, during three sprints. We conducted this case study in two phases.
Introductory Course - During this phase, we trained the subjects to
620
M
use the model and procedure, and presented them the concepts related to this research. The training format was a lecture, in which the lecturer was one of
ED
the authors. This phase lasted one hour. Procedure Usage - After the first phase, the subjects used the procedure 625
presented in Section 6 during three sprints, each one lasting two weeks. To feed
PT
the model, a researcher interviewed the Scrum Masters at the end of each sprint to collect the data. In these interviews, a researcher was responsible for asking the questions regarding the quality of each model’s node, as well as collecting the
CE
data for calculating the team’s performance for that sprint. Then, the researcher
630
used the AgenaRisk tool to perform the model calculations. These results, were
AC
compared with the teams’ performance, as indicator of it (i.e., Section 6.4), to elaborate the corrective and preventive plan. Since the units of analysis adopt Scrum, we used the performance measure
proposed by Kumar [56] to calculate the teams’ performance in each sprint.
635
This approach is based on three parameters: 1. # user stories planned vs # user stories delivered - Measures planning 34
ACCEPTED MANUSCRIPT
effectiveness, efficiency; 2. # user stories delivered vs # user stories accepted - Measures quality of the deliverables; 3. # bugs found vs # bugs fixed - Measures defects-removal rate;
CR IP T
640
With these measures, we calculate the overall percentage, which is the mean of these measures. The resulting percentage is considered the team’s performance for a given sprint. Moreover, Kumar [56] provide a scale for each one of the parameters, as well as for the overall performance. However, we can define
our own scales based on our requirements. The average time the subjects 1,
AN US
645
2, and 3 took to input the data to the model was, respectively, 15, 15, and 5 minutes.
At the end of the three sprints, the subjects responded the questions related to the research questions. 7.2. Results
M
650
We present the model outputs for each unit of analysis, for each sprint, in Figure 8, Figure 9, and Figure 10. We present the calculated performances of
AC
CE
PT
ED
the teams in Table 7.
35
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 8: Model outputs for Team A.
36
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 9: Model outputs for Team B.
37
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 10: Model outputs for Team C.
38
ACCEPTED MANUSCRIPT
Table 7: Teams’ Performance in each sprint.
Team A
B
C
Sprint 1
98,610%
95,230%
73,330%
Sprint 2
69,230%
100%
100%
Sprint 3
70,510%
100%
77,770%
CR IP T
Sprint
We describe the answers provided for each one of the Likert scale questions designed to directly answer the research questions as follows:
AN US
655
1. “Do you agree with the model calculated results? ” - related to RQ1 : One subject answered True, and the other two answered More True than False.
2. “Does the model assist on the detection of opportunities for improving agile team’s TWQ? ” - related to RQ2 :
660
M
One subject answered True, and the other two answered More True than False.
ED
3. “Is it easy to implement and use the procedure in Scrum-based teams routine? ” - related to RQ3 :
All three subjects answered True.
665
PT
4. “Is the cost-benefit of applying the procedure positive? ” - related to RQ4 : Two subjects answered True, and the other one answered More True than
CE
False.
7.3. Discussion In this section, we discuss each research question given the collected data
AC
670
presented in Section 7.2. Furthermore, we present benefits and challenges of using the proposed model. To answer this RQ1 , we triangulated the model calculated results, the
teams’ performances, and facts that occurred during the sprints, which could
39
ACCEPTED MANUSCRIPT
675
explain the model calculated results as well as the teams’ performances. These facts were provided by the open-ended question regarding this research question. The Subject A reported that, for the Sprint 1, they had no challenges, since
CR IP T
most of the risks were avoided during the sprint planning and no external agents affected their execution. However, for the Sprint 2, some members had to be 680
reallocated to other tasks in the middle of the sprint, which had a huge impact on their Autonomy, affecting negatively on the TWQ. Moreover, similarly to the
Sprint 2, on the Sprint 3, the Team Autonomy was negatively affected because
they needed to re-prioritize some issues due to a delivery that was not discussed
685
AN US
with the team members during the planning meeting. These facts are consistent with the model calculated results and team’s performance on each sprint. The Team B was the most regular one, despite the low quality reported in each one of the three sprints. According to the Subject B, it did not affect much the overall TWQ because they were already aware of the PO’s behavior of changing the scope and could anticipate him on the sprint planning meetings. Therefore, the model calculated results, the team’s performance, and the TWQ
M
690
are consistent with each other.
ED
The Subject C was responsible for managing a team composed with parttime and full-time developers. According to him, some part-time developers are not mature enough to handle some responsibilities. In addition, it is also a problem to get the whole team united during the Daily Meetings, which affected
PT
695
the team’s Communication. For the Sprints 1 and 2, the Team C worked on
CE
code refactoring and preparing the code for upcoming requirements. However, Subject C reported the difference between the performance on Sprint 1 and Sprint 2 occurred because the planning for the first sprint was not efficient as the planning for the second sprint. For the Sprint 3, the Subject C reported that
AC
700
the web services started to be handled by the client’s team. Since the client’s team did not follow exactly what was documented and the communication with them was not efficient, it led the Team C to compromise the sprint, decreasing its performance.
705
Therefore, given that all facts, model calculated results, and teams’ perfor40
ACCEPTED MANUSCRIPT
mances, on each one of the sprints, are consistent with each other, it is possible to answer that the model measurements of the TWQ are acceptable, given the case study context. Moreover, the subjects agreed with the model calculated
CR IP T
results, given that they matched their thoughts about the teams’ TWQ. Regarding RQ2 , by applying the procedure and using the proposed model,
710
the subjects reported that model outputs are useful on detecting weaknesses
and strengths to allow continuous improvement of the TWQ. Since the outputs are quantitative, they claimed that this helped on prioritizing corrective actions and defining a plan to execute them.
Moreover, in the Likert scale answers, the subjects reported that the model
AN US
715
assists on the detection of opportunities for improving agile team’s TWQ. In regards to RQ3 , the subjects agree that the procedure adoption and usage is simple and beneficial, as answered in the Likert scale questions. In addition to that, the average time they took to input data to the model indicates that this 720
process is simple, and the model structure is well defined. However, as stated by
M
them, the only problem on applying the procedure was the fact that the model calculations could only be performed on the computer of one of the researchers.
ED
According to them, if they had access to the tool, it would be much easier and better to apply the procedure. Based on these answers, we concluded that it is 725
easy to implement and apply the procedure in Scrum-based teams routine at
PT
Virtus.
To answer RQ4 , we analyzed the time needed for the subjects to input data
CE
to model, the duration of time for the introductory course, and the answers regarding the benefits of using the model and applying the procedure. Based
730
on these data, we concluded that the cost-benefit of applying the procedure
AC
positive at Virtus is positive. Furthermore, the Likert scale answers given by the subjects enhance our conclusion. In Table 8, we present the benefits and challenges of using the proposed
model and procedure in a daily basis, according to the collected data and our
735
observations.
41
ACCEPTED MANUSCRIPT
Table 8: Summarized Results.
Benefits
Shortcomings • The input nodes are subjective.
• Outputs of the model are
CR IP T
• The model still requires more
acceptable.
empirical validation.
• Assists on the detection of
• It requires person/people
TWQ improvement opportunities.
performing the
• Helps on prioritizing corrective
evaluation to be impartial.
actions regarding TWQ.
• It requires the AgenaRisk tool to perform the results calculations.
AN US
• Adoption is simple, and beneficial.
7.4. Threats to Validity
Runeson & H¨ ost [53] state that there are different ways to classify threats to validity aspects in the literature. The authors suggest a classification schema
740
M
that distinguish four validity aspects of case studies. These aspects are: Construct Validity, Internal Validity, External Validity, and Reliability. Based on
ED
what is stated in [53], we identified the following threats to validity: • Construct Validity: The structure of the proposed model presents a new
PT
set of indicators that are not specifically based on prior work, although the questions regarding the input nodes are inspired on previous work. There-
745
fore, the measure of teamwork quality is uncertain. To reduce the effort
CE
of using the proposed model, we only defined one question for each one of the input nodes, which can cause bias on the input. We tried to minimize
AC
this threat by providing an introductory course and defining meaning for
750
the Likert scales. Another critical point is the lack of validation regarding the measures proposed on Kumar [56] to calculate the performance of the teams; • Internal Validity: The model selection problem is a grievous concern to the internal validity because, since we did not have data, the parameters 42
ACCEPTED MANUSCRIPT
of the model are based on the knowledge of only one practitioner. In the presence of data, it would be able to perform independence test such
755
as applying the PC algorithm [57]. Furthermore, the TWQ was adopted
CR IP T
as an indicator of the teams’ performances, but there are other factors that influence this measure such as business stability, company structure
and organizational maturity. Given that these factors were ignored, the interpretation of the model outputs might be imprecise. Finally, since
760
the units of analysis did not use a formal process to assess teamwork, our conclusions might not be specific to our model and procedure, but to using
AN US
a process to assess teamwork;
• External Validity: Since the study objects were observed in only three Scrum-based projects with small teams and we only collected data from
765
one of the team members, we can conclude that the model has not been empirically validated and, therefore, it cannot be generalized. For an
M
empirically validated causal model, it is necessary to collect enough data to evaluate the hypotheses for each relationship using an approach such as SEM. Anderson and Vastag [29] speculate that a sample size of 100
770
ED
cases would be reasonable, but it depends on the number of states in the variables and the number of parents of the variables. On the other hand,
PT
the types of products developed by the teams were different, which we believe can give more credibility to our conclusions
• Reliability: Since the subjects are providing data regarding their teams,
CE
775
the data might be biased. In addition to that, the questions asked during the interviews and defined in the questionnaires might not be clear enough
AC
to avoid biased input.
8. Conclusions and Future Works
780
In this paper, we presented an analytically practitioner-based Bayesian net-
work model to assess and improve the TWQ of agile teams. In addition to the
43
ACCEPTED MANUSCRIPT
model, we also present a procedure to use and adapt it. To build the model, we performed a literature review to identify the factors that influence the TWQ and identified the relationships between by interviewing an expert. The procedure is based on [19], which is composed of five steps: (i) Bayesian network
CR IP T
785
Construction, (ii) Bayesian network Evaluation, (iii) Bayesian network Data Input, (iv) Bayesian network outputs analysis and (v) execution of corrective and preventive actions.
To evaluate both model and procedure, we performed a case study in three 790
Scrum-based software development teams, during three sprints. According to
AN US
our results, in the context of the case study, the model assists agile teams on
assessing teamwork quality and identifying improvement opportunities, and the cost-benefit for using with the proposed procedure is positive. Moreover, the subjects of the case study reported they found easy to implement and apply 795
the procedure on their teams’ routines. Even though the relationships between the nodes of the model are not empirically validated, we believe that we have
M
achieved promising results regarding the model, based on the discussion about RQ1, presented in Section 7.3.
800
ED
The limitations of this study are related to the Bayesian network construction and the case study’s threats to validity. We executed the case study with only three teams in one company. Furthermore, the teams did not use a formal
PT
process to assess teamwork. Therefore, our conclusions might not be specific to both model and procedure, but to using a tool to assess teamwork.
CE
For future works, we plan to apply both model and procedure in more soft805
ware development teams, in different contexts and maturity levels to validate it and continuously calibrate it. Moreover, we plan on evaluating our solution by
AC
collecting data from the other members of the development team, not only the team leader or Scrum Master. With this, we intend to enhance the reliability of the model and procedure, as well as verify divergences regarding the different
810
results for different roles in a team. Once we have more data, we intend to explore the use of SEM to verify [58] and, possibly, refine the model structure. In addition to that, we encourage researchers to apply the procedure to other 44
ACCEPTED MANUSCRIPT
agile methods, especially in immature teams. Finally, we plan on developing a tool to implement the model presented in this paper to be used by practitioners, so we can avoid licensing issues.
CR IP T
815
References
[1] B. Boehm, R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2003.
[2] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham,
AN US
820
M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, D. Thomas, Manifesto for agile software development, http://www.agilemanifesto.org/ (2001).
[3] V. Stray, D. I. Sjøberg, T. Dyb˚ a, The daily stand-up meeting: A grounded
M
825
theory study, Journal of Systems and Software 114 (2016) 101–124. doi:
ED
http://dx.doi.org/10.1016/j.jss.2016.01.004. [4] L. Williams, R. R. Kessler, W. Cunningham, R. Jeffries, Strengthening the case for pair programming, IEEE Software 17 (4) (2000) 19–25. doi: 10.1109/52.854064.
PT
830
[5] K. Molkken-stvold, N. C. Haugen, H. C. Benestad, Using planning poker
CE
for combining expert estimates in software projects, Journal of Systems and Software 81 (12) (2008) 2106–2117. doi:http://dx.doi.org/10.1016/j.
AC
jss.2008.03.058.
835
[6] J. R. Katzenbach, D. K. Smith, The wisdom of teams: Creating the highperformance organization, Harvard Business Press, 1993.
[7] T. Chow, D.-B. Cao, A survey study of critical success factors in agile software projects, Journal of Systems and Software 81 (6) (2008) 961–971. doi:http://dx.doi.org/10.1016/j.jss.2007.08.020. 45
ACCEPTED MANUSCRIPT
840
[8] N. B. Moe, T. Dingsøyr, T. Dyb˚ a, A teamwork model for understanding an agile team: A case study of a scrum project, Information and Software Technology 52 (5) (2010) 480 – 491. doi:http://dx.doi.org/10.1016/
CR IP T
j.infsof.2009.11.004. [9] M. A. Ringstad, T. Dingsøyr, N. Brede Moe, Systems, Software and Service Process Improvement: 18th European Conference, EuroSPI 2011,
845
Roskilde, Denmark, June 27-29, 2011. Proceedings, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, Ch. Agile Process Improvement: Diag-
978-3-642-22206-1\_15. 850
AN US
nosis and Planning to Improve Teamwork, pp. 167–178. doi:10.1007/
[10] R. M. Fontana, I. M. Fontana, P. A. da Rosa Garbuio, S. Reinehr, A. Malucelli, Processes versus people: How should agile software development maturity be defined?, Journal of Systems and Software 97 (2014) 140 – 155.
M
doi:http://dx.doi.org/10.1016/j.jss.2014.07.030.
[11] L. Williams, K. Rubin, M. Cohn, Driving process improvement via comparative agility assessment, in: Proceedings of the 2010 Agile Conference,
855
ED
AGILE ’10, IEEE Computer Society, Washington, DC, USA, 2010, pp. 3–10. doi:10.1109/AGILE.2010.12.
PT
[12] S. Wood, G. Michaelides, C. Thomson, Successful extreme programming: Fidelity to the methodology or good teamworking?, Information and Software Technology 55 (4) (2013) 660 – 672. doi:http://dx.doi.org/10.
860
CE
1016/j.infsof.2012.10.002.
AC
[13] E. Amengual, A. Mas, A. Mesquida, Team spice: A spice-based teamwork
865
assessment model, in: A. Riel, R. OConnor, S. Tichkiewitch, R. Messnarz (Eds.), Systems, Software and Services Process Improvement, Vol. 99 of Communications in Computer and Information Science, Springer Berlin Heidelberg, 2010, pp. 37–47. doi:10.1007/978-3-642-15666-3\_4. URL http://dx.doi.org/10.1007/978-3-642-15666-3_4
46
ACCEPTED MANUSCRIPT
[14] M. Hoegl, H. G. Gemuenden, Teamwork quality and the success of innovative projects: A theoretical concept and empirical evidence, Organization science 12 (4) (2001) 435–449.
870
CR IP T
[15] N. Moe, T. Dingsøyr, E. Røyrvik, Putting agile teamwork to the test an preliminary instrument for empirically assessing and improving ag-
ile software development, in: P. Abrahamsson, M. Marchesi, F. Maurer
(Eds.), Agile Processes in Software Engineering and Extreme Programming, Vol. 31 of Lecture Notes in Business Information Processing, Springer
875
Berlin Heidelberg, 2009, pp. 114–123. doi:10.1007/978-3-642-01853-4\
AN US
_14.
[16] Y. Lindsjørn, D. I. Sjberg, T. Dingsøyr, G. R. Bergersen, T. Dyb˚ a, Teamwork quality and project success in software development: A survey of agile development teams, Journal of Systems and Software 122 (2016)
880
274 – 286. doi:http://dx.doi.org/10.1016/j.jss.2016.09.028. http://www.sciencedirect.com/science/article/pii/
S016412121630187X
M
URL
ED
[17] I. Ben-Gal, Bayesian Networks, John Wiley & Sons, Ltd, 2008. doi:10. 1002/9780470061572.eqr089.
885
PT
URL http://dx.doi.org/10.1002/9780470061572.eqr089 [18] K. Verbert, R. Babuka, B. D. Schutter, Bayesian and dempstershafer reasoning for knowledge-based fault diagnosisa comparative study, En-
CE
gineering Applications of Artificial Intelligence 60 (2017) 136 – 150. doi:http://doi.org/10.1016/j.engappai.2017.01.011.
890
AC
URL
http://www.sciencedirect.com/science/article/pii/
S0952197617300118
[19] M. Perkusich, G. Soares, H. Almeida, A. Perkusich, A procedure to de-
895
tect problems of processes in software development projects using bayesian networks, Expert Systems with Applications 42 (1) (2015) 437 – 450.
47
ACCEPTED MANUSCRIPT
[20] P. Hearty, N. Fenton, D. Marquez, M. Neil, Predicting project velocity in xp using a learning dynamic bayesian network model, IEEE Transactions on Software Engineering 35 (1) (2009) 124–137. doi:10.1109/TSE.2008.76.
CR IP T
[21] A. Rodrguez, F. Ortega, R. Concepcin, A method for the evaluation of risk in {IT} projects, Expert Systems with Applications 45 (2016) 273 –
900
285. doi:http://dx.doi.org/10.1016/j.eswa.2015.09.056. URL
http://www.sciencedirect.com/science/article/pii/
S0957417415006855
AN US
[22] C.-G. Bai, Bayesian network based software reliability prediction with an operational profile, J. Syst. Softw. 77 (2) (2005) 103–112. doi:10.1016/
905
j.jss.2004.11.034.
[23] T. Schulz, L. Radli´ nski, T. Gorges, W. Rosenstiel, Predicting the flow of defect correction effort using a bayesian network model, Empirical Software
910
M
Engineering 18 (3) (2011) 435–477. doi:10.1007/s10664-011-9175-7. [24] A. Silva Freire, R. Da Silva, M. Perkusich, H. Almeida, A. Perkusich, A bayesian network model to assess agile teams’ teamwork quality, in: Soft-
ED
ware Engineering (SBES), 2015 29th Brazilian Symposium on, 2015, pp. 191–196. doi:10.1109/SBES.2015.29.
PT
[25] N. Friedman, D. Geiger, M. Goldszmidt, Bayesian network classifiers, Machine Learning 29 (2-3) (1997) 131–163.
915
CE
[26] J. Pearl, S. Russell, Bayesian networks, Handbook of brain theory and neural networks.
AC
[27] N. Fenton, M. Neil, J. G. Caballero, Using ranked nodes to model qual-
920
itative judgments in bayesian networks, Knowledge and Data Engineering, IEEE Transactions on 19 (10) (2007) 1420–1432. doi:10.1109/TKDE. 2007.1073.
[28] D. Stankovic, V. Nikolic, M. Djordjevic, D.-B. Cao, A survey study of critical success factors in agile software projects in former yugoslavia {IT} 48
ACCEPTED MANUSCRIPT
companies, Journal of Systems and Software 86 (6) (2013) 1663–1678. doi: http://dx.doi.org/10.1016/j.jss.2013.02.027.
925
[29] R. D. Anderson, G. Vastag, Causal modeling alternatives in operations
CR IP T
research: Overview and application, European Journal of Operational
Research 156 (1) (2004) 92 – 109, eURO Excellence in Practice Award 2001. doi:https://doi.org/10.1016/S0377-2217(02)00904-9. URL
930
http://www.sciencedirect.com/science/article/pii/
S0377221702009049
AN US
[30] A. Qumer, B. Henderson-Sellers, A framework to support the evaluation,
adoption and improvement of agile methods in practice, Journal of Systems and Software 81 (11) (2008) 1899 – 1919. doi:http://dx.doi.org/10. 1016/j.jss.2007.12.806.
935
[31] T. J. Gandomani, M. Z. Nafchi, An empirically-developed framework for
M
agile transition and adoption: A grounded theory approach, Journal of Systems and Software 107 (2015) 204 – 219. doi:http://dx.doi.org/10.
940
ED
1016/j.jss.2015.06.006.
[32] O. Salo, P. Abrahamsson, An iterative improvement process for agile software development, Software Process: Improvement and Practice 12 (1)
PT
(2007) 81–100. doi:10.1002/spip.305. URL http://dx.doi.org/10.1002/spip.305
CE
[33] I. Aaen, J. Arent, L. Mathiassen, O. Ngwenyama, A conceptual map of software process improvement, Scandinavian Journal of Information Sys-
945
AC
tems 13 (2001) 123–146. URL http://dl.acm.org/citation.cfm?id=565431.565437
[34] E. Amengual, A. Mas, Teamwork best practices in iso/iec 15504, in: Pro-
950
ceedings of the 9th International Conference on Software Process Improvement and Capability Determination, 2009, pp. 106–112.
49
ACCEPTED MANUSCRIPT
[35] E. Amengual-Alcover, A. Mas-Picacho, Can teamwork management help in software quality and process improvement?, Improving Quality in Business Processes, Products and Organizational Systems (2009) 26.
CR IP T
[36] B. Kitchenham, S. Charters, Guidelines for performing Systematic Literature Reviews in Software Engineering, Tech. Rep. EBSE 2007-001, Keele
955
University and Durham University Joint Report (2007).
[37] E. Weimar, A. Nugroho, J. Visser, A. Plaat, Towards high performance software teamwork, in: Proceedings of the 17th International Conference
AN US
on Evaluation and Assessment in Software Engineering, EASE ’13, ACM,
New York, NY, USA, 2013, pp. 212–215. doi:10.1145/2460999.2461030.
960
URL http://doi.acm.org/10.1145/2460999.2461030
[38] Y. KOZAK, Barriers against better team performance in agile software projects, Master’s thesis, Chalmers University of Technology, Sweden
965
M
(2013).
[39] C. T. Schmidt, T. Kude, A. Heinzl, S. Mithas, How agile practices influence
ED
the performance of software development teams: The role of shared mental models and backup, in: ICIS 2014 Proceedings, AISeL, Atlanta, Ga., 2014, p. Paper 15.
970
PT
URL http://ub-madoc.bib.uni-mannheim.de/37362/ [40] A. Cockburn, J. Highsmith, Agile software development, the people factor,
CE
Computer 34 (11) (2001) 131–133. doi:10.1109/2.963450. [41] C. Gurram, S. G. Bandi, Teamwork in distributed agile software develop-
AC
ment, Master’s thesis, Blekinge Institute of Technology, School of Comput-
975
ing (2013).
[42] A. Johansson, Toward improvements of teamwork in globally distributed agile teams, Bachelor of science thesis in software engineering and management, University of Gothenburg (2013).
50
ACCEPTED MANUSCRIPT
[43] G. Tjørnehøj, M. Fransg˚ ard, S. Skalkam, Trust in agile teams in distributed software development, in: Information System Research Seminar in Scandinavia 2012, 2012.
980
CR IP T
[44] L. M. R. Haraldsen, An investigation of team effectiveness in agile soft-
ware development, Master’s thesis, Norwegian University of Science and Technology (2012).
[45] V. Gulliksen Stray, N. B. Moe, T. Dingsøyr, Agile Processes in Software Engineering and Extreme Programming: 12th International Confer-
985
AN US
ence, XP 2011, Madrid, Spain, May 10-13, 2011. Proceedings, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, Ch. Challenges to Teamwork: A Multiple Case Study of Two Agile Teams, pp. 146–161. doi: 10.1007/978-3-642-20677-1_11.
URL http://dx.doi.org/10.1007/978-3-642-20677-1_11
990
M
[46] V. Lalsing, S. Kishnah, S. Pudaruth, People factors in agile software development and project management, International Journal of Software Engi-
ED
neering & Applications (IJSEA) 3 (1) (2012) 117–137. [47] E. Whitworth, R. Biddle, The social nature of agile teams, in: Agile Conference (AGILE), 2007, 2007, pp. 26–36. doi:10.1109/AGILE.2007.60.
995
PT
[48] M. Perkusich, H. O. de Almeida, A. Perkusich, A model to detect problems on scrum-based software development projects, in: Proceedings of the 28th
CE
Annual ACM Symposium on Applied Computing, ACM, 2013, pp. 1037– 1042.
[49] B. Mullen, C. Copper, The relation between group cohesiveness and per-
AC
1000
formance: An integration., Psychological bulletin 115 (2) (1994) 210.
[50] E. Salas, D. E. Sims, C. S. Burke, Is there a big five in teamwork?, Small Group Research 36 (5) (2005) 555–599. arXiv:http://sgr.sagepub.com/ content/36/5/555.full.pdf+html, doi:10.1177/1046496405277134.
51
ACCEPTED MANUSCRIPT
1005
[51] S. R. Bustamante, A., Agile xxl:
Scaling agile for project teams,
seapine software, inc., http://downloads.seapine.com/pub/ebooks/ AgileScaling_eBook.pdf, accessed in 01/10/2017 (2011).
CR IP T
[52] C. Melo, D. Cruzes, F. Kon, R. Conradi, Agile team perceptions of productivity factors, in: Agile Conference (AGILE), 2011, 2011, pp. 57–66. doi:10.1109/AGILE.2011.35.
1010
[53] P. Runeson, M. H¨ ost, Guidelines for conducting and reporting case study
research in software engineering, Empirical Software Engineering 14 (2)
AN US
(2009) 131–164. doi:10.1007/s10664-008-9102-8.
[54] T. C. Lethbridge, S. E. Sim, J. Singer, Studying software engineers: Data collection techniques for software field studies, Empirical Software Engi-
1015
neering 10 (3) (2005) 311–341. doi:10.1007/s10664-005-1290-x. URL http://dx.doi.org/10.1007/s10664-005-1290-x
M
[55] John Wiley and Sons, New York, Introduction to qualitative research methods: The search for meanings (1984). [56] M. P. Kumar,
A simple way to measure the performance of
ED
1020
scrum teams, https://www.scrumalliance.org/community/articles/ 2014/may/simple-way-to-measure-performance-of-scrum-teams, ac-
PT
cessed in 01/10/2017 (2014).
CE
[57] S. P. G. C. M. C. Scheines, R., Lawrence Erlbaum, 1994. 1025
[58] S. Gupta, H.-W. Kim, Application of bayesian modeling to management
AC
information systems: a latent scores approach, in: Bayesian Network Technologies: Applications and Graphical Models, IGI Global, 2007, pp. 103– 126.
52
A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams Arthur Freire, Mirko Perkusich, Renata Saraiva, Hyggo Almeida, Angelo Perkusich PII: DOI: Reference:
S0950-5849(17)30020-4 10.1016/j.infsof.2018.04.004 INFSOF 5980
To appear in:
Information and Software Technology
Received date: Revised date: Accepted date:
11 January 2017 8 March 2018 13 April 2018
Please cite this article as: Arthur Freire, Mirko Perkusich, Renata Saraiva, Hyggo Almeida, Angelo Perkusich, A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams, Information and Software Technology (2018), doi: 10.1016/j.infsof.2018.04.004
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
highlights • We present a Bayesian network-based model to assess the quality of agile
CR IP T
teamwork; • A procedure to use and adapt the model is presented;
• The model practical utility and the procedure were evaluated with a case study executed in three Scrum teams;
• The usage of both model and procedure assists on the continuous improve-
AC
CE
PT
ED
M
AN US
ment of the teamwork quality with positive cost-benefit.
1
ACCEPTED MANUSCRIPT
A Bayesian Networks-based approach to assess and improve the teamwork quality of agile teams
CR IP T
Arthur Freirea , Mirko Perkusichb , Renata Saraivaa , Hyggo Almeidaa , Angelo Perkusichc a Department
AN US
of Computing and Systems, Federal University of Campina Grande, Campina Grande, PB, Brazil b Federal Institute of Paraiba, Monteiro, PB, Brazil c Department of Electrical Engineering, Federal University of Campina Grande, Campina Grande, PB, Brazil
Abstract
CONTEXT: According to the agile principles and values, as well as recent research articles, teamwork factors are critical to achieve success in agile projects. However, teamwork does not automatically arise. There are some existing instruments with the purpose of assessing the teamwork quality based on
M
Structural Equation Modeling (i.e., empirically derived) and Radar Plots, but they may not be useful in a concrete situation because these techniques are not
ED
advised for prediction and diagnosis purposes. OBJECTIVE: Analytically derive a Bayesian network model based on a literature review and a practitioner’s knowledge; and to assess its practical
PT
utility through a case study.
METHOD: To build the model, we executed a top-down approach using data
CE
collected through a literature review and a domain practitioner. We assessed the model with a case study executed in three Scrum teams. RESULTS: Given the context of the case study, the model assists agile teams
AC
on assessing teamwork quality and identifying improvement opportunities, is easy to learn, and the cost-benefit for using it with the proposed procedure is Email addresses: [email protected] (Arthur Freire), [email protected] (Mirko Perkusich), [email protected] (Renata Saraiva), [email protected] (Hyggo Almeida), [email protected] (Angelo Perkusich)
Preprint submitted to Information and Software Technology
April 13, 2018
ACCEPTED MANUSCRIPT
positive. CONCLUSION: We concluded that we achieved promising results with the presented solution. However, it needs more evaluation and validation to gener-
CR IP T
alize the obtained results. Keywords: Teamwork, Agile Teams, Bayesian Network, Assessment, Measurement, Improvement
1. Introduction
Agile software development consists of a lightweight approach to deliver
AN US
software continuously and has recently become popular in the industry. It is an
alternative approach to plan-driven development, in which the project plan and 5
system architecture accommodate foreseeable change with the goal of achieving predictability, stability and high assurance. Conversely, agile methods are reactive with the goal of achieving rapid value and responsiveness to change.
or environment changes [1].
M
Therefore, they are a better fit in contexts with rapid marketplace, technology
To achieve their goals, agile methods consider individuals and interactions as
10
ED
more important than processes and tools [2]. This is reflected on its principles: six out of the twelve principles are related to the individuals involved in the
PT
product development. These principles focus on factors such as team motivation, communication, collaboration and team management. This is also reflected by 15
popular agile practices such as Daily Stand-up Meetings [3], Pair Programming
CE
[4] and Planning Poker [5]. According to the Agile Manifesto [2], the best architecture, requirements and
design emerge from self-organized teams. For this purpose, it is necessary that
AC
team members collaborate to embrace the concept of whole-team responsibility
20
and commitment. Katzenbach & Smith [6] define team as a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually accountable.
3
ACCEPTED MANUSCRIPT
Several researchers [7, 8, 9, 10, 11] show that teamwork factors are critical to 25
achieve success in agile projects. On the other hand, good quality teamwork does not automatically arise [12]. In this paper, the concept of teamwork is defined
CR IP T
as: the capability of team members to work in a synchronous and cohesive way, keeping the team’s goals as priority, without unwanted interference of an outside agent (e.g. client or upper management) on the executed tasks.
In the literature, there are instruments with the purpose of assessing team-
30
work quality (TWQ) [13, 14, 15]. Amengual et al. [13] focused on ISO/IEC
15504 (i.e., SPICE); Hoegl & Gemuenden [14], on teams working on innovative
AN US
projects; and Moe et al. [15], on agile teams. However, there is still a need
for more research efforts be made to validate the agile TWQ construct and to 35
advance the measurement of team performance [16].
In this paper, we complement the current state-of-art by: (i) presenting a list of agile teamwork key factors identified on a literature review; (ii) constructing an analytically derived Bayesian networks based on the literature review and a
40
M
practitioners knowledge; and (iii) performing a case study to assess the practical utility of the proposed solution.
ED
We used Bayesian networks to model the causal relationships between the key factors of teamwork in agile teams and to handle uncertainty in the measurements and relationships [17]. The context of our problem is subject to
45
PT
randomness (i.e., uncertain future outcome) and incompleteness, which results in considering techniques for uncertainty reasoning such as fuzzy logic, neural
CE
networks, Bayesian networks, Dempster-Shafer, possibility theory and Kalman filters. The lack of available data invalidates the usage of a data-based approach such as Kalman filters, neural networks or any statistical model, but motivates
AC
the usage of knowledge-based approaches. More specifically, in our context, we
50
need to handle causal reasoning, adaptability, clarity of inference and support decision-making. According to Verbert et al. [18], Bayesian networks is the most suitable approach for this purpose. Researchers have applied this technique in software engineering for several purposes such as process improvement [19], productivity prediction of eXtreme 4
ACCEPTED MANUSCRIPT
55
programming (XP) teams productivity [20], risk management [21] and product quality prediction [22, 23]. A first version of the model was is presented in [24]. In [24], the model is
CR IP T
based primarily on factors extracted from two papers and it was not evaluated in real projects. Now, we increment our research by extracting the teamwork 60
key factors from a literature review, and evaluating the model on a case study with three software development teams.
The research method applied is composed of three main activities: (i) per-
form a literature review to identify the factors that influence the quality of agile
65
AN US
teamwork; (ii) construct the Bayesian network model and define the procedure
to use it; and (iii) perform a case study to assess the practical utility of the model and evaluate the proposed procedure for using the model. To build the Bayesian network, we executed a top-down approach to build the model according to knowledge extracted from both a practitioner and the literature. The practitioner has over five years of experience with Scrum projects, holds an MBA on project management and certifications from Scrum Alliance1 .
M
70
To use the model, we propose to apply a procedure that consists of four stages.
ED
This procedure is based on the procedure presented by Perkusich et al. [19]. However, we performed some adaptations on it to fit the context of our problem. We evaluated both model and procedure with a case study applied to three Scrum-based software development teams at Virtus 2 . For each team, we col-
PT
75
lected data from its leader and documentation. The goal of the study was to
CE
evaluate the practical utility of the proposed solution. According to the case study results, the model assists agile teams on assessing
teamwork quality and identifying improvement opportunities, is easy to learn, and the cost-benefit of using it with the presented procedure is positive. Given
AC
80
the case study results, we concluded that we achieved promising results with the proposed solution, but it needs more evaluation and validation. 1 www.scrumalliance.org 2 http://www.virtus.ufcg.edu.br
5
ACCEPTED MANUSCRIPT
In what follows, in Section 2, we present an overview on Bayesian networks and justify its use; in Section 3, we present background on agile teamwork; 85
in Section 4, we present results of a literature review to identify factors to
CR IP T
assess the teamwork quality of agile teams; in Section 5, we present the details regarding the model construction; in Section 6, we present the procedure to use the proposed model; in Section 7, we present the results of the case study; finally, in Section 8, we present our conclusions including our work’s current 90
limitations and future works.
AN US
2. Bayesian Networks Overview
Bayesian networks belong to the family of probabilistic graph models and are used to represent knowledge about an uncertain domain [17]. A Bayesian network, B, is a directed acyclic graph that represents a joint probability dis95
tribution over a set of random variables V [25]. The network is defined by the
M
pair B = {G, Θ}. G is the directed acyclic graph in which the nodes X1 , . . . , Xn represent random variables and the arcs represent the direct dependencies between these variables. Θ represents the set of the probability functions. This set
100
ED
contains the parameter θxi |πi = PB (xi |πi ) for each xi in Xi conditioned by πi , the set of the parameters of Xi in G. Equation 1 presents the joint distribution
PT
defined by B over V .
CE
PB (X1 , . . . , Xn ) =
n Y
i=1
PB (xi |πi ) =
n Y
i=1
θXi |πi
(1)
We present an example of a Bayesian network in Figure 1. Circles represent
the nodes and arrows represent the arcs. The probability functions are usually
AC
represented as node probability tables (NPT). Even though the arcs represent
105
the casual connection’s direction between the variables, the information can propagate in any direction [26]. In many cases, there is not enough data to define the NPT and it is necessary
to collect them from domain experts. To reduce this effort, Fenton et al. [27] presented the concept of ranked nodes, which are based on the doubly truncated 6
ACCEPTED MANUSCRIPT
X2
X1 T
F
0.7
0.3
T
F
0.2
0.8
CR IP T
MATURE TESTING PROCESS
AUTOMATED TESTS
X3
X2
F
F
F
T
T
F
T
T
T
F
0.1
0.9
0.7
0.3
0.4
0.6
0.9
0.1
AN US
STABLE RELEASE VERSION
X1
Figure 1: A Bayesian network example [24], used by permission.
110
Normal distribution (TNormal) limited in the [0, 1] region. This distribution is
M
based on four parameters: µ, mean (i.e., central tendency); σ 2 , variance (i.e., confidence in the results); a, lower bound (i.e., 0); and, b, upper bound (i.e.,
ED
1). This distribution enables us to model a variety of shapes (i.e., relationships) such as a uniform distribution, achieved when σ 2 → ∞, and highly skewed
distributions, achieved when σ 2 → 0. We show TNormal examples in Figure 2.
AC
CE
PT
115
µ = 0.5 σ2=1 a=0 b=1
µ = 0.5 σ 2 = 0.01 a=0 b=1
µ = 0.2 σ 2 = 0.1 a=0 b=1
Figure 2: TNormal examples.
In this approach, µ is defined by a weighted function of the parent nodes.
7
ACCEPTED MANUSCRIPT
There are four weighted functions: weighted mean (WMEAN), weighted minimum (WMIN), weighted maximum (WMAX) and the mixture of WMIN and WMAX (MIXMINMAX). According to the authors, these functions are enough to represent the types of relationship necessary for defining NPT. In Figure 3,
CR IP T
120
we show examples of NPT calculated with these functions.
In the examples presented in Figure 3, WMEAN and MIXMINMAX have the same values. The difference between the two functions is that WMEAN
calculates the weighted mean of the parent nodes values (i.e., the weights are 125
set for the parent nodes) and MIXMINMAX mixes the minimum and maximum
AN US
functions (i.e., the weights are set for the functions).
WMIN
M
WMEAN
MIXMINMAX
AC
CE
PT
ED
WMAX
Figure 3: Weighted functions examples.
To define which function should be used, the model developer must perform
”what if” analysis with the expert by defining questions and collecting answers to define a truth table [27]. The model developer must analyze the answers and
130
define the most appropriate function. The variance is defined empirically, and 8
ACCEPTED MANUSCRIPT
it should reflect the confidence of the domain expert in the results [27]. We show an example of questions and answers in Table 1, in which the node C has two parents, A and B. In the example shown, the most appropriate function
135
CR IP T
is WMIN. We asked the domain expert to, for the questions asked, define the expected probabilities for each state, based on the parent nodes states. Given
this, since Teamwork depends on Cohesion and Team Autonomy, we present the calibration results for Teamwork in Table 4. Table 1: Calibration example.
B
C
Very High
Very High
Very High
Very Low
Very Low
Very Low
Very Low
Very High
Low
Very High
Very Low
Low
M
3. Background on teamwork
AN US
A
3.1. Teamwork in agile software development
ED
Even though so much of some agile methodologies assume that collaborative
140
behavior will happen, so the teamwork quality can be increased, as stated in [12],
PT
teamwork does not automatically arise. Because of the different characteristics between agile and plan-driven development, teamwork in agile contexts requires much more than each team member doing its part of the work, and making sure everything piece of the puzzle will be integrated to work together as whole.
CE
145
Teamwork in agile requires a bigger sense of collective ownership, in which
every member can replace another without major issues, and shared leadership
AC
really exists. Moreover, the agile practices provide opportunities for team members to collaborate with each member and increase communication. As stated
150
in Section 1, in this paper, we defined agile teamwork as: the capability of team members to work in a synchronous and cohesive way, keeping the team’s goals as priority, without unwanted interference of an outside agent (e.g. client or
9
ACCEPTED MANUSCRIPT
upper management) on the executed tasks. In this Section, we give an overview of teamwork key factors, challenges, and importance. Chow & Cao [7] analyzed data collected through a survey from 109 agile
155
CR IP T
projects from 25 countries around the world and identified Team Capability as a factor that impacts the time and cost of agile projects. These finding agree
with the fifth principle of the Agile Manifesto [2], which is to build projects around motivated individuals. However, Stankovic et al. [28] showed data 160
invalidating the conclusions of Chow & Cao [7], but their study is limited to former Yugoslavia IT companies.
AN US
In a case study in one company, Moe et al. [8] show that problems with team orientation, team leadership and coordination in addition to highly specialized skills and corresponding division of work are important barriers for achieving 165
team effectiveness. Fontana et al. [10] present results of an exploratory study in which they concluded that, in agile context, maturity means fostering more subjective capabilities such as collaboration, communication, commitment, care,
M
sharing and self-organization. Williams et al. [11] presented a tool to measure the agility of companies and consider teamwork factors (e.g., team composition) as one of the dimensions to be evaluated.
ED
170
In Lindsjørn et al. [16], the authors present result of applying Structural Equation Modeling (SEM) to evaluate the results of a survey with 477 respon-
PT
dents from 26 companies. They concluded that TWQ significantly affects team performance, when rated by team members and leaders, and the team members’ success. SEM is a widely used method for quantifying and evaluating an as-
CE
175
sumed causal process. Therefore, its primary goal is to evaluate if a theoretical network is a reasonable representation of the process that generated the study
AC
data [29]. Since this study’s goal is to assess agile teamwork quality and assist the team on predicting and diagnosing issues (i.e, risks) within the team, we
180
used Bayesian networks. In the literature, there are studies proposing the usage of frameworks based
on assessments to assist on the adoption (i.e., usage) of agile methods such as [30] and [31]. Qumer & Henderson-Sellers [30] presented a framework to 10
ACCEPTED MANUSCRIPT
support the evaluation, adoption and improvement of agile methods to guide the 185
behavior of self-organizing and empowered agile teams. Gandomani & Nafchi [31] presented a framework based on PDCA (Plan-Do-Check-Act cycle) to adopt
CR IP T
agile methods. According to the authors, assessments are important during the adoption (i.e., transition) process, because it helps to find the challenges of agile teams and monitor the processes and progresses of the transition. However, none 190
of these solutions present an instrument to assess teamwork quality.
According to Salo & Abrahamsson [32], since the focus of agile methods is different than plan-driven development focus, there is a need of new software
AN US
process improvement (SPI) (i.e., initiatives that can be used in software organizations to mature their operations [33]) mechanisms for them. In addition, 195
after executing an action research in two companies, Ringstad et al. [9] concluded that the use of diagnosis and action planning to improve teamwork in agile projects improves shared leadership, team orientation and learning. Therefore, there is a need for instruments to assess agile teamwork. These
200
M
instruments can be used to identify improvement opportunities and support the agile principle “at regular intervals, the team reflects on how to become
ED
more effective, then tune and adjusts its behavior accordingly” [2]. The results of the instruments would guide the improvement of agile teams and would be especially beneficial for immature teams (i.e., organizations). Moreover, since
205
PT
these instruments help on the TWQ improvement, they can be considered new SPI mechanisms for agile methods.
CE
3.2. Existing instruments to measure the TWQ of software teams Amengual et al. [13] state that software engineering is currently paying
AC
special attention to cooperative and human aspects of software development. Given this statement, the authors present a SPICE-based Teamwork Assessment
210
Model for software teams. This assessment model is based upon a Teamwork Reference Model and a Measurement Framework presented, respectively, in [34] and [35]. According to the authors, in the context of software development, before publishing their work, there did not exist a common framework that 11
ACCEPTED MANUSCRIPT
could be used as reference for teamwork assessment in software teams. So, 215
the authors expected their model to be used as a reference framework for the development of new software tools for the simulation of the behavior of software
CR IP T
teams. However, their model does not relate the teamwork key factors to output the current teamwork status. The users are required to respond to questions regarding the teamwork key factors and analyze their own answers. In addition 220
to that, since this model does not focus on agile teams, it does not address important agile teamwork factors such as Self-Organizing and Redundancy.
Hoegl & Gemuenden [14] present a comprehensive concept of the collabora-
AN US
tion in teams called Teamwork Quality (TWQ). This construct has six facets
(i.e., Communication, Coordination, Balance of Member Contributions, Mutual 225
Support, Effort, and Cohesion). Based on these facets and on data collected in this study, the authors propose a way for measuring the TWQ through a linear regression model, where the high order factor (i.e., TWQ) is the dependent variable, and the construct facets are the independent variable. According
230
M
to the authors, the TWQ construct provides a comprehensive measure of the collaborative team-task process focusing on the quality of interactions rather
ED
than on activities in teams. In addition to that, to avoid bias, the data was collected from multiple team members. However, in the context of this paper, this instrument is limited because it only measures the teams’ degree of Collab-
235
PT
oration. So, it does not address some important agile teamwork factors such as Self-Organizing and Team Autonomy.
CE
Moe et al. [15] propose an instrument that addresses key concerns and characteristics of agile teamwork and presents them along five dimensions: Shared
Leadership, Team Orientation, Redundancy, Learning, and Autonomy. The in-
AC
strument outputs a radar plot of the status of the teamwork. An expert group
240
to whom the model was presented, composed of 35 people, found the model useful for understanding team problems such as the team agreeing on using test-driven development, but not applying it in practice; the team having no control over the members assigned to the team; and management not trusting the team. According to the authors, the instrument gives researches and de12
ACCEPTED MANUSCRIPT
245
velopers a common language for discussing teamwork. To assess the teamwork current status, it is necessary to answer a set of questions for each dimension, and, based on these answers, assign a score on a scale from 0 to 10 for the
CR IP T
dimension. In our prior work [24], we have proposed a Bayesian networks-based model 250
to assess agile teams’ teamwork quality. Since most of the model nodes are subjective, we used Bayesian networks given its capability to handle uncertainty [17]. Although our prior work has yielded several promising results, there are
AN US
some limitations with respect to the objective of our current work:
• It was built only based on the factors presented in [14, 15], and evolved with the help of practitioners. So, important agile teamwork key factors
255
are missing;
• The probability functions of the model were defined using only the one of
M
the four possible functions presented in [27]; • It was not evaluated in real projects; • It lacks a process to assist on its usage.
ED
260
In this paper, we seek to eliminate these threats by proposing a new model,
PT
presented in Section 5, and a procedure to apply it to agile projects, presented in Section 6. Furthermore, we applied both model and procedure to real projects
CE
through a case study as presented in Section 7.
265
4. Literature Review
AC
To perform the literature review, we executed a process based on the Guide-
lines for performing Systematic Literature Reviews in Software Engineering [36]. The main goal of this review is to identify the as many key factors that influence the quality of the Teamwork in agile teams as possible. Due to this fact, we
270
perceived it to be unnecessary to assess the quality of the papers. Moreover,
13
ACCEPTED MANUSCRIPT
the selection and extraction steps were performed by only one person, but the aggregation of the results was performed by three people. This process was composed of four major steps: (i) planning; (ii) search
275
CR IP T
strategy definition; (iii) study selection; and (iv) study extraction. In step (i), we defined the following research question: “What key factors influence the quality
of the Teamwork in agile teams?”. Then, we defined the inclusion/exclusion criteria as well as the study selection and extraction strategy. In step (ii), we
defined the search engines to perform the searching for papers, the search string, and extracted the .bib files to import all papers information to the StArt tool3 .
In steps (iii) and (iv), we applied the inclusion/exclusion criteria according to the search strategy defined in (i).
AN US
280
For brevity, in this paper, we briefly present details regarding the literature review process. It was based on automatic search on digital libraries, namely: ACM, IEEEXplore, Scopus, ScienceDirect and Google Scholar. We defined a 285
search string based on the keywords agile, teamwork, team and factor, and the
M
considered period of publication was from 2009 to 2015. To select the studies, we analyzed the title, abstract and keywords and included the ones that were in
ED
the scope of agile teamwork. Afterwards, for the remaining studies, we analyzed the entire text and selected papers that presented, at least, one factor to assess 290
agile teamwork or identified any relationship between teamwork factors. At the
PT
end of the process, we extracted 20 agile teamwork key factors from 15 studies.
CE
We present the extracted factors in Table 2.
AC
Factor
Communication
Table 2: Agile Teamwork Key Factors. Definition Information sharing across team members.
Reference [8] [14] [37] [38] [39] [40] [41] [42]
3 http://lapes.dc.ufscar.br/tools/start_tool
14
ACCEPTED MANUSCRIPT
Coordination
Refers to the tasks execution by team mem-
[8] [14] [37] [39] [41]
bers in a synchronized and integrated man-
Cohesion
CR IP T
ner.
Interpersonal attraction between the team
[14] [37] [39]
members, their commitment to the team
tasks, to the team itself, and group pride spirit. Trust
The vulnerability from one part to another
[37] [39] [40] [43]
AN US
based on the expectation that the other will execute a given action, regardless the monitoring or controlling capacity. Collaboration
Refers to the commitment that the team
[14] [37] [40] [42]
members have with each other to achieve the common goals.
The team members share the same values
M
Value Diversity
[37]
ED
and goals. Shared Leadership
The authority on the decision making pro-
[41] [44]
cess is shared.
CE
PT
Team Orientation
Redundancy
AC
Team Autonomy
Team Learning
[8] [9] [15] [39]
Refers to the respect that the team members have with each other
[8] [9] [15] [38] [41] [44] [45]
Team members capacity to replace each other on the tasks execution without the
[8] [15] [38] [44]
need of training. Refers to the influence external agents have
[9] [15] [41]
on the team tasks execution. The team’s capability to identify changes on its environment and adjust its strategies as necessary.
15
[9] [15] [39] [41] [44] [45]
ACCEPTED MANUSCRIPT
Monitoring
Team synchronization regarding the tasks
[8] [41] [44]
and barriers.
Feedback
CR IP T
Refers to the act of providing, forwarding, and receiving information about the team members’ performance.
Set of experiences, comprehensions, and
Culture
[8] [41]
[38]
shared meanings among the team members. Team members personalities.
Team Distribution
The team’s physical distribution.
Team Size
The quantity of members of a team.
AN US
Personality
[38] [39] [46] [46] [46]
The team members’ capacity to contribute Balance of Mem-
with all their necessary knowledge for the
bers Contribution
development of the tasks.
[14]
Workload sharing and team tasks
M
prioritization according to the others in Effort
relation to other obligations are indicators
[14]
ED
of the team members’ effort to execute the common tasks. Team members’ motivation to execute the
[47]
team tasks and work together.
CE
PT
Motivation
AC
5. Proposed Model
295
In this paper, we propose a Bayesian networks-based model to assess the
teamwork quality of agile teams. To build the model, we used the factors presented in Table 2 as a foundation. However, since there are many different agile methods, it is not possible to build a model for all cases due to the differences between these methods and the context of the projects. Therefore, the model
16
ACCEPTED MANUSCRIPT
300
presented in this paper is generic, focused on collocated teams. Moreover, the proposed model is adaptable to fit a given team’s context (e.g., a geographically distributed team or a team that uses Pair Programming), as presented
CR IP T
in Section 6.2. On the other hand, we attempted to minimize the adaptation effort.
We decided to use Bayesian networks due its capability to handle uncertainty
305
[17], formally represent knowledge and the easiness to model and quantify the relationships between the key factors that have impact on the teamwork quality.
Furthermore, it has adequate tool support. We used AgenaRisk 4 , due to its
AN US
support to ranked nodes.
The Bayesian network construction can be divided in two steps: the directed
310
acyclic graph (DAG) construction and the probability functions definition. Each node of the DAG represents a TWQ key factor and there is an edge between two nodes whenever they relate to each other. For each edge, the influenced TWQ key factor on the relationship is the edge’s endpoint. Furthermore, each key factor has possible states and each one of them has an associated probabil-
M
315
ity. Given that, as presented in [48], each key factor represents a set of tuples
ED
N = {(s1 , p1 ), . . . , (s|N | , p|N | )}, where si is a possible state and pi is the associated probability. Also, we have the set of key factors F = {N1 , . . . , N|F | }. Furthermore, we have the set of edges R = {(Nj , Nk ) | Nj ⊂ F ∧ Nk ⊂ F }, where Nj is the initial point and Nk is the endpoint of the edge. Therefore, the
PT
320
problem is to find all elements of the sets F and R. To find all elements of the
CE
set F , we need to identify all key factors Na , and for each key factor Na , find all the possible states si and associated probability pi , where a ≤ |F | and i ≤ |Na |.
Finally, to find all elements of the set R, we need to identify all fj and fk where
fj and fk ∈ F .
AC
325
According to Perkusich et al. [48], a probability function can be represented
by P r = {A|B}, where A is a dependent variable and B is a set of parent nodes.
Thus, the set P = {P ri , . . . , P r|P | } represents all the probability functions for 4 http://www.agena.co.uk/
17
ACCEPTED MANUSCRIPT
the Bayesian network. So, the problem is to find all the elements of the set 330
P . To find them, for each P ri where 1 ≤ i ≤ |P |, we need to quantify the relationship between A and Bj , where 1 ≤ j ≤ |B| and Bj ∈ B.
CR IP T
We interviewed an agile practitioner to help on the process of building the model. Since the construction process was divided, we interviewed the practitioner according to the two phases of this process: an interview to build the 335
DAG, in which the practitioner was responsible for deriving some relationships between the factors; and other one to define the probability functions. Be-
sides the knowledge on agile methods, the practitioner also has knowledge on
AN US
Bayesian networks, which was a key factor to achieve success in building the model.
The practitioner is a certified Scrum Master and Product Owner (PO) by
340
the Scrum Alliance and has a MBA degree in project management. Moreover, he has over five years working as team leader in agile software development
5.1. DAG Construction
M
projects.
To define the DAG, we broke down the problem into two sub-problems:
ED
345
identify the elements of F and R, and identify the elements of N . On the first sub-problem we are only interested on TWQ factors and their relationship. On
PT
the second sub-problem, we identify the possible states and their probabilities for each node on the DAG. We used a top-down approach to, along with the practitioner, decompose
350
CE
the top-level node, Teamwork, into factors that we judged to be observable. To decompose the nodes, we used reasoning and data extracted from the literature
AC
review.
355
According to Mullen & Copper [49], Cohesion refers to the interpersonal
attraction of team members, their commitment to the team task, and grouppride spirit. Other researchers [14, 37, 39] claim that Cohesion is a key factor
to assess Teamwork. Furthermore, Team Autonomy is an important factor to guarantee that the team has an efficient Teamwork [9, 15, 41]. Moe et al. [15] 18
ACCEPTED MANUSCRIPT
describe it as the team’s ability to regulate their boundary conditions, which 360
refers to the external influences of management and other individuals on the activities of the team. Therefore, we can say that if a team is cohesive and
dependent on Cohesion and Team Autonomy.
CR IP T
has autonomy, the TWQ is high. Given this, we decided to make Teamwork
The Agile Manifesto [2] claims that the best solutions come from self-organized 365
teams. Therefore, we considered that, to be cohesive, agile teams must be self-organized. In addition to that, it is necessary that team members collab-
orate to embrace the concept of whole-team responsibility and commitment
Organizing and Collaboration.
AN US
[14, 37, 40, 42]. Therefore, we defined that Cohesion is dependent on Self-
Team Learning, Shared Leadership, and Redundancy are important char-
370
acteristics for self-organized teams [15]. The authors describe them, respectively, as the ability to identify the changes in the team environment and adjust the strategies as needed; the sharing of decision authority and leader-
375
M
ship; and the team members’ capacity to perform (parts of) each others jobs and substitute each other as circumstances demand. Other studies suggest
ED
that these factors need to be addressed for achieving high quality Teamwork [8, 9, 15, 39, 41, 44, 45]. Other works [8, 44, 38] present the concept of Team expertise, the sum of the knowledge of the individuals composing the team, as
380
PT
important for self-organized teams. Given their similarity, we merged the concepts Redundancy and Team expertise into the factor Expertise. Given this, we
CE
added Team Learning, Shared Leadership, and Expertise to the model as parent nodes of Self-Organizing. For a team to collaborate, it must be coordinated. Hoegl & Gemuenden
AC
[14] state that Coordination refers to team members executing their activities
385
in a timely and integrated manner. It is described as a key-factor for Teamwork by several studies [8, 14, 37, 39, 41]. On the other hand, only coordination is not enough. Team Orientation refers to the mutual respect among the team members as well as prioritization of team goals over individual goals [50]. Several researchers [8, 9, 15, 41, 44, 45, 38] identified it as an important indicator of 19
ACCEPTED MANUSCRIPT
390
team collaboration. Given this, we added Coordination and Team Orientation as parents of Collaboration. For a team to be oriented is a result of mixing the personalities of the team
CR IP T
members [39, 38, 46]. Furthermore, the Expertise of the team members with Redundancy, increases the mutual respect among the team members and, con395
sequently, the quality of the Team Orientation. Therefore, we added Personal Attributes and Expertise as parents of Team Orientation.
For a team to coordinate its task, it must communicate. Since agile is based
on tacit knowledge sharing [1], Communication is a must factor to assess agile
400
AN US
teams [8, 14, 37, 39, 41, 40, 42, 38]. Furthermore, Daily Meetings play an important role on synchronizing the team members’ tasks, as well as removing impediments and mitigating risks. Given the definition of Coordination we
believe that these two factors improve the Coordination of the team. So, we introduced both factors in the model as parent nodes of Coordination. The communication of the team depends on the communication channels and team distribution. Furthermore, according to Bustamante [51], the ideal
M
405
agile team is co-located and communicates face-to-face daily. So, the Team
ED
Distribution and the Means of Communication adopted by the team members are factors that influence the quality of the Communication. Therefore, we added these factor to the model and made them parent nodes of Communication. In Daily Meetings, agile teams synchronize their tasks and problems [8, 41,
PT
410
44]. However, it is important that all team members are present on the Daily
CE
Meetings, so they can be aware of what is the current status. Therefore, we added Monitoring and All Members Present as parent nodes of Daily Meetings
AC
in the model. Finally, we present the resulting DAG in Figure 4.
20
ACCEPTED MANUSCRIPT
Teamwork
Team Autonomy
Collaboration
Self-Organizing
Team Orientation
Coordination
Team Distribution
Means of Communication
Personal Attributes
Monitoring
Expertise
Shared Leadership
Team Learning
AN US
Daily Meetings
Communication
CR IP T
Cohesion
All Members Present
M
Figure 4: Proposed model.
For the second sub-problem (i.e., identify the elements of N ), we defined that
415
ED
each node is composed of a 5-point Likert scale (e.g., Very Low, Low, Medium, High, and Very High). Each state corresponds to the quality level of the given
PT
node (i.e., factor). To avoid misunderstanding with the scale, for each node, we defined the expected observation for the extreme states (i.e., Very Low and Very High) in Table 3.
AC
CE
420
Table 3: DAG nodes states definition.
Node
Teamwork
State Very
The team works cohesively and there is no ex-
High
ternal agent affecting the work.
Very
The team does not work cohesively and there
Low
is an external agent affecting the work.
21
ACCEPTED MANUSCRIPT
Node Team Autonomy
State Very
There is no external agent interfering on how
High
the team execute its tasks. The external agent
CR IP T
collaborate with them to define what will be done and only when adequate.
Cohesion
Very
There is an external agent interfering on how
Low
the team execute its tasks.
Very
The team works cohesively and synchronously,
High
prioritizing the team goals, and self-organize
Very Low
AN US
efficiently.
The team does not work cohesively and synchronously, does not prioritize the team goals, and does not self-organize efficiently.
Collaboration
Very High
There is a high degree of collaboration in the team for achieving success on the project de-
M
velopment.
There is no collaboration in the team for
Low
achieving success on the project development.
Very
The team is capable to self-organize effi-
High
ciently in order to face challenges and complex
ED
Very
CE
PT
Self-Organizing
AC
Coordination
Team Orientation
changes.
Very
The team is not capable to self-organize in or-
Low
der to face challenges and complex changes.
Very
The team execute its tasks in a synchronous
High
and integrated manner.
Very
The team does not execute its tasks in a syn-
Low
chronous and integrated manner.
Very
The team members trust each other and feel
High
motivated to work together for achieving the team goals.
22
ACCEPTED MANUSCRIPT
Node Very
The team members do not trust each other and
Low
do not feel motivated to work together.
Very
The communication is effective.
CR IP T
Communication
State
High Very
The communication is not effective.
Low Very
It was possible to gather all obstacles, find a
High
way out of them, and synchronize the team.
Very
It was not possible to gather all obstacles, find
Low Team Distribution
Very High Very
a way out of them, and synchronize the team. All members share the same work place.
None of the members share the same work place.
M
Low
AN US
Daily Meetings
The team members communicate face-to-face
Very
The team members do not communicate face-
Low
to-face.
Very
The team members expose their obstacles and
High
progress regarding their tasks in a clear and
AC
CE
PT
Monitoring
ED
Very Means of Communication High
All Members Present
whenever possible.
objective way. Very
The team members do not expose their ob-
Low
stacles and progress regarding their tasks in a clear and objective way. Instead, they enjoy the opportunity to justify themselves about their decisions [3].
Very
All members attended to the daily meetings.
High
23
ACCEPTED MANUSCRIPT
Node Very
There was no daily meeting in which all mem-
Low
bers attended.
Very
The mixture of personalities contributes for a
High
good relationship of the team members.
Very
The mixture of personalities does not con-
Low
tribute for a good relationship of the team members.
Very
The team members have the necessary knowl-
High
edge for developing the tasks with redundancy.
Very Low
AN US
Expertise
CR IP T
Personal Attributes
State
The team members do not have the necessary knowledge for developing the tasks with redundancy.
Shared Leadership
Very High
shared.
The decision authority and leadership is not
M
Very
The decision authority and leadership is
shared.
Very
The team adapts itself to changes in the
High
team environment and adjust the strategies as
ED
Low
needed.
Very
The team does not adapt itself to changes in
Low
the team environment and adjust the strategies as needed.
AC
CE
PT
Team Learning
5.2. Probability Tables Definition To define the NPT, since we only used ranked nodes, we applied a modified
version of the method presented in [27], which we present an overview in Section
425
2. For each dependent node in the DAG, we defined a set of questions for the domain practitioner. For each question, the practitioner replied with a 24
ACCEPTED MANUSCRIPT
probability for each state of the child node. Afterwards, we analyzed the data collected and calibrated the NPT of the nodes (i.e., defined σ 2 and µ). For brevity, in this paper, we only show the process to define one NPT: Teamwork. We created a webpage5 to present the data collected to define all the NPTs.
CR IP T
430
In Table 4, we show the data collected for Teamwork. By analyzing Table 4,
we concluded that if any of the attributes value is low, Teamwork quality tends to be low. Therefore, we decided to use WMIN function with different weights
for each attribute. Empirically, we chose the weights to be 5 and 1, for Cohesion 435
and Team Autonomy, respectively, and σ 2 = 0.0005, because we observed that
AN US
WMIN tends to be a minimum function reflecting the practitioner’s confidence in the results with this configuration.
Table 4: Data collected to calibrate the NPT of Teamwork.
Teamwork
Team Autonomy
Very Low
Low
Medium
High
Very High
Very High
Very Low
0
0
0
0.8
0.2
Very Low
Very High
0.2
0.8
0
0
0
Very High
Medium
0
0
0
0.2
0.8
Medium
Very High
0
0
0.8
0.2
0
ED
M
Cohesion
PT
6. Procedure to use the Model One of the goals of this paper is to present a procedure to use the model presented in Section 5 with the purpose of assisting on the improvement of the
CE
440
TWQ of agile teams. Perkusich et al. [19] proposed a procedure for detecting problems of processes in software development projects based on Bayesian net-
AC
works. Given that both works are based on Bayesian networks and continuous improvement, we adapt the given procedure to our context.
445
The procedure presented by Perkusich et al. [19], shown in Figure 5, contains 5 https://sites.google.com/site/istbnteamwork/
25
ACCEPTED MANUSCRIPT
five steps. We apply all the steps, but adapt their goals to fit the purposes of
M
AN US
CR IP T
our work. More specifically, we adapted steps I, II and IV.
Figure 5: Perkusich et al. proposed procedure for using the model.
ED
6.1. Step I - Model Construction
The goal of this step is to have a Bayesian network that represents the 450
teamwork key factors, as well as the relationships between them, given the
PT
context of the team in which the procedure will be applied. In Section 5, we presented details regarding how we executed this step. However, in this paper
CE
we propose a generic model that represents the essence of agile teams. Since the difference of agile teams is only on some practices adoption and the geographical
455
distribution, there is no need to perform this step because the model presented
AC
in Section 5 is already considered its output. 6.2. Step II - Model Evaluation This is the initial stage of the cyclic part of the procedure. After constructing
the model, the next step is to define how long the cycles will last (e.g., for 460
iterative processes, a procedure’s cycle could last one iteration), and when each 26
ACCEPTED MANUSCRIPT
stage will be executed. Then, it is necessary to evaluate the model to check if it still suits the project’s current context (i.e., DAG and probability functions). Since the only factors that may differ from one agile team to another are the
465
CR IP T
team members distribution and the adopted agile practices, only the parent nodes of Coordination should be modified to make the model fit the team’s context.
We present two examples on how to modify the model according to a give team as follows: 1. Distributed Team
In distributed teams, it is not possible to consider that the main means
AN US
470
of communication will be a face-to-face conversation. Thereby, it would be possible to replace the parent nodes of Communication (i.e., Team Distribution and Means of Communication) for Frequency and Efficiency. These factors would need to be periodically observed with a certain caution because they are more subjective than Team Distribution and Means of Communication.
M
475
Usually, in distributed teams, the members use email groups and online in-
ED
stant messaging and video conference applications to centralize the team’s communication. Therefore, it would be necessary to observe the Frequency and the Efficiency of the Communication on these applications. By doing
PT
480
so, the resulting modified model would be as represented in Figure 6.
2. Extreme Programming (XP) Teams
CE
In XP teams, it is usually adopted the practices of Pair Programming and Code Review. So, in XP teams that adopt these practices, it would be
AC
485
necessary to add nodes corresponding to these factors to the model. It could be done by creating the node Practices and adding it as child of Daily Meetings, and parent of Coordination. Then, the nodes Pair Programming and Code Review would also be added as parent nodes of Practices. By doing so, the model would represent an XP team that adopts, besides
490
Daily Meetings, PP and Code Review. It would also be possible to add
27
ACCEPTED MANUSCRIPT
Coordination
Daily Meetings
Efficiency
Frequency
AN US
Communication
CR IP T
...
Monitoring
All Members Present
M
Figure 6: DAG example for a distributed team.
parent nodes to PP and Code Review in order to decompose them into
ED
less subjective factors.
Thereby, we present the modified DAG for an XP team as described above
495
PT
in Figure 7.
6.3. Step III - Model Data Input At this stage, the user inputs data (i.e., evidences) to the model. Ideally,
CE
all of the input nodes should have evidences. For the input nodes that data could not be measured from, and therefore, evidences could not be set, the
AC
uncertainty should be the same for all possible states (i.e., a uniform distribu-
500
tion in the interval [0, 1]). After inputting all evidences into the model, the outputs should be calculated using a Bayesian networks specific tool such as
28
ACCEPTED MANUSCRIPT
...
CR IP T
Coordination Practices
Communication Corde Review
Pair Programming
Team Distribution
Means of Communication
AN US
... ... ... ...
Monitoring
Daily Meetings
All Members Present
M
Figure 7: DAG example for an XP team.
GeNIe6 , AgenaRisk7 or Netica8 . For this research, we used AgenaRisk because
ED
it implements the algorithm of ranked nodes, which was adopted in the model construction. The outputs are data with probability values for each state of 505
each node that represent the current quality status of each factor.
PT
This stages outputs are that the calculated results for each node of the model, except for the input nodes. We propose that the outputs of this stage are presented in an iteration retrospective, so all members can be aware of the
CE
team’s problems.
6.4. Step IV - Model Output’s Analysis
AC
510
At this stage, the user should analyze the model’s calculated outputs to
detect problems impacting the TWQ. The goal of this stage is to assess the 6 http://genie.sis.pitt.edu/ 7 http://www.agenarisk.com 8 http://www.norsys.com/
29
ACCEPTED MANUSCRIPT
current TWQ and develop a plan to apply corrective and preventive actions to improve the projects chances of success. There is also the possibility to use 515
sensitivity analysis to prioritize the problems resolution.
CR IP T
Since the factors that influence the TWQ are subjective, there might be some difficulties on the interpretation of the outputs. However, it is known that the TWQ has positive impact on the team’s performance. Yet, the team’s performance, not only depends on the TWQ, but also on other factors such 520
as planning, product complexity, usage agile software engineering techniques, resources, programming languages, and tools [52]. Given this, to analyze the
mance, as an indicator of it.
AN US
model outputs, we propose the users to compare them to the team’s perfor-
The output of this stage is a plan with the corrective and preventive actions 525
identified during it. We propose to use this plan as basis for the planning of new iterations.
M
6.5. Step V - Corrective and Preventive Actions
During this stage, the plan defined in Step IV is executed and managed by
530
ED
the project manager or another project member. At the end of this stage, it is necessary to verify if the actions that were committed to were executed and
PT
analyze their consequences.
7. Case Study
CE
Case study is a research methodology suitable for studying contemporary phenomena in its natural context [53]. To evaluate the proposed procedure, we conducted a case study at Virtus 9 . Virtus is located at the Federal University
of Campina Grande, and we chose it due to its academia-industry relations.
AC
535
Due to availability, we chose three agile software development teams as units
of analysis. The projects in which these teams were working on are executed in partnership with multinational technology companies to develop software 9 http://www.virtus.ufcg.edu.br
30
ACCEPTED MANUSCRIPT
540
products. These projects are managed with Scrum 10 . The duration of the case study was forty-five days (i.e., three iterations). The goal of the case study is to evaluate the practical utility of the proposed
CR IP T
solution to assist on the assessment and improvement of the TWQ of agile teams. With this purpose, we defined the following research questions:
RQ1 : Are the model calculated results acceptable by the subjects at
545
Virtus?
RQ2 : Does the model assist on the detection of opportunities for improv-
AN US
ing the TWQ of an agile team at Virtus?
RQ3 : Is it easy to implement and apply the procedure in Scrum-based teams routine at Virtus?
550
RQ4 : Is the cost-benefit of applying the procedure positive at Virtus?
7.1.1. Units of Analysis
M
7.1. Case Study Design
555
ED
We executed this case study with three agile software development teams. Each unit of analysis was an agile software development team executing projects at Virtus. These units of analysis were nominated Team A, Team B, and Team
PT
C.
The Team A was working on an app for Desktop/Tablet x(86). Team B’s goal was to develop a tool to monitor and control equity security assets using Android. Team C’s goal was to develop a web app that needs to be integrated
CE 560
to other hardware and software components being developed at Virtus and by
AC
their client.
In the context of all teams, the teamwork quality is assessed informally.
Therefore, they do not use any instrument or defined process with the purpose
565
of assessing the teamwork quality. Conversely, they rely on the experience of 10 http://www.scrumguides.org/scrum-guide.html
31
ACCEPTED MANUSCRIPT
the Scrum Masters and daily interaction between the team members and Daily Scrum meetings to assess if there are opportunities to improve teamwork quality. Furthermore, as described by Scrum, at the end of the iterations (i.e., sprints),
570
CR IP T
the team meets to discuss how to improve the way they work during the Sprint Retrospective meeting. In Table 5, we show descriptive data of the units of analysis. Table 5: Units of Analysis Information.
Team
A
B
C
4
6
5
2.5
2.0
2.0
1.0
1.0
2.0
AN US
Characteristic Team size
Mean experience, in years, working on software development projects
Mean experience, in years, working in agile teams
M
7.1.2. Subjects - Who uses the model and the procedure?
For each unit of analysis (i.e., team), the subjects are Scrum Masters. At
575
ED
Virtus, the Scrum Masters execute tasks related to the process and team leadership (i.e., management), design and implementation. Therefore, in practice, one person shares the roles of Scrum Master and technical leader. In Table 6,
PT
we present the profile of the subjects. Table 6: Subjects Profile.
Subject 2
3
Experience, in years, working on software development projects
5.0
10.0
10.0
Experience, in years, leading software development teams
0.5
3.0
2.0
Experience, in years, using metrics for support the decision making process
1.5
2.0
6.0
Experience, in years, using agile methods
5.0
2.0
7.0
CE
1
AC
Characteristic
7.1.3. Methods The data collection is a necessary step to answer the research questions in
580
a case study. According to Lethbridge et al. [54], there are three different 32
ACCEPTED MANUSCRIPT
categories of data collection methods: direct (e.g., interviews), indirect (e.g., questionnaires), and independent (e.g., documentation analysis). In this case study, we collected the data through all methods presented by
585
CR IP T
Lethbridge et al. [54]. The input data for the model was collected through interviews and the performances of the teams on each sprint were calculated through documentation analysis. The research questions were answered by col-
lecting data through questionnaires and analyzing the calculations of the model. The questions to input data to the model and to answer the research questions are available on a website11 .
AN US
Each question to feed the model was created to define the status of the given
590
factor and could be answered in a five-point Likert scale. For example, for the node Expertise, we created the following question: “Do all the team members have the enough knowledge to complete all stories with intersection (i.e., ability to replace each other with minimum effort)?”. The possible answers for this question are given below, as well as the corresponding state of the given node according to each answer:
ED
False → Very Low
M
595
More False than True → Low
PT
Neither False nor True → Medium More True than False → High
600
CE
True → Very High
To collect data to directly answer the research questions, we created a ques-
AC
tionnaire composed of four Likert scale questions with five points, one for each research question. In addition to that, we added three open-ended questions,
605
each one of them related to the first three research questions, in which the subjects could comment and add more information regarding their Likert scale 11 https://sites.google.com/site/istbnteamwork/
33
ACCEPTED MANUSCRIPT
answers. However, the open-ended question for RQ1 was asked at the end of each sprint, which allowed us to collect more information regarding the acceptance of the model calculated results, as well as to help on the analysis process. We analyzed the open-ended data using thematic analysis [55].
CR IP T
610
The possible answers for each Likert scale question were the same as the ones to feed model, as described before. 7.1.4. Case Study Procedure
We used AgenaRisk to execute the Bayesian networks calculations because it implements the algorithm of ranked nodes, which we applied to our model. Due
AN US
615
to licensing limitations, the models were built and executed on the computer of one of the researchers. For each unit of analysis, we executed the procedure presented in Figure 5, during three sprints. We conducted this case study in two phases.
Introductory Course - During this phase, we trained the subjects to
620
M
use the model and procedure, and presented them the concepts related to this research. The training format was a lecture, in which the lecturer was one of
ED
the authors. This phase lasted one hour. Procedure Usage - After the first phase, the subjects used the procedure 625
presented in Section 6 during three sprints, each one lasting two weeks. To feed
PT
the model, a researcher interviewed the Scrum Masters at the end of each sprint to collect the data. In these interviews, a researcher was responsible for asking the questions regarding the quality of each model’s node, as well as collecting the
CE
data for calculating the team’s performance for that sprint. Then, the researcher
630
used the AgenaRisk tool to perform the model calculations. These results, were
AC
compared with the teams’ performance, as indicator of it (i.e., Section 6.4), to elaborate the corrective and preventive plan. Since the units of analysis adopt Scrum, we used the performance measure
proposed by Kumar [56] to calculate the teams’ performance in each sprint.
635
This approach is based on three parameters: 1. # user stories planned vs # user stories delivered - Measures planning 34
ACCEPTED MANUSCRIPT
effectiveness, efficiency; 2. # user stories delivered vs # user stories accepted - Measures quality of the deliverables; 3. # bugs found vs # bugs fixed - Measures defects-removal rate;
CR IP T
640
With these measures, we calculate the overall percentage, which is the mean of these measures. The resulting percentage is considered the team’s performance for a given sprint. Moreover, Kumar [56] provide a scale for each one of the parameters, as well as for the overall performance. However, we can define
our own scales based on our requirements. The average time the subjects 1,
AN US
645
2, and 3 took to input the data to the model was, respectively, 15, 15, and 5 minutes.
At the end of the three sprints, the subjects responded the questions related to the research questions. 7.2. Results
M
650
We present the model outputs for each unit of analysis, for each sprint, in Figure 8, Figure 9, and Figure 10. We present the calculated performances of
AC
CE
PT
ED
the teams in Table 7.
35
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 8: Model outputs for Team A.
36
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 9: Model outputs for Team B.
37
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 10: Model outputs for Team C.
38
ACCEPTED MANUSCRIPT
Table 7: Teams’ Performance in each sprint.
Team A
B
C
Sprint 1
98,610%
95,230%
73,330%
Sprint 2
69,230%
100%
100%
Sprint 3
70,510%
100%
77,770%
CR IP T
Sprint
We describe the answers provided for each one of the Likert scale questions designed to directly answer the research questions as follows:
AN US
655
1. “Do you agree with the model calculated results? ” - related to RQ1 : One subject answered True, and the other two answered More True than False.
2. “Does the model assist on the detection of opportunities for improving agile team’s TWQ? ” - related to RQ2 :
660
M
One subject answered True, and the other two answered More True than False.
ED
3. “Is it easy to implement and use the procedure in Scrum-based teams routine? ” - related to RQ3 :
All three subjects answered True.
665
PT
4. “Is the cost-benefit of applying the procedure positive? ” - related to RQ4 : Two subjects answered True, and the other one answered More True than
CE
False.
7.3. Discussion In this section, we discuss each research question given the collected data
AC
670
presented in Section 7.2. Furthermore, we present benefits and challenges of using the proposed model. To answer this RQ1 , we triangulated the model calculated results, the
teams’ performances, and facts that occurred during the sprints, which could
39
ACCEPTED MANUSCRIPT
675
explain the model calculated results as well as the teams’ performances. These facts were provided by the open-ended question regarding this research question. The Subject A reported that, for the Sprint 1, they had no challenges, since
CR IP T
most of the risks were avoided during the sprint planning and no external agents affected their execution. However, for the Sprint 2, some members had to be 680
reallocated to other tasks in the middle of the sprint, which had a huge impact on their Autonomy, affecting negatively on the TWQ. Moreover, similarly to the
Sprint 2, on the Sprint 3, the Team Autonomy was negatively affected because
they needed to re-prioritize some issues due to a delivery that was not discussed
685
AN US
with the team members during the planning meeting. These facts are consistent with the model calculated results and team’s performance on each sprint. The Team B was the most regular one, despite the low quality reported in each one of the three sprints. According to the Subject B, it did not affect much the overall TWQ because they were already aware of the PO’s behavior of changing the scope and could anticipate him on the sprint planning meetings. Therefore, the model calculated results, the team’s performance, and the TWQ
M
690
are consistent with each other.
ED
The Subject C was responsible for managing a team composed with parttime and full-time developers. According to him, some part-time developers are not mature enough to handle some responsibilities. In addition, it is also a problem to get the whole team united during the Daily Meetings, which affected
PT
695
the team’s Communication. For the Sprints 1 and 2, the Team C worked on
CE
code refactoring and preparing the code for upcoming requirements. However, Subject C reported the difference between the performance on Sprint 1 and Sprint 2 occurred because the planning for the first sprint was not efficient as the planning for the second sprint. For the Sprint 3, the Subject C reported that
AC
700
the web services started to be handled by the client’s team. Since the client’s team did not follow exactly what was documented and the communication with them was not efficient, it led the Team C to compromise the sprint, decreasing its performance.
705
Therefore, given that all facts, model calculated results, and teams’ perfor40
ACCEPTED MANUSCRIPT
mances, on each one of the sprints, are consistent with each other, it is possible to answer that the model measurements of the TWQ are acceptable, given the case study context. Moreover, the subjects agreed with the model calculated
CR IP T
results, given that they matched their thoughts about the teams’ TWQ. Regarding RQ2 , by applying the procedure and using the proposed model,
710
the subjects reported that model outputs are useful on detecting weaknesses
and strengths to allow continuous improvement of the TWQ. Since the outputs are quantitative, they claimed that this helped on prioritizing corrective actions and defining a plan to execute them.
Moreover, in the Likert scale answers, the subjects reported that the model
AN US
715
assists on the detection of opportunities for improving agile team’s TWQ. In regards to RQ3 , the subjects agree that the procedure adoption and usage is simple and beneficial, as answered in the Likert scale questions. In addition to that, the average time they took to input data to the model indicates that this 720
process is simple, and the model structure is well defined. However, as stated by
M
them, the only problem on applying the procedure was the fact that the model calculations could only be performed on the computer of one of the researchers.
ED
According to them, if they had access to the tool, it would be much easier and better to apply the procedure. Based on these answers, we concluded that it is 725
easy to implement and apply the procedure in Scrum-based teams routine at
PT
Virtus.
To answer RQ4 , we analyzed the time needed for the subjects to input data
CE
to model, the duration of time for the introductory course, and the answers regarding the benefits of using the model and applying the procedure. Based
730
on these data, we concluded that the cost-benefit of applying the procedure
AC
positive at Virtus is positive. Furthermore, the Likert scale answers given by the subjects enhance our conclusion. In Table 8, we present the benefits and challenges of using the proposed
model and procedure in a daily basis, according to the collected data and our
735
observations.
41
ACCEPTED MANUSCRIPT
Table 8: Summarized Results.
Benefits
Shortcomings • The input nodes are subjective.
• Outputs of the model are
CR IP T
• The model still requires more
acceptable.
empirical validation.
• Assists on the detection of
• It requires person/people
TWQ improvement opportunities.
performing the
• Helps on prioritizing corrective
evaluation to be impartial.
actions regarding TWQ.
• It requires the AgenaRisk tool to perform the results calculations.
AN US
• Adoption is simple, and beneficial.
7.4. Threats to Validity
Runeson & H¨ ost [53] state that there are different ways to classify threats to validity aspects in the literature. The authors suggest a classification schema
740
M
that distinguish four validity aspects of case studies. These aspects are: Construct Validity, Internal Validity, External Validity, and Reliability. Based on
ED
what is stated in [53], we identified the following threats to validity: • Construct Validity: The structure of the proposed model presents a new
PT
set of indicators that are not specifically based on prior work, although the questions regarding the input nodes are inspired on previous work. There-
745
fore, the measure of teamwork quality is uncertain. To reduce the effort
CE
of using the proposed model, we only defined one question for each one of the input nodes, which can cause bias on the input. We tried to minimize
AC
this threat by providing an introductory course and defining meaning for
750
the Likert scales. Another critical point is the lack of validation regarding the measures proposed on Kumar [56] to calculate the performance of the teams; • Internal Validity: The model selection problem is a grievous concern to the internal validity because, since we did not have data, the parameters 42
ACCEPTED MANUSCRIPT
of the model are based on the knowledge of only one practitioner. In the presence of data, it would be able to perform independence test such
755
as applying the PC algorithm [57]. Furthermore, the TWQ was adopted
CR IP T
as an indicator of the teams’ performances, but there are other factors that influence this measure such as business stability, company structure
and organizational maturity. Given that these factors were ignored, the interpretation of the model outputs might be imprecise. Finally, since
760
the units of analysis did not use a formal process to assess teamwork, our conclusions might not be specific to our model and procedure, but to using
AN US
a process to assess teamwork;
• External Validity: Since the study objects were observed in only three Scrum-based projects with small teams and we only collected data from
765
one of the team members, we can conclude that the model has not been empirically validated and, therefore, it cannot be generalized. For an
M
empirically validated causal model, it is necessary to collect enough data to evaluate the hypotheses for each relationship using an approach such as SEM. Anderson and Vastag [29] speculate that a sample size of 100
770
ED
cases would be reasonable, but it depends on the number of states in the variables and the number of parents of the variables. On the other hand,
PT
the types of products developed by the teams were different, which we believe can give more credibility to our conclusions
• Reliability: Since the subjects are providing data regarding their teams,
CE
775
the data might be biased. In addition to that, the questions asked during the interviews and defined in the questionnaires might not be clear enough
AC
to avoid biased input.
8. Conclusions and Future Works
780
In this paper, we presented an analytically practitioner-based Bayesian net-
work model to assess and improve the TWQ of agile teams. In addition to the
43
ACCEPTED MANUSCRIPT
model, we also present a procedure to use and adapt it. To build the model, we performed a literature review to identify the factors that influence the TWQ and identified the relationships between by interviewing an expert. The procedure is based on [19], which is composed of five steps: (i) Bayesian network
CR IP T
785
Construction, (ii) Bayesian network Evaluation, (iii) Bayesian network Data Input, (iv) Bayesian network outputs analysis and (v) execution of corrective and preventive actions.
To evaluate both model and procedure, we performed a case study in three 790
Scrum-based software development teams, during three sprints. According to
AN US
our results, in the context of the case study, the model assists agile teams on
assessing teamwork quality and identifying improvement opportunities, and the cost-benefit for using with the proposed procedure is positive. Moreover, the subjects of the case study reported they found easy to implement and apply 795
the procedure on their teams’ routines. Even though the relationships between the nodes of the model are not empirically validated, we believe that we have
M
achieved promising results regarding the model, based on the discussion about RQ1, presented in Section 7.3.
800
ED
The limitations of this study are related to the Bayesian network construction and the case study’s threats to validity. We executed the case study with only three teams in one company. Furthermore, the teams did not use a formal
PT
process to assess teamwork. Therefore, our conclusions might not be specific to both model and procedure, but to using a tool to assess teamwork.
CE
For future works, we plan to apply both model and procedure in more soft805
ware development teams, in different contexts and maturity levels to validate it and continuously calibrate it. Moreover, we plan on evaluating our solution by
AC
collecting data from the other members of the development team, not only the team leader or Scrum Master. With this, we intend to enhance the reliability of the model and procedure, as well as verify divergences regarding the different
810
results for different roles in a team. Once we have more data, we intend to explore the use of SEM to verify [58] and, possibly, refine the model structure. In addition to that, we encourage researchers to apply the procedure to other 44
ACCEPTED MANUSCRIPT
agile methods, especially in immature teams. Finally, we plan on developing a tool to implement the model presented in this paper to be used by practitioners, so we can avoid licensing issues.
CR IP T
815
References
[1] B. Boehm, R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2003.
[2] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham,
AN US
820
M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, D. Thomas, Manifesto for agile software development, http://www.agilemanifesto.org/ (2001).
[3] V. Stray, D. I. Sjøberg, T. Dyb˚ a, The daily stand-up meeting: A grounded
M
825
theory study, Journal of Systems and Software 114 (2016) 101–124. doi:
ED
http://dx.doi.org/10.1016/j.jss.2016.01.004. [4] L. Williams, R. R. Kessler, W. Cunningham, R. Jeffries, Strengthening the case for pair programming, IEEE Software 17 (4) (2000) 19–25. doi: 10.1109/52.854064.
PT
830
[5] K. Molkken-stvold, N. C. Haugen, H. C. Benestad, Using planning poker
CE
for combining expert estimates in software projects, Journal of Systems and Software 81 (12) (2008) 2106–2117. doi:http://dx.doi.org/10.1016/j.
AC
jss.2008.03.058.
835
[6] J. R. Katzenbach, D. K. Smith, The wisdom of teams: Creating the highperformance organization, Harvard Business Press, 1993.
[7] T. Chow, D.-B. Cao, A survey study of critical success factors in agile software projects, Journal of Systems and Software 81 (6) (2008) 961–971. doi:http://dx.doi.org/10.1016/j.jss.2007.08.020. 45
ACCEPTED MANUSCRIPT
840
[8] N. B. Moe, T. Dingsøyr, T. Dyb˚ a, A teamwork model for understanding an agile team: A case study of a scrum project, Information and Software Technology 52 (5) (2010) 480 – 491. doi:http://dx.doi.org/10.1016/
CR IP T
j.infsof.2009.11.004. [9] M. A. Ringstad, T. Dingsøyr, N. Brede Moe, Systems, Software and Service Process Improvement: 18th European Conference, EuroSPI 2011,
845
Roskilde, Denmark, June 27-29, 2011. Proceedings, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, Ch. Agile Process Improvement: Diag-
978-3-642-22206-1\_15. 850
AN US
nosis and Planning to Improve Teamwork, pp. 167–178. doi:10.1007/
[10] R. M. Fontana, I. M. Fontana, P. A. da Rosa Garbuio, S. Reinehr, A. Malucelli, Processes versus people: How should agile software development maturity be defined?, Journal of Systems and Software 97 (2014) 140 – 155.
M
doi:http://dx.doi.org/10.1016/j.jss.2014.07.030.
[11] L. Williams, K. Rubin, M. Cohn, Driving process improvement via comparative agility assessment, in: Proceedings of the 2010 Agile Conference,
855
ED
AGILE ’10, IEEE Computer Society, Washington, DC, USA, 2010, pp. 3–10. doi:10.1109/AGILE.2010.12.
PT
[12] S. Wood, G. Michaelides, C. Thomson, Successful extreme programming: Fidelity to the methodology or good teamworking?, Information and Software Technology 55 (4) (2013) 660 – 672. doi:http://dx.doi.org/10.
860
CE
1016/j.infsof.2012.10.002.
AC
[13] E. Amengual, A. Mas, A. Mesquida, Team spice: A spice-based teamwork
865
assessment model, in: A. Riel, R. OConnor, S. Tichkiewitch, R. Messnarz (Eds.), Systems, Software and Services Process Improvement, Vol. 99 of Communications in Computer and Information Science, Springer Berlin Heidelberg, 2010, pp. 37–47. doi:10.1007/978-3-642-15666-3\_4. URL http://dx.doi.org/10.1007/978-3-642-15666-3_4
46
ACCEPTED MANUSCRIPT
[14] M. Hoegl, H. G. Gemuenden, Teamwork quality and the success of innovative projects: A theoretical concept and empirical evidence, Organization science 12 (4) (2001) 435–449.
870
CR IP T
[15] N. Moe, T. Dingsøyr, E. Røyrvik, Putting agile teamwork to the test an preliminary instrument for empirically assessing and improving ag-
ile software development, in: P. Abrahamsson, M. Marchesi, F. Maurer
(Eds.), Agile Processes in Software Engineering and Extreme Programming, Vol. 31 of Lecture Notes in Business Information Processing, Springer
875
Berlin Heidelberg, 2009, pp. 114–123. doi:10.1007/978-3-642-01853-4\
AN US
_14.
[16] Y. Lindsjørn, D. I. Sjberg, T. Dingsøyr, G. R. Bergersen, T. Dyb˚ a, Teamwork quality and project success in software development: A survey of agile development teams, Journal of Systems and Software 122 (2016)
880
274 – 286. doi:http://dx.doi.org/10.1016/j.jss.2016.09.028. http://www.sciencedirect.com/science/article/pii/
S016412121630187X
M
URL
ED
[17] I. Ben-Gal, Bayesian Networks, John Wiley & Sons, Ltd, 2008. doi:10. 1002/9780470061572.eqr089.
885
PT
URL http://dx.doi.org/10.1002/9780470061572.eqr089 [18] K. Verbert, R. Babuka, B. D. Schutter, Bayesian and dempstershafer reasoning for knowledge-based fault diagnosisa comparative study, En-
CE
gineering Applications of Artificial Intelligence 60 (2017) 136 – 150. doi:http://doi.org/10.1016/j.engappai.2017.01.011.
890
AC
URL
http://www.sciencedirect.com/science/article/pii/
S0952197617300118
[19] M. Perkusich, G. Soares, H. Almeida, A. Perkusich, A procedure to de-
895
tect problems of processes in software development projects using bayesian networks, Expert Systems with Applications 42 (1) (2015) 437 – 450.
47
ACCEPTED MANUSCRIPT
[20] P. Hearty, N. Fenton, D. Marquez, M. Neil, Predicting project velocity in xp using a learning dynamic bayesian network model, IEEE Transactions on Software Engineering 35 (1) (2009) 124–137. doi:10.1109/TSE.2008.76.
CR IP T
[21] A. Rodrguez, F. Ortega, R. Concepcin, A method for the evaluation of risk in {IT} projects, Expert Systems with Applications 45 (2016) 273 –
900
285. doi:http://dx.doi.org/10.1016/j.eswa.2015.09.056. URL
http://www.sciencedirect.com/science/article/pii/
S0957417415006855
AN US
[22] C.-G. Bai, Bayesian network based software reliability prediction with an operational profile, J. Syst. Softw. 77 (2) (2005) 103–112. doi:10.1016/
905
j.jss.2004.11.034.
[23] T. Schulz, L. Radli´ nski, T. Gorges, W. Rosenstiel, Predicting the flow of defect correction effort using a bayesian network model, Empirical Software
910
M
Engineering 18 (3) (2011) 435–477. doi:10.1007/s10664-011-9175-7. [24] A. Silva Freire, R. Da Silva, M. Perkusich, H. Almeida, A. Perkusich, A bayesian network model to assess agile teams’ teamwork quality, in: Soft-
ED
ware Engineering (SBES), 2015 29th Brazilian Symposium on, 2015, pp. 191–196. doi:10.1109/SBES.2015.29.
PT
[25] N. Friedman, D. Geiger, M. Goldszmidt, Bayesian network classifiers, Machine Learning 29 (2-3) (1997) 131–163.
915
CE
[26] J. Pearl, S. Russell, Bayesian networks, Handbook of brain theory and neural networks.
AC
[27] N. Fenton, M. Neil, J. G. Caballero, Using ranked nodes to model qual-
920
itative judgments in bayesian networks, Knowledge and Data Engineering, IEEE Transactions on 19 (10) (2007) 1420–1432. doi:10.1109/TKDE. 2007.1073.
[28] D. Stankovic, V. Nikolic, M. Djordjevic, D.-B. Cao, A survey study of critical success factors in agile software projects in former yugoslavia {IT} 48
ACCEPTED MANUSCRIPT
companies, Journal of Systems and Software 86 (6) (2013) 1663–1678. doi: http://dx.doi.org/10.1016/j.jss.2013.02.027.
925
[29] R. D. Anderson, G. Vastag, Causal modeling alternatives in operations
CR IP T
research: Overview and application, European Journal of Operational
Research 156 (1) (2004) 92 – 109, eURO Excellence in Practice Award 2001. doi:https://doi.org/10.1016/S0377-2217(02)00904-9. URL
930
http://www.sciencedirect.com/science/article/pii/
S0377221702009049
AN US
[30] A. Qumer, B. Henderson-Sellers, A framework to support the evaluation,
adoption and improvement of agile methods in practice, Journal of Systems and Software 81 (11) (2008) 1899 – 1919. doi:http://dx.doi.org/10. 1016/j.jss.2007.12.806.
935
[31] T. J. Gandomani, M. Z. Nafchi, An empirically-developed framework for
M
agile transition and adoption: A grounded theory approach, Journal of Systems and Software 107 (2015) 204 – 219. doi:http://dx.doi.org/10.
940
ED
1016/j.jss.2015.06.006.
[32] O. Salo, P. Abrahamsson, An iterative improvement process for agile software development, Software Process: Improvement and Practice 12 (1)
PT
(2007) 81–100. doi:10.1002/spip.305. URL http://dx.doi.org/10.1002/spip.305
CE
[33] I. Aaen, J. Arent, L. Mathiassen, O. Ngwenyama, A conceptual map of software process improvement, Scandinavian Journal of Information Sys-
945
AC
tems 13 (2001) 123–146. URL http://dl.acm.org/citation.cfm?id=565431.565437
[34] E. Amengual, A. Mas, Teamwork best practices in iso/iec 15504, in: Pro-
950
ceedings of the 9th International Conference on Software Process Improvement and Capability Determination, 2009, pp. 106–112.
49
ACCEPTED MANUSCRIPT
[35] E. Amengual-Alcover, A. Mas-Picacho, Can teamwork management help in software quality and process improvement?, Improving Quality in Business Processes, Products and Organizational Systems (2009) 26.
CR IP T
[36] B. Kitchenham, S. Charters, Guidelines for performing Systematic Literature Reviews in Software Engineering, Tech. Rep. EBSE 2007-001, Keele
955
University and Durham University Joint Report (2007).
[37] E. Weimar, A. Nugroho, J. Visser, A. Plaat, Towards high performance software teamwork, in: Proceedings of the 17th International Conference
AN US
on Evaluation and Assessment in Software Engineering, EASE ’13, ACM,
New York, NY, USA, 2013, pp. 212–215. doi:10.1145/2460999.2461030.
960
URL http://doi.acm.org/10.1145/2460999.2461030
[38] Y. KOZAK, Barriers against better team performance in agile software projects, Master’s thesis, Chalmers University of Technology, Sweden
965
M
(2013).
[39] C. T. Schmidt, T. Kude, A. Heinzl, S. Mithas, How agile practices influence
ED
the performance of software development teams: The role of shared mental models and backup, in: ICIS 2014 Proceedings, AISeL, Atlanta, Ga., 2014, p. Paper 15.
970
PT
URL http://ub-madoc.bib.uni-mannheim.de/37362/ [40] A. Cockburn, J. Highsmith, Agile software development, the people factor,
CE
Computer 34 (11) (2001) 131–133. doi:10.1109/2.963450. [41] C. Gurram, S. G. Bandi, Teamwork in distributed agile software develop-
AC
ment, Master’s thesis, Blekinge Institute of Technology, School of Comput-
975
ing (2013).
[42] A. Johansson, Toward improvements of teamwork in globally distributed agile teams, Bachelor of science thesis in software engineering and management, University of Gothenburg (2013).
50
ACCEPTED MANUSCRIPT
[43] G. Tjørnehøj, M. Fransg˚ ard, S. Skalkam, Trust in agile teams in distributed software development, in: Information System Research Seminar in Scandinavia 2012, 2012.
980
CR IP T
[44] L. M. R. Haraldsen, An investigation of team effectiveness in agile soft-
ware development, Master’s thesis, Norwegian University of Science and Technology (2012).
[45] V. Gulliksen Stray, N. B. Moe, T. Dingsøyr, Agile Processes in Software Engineering and Extreme Programming: 12th International Confer-
985
AN US
ence, XP 2011, Madrid, Spain, May 10-13, 2011. Proceedings, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, Ch. Challenges to Teamwork: A Multiple Case Study of Two Agile Teams, pp. 146–161. doi: 10.1007/978-3-642-20677-1_11.
URL http://dx.doi.org/10.1007/978-3-642-20677-1_11
990
M
[46] V. Lalsing, S. Kishnah, S. Pudaruth, People factors in agile software development and project management, International Journal of Software Engi-
ED
neering & Applications (IJSEA) 3 (1) (2012) 117–137. [47] E. Whitworth, R. Biddle, The social nature of agile teams, in: Agile Conference (AGILE), 2007, 2007, pp. 26–36. doi:10.1109/AGILE.2007.60.
995
PT
[48] M. Perkusich, H. O. de Almeida, A. Perkusich, A model to detect problems on scrum-based software development projects, in: Proceedings of the 28th
CE
Annual ACM Symposium on Applied Computing, ACM, 2013, pp. 1037– 1042.
[49] B. Mullen, C. Copper, The relation between group cohesiveness and per-
AC
1000
formance: An integration., Psychological bulletin 115 (2) (1994) 210.
[50] E. Salas, D. E. Sims, C. S. Burke, Is there a big five in teamwork?, Small Group Research 36 (5) (2005) 555–599. arXiv:http://sgr.sagepub.com/ content/36/5/555.full.pdf+html, doi:10.1177/1046496405277134.
51
ACCEPTED MANUSCRIPT
1005
[51] S. R. Bustamante, A., Agile xxl:
Scaling agile for project teams,
seapine software, inc., http://downloads.seapine.com/pub/ebooks/ AgileScaling_eBook.pdf, accessed in 01/10/2017 (2011).
CR IP T
[52] C. Melo, D. Cruzes, F. Kon, R. Conradi, Agile team perceptions of productivity factors, in: Agile Conference (AGILE), 2011, 2011, pp. 57–66. doi:10.1109/AGILE.2011.35.
1010
[53] P. Runeson, M. H¨ ost, Guidelines for conducting and reporting case study
research in software engineering, Empirical Software Engineering 14 (2)
AN US
(2009) 131–164. doi:10.1007/s10664-008-9102-8.
[54] T. C. Lethbridge, S. E. Sim, J. Singer, Studying software engineers: Data collection techniques for software field studies, Empirical Software Engi-
1015
neering 10 (3) (2005) 311–341. doi:10.1007/s10664-005-1290-x. URL http://dx.doi.org/10.1007/s10664-005-1290-x
M
[55] John Wiley and Sons, New York, Introduction to qualitative research methods: The search for meanings (1984). [56] M. P. Kumar,
A simple way to measure the performance of
ED
1020
scrum teams, https://www.scrumalliance.org/community/articles/ 2014/may/simple-way-to-measure-performance-of-scrum-teams, ac-
PT
cessed in 01/10/2017 (2014).
CE
[57] S. P. G. C. M. C. Scheines, R., Lawrence Erlbaum, 1994. 1025
[58] S. Gupta, H.-W. Kim, Application of bayesian modeling to management
AC
information systems: a latent scores approach, in: Bayesian Network Technologies: Applications and Graphical Models, IGI Global, 2007, pp. 103– 126.
52