Simulating evolution in model-based product line engineering

Information and Software Technology 52 (2010) 758–769

Contents lists available at ScienceDirect

Information and Software Technology journal homepage: www.elsevier.com/locate/infsof

Simulating evolution in model-based product line engineering Wolfgang Heider a,*, Roman Froschauer b,1, Paul Grünbacher a, Rick Rabiser a, Deepak Dhungana c,2 a

Johannes Kepler University Christian Doppler Laboratory for Automated Software Engineering, Altenberger Str. 69, 4040 Linz, Austria Upper Austrian University of Applied Sciences, Stelzhamerstr. 23, 4600 Wels, Austria c Lero – The Irish Software Engineering Research Centre, University of Limerick, Limerick, Ireland b

a r t i c l e Article history: Received 5 October Received in revised Accepted 20 March Available online 29

i n f o 2009 form 15 March 2010 2010 March 2010

Keywords: Product line engineering Maintenance and evolution Model-based development Simulation Industrial automation systems

a b s t r a c t Context: Numerous approaches are available for modeling product lines and their variability. However, the long-term impacts of model-based development on maintenance effort and model complexity can hardly be investigated due to a lack of empirical data. Conducting empirical research in product line engineering is difficult as companies are typically reluctant to provide access to data from their product lines. Also, many benefits of product lines can be measured only in longitudinal studies, which are difficult to perform in most environments. Objective: In this paper, we thus aim to explore the benefit of simulation to investigate the evolution of model-based product lines. Method: We present a simulation approach for exploring the effects of product line evolution on model complexity and maintenance effort. Our simulation considers characteristics of product lines (e.g., size, dependencies in models) and we experiment with different evolution profiles (e.g., technical refactoring vs. placement of new products). Results: We apply the approach in a simulation experiment that uses data from real-world product lines from the domain of industrial automation systems to demonstrate its feasibility. Conclusion: Our results demonstrate that simulation contributes to understanding the effects of maintenance and evolution in model-based product lines. Ó 2010 Elsevier B.V. All rights reserved.

1. Introduction Software product lines are typically developed and maintained using model-based approaches. For example, feature-based [1–3], decision-oriented [4,5], or orthogonal [6] approaches have been proposed to define and manage product lines’ variability. Similarly, architecture description languages [6–8] and UML-based techniques [9,10] have been adopted for this purpose. It is however difficult to evaluate the benefits and drawbacks of these approaches in product line engineering (PLE) [11]. For instance, product line models need to be observed over a long time period to unveil their real benefits. This is infeasible in many practical settings. Also, product lines represent success-critical assets and companies are typically reluctant to give researchers access to data from their product lines. In contrast to economic approaches [12] for determining the return on investment of adopting product lines, we aim at

* Corresponding author. Tel.: +43 (0) 732 2468 8873; fax: +43 (0) 732 2468 8878. E-mail addresses: [email protected] (W. Heider), [email protected] (R. Froschauer), [email protected] (P. Grünbacher), [email protected] (R. Rabiser), [email protected] (D. Dhungana). 1 Tel.: +43 (0) 7242 72811 4416; fax: +43 (0) 7242 72811 94416. 2 Tel.: +353 (0) 61 233704; fax: +353 (0) 61 213036. 0950-5849/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.infsof.2010.03.007

investigating the effects of long-term maintenance and evolution of product lines [13–15]. More specifically, in this work we use simulation to explore the effects of different product line characteristics (e.g., size, dependencies in models) and different product line evolution profiles (e.g., technical refactoring vs. placement of new products) on model maintenance effort and model complexity. Simulations are attractive for providing empirical support and for justifying software engineering methods. Simulations are also common and accepted as evidence within the management research literature where evaluation challenges are often similar to those in software engineering [16]. They provide a comparably inexpensive way to gain insights as the costs and risks of experimenting with real product lines (especially the costs of conducting longitudinal studies) are prohibitively high. Controlled experiments are difficult and often infeasible to perform in PLE due to the long time span and a lack of data points caused by highly uncontrollable conditions and circumstances. Therefore, only few authors report on studies about real product lines and their evolution [17]. Product lines are long-living entities and new customer requirements, changing business strategies, or technical changes inevitably require continuous changes to product line models [18]. Our simulation approach thus assists product line engineers in assessing model evolution and estimating maintenance effort for their

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

product line environments. We present an extensible simulation tool that allows defining input parameters about a product line’s characteristics and specific evolution profiles. The result variables of the simulation are also customizable to allow computing relevant maintenance metrics (e.g., model complexity) as required in different domains. The paper is structured as follows: In Section 2 we present challenges in the domain of component-based industrial automation systems to further motivate the need for this research. In Section 3 we describe our overall approach. Our tool for simulating product line evolution is presented in Section 4. We evaluate the feasibility of our simulation approach by applying it to concrete product lines in the domain of industrial automation systems (Section 5). Finally, we discuss results, threats to validity, and related work and then conclude the paper. 2. Problem illustration: evolving industrial automation system product lines We use the domain of industrial automation systems (IAS) later in this paper to evaluate our simulation approach. To motivate our research, we briefly describe IAS characteristics, discuss models used to define product lines in IAS, and illustrate the vital role of evolution in this domain. 2.1. IAS and components IAS consist of numerous complex sensor units cooperating with actuators to perform measurement and control in industrial applications [17,20]. Compared to more mainstream software systems IAS are characterized by their distributed and hardware-related nature. IAS are often based on the standard IEC 61131-3 [19], which is dedicated to systems based on programmable logic controllers (PLCs). In such environments function blocks are used to define robust and reusable software components. More recently, the emerging standard IEC 61499 [21] strengthened this component-based view by defining a reference architecture for distributed, modular, and flexible control systems. The standard specifies a highly generic architectural model for distributed applications in industrial process measurement and control systems as well as guidelines for developing such systems. Within the last few years IAS have advanced significantly with regard to the clear identification and definition of components and enabling reuse being one of the main aims [22–27]. Function blocks serve as the basic building blocks in IAS and define inputs and outputs for events and data. The standard IEC 61499 defines a type/instance concept as also common in programming languages. Function blocks are instances of function block types which define both the external interfaces and the internal execution. Suender [28] recognizes IEC 61499 function blocks as software components based on common definitions similar to those by Weck and Szyperski [29] in the area of software engineering. Developing IEC 61499 applications means defining function block networks that determine the dependencies among function blocks. IAS consist of one or multiple function block networks with interconnected function block instances. Creating and evolving such applications means introducing new types and wiring their instances.

759

reconfiguration at runtime (dynamic product lines [31]). Defining IAS as product lines requires modeling the problem space and the solution space (cf. Fig. 1). Problem space models define the externally visible variability offered by the system. For instance, an engineer can take customization decisions at design time or runtime to modify an existing IAS, e.g., to activate an existing feature which is already available in a plant (i.e., the move parts feature). Solution space models specify the product line’s components (i.e., the function block FBMoveBottle_Slow) and their interdependencies. Distinguishing between design time and runtime aspects allows reflecting component types and component instances in IAS [31]. Problem space decisions lead to the selection of component types for a particular domain at design time or the inclusion of component instances at runtime (cf. Fig. 1). The examples in Fig. 1 are taken from earlier work [31], in which we applied the existing product line modeling approach DOPLER [4] to a bottle-sorting plant. The product line model needed to consider customer requirements such as ‘‘The system should allow transporting two types of bottles from a defined user-interaction point, detecting the type of bottle, and sorting it into one of two boxes based on its type.” The model also needed to take into account the existing plant layout, i.e., two conveyor belts, a pick & place unit, a color sensor, and a switchblade. For the bottle-sorting example, we modeled the hardware and software variability of the plant to support customer-specific developments and adaptations (Fig. 1 shows a simplified part of the variability model). Although the system consists of only eight components, numerous variations exist regarding the placement of each component and their order of execution. In the DOPLER approach the problem space is defined through decision models and the solution space is defined through asset models. Decisions represent the variability of a system by defining the set of choices available at a certain point in time. Taking a decision involves judging the merits of multiple options and selecting one of them for action. This sets the value of the decision. Possible values are limited by the decision type (e.g., Boolean, number, enumeration). The decision model defines dependencies among decisions. For example, a decision might only become relevant after another decision has been taken in a particular way. For instance, in the example shown in Fig. 1, answering the design decision ‘‘ Do you want to move parts?” with ‘‘yes” makes the runtime decision ‘‘At which speed do you want to move?” relevant. Assets in DOPLER represent the reusable elements of a product line. In the IAS domain, the asset types are component types used in the design time model (i.e., MovePart) and component instances used in the runtime model (i.e., FBMoveBottle_Slow or FBMoveBottle_Fast). Assets can depend on each other, for example, a component may require another component to function properly. Decisions and assets can be mapped by defining for an asset under which conditions it will be part of a system. This allows to automatically derive and (re-)configure products from the product line: Based on the values of taken decisions, components can be automatically selected for inclusion in the configured system and components can be instantiated and composed as required. In our example, MovePart will be included if the decision ‘‘ Do you want to move parts?” is answered with ‘‘yes”. The speed to move the parts can be adjusted at runtime by answering the decision ‘‘ At which speed do you want to move?” with ‘‘slow” or ‘‘fast” thereby leading to the selection of either FBMoveBottle_Slow or FBMoveBottle_Fast.

2.2. Product line models of IAS 2.3. The evolution of IAS and effects on product line models Recent research demonstrates the use of product line models to support the definition and evolution of IAS and their variability [30]. In particular, such models have proven useful for documenting the systems’ flexibility at design time and for automating

IAS control the production process of goods and products and are highly affected by changing market and business needs. IAS thus require a high level of adaptability even at runtime to ensure

760

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

Fig. 1. Modeling spaces for defining product lines of IAS. The figure shows examples of model elements and their relations derived from an existing IAS product line of a bottle-sorting plant [31].

flexibility for dealing with changes in a continuous production process [32]. Changes can be necessary as a result of changing technologies, new customer requirements, or code refactoring. Models describing IAS are thus not stable but subject to continuous evolution. Examples of changes to models are the removal of components from a system, the addition of new customization options (i.e., decisions) for customers, the refactoring and splitting of components, or the definition of dependencies between components. Such maintenance actions affect IAS function blocks (types and instances) as well as their connections. More specifically, examples of maintenance operations include modifying function block types (affecting all running and persisted instances), creating new function block types and corresponding instances, adding/removing function block instances from existing types, adding new connections (data and event dependencies), and rewiring/removing existing connections (data and event dependencies). Like many other systems, current IAS are typically maintained by carrying out these operations manually based on intuition and tacit knowledge of the involved stakeholders. Engineers need detailed knowledge about the application for this purpose. Evolution is very risky especially in safety-critical environments as it is difficult to understand the impact of changes given the complexity of many IAS. Thus, engineers can benefit from approaches that utilize models for defining an application’s variability to guide its evolution. For instance, dynamic product lines allow managing the variability and customizability of component-based IAS and support their automated reconfiguration based on models. Froschauer et al. [30] show how variability models can be used in different development activities for designing and implementing IAS components. In such an environment two types of maintenance can be distinguished:  Design-time maintenance deals with adding, modifying, or deleting the components for a specific domain and the decisions addressing stakeholder’s requirements in the design time variability model. The design time variability model defines the basic activities of an industrial process but does not contain information about the implementation of components. It can be compared to a platform-independent model common in Model Driven Architecture (MDA) [33].

 Runtime maintenance deals with changes to artifacts stored in the runtime variability model which represents the component instances of the technical solution, their dependencies, and their variability, i.e., decisions related to component instances. It is comparable to the platform-specific model in MDA [33]. 3. Simulating product line maintenance and evolution We developed an approach for simulating the maintenance and evolution of model-based product lines. We define our simulation approach following the general framework proposed by Kellner et al. [34], i.e., the purpose of the simulation is to understand the long-term impact of evolution on maintenance effort and model complexity in different product line environments. The scope of the simulation is the evolution of product line models and their maintenance. Input parameters to the simulation are domain-specific product line profiles and evolution profiles. Result variables are maintenance effort and model complexity. We first state basic definitions and assumptions. We then describe relevant product line profiles as simulation inputs. We discuss possible maintenance operations and their relationship to different evolution profiles. Finally, we discuss how to analyze the simulation results and present metrics that can be used to compute the result variables in our simulations. 3.1. Product line profiles Simulations need proper abstractions to be manageable and comparable. Dependency graphs provide an abstraction of product line models that allow comparing different systems possibly modeled using different approaches. Model size and the probabilities of dependencies between model elements are basic characteristics defining product line dependency graphs. These characteristics can be easily measured for existing product lines and can be used in maintenance simulations with graph generation and modification. Depending on the product line, models can be highly interconnected or elements can be rather independent of one another. We consider such characteristics as independent from the size of the model when creating random graphs for the simulation. Using product line profiles we define P(SD), P(DD),

761

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

and P(PSD) probabilities for dependencies among different model elements that are utilized by an Erdõs and Renyi [35] random graph generator:  P(SD) defines the probability of a dependency between two arbitrary solution space elements (e.g., components).  P(DD) defines the probability of a dependency between two arbitrary problem space elements (e.g., decisions).  P(PSD) defines the probability of a dependency between an arbitrary problem space element and an arbitrary solution space element. A probability value of 0.33 means that 33% of the (combinatorial) possible dependencies are set in the model. For example, if 10 new components are added to a model already containing 20 other components, the number of possible relationships will increase from 190 to 435. Therefore a probability value of 33% will result in 63 established dependencies before and 144 dependencies after adding the 10 new components (assuming the number of possible dependencies (=edges) between nodes in a undirected graph is 0.5  n  (n  1)). The values for P(SD), P(DD) and P(PSD) can only be set after analyzing existing product lines in a specific domain. Therefore, domain experts are required in the definition process. In the simulation experiment described in Section 5 we defined IAS-specific profiles on the basis of existing applications (for P(SD) and P(DD)) and domain experts’ knowledge (for P(PSD)). 3.2. Elementary maintenance operations Our simulator generates random maintenance operations based on evolution profiles (see Section 3.3) and applies them to the random graphs used to represent product line models. Removing a decision, adding a component, or creating several dependencies to other components are examples of maintenance operations. The maintenance operations simulate repeated user interactions evolving a product line model. The elementary maintenance operations are shown in Table 1. 3.3. Evolution profiles The elementary maintenance operations occur with different intensity depending on the stage of product line development and the business environment. For example, a company refactoring the architecture of its product line will typically face many changes to the solution space. As a result AddComp/RemComp and AddCompDep/RemCompDep operations will be more likely than AddDec/RemDec. A company in a highly dynamic business environment will be forced to frequent updates of the problem space when increasing the number of available system features. The probabilities of elementary maintenance operations can be defined in evolution profiles. We explain three example profiles which we will use later in our simulation experiment (cf. Section 5). They represent two extreme scenarios and one middle-ground situation. Users can define arbitrary scenarios for their own specific development context and probabilities of elementary maintenance operations can be measured by monitoring the engineers’ modeling activities [14]. 3.3.1. Product line refactoring Refactoring a product line – e.g., due to evolving technology – might lead to the adaptation of existing components or the development of new components. In this case, changes to the solution space are more likely than changes to the problem space. New customer requirements that cannot be fulfilled with the existing solution components will require the development of new components. New components and dependencies will have to be

Table 1 Elementary model maintenance operations and examples from the IAS domain. Operation

Description

Example (cf. Fig. 1)

Modeling space

AddDec, RemDec

Adding or removing a decision Adding or removing a component

Add a decision ‘‘At which speed do you want to move?” Add a component MovePart or component instances such as FBMoveBottle_Fast Add a dependency from FBMoveBottle_Fast to MovePart

Problem space

Link the decision about whether to move parts with the decision regarding the speed of the parts Assign the decision about the speed to the component instances FBMoveBottle_Slow and FBMoveBottle_Fast

Problem space

AddComp, RemComp

AddCompDep, RemCompDep

AddDecDep, RemDecDep

AddDecCompDep, RemDecCompDep

Adding or removing a dependency between components Adding or removing a dependency between decisions

Adding or removing a dependency between a decision and a component and/or component instance

Solution space

Solution space

Problem and solution space

added to the product line model, meaning that AddComp and AddCompDep are more likely than other operations. Also, links have to be established between new components and existing decisions (AddDecCompDep). In some cases, it might also be necessary to add new decisions (AddDec) if for instance new features are made available as a result of the refactoring effort. 3.3.2. Product placement This scenario assumes that the problem space is changed to allow different configurations of the existing components, without changing the components. This is often caused by changing market situations and other business considerations which require modifications of the problem space to capture new requirements as decisions (AddDec) or to remove existing ones (RemDec). Such business considerations do not necessarily result in adapting the solution space as they can often be incorporated by restricting or extending the possible choices in the problem space (AddDec, RemDec, AddDecDep, RemDecDep). Changes to the decisions can, however, also require developing new components (AddComp). It might also be required to update dependencies if the solution space is affected ( AddDecCompDep, RemDecCompDep). 3.3.3. Continuous evolution This profile represents a middle-ground between the two other profiles meaning that product line models are subject to continuous changes in both spaces by adding/removing components or reorganizing decisions. For example, if two dependencies contradict, one of them may have to be removed. In other cases adding new dependencies might be necessary if decisions or components require each other and no dependency has yet been defined. Table 2 depicts the three evolution profile examples used for our simulation in Section 5. The values are estimated by domain experts. The probabilities are used by our random Boolean generator to decide whether a specific maintenance operation needs to be performed in one simulation step. The values can also be obtained by measuring the number of model maintenance operations using instrumented tools. P(AddDec) and P(RemDec) determine the

762

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

Table 2 Probability values for the three evolution profiles.

Table 3 A Goal-Question-Metric model relating simulation goals and result variables.

Probabilities of elementary operations

Refactoring (%)

Product placement (%)

Continuous evolution (%)

P(AddDec) P(RemDec) P(AddComp) P(RemComp)

5 5 90 60

80 40 15 10

30 20 50 30

probability for decisions to be created or deleted in a single simulation step. P(AddComp) and P(RemComp) determine the probability that a component or an instance thereof is added/removed in a single simulation step. The dependencies between components and decisions are added based on a product line profile as introduced in Section 3.1.

GQM

Description

Goal

Purpose: Analyzing the effects of product line evolution Object: Product line models Issue: Maintenance effort and model complexity Viewpoint: Product line engineers

Question 1

How complex are the models?

Metric MC: model complexity

We use the Floyd–Warshall algorithm [37] to compute the average shortest path between all pairs of nodes. Thus we can investigate significant changes to the overall model complexity independent from the model size. For a fully connected graph (representing the unrealistic case of a product line model in which every model element is connected to all other model elements) this algorithm returns the value 1. The overall model complexity of solution and problem space models stored in a graph G with vertices V and edges E is defined as the inverse average shortest path of all vertex pairs vi and vj in relation to the number of vertices G = (V, E)

3.4. Analysis concept We simulate the maintenance and evolution of product lines which are represented as models. For these models and the dependency graphs we can use standard graph metrics to determine their evolving characteristics. We adopt the analysis approach by Zimmerman and Nagappan [36] and use common graph complexity measures [37] to determine the evolving complexity of product line models. Graph complexity measures are widely used in software engineering, e.g., for determining the complexity of software modules [38]. We consider model elements (assets and decisions) as the vertices V and dependencies between model elements as the edges E of a graph. A decision can be connected to an arbitrary number of components as well as other decisions. If there are transitive dependencies to additional model elements, these relations are also resolved to edges in the dependency graph. The edges in our generated models represent the navigation path relevant for an engineer maintaining a model. Our simulation tools (cf. Section 4) can be configured with domain-specific metrics representing the result variables in our simulation. These metrics define the kinds of measurements to be made (during and after) simulating maintenance operations. The Goal-Question-Metric (GQM) [39] model shown in Table 3 illustrates the derivation of domain-specific metrics (result variables) from our more general simulation goals. After each simulation step consisting of several maintenance operations we measure the effects of these changes: Model complexity (MC) defines the normalized complexity of solution and problem space models determined using a Shortest Path graph algorithm [37]. Maintenance effort (ME) defines the number of dependency links added, changed, deleted, or inspected during each maintenance step. This metric reflects that adding or removing a model element is typically not possible without also adding, inspecting, changing, or deleting related elements. Effort increases even if no actual change is made, e.g., several links need to be inspected before an element may eventually be deleted. 4. Simulator tool We have developed a tool for simulating modifications of product line models in different contexts. This Eclipse-based tool provides extension points that allow implementing Java code for arbitrary operations on a product line model represented in a DOPLER model [4] and computing arbitrary metrics after each operation. For the purpose of this paper we developed extensions to generate DOPLER product line models based on product line profiles and to perform maintenance operations based on evolution profiles. During the simulation process scripts automate the

PjV i j PjV j j i¼0

j¼0

jV j minðpathðv i ;v j ÞÞ

path(i, j) returns the set of all path lengths between i and j Question 2

How big is the maintenance effort for performing maintenance operations?

Metric ME: maintenance effort

The maintenance effort is affected by the number of edges per vertex, because when maintaining one model element, dependent model elements also need to be inspected. Therefore we consider dependency links added, changed, inspected, or deleted during simulation in relation to the overall number of components G = (V, E) jEj ME ¼ jVj

execution of the extensions that implement model generation and maintenance operations. The simulator tool (cf. Fig. 2) allows generating random graph models based on the given product line profiles and defined initial size (number of problem space and solution space elements). However, simulation models can also be created by parsing and analyzing the characteristics of existing product line models as we will show in Section 5. The tool computes result metrics after each simulated step (cf. the results presented in Section 5). Such results metrics can also be developed as extensions to the tool to allow defining arbitrary result metrics for simulations. We have developed extensions that compute our two result metrics (cf. Section 3.4). The inputs for a specific simulation run with our extensions for model generation and maintenance operations are: the product line profile (cf. Section 3.1), the evolution profile (cf. Section 3.3) and the number of maintenance iterations given with the simulation script file. Simulation outputs are the list of performed maintenance operations and sets of values with one value per defined metric (cf. Section 3.4) per maintenance step. A schematic representation of the simulation process is depicted in Fig. 3. Step 0: Based on the product line profile our simulation tool first initializes a product line model (a random graph) with a predefined number of solution space and problem space elements. The dependencies between elements are created according to the probabilities defined in the product line profile. Steps 1–n: In each simulation step, the probability values of the evolution profile are used with the Boolean random generator

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

763

Fig. 2. Overview of our extensible simulator tool.

operation. The number of simulation steps given in the simulation script determines the total number of iterations to be conducted. 5. Simulating product line evolution in the domain of IAS

Fig. 3. Simulation process (schematic representation).

to create sets of elementary maintenance operations. Valid sequences are for example [AddDec, RemDec]; [AddDec, RemDec, AddComp, RemDec]; [RemDec]; or [empty]. The sequence of operations is applied to the model. If elements are added, they are weaved into the model with dependency links generated according to the dependency probabilities in the product line profile. If elements are deleted, they are randomly selected and removed from the model together with the dependencies they have to other elements. After each simulation step, the tool computes all implemented result metrics for the model. One value is computed and recorded per metric after each maintenance

We evaluate the proposed simulation approach by conducting a simulation experiment to analyze the effects of different evolution profiles when adopting PLE in the IAS domain. In particular, we aim to address the following simulation goals relevant for experts in the field of IAS who need to assess the value of model-based PLE: SG1: Understand the impact of the degree of variability modeling on the maintenance effort. We simulate different variability modeling scenarios, i.e., different ratios of problem space elements (decisions) to solution space elements (components) (cf. Section 5.2). For example, a high P(PSD) means that decisions are linked to a large number of components, which may be caused by modeling products providing a high degree of variability. A lower P(PSD) and therefore a lower amount of modeled dependencies between decisions and components means that only few components are directly influenced by decisions. We also investigate the effects of different evolution profiles on the product line models. Achieving this goal would typically involve empirical studies on how different engineers model applications in their domain. SG2: Understanding the impact of the IAS characteristics on maintenance effort and model complexity. Addressing simulation goal SG2 would normally require longitudinal studies. Instead, we investigate these issues by simulating the evolution of IAS of different size and with different dependency probabilities in models. 5.1. Simulation setup We generate random graphs [35] for the simulation based on data from real-world systems and derive the application characteristics from 128 IAS of different size and with different dependency probabilities (cf. Section 5.3). The IAS applications were developed manually by IAS domain experts other than the authors to preserve independence and ensure the data is representative. The simulation models represent IEC 61499-based IAS and follow the structure introduced in Section 2. The solution space reflects an

764

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

already deployed application, respectively its component types (function block types) and instances (function blocks). The problem space of the product line is set up by defining decisions on top of the solution space. With regard to simulation goal SG1 the decisions in the problem space are linked to one or more solution space elements, i.e., components and instances. P(PSD) represents the probability of these links and defines the degree of variability modeling for a particular product line. P(PSD) is supposed to have a significant influence on a product line’s maintenance as it reflects the granularity of modeling. Besides the probabilities defined in the evolution profiles, we also consider the characteristics of existing IAS by empirically gathering how problem space elements are linked to solution space elements and how design models are linked to runtime models. For this purpose, we analyzed existing IAS applications and case studies [31,40] to define two values used to setup the modeling space for our simulations: The Reuse Factor determines the number of component instances in already deployed IEC 61499 applications in relation to the total number of available component types. Gathering values from existing applications gives an estimate on how many component types will typically be necessary in an application to be maintained as a software product line. We identified this value by analyzing the 128 applications. The Variability Rate determines the number of problem space elements (i.e., decisions) in relation to the number of solution space elements (i.e., components). To find plausible values we used the results of two case studies we conducted on modeling IAS as software product lines [31,40]. Based on the models resulting from these case studies we determined the numbers of problem and solution space elements and used them to calculate a mean variability rate for the simulation. Table 4 depicts the parameters characterizing the maintenance operations and the different sources they have been derived from. The probabilities for maintenance operations AddComp/RemComp and AddDec/RemDec were defined based on the evolution profiles introduced in Section 3.3. The probability for dependencies between problem and solution space elements P(PSD) was defined based on the results of case studies as explained above [31,40]. The probability for AddCompDep is taken from the currently loaded application and the calculated value for component dependencies within the product line profile P(SD).

5.2. Simulation goal SG1: understand the impact of the degree of variability modeling on the maintenance effort We use the probability of dependencies existing between problem and solution space elements P(PSD) to define the degree of variability modeled in a product line. Varying this value directly expresses different degrees of variability modeled by an engineer.

Values

Source

Evolution profiles

P(AddComp), P(RemComp), P(AddDec), P(RemDec) P(PSD)

Defined by analyzing different scenarios of our industry partners

Reuse Factor

Whenever a model-based approach is used to develop and maintain a product line a key question is whether the models will remain maintainable during the evolution of the product line. We address simulation goal SG2 by simulating maintenance and evolution on the basis of existing applications. We gathered data from 128 IAS as an input for our simulation. After classifying these applications according to their dependency probabilities and size (cf. Fig. 5), we simulated a life cycle of 100 maintenance operations using the evolution profiles we have defined in Section 3.3. Table 6 shows the simulation setup for our simulations.

Table 5 Simulation setup for simulation goal SG1.

Reuse Factor

Context

Deployed applications from industry partners

5.3. Simulation goal SG2: understanding the impact of the IAS characteristics on maintenance effort and model complexity

Parameter

Table 4 Data sources for maintenance operations.

IAS case studies

Different evolution profiles (cf. Section 3.3) are considered. As Table 5 shows, the simulation setup requires several parameters from different sources. We present results of simulating the evolution of one representative IAS product line – a function block application controlling a bottle-sorting plant [31] (cf. Section 2). During the simulation, we applied the three evolution profiles and experimented with different P(PSD) values (independent variables). We computed the model complexity and the maintenance effort (dependent variables) as defined in Section 3.4. Fig. 4 depicts the simulation results for the evolution profiles refactoring and product placement as the two ‘‘extreme” cases. The simulation for the three evolution profiles led to similar results for each profile and model complexity stayed nearly constant in all cases. The simulation shows that a higher PSD, e.g., caused by a model containing more diverse products, increases the complexity of the underlying model. Looking into more detail the product placement profile leads to more interconnected and more complex systems. PSD values of 10% and 25% have a greater increase rate towards more complexity than PSD values of 15% for which complexity remains quite constant. Regarding simulation goal SG1 we see that for the purpose of refactoring the degree of variability has no significant impact on maintenance effort; higher PSD values tend to result in higher component interface complexity (indicated by the average number of dependencies per component). For the purpose of product placement the probability of existing dependencies between problem space and solution space was optimal in the case of 15%. Lower or higher values lead to a stronger increase of component interface complexity and thereby increase maintenance effort. One interpretation of this simulation result is that for long-term maintainability of the investigated systems modelers should carefully monitor the number of mappings between the problem and solution space to keep complexity and effort manageable.

Probability of relationships existing between decisions and components Number of component instance in relation to number of component types

Variability Rate P(AddComp)

Fixed

Value

Source

2.077

Existing applications IAS case studies

3.6166 Varied by profile (cf. Section 3.3)

Various

P(RemComp)

Various

P(AddDec)

Various

P(RemDec)

Various

P(PSD)

Varied by simulation

10%, 15%, 25%

Evolution profile Evolution profile Evolution profile Evolution profile Scope of simulation

765

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

Fig. 4. Simulation results (MC and ME) of the bottle-sorting IAS product line for different P(PSD) values for the evolution profiles refactoring and product placement.

We again analyze model complexity and maintenance effort metrics (dependent variables) for the three evolution profiles (independent variable), but during this simulation we keep P(PSD) fixed at 15% thus taking into account the SG1 simulation results. We simulate maintenance operations for a set of applications classified into four classes by their mean dependency probability and mean application size as shown in Fig. 5. Figs. 6–8 show the results of our simulations. We analyze the evolution with our metrics in relation to small and simple systems. In the figures we use the results for small and simple systems as the baseline and show the other three types of IAS in relation to that baseline. This allows us to compare the long-term effects of maintenance of models of different characteristics. We discuss the results of the simulations grouped by evolution profile: Continuous evolution leads to increased maintenance effort and model complexity for large systems in relation to small and

Fig. 5. Classification of existing IAS applications.

Table 6 Simulation setup for simulation goal SG2. Parameter Reuse Factor

Fixed

Variability Rate P(PSD) P(AddComp) P(RemComp) P(AddDec) P(RemDec)

Varied by profile (cf. Section 3.3)

Value

Source

2.077 3.6166

Existing applications IAS case study

15

RQ1

Various Various Various Various

Evolution Evolution Evolution Evolution

profile profile profile profile

simple systems, whereas the maintenance effort stays rather constant over time. For the investigated domain of IAS this may be due to continuous enhancements and adaptations of a manufacturing process and its process steps. Although the process steps’ complexity increases, the overall structure of the process is not changed, which results in a rather constant maintenance effort. In contrast to this scenario the Product placement scenario leads to a decrease of model complexity and maintenance effort over time. A possible reason for this is the adjustment of generic requirements to more specific customer and process requirements. However, Product placement has no noticeable effect for small and complex systems over time. Evolving the models by applying Product placement decreases the overall complexity and maintenance effort. This may be caused by removing unnecessary decisions in the problem space, which also decreases the maintenance effort. Refactoring has a higher impact on larger systems than on smaller systems. In a large system the complexity and the maintenance effort increases slightly over time, whereas for small systems the complexity decreases and the maintenance effort stays constant. For IAS applications the increase in complexity and maintenance effort may result from adding additional functionality while refactoring the application.

766

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

Fig. 6. Simulation results for the four application classes for the continuous evolution profile (Small and simple systems are used as baseline for comparison).

Fig. 7. Simulation results for the four application classes for the product placement profile (Small and simple systems are used as baseline for comparison).

Fig. 8. Simulation results for the four application classes for the refactoring profile (Small and simple systems are used as baseline for comparison).

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

Summarizing the results of this simulation on the basis of our so far collected IAS applications we show that the small and complex applications have a more significant feedback in terms of complexity compared to large applications. Independent of the dependency probabilities large applications show the same principal behavior in relation to small and simple systems. Therefore we consider the type of variability models as suitable for all kinds of large systems, as their maintenance effort does not evolve excessively in relation to small systems over time. While these simulation results may seem obvious and intuitive, they confirm the suitability of our simulation approach. We focused on two simulation goals relevant for the IAS domain to demonstrate our simulation approach. Other goals may be considered by defining customer- and domain-specific product line profiles, evolution profiles, and metrics reflecting typical maintenance strategies in companies. 6. Threats to validity 6.1. Random graphs We use product line models based on random graphs in our simulations. Some caveats are reported in the literature regarding the use of random graphs. For instance, there is a risk of creating cliques and isolated elements during graph generation [35]. Similarly, the different dependency types in the models are not fully independent. However, we used graph visualization techniques and found out that these cliques represent realistic modeling situations of stronger connected components. 6.2. Metrics For our simulations we currently do not distinguish between different dependency types and use bidirectional dependency links of equal importance (which was sufficient for the IAS domain). Model elements and dependency types may vary in product line models. Our ME and MC metrics may thus not be applicable in other domains. For instance, it might be necessary to distinguish different types of dependencies when computing ME. However, the presented simulation framework allows defining arbitrary metrics that can meet the demands of other domains. 6.3. Generated maintenance operations We use random generators to create the elementary maintenance operations in each simulation step. However, we currently do not consider the logical order of the maintenance operations themselves. In reality however, it is likely that the probability of certain operations depends on the preceding operations. For example, after a solution space element is added, the probability of adding solution space dependencies will most likely increase. In future work, we will refine our approach to consider logical dependencies of the operations. Additionally, the solution space may be considered being split into design and runtime model (cf. Fig. 1), which requires an additional definition of maintenance operations explicitly dealing with component instances, such as AddInstance or RemInstance. For the presented simulation we assumed that AddComp and RemComp deal with both components and component instances. 6.4. Evolution profiles Our simulation tool uses profiles defining the probability of maintenance operations (cf. Section 3.3). We defined these probabilities based on our own and our industry partners’ experience.

767

While these numbers might deviate from reality we stress that the purpose of the profiles is to experiment with different possible trends. We thus tried to reflect extreme profiles (e.g., product line refactoring and product placement) as well as a middle-ground profile (continuous evolution). 7. Related work Our discussion of related work covers the areas of maintenance and evolution of product lines, empirical research based on simulation, and simulation in software engineering. 7.1. Maintenance and evolution of product lines Many product line approaches assume that activities in domain and application engineering can take a fairly stable product line for granted. However, real-world product lines inevitably and continuously evolve. Despite its importance, comparably few papers discuss product line evolution [14,41,42]. Managing evolution is success-critical in model-based approaches to ensure consistency after changes to meta-models, models, and actual development artifacts. Several authors [17,43,44] have stressed the importance of approaches for product line evolution to avoid the erosion of a product line, i.e., the deviation from the product line model up to the point where key properties no longer hold. Svahnberg and Bosch [45] report on experiences regarding the evolution of products, software components, and software architecture in product lines. Ajila and Dumitrescu [46] report on a study they conducted on the economic impact of market reposition on a particular product line from a telecommunications equipment supplier. They conclude that there is no relationship between size of code added to the product line in course of the reposition and the number of designers required to develop and test it but that there is a relationship between designer turnover and impact of change. Few approaches provide explicit support for product line evolution. For example, Deng et al. [47] describe a model-driven product line approach that explicitly focuses on the issue of domain evolution and product line architectures. They discuss several challenges for the evolution of model-driven software product line architectures and present their solution of supporting evolution with automated domain model transformations. Another example is the work by Mende et al. [48], who present tool support for the evolution of software product lines based on the ‘‘grow-and-prune’’ model. They support identifying and refactoring code that has been created by copy and paste and which might be moved from product level to product line level. 7.2. Empirical research based on simulation Simulation is increasingly used by academics and practitioners as an approach for analyzing complex business and engineering questions [34]. For instance, Port et al. [16] use simulation for understanding the effects of different requirements prioritization strategies. Gokhale et al. [49] use discrete-event simulation to analyze component-based systems regarding reliability in two case studies. Lehman and Kahen [50] describe a systems dynamics model to support decision-making regarding personnel allocation in long-term software evolution. Chen et al. [51] present a process evaluation approach for product lines including a process simulator tool supporting the simulation of software product line engineering with a focus on strategic management and long-term forecasting. The simulator allows predicting development costs, schedule, and resource usage rates for a particular software product line process at a high level using discrete-event simulation with general parameters (e.g., desired maximum number of prod-

768

W. Heider et al. / Information and Software Technology 52 (2010) 758–769

ucts to be supported by the product line) and product parameters (e.g., lines of code) as inputs. 7.3. Simulation in software engineering Simulations have also been proposed and used as a teaching tool in software engineering and education. Navarro and van der Hoek [52] propose the SimSE simulation game that allows players to take on the role of a project manager to experience fundamentals of software engineering through cause-effect models. Jain and Boehm [53] present SimVBSE, a game for students to explore value-based software engineering. The SESAM simulation tool provides a training environment for future project managers based on models of the software development process [54]. 8. Conclusions and future work We presented a simulation approach that aims at supporting product line engineers to arrive at more informed decisions regarding the suitability of product line modeling considering different evolution profiles and product line characteristics. We illustrated the approach and its feasibility by applying it to concrete product lines in the domain of industrial automation systems. Conducting empirical research in product line engineering is challenging as companies are typically reluctant to give researchers access to their product line data. Also, many benefits of product lines can be measured only in longitudinal studies which are infeasible or at least difficult to perform in most environments. We thus believe that simulation is a promising approach to gain an understanding regarding the utility of product line approaches. The simulation results presented are only relevant for a particular domain – respectively the IAS domain, where we gathered input parameters like the Reuse Factor and the variability rate. We however demonstrated the applicability of our simulation approach by using product lines of different size and dependency probabilities from different stages of the life cycle. We believe that our approach is applicable to other product line environments, especially due to the possibility to configure our simulation tool with different parameters describing the environments and the maintenance of models in these environments. In future work we plan to refine our simulation approach to better address specifics of particular evolution profiles and plan to conduct further simulations in other domains and environments. Acknowledgements This work has been supported by the Christian Doppler Forschungsgesellschaft, Austria and Siemens VAI Metals Technologies. We also want to thank Andreas Schimmel from Technical University of Vienna for providing data of IEC 61499 applications used in our simulations. References [1] K. Kang, S. Cohen, J. Hess, W. Nowak, S. Peterson, Feature-oriented domain analysis feasibility study, Technical report, CMU/SEI TR-21, USA, 1990. [2] K. Czarnecki, U. Eisenecker, Generative Programming: Methods, Techniques, and Applications, Addison-Wesley, 1999. [3] T. Asikainen, T. Männistö, T. Soininen, A unified conceptual foundation for feature modeling, in: Proc. of 10th International Software Product Line Conference, IEEE CS, 2006, pp. 31–40. [4] D. Dhungana, P. Grünbacher, R. Rabiser, Domain-specific adaptations of product line variability modeling, in: J. Ralyté, S. Brinkkemper, B. Henderson-Sellers (Eds.), Proc. of IFIP WG 8.1 Working Conf. on Situational Method Engineering: Fundamentals and Experiences, International Federation for Information Processing, Springer Series in Computer Science, 2007, pp. 238–251. [5] K. Schmid, I. John, A customizable approach to full-life cycle variability management, Journal of the Science of Computer Programming 53 (2004) 259– 284 (Special Issue on Variability Management).

[6] K. Pohl, G. Bockle, F. Van der Linden, Software Product Line Engineering: Foundations, Principles and Techniques, Springer, 2005. [7] M. Matinlassi, Comparison of software product line architecture design methods: COPA, FAST, FORM, KobrA and QADA, in: Proc. of the 26th International Conference on Software Engineering (ICSE’04), Edinburgh, Scotland, IEEE CS, 2004, pp. 127–136. [8] E.M. Dashofy, A. van der Hoek, R.N. Taylor, A Highly-extensible, XML-based architecture description language, in: Proc. of the Working IEEE/IFIP Conference on Software Architecture (WICSA’01), Amsterdam, The Netherlands, IEEE Computer Society, 2001, pp. 103–112. [9] C. Atkinson, J. Bayer, C. Bunse, E. Kamsties, O. Laitenberger, R. Laqua, D. Muthig, B. Paech, J. Wüst, J. Zettel, Component-Based Product Line Engineering with UML, Addison-Wesley, 2002. [10] H. Gomaa, Designing Software Product Lines with UML, Addison-Wesley, 2005. [11] M. Sinnema, S. Deelstra, Classifying variability modeling techniques, Journal on Information and Software Technology 49 (2007) 717–739. [12] B. Boehm, A.W. Brown, R. Madachy, Y. Yang, A software product line life cycle cost estimation model, in: Proc. of the 2004 International Symposium on Empirical Software Engineering, Redondo Beach, CA, USA, IEEE Computer Society, 2004, pp. 156–164. [13] D. Dhungana, T. Neumayer, P. Grünbacher, R. Rabiser, Supporting evolution in model-based product line engineering, in: Proc. of the 12th International Software Product Line Conference (SPLC 2008), Limerick, Ireland, IEEE Computer Society, 2008, pp. 319–328. [14] W. Heider, R. Rabiser, D. Dhungana, P. Grünbacher, Tracking evolution in model-based product lines, in: Proc. of 1st International Workshop on Modeldriven Approaches in Software Product Line Engineering (MAPLE 2009), 13th International Software Product Line Conference (SPLC 2009), San Francisco, CA, USA, Software Engineering Institute, Carnegie Mellon, 2009, pp. 59–63. [15] W. Heider, R. Rabiser, Tool support for evolution of product lines through rapid feedback from application engineering, in: Proc. of the 4th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS 2010), Linz, Austria, ICB-Research Report No. 37, University of Duisburg Essen, 2010, pp. 167–170. [16] D. Port, A. Olkov, T. Menzies, Using simulation to investigate requirements prioritization strategies, in: Proc. of 23rd IEEE/ACM International Conf. on Automated Software Engineering (ASE 2008), L’Aquila, Italy, 2008, pp. 268– 277. [17] S. Deelstra, M. Sinnema, J. Bosch, Variability assessment in software product families, Journal of Information and Software Technology 51 (2009) 195–218. [18] S. Deelstra, M. Sinnema, J. Bosch, Product derivation in software product families: a case study, Journal of Systems and Software 74 (2005) 173–194. [19] International Electrotechnical Commission, IEC 61131-3: Programmable Controllers – Part 3: Programming Languages, 2003. [20] DG-Research, Working document for the manufacture 2003 conference, Technical report, European Commission Directorate General dealing with Research, 2003. [21] International Electrotechnical Commission, IEC 61499: Function Blocks for Industrial-process Measurement and Control Systems, 2005. [22] C. Suender, A. Zoitl, C. Dutzler, Functional structure-based modeling of automation systems, International Journal of Manufacturing Research 1 (2006) 405–420. [23] K. Thramboulidis, Model integrated mechatronics: an architecture for the model driven development of manufacturing systems, in: Proc. of IEEE International Conf. on Mechatronics (ICM ‘04), Istanbul, Turkey, IEEE Computer Society, 2004, pp. 497–502. [24] V. Vyatkin, H.-M. Hanisch, A modeling approach for verification of IEC1499 function blocks using net condition/event systems, in: Proc. of the 7th IEEE International Conf. on Emerging Technologies and Factory Automation (ETFA’99), Barcelona, Spain, IEEE Computer Society, 1999, pp. 261–270. [25] A. Zoitl, Real-time Execution for IEC 61499, ISA, 2008. [26] R.W. Lewis, Modeling Control Systems using IEC 61499, IEEE Publishing, 2001. [27] J.H. Christensen, Function Block- Based, Holonic Systems Technology, 2007, (online). [28] C. Suender, Evaluation of downtimeless system evolution in automation and control systems, PhD thesis, Technical University of Vienna, 2008. [29] W. Weck, C. Szyperski, Do we need inheritance? in: Proc. of Workshop on Composability Issues in Object-Orientation, 10th European Conf. on Objectoriented Programming (ECOOP’96), Linz, Austria, 1996. [30] R. Froschauer, D. Dhungana, P. Grünbacher, Managing the life-cycle of industrial automation systems with product line variability models, in: Proc. of the 34th EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO-SEAA 2008), Parma, Italy, IEEE Computer Society, 2008, pp. 35–42. [31] R. Froschauer, A. Zoitl, P. Grünbacher, Development and adaptation of IEC 61499 automation & control applications with runtime variability models, in: Proc. of the 7th IEEE Int’l Conference on Industrial Informatics (INDIN 2009), Cardiff, UK, IEEE CS, 2009, pp. 905–910. [32] T. Strasser, I. Muller, C. Sunder, O. Hummer, H. Uhrmann, Modeling of reconfiguration control applications based on the IEC 61499 reference model for industrial process measurement and control systems, in: Proc. of IEEE Workshop on Distributed Intelligent Systems: Collective Intelligence and Its Applications (DIS), Prague, Czech Republic, IEEE Computer Society, 2006, pp. 127–132. [33] OMG Object Management Group, MDA Guide Version 1.0.1, omg/2003-06-01, 2003, (online).

W. Heider et al. / Information and Software Technology 52 (2010) 758–769 [34] M.I. Kellner, R.J. Madachy, D. Raffo, Software process simulation modeling: why? what? how?, Journal of Systems and Software 46 (1999) 91–105 [35] P. Erdõs, A. Rényi, On random graphs, Publicationes Mathematicae 6 (1959) 290–297. [36] T. Zimmermann, N. Nagappan, Predicting subsystem failures using dependency graph complexities, in: Proc. of the 18th IEEE International Symposium on Software Reliability (ISSRE’07), Washington DC, USA, 2007. [37] R.W. Floyd, Algorithm 97 (SHORTEST PATH), Communications of the ACM 5 (1962) 345. [38] T.J. McCabe, C.W. Butler, Design complexity measurement and testing, Communications of the ACM 32 (1989) 1415–1425. [39] V. Basili, G. Caldiera, D. Rombach, Goal/question/metric paradigm, in: J. Marciniak (Ed.), Encyclopedia of Software Engineering, vol. 1, John Wiley and Sons, New York, 1994, pp. 528–532. [40] R. Froschauer, D. Dhungana, P. Grünbacher, Runtime adaptation of IEC 61499 applications using domain-specific variability models, in: Proc. of the 12th International Software Product Line Conference, Second Volume, 2nd International Workshop on Dynamic Software Product Lines (DSPL 2008), Limerick, Ireland, Lero, 2008, pp. 39–44. [41] J. Bosch, Design and Use of Software Architectures: Adopting and Evolving a Product Line Approach, ACM Press/Addison-Wesley Publishing Co., New York, USA, 2000. [42] J.D. McGregor, The evolution of product line assets, Technical report CMU/SEI2003-TR-005 ESC-TR-2003-005, CMU/SEI, 2003. [43] S. Johnson, J. Bosch, Quantifying software product line ageing, in: Proc. of the 1st ICSE 2000 Workshop on Software Product Lines: Economics, Architectures, and Implications, Limerick, Ireland, pp. 27–32. [44] H. Siy, D. Perry, Challenges in evolving a large scale software product, in: Proc. of the Principles of Software Evolution Workshop, Collocated with the International Software Engineering Conference (ICSE98), Kyoto, Japan, 1998, pp. 29–32. [45] M. Svahnberg, J. Bosch, Evolution in software product lines: two cases, Journal of Software Maintenance: Research and Practice 11 (1999) 391–422.

769

[46] S.A. Ajila, R.T. Dumitrescu, Experimental use of code delta, code churn, and rate of change to understand software product line evolution, Journal of Systems and Software 80 (2007) 74–91. [47] G. Deng, J. Gray, D.C. Schmidt, Y. Lin, A. Gokhale, G. Lenz, Evolution in model-driven software product-line architectures, in: P.F. Tiako (Ed.), Designing Software-intensive Systems, Idea Group Inc. (IGI), 2008, pp. 1280–1312. [48] T. Mende, F. Beckwermert, R. Koschke, G. Meier, Supporting the grow-andprune model in software product lines evolution using clone detection, in: Proc. of the 12th European Conference on Software Maintenance and Reengineering (CSMR ‘08), IEEE Computer Society, Washington, DC, USA, 2008, pp. 163–172. [49] S. Gokhale, M. Lyu, K. Trivedi, Reliability simulation of component-based software systems, in: Proceedings of The Ninth International Symposium on Software Reliability Engineering (ISSRE ’98), Paderborn, Germany, IEEE Computer Society, 1998, p. 192. [50] M.M. Lehman, F.-R.J.G. Kahen, Simulation process modelling for managing software evolution, in: N. Juristo, S. Acuna (Eds.), Software Process Modelling, Springer Verlag, 2005, pp. 87–109. [51] Y. Chen, G.C. Gannod, J.S. Collofello, A software product line process simulator, Software Process: Improvement and Practice 11 (2006) 385–409 (Special Issue on Working in a Complex Environment). [52] E.O. Navarro, A. van der Hoek, Teaching software engineering using simulation games, in: Proc. International Conf. on Simulation in Education, Huntsville, AL, USA, 2006. [53] A. Jain, B. Boehm, SimVBSE. Developing a game for value-based software engineering, in: Proc. of the 19th Conf. on Software Engineering Education Training (CSEET 2006), Turtle Bay, HI, USA, IEEE Computer Society, 2006, pp. 103–114. [54] A. Drappa, J. Ludewig, Quantitative modeling for the interactive simulation of software projects, Journal of Systems and Software 46 (1999) 113–122.