Practice-driven approach for creating project-specific software development methods

Information and Software Technology 49 (2007) 345–365 www.elsevier.com/locate/infsof

Practice-driven approach for creating project-specific software development methods Marko Bajec *, Damjan Vavpoticˇ, Marjan Krisper University of Ljubljana, Faculty of Computer and Information Science, Trzaska 25, 1000 Ljubljana, Slovenia Received 16 December 2005; received in revised form 6 May 2006; accepted 25 May 2006 Available online 3 July 2006

Abstract Both practitioners and researchers agree that if software development methods were more adjustable to project-specific situations, this would increase their use in practice. Empirical investigations show that otherwise methods exist just on paper while in practice developers avoid them or do not follow them rigorously. In this paper we present an approach that deals with this problem. Process Configuration, as we named the approach, tells how to create a project-specific method from an existing one, taking into account the project circumstances. Compared to other approaches that deal with the creation of project-specific methods, our approach tends to be more flexible and easier to implement in practice as it introduces few simplifications. The proposed approach is practice-driven, i.e. it has been developed in cooperation with software development companies.  2006 Elsevier B.V. All rights reserved. Keywords: Method engineering; Process engineering; Project-specific method

1. Introduction It is a common belief among the researchers of the IS community that the use of methods in software development is beneficial, since it improves the process and its product. Interestingly, investigations in practice show rather different picture. It seems that methods are actually underused and what is more their use is not increasing (see e.g. [13,19,20,10,26]). One reason contributing to this rather paradoxical situation is inflexibility, which is a characteristic of methods that permits virtually no adjustment to organisation-specific or project-specific circumstances. As stated by Fitzgerald, in software development, methods are not applied rigorously nor uniformly, even when training in their use has been provided. This supports the view that a unique method-in-action is created for each development project [10]. *

Corresponding author. Tel.: +386 14768 814; fax: +386 14768 705. E-mail addresses: [email protected] (M. Bajec), damjan.vavpotic@ fri.uni-lj.si (D. Vavpoticˇ), [email protected] (M. Krisper). 0950-5849/$ - see front matter  2006 Elsevier B.V. All rights reserved. doi:10.1016/j.infsof.2006.05.007

In this paper, we discuss how flexibility can be integrated into the methods that software development organisations are using in their every-day practice. The approach that we present (Process Configuration Approach or in short PCA) builds on the previous research work known from the literature as Method Engineering. The method engineering deals with the creation of methods that are specifically attuned to the needs of a particular organisation or project. While there were several approaches proposed in the past that suggest how to create methods on-the-fly, their use in practice is somewhat low, which seems to be a result of their inherent complexity. In the PCA we tried to avoid the problems that hinder the use of traditional assemblybased method engineering in practice by introducing some limitations. We believe the PCA contributes a very useful architecture that can be used to implement any approach to method engineering that is based on the tailoring of a single method. The PCA was developed as a part of the MasterProc project, aimed at developing a framework and supporting

346

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

tools for reengineering software development methods in Slovenian companies.1 The paper is organised as follows. In Section 2 we describe the research approach adopted in our work. Next is the related works section which briefly describes other approaches to method construction for the needs of a particular organisation or project. The core of the paper is in Sections 4 and 5. Section 4 gives a theoretical background of the approach while Section 5 describes the results of the application of the approach in practice. In Section 6 we compare our approach with other SME approaches and illustrate how the PCA could be used in combination with some of the existing approaches to method engineering. Finally, in Section 7, concluding remarks are given.

practical situations, which gives him/her optimal access to practice. The most important weakness of this approach is that it makes the research process and findings hard to structure and manage. Better option in this sense are experiments, which provide access to practice that are controlled or partially controlled by researchers. In contrast to action research they provide weaker relation to practice. Finally, practice studies present a research approach to study software development without researchers being actively involved in practice. The practice studies cover different types of study: direct study, e.g. field studies or case studies, and indirect studies, e.g. surveys or interviews.

2. Research approach

The PCA approach that we describe in this paper has been developed as one of the deliverables of the aforementioned MasterProc project. The aim of the MasterProc was to improve software development practice in Slovenian companies by developing a framework and supporting tools for reengineering software development methods. The project was organised as a collaborative practice research using a combination of different research approaches. In this subsection we describe in more detail how the MasterProc was organised. Please note that the development of the PCA was just an activity of the MasterProc project. In the organisation of the MasterProc project the principles of a general learning cycle have been adopted, i.e. interpret current situation, find ways to improve practice, plan and implement improvements, and learn from the actions taken. As described above the CPR supports such learning cycle by the three goals it identifies: to understand the current state of software development, to build new knowledge that can support practice, and finally to plan changes and implement them as necessary. After implementing the improvements, the interpretation of the lessons learned have to take place, hopefully leading into the next learning cycle. At the time this paper was written, we were still undergoing the first learning cycle of the MasterProc project. According to project plan the first learning cycle should be finished till September 2006, while the next is scheduled for the beginning of 2007.2 To describe the organisation of the MasterProc project clearly as to present how the development of the PCA that is described in this paper fits into the whole project we provide detailed descriptions in Table 1. For the timeline of the project see Fig. 1. The activities that were carried out to develop the PCA approach are deliberately described in more detail and are coloured in green. (For interpretation of the references in colour in the text, the reader is referred to the web version of this article.)

2.1. Collaborative practice research The research approach adopted in the MasterProc project is founded on the Collaborative practice research (CPR) [25] that proved to be very useful for projects aimed at understanding, supporting and improving software development practice. The CPR defines three goals that are important in software process improvement and suggests activities that help to achieve them. First, to understand the current state of software development (the goal) the interpretation of practice has to be engaged in (the activity). Second, to build new knowledge that can support practice (the goal), normative propositions, such as guidelines, standards, methods, tools, etc. has to be designed (the activity). Finally, to learn what is needed to actually improve practice (the goal), various social and technical interventions have to be employed (the activity). Although the three goals are distinct in nature and can be pursuit in isolation this is not suggested as it would seriously reduce the opportunities to learn about practice. The CPR takes as the main consideration the difficulty of establishing effective and well functioning relations between research and practice. Ideally, the research process should be tightly connected with practice to get firsthand information. At the same time the research needs to be well structured and managed to produce rigorous and publishable results. This, of course, is difficult if the project agenda strongly depends on how practice evolves. The CPR recognises three basic research approaches that help to take into account the dilemmas how to fulfil the two criteria. These are: Action research, Experiments and Practice studies. Action research is based on strong integration between research and practice. The researcher is involved in 1 The MasterProc is a research project which is carried out under the umbrella of the Centre of excellence. One of the missions of the Centre of excellence is to improve software development methods in Slovenian companies. The project is co-founded by the Slovenian Ministry of Higher Education, Science and Technology, European Commission and the participating Software Companies.

2.2. Organisation of the MasterProc project

2

The timeline and learning cycles of the MasterProc project are influenced by the dynamics of the Centre of Excellence. According to the currently known information the Centre of Excellence will finish its first programme period in the end of 2006 and start with the next period in the beginning of 2007.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

347

Table 1 The research approach Goal 1: to understand the current state of software development Activities: for each of the participating companies: – [A1] Acquire information on existing ways of working (how the company performs its software development projects). – [A2] Determine the level of flexibility a software development method would require to appropriately support typical projects of the company. Research approaches: – interviews and questionnaires. – action research. Description: to carry out an assessment of the existing state of the art of software development methods in each of the participating software companies several meetings were organised. A combination of interviews and questioners was used to acquire information on how the companies currently develop software. The main objective of the assessment was to determine how socio-technically suitable are the methods for typical projects carried out by each of the software companies and to identify deficiencies that would need improvements. Furthermore, the goal was to identify, for each of the participating companies, how different in terms of the ways of working are the projects it performs. This information indicated what level of flexibility would be required for the company’s method in order to provide appropriate support for its ways of working. The information that we received from the interviews and surveys was complemented by action research. For each of the participating software companies a working team was set up comprising two researchers and two practitioners. The main responsibility of the team was to take part in real projects to get firsthand information on how a particular company is developing software. The practitioners acted as project managers and method engineers, while the researchers were more or less observers. Goal 2: to build new knowledge that can support practice Activities: – [A3] For each of the participating companies conduct a research to understand how to resolve identified deficiencies in its ways of working; thus to improve current practice. – [A4] Conduct a research to understand how to create flexible software development methods. Research approaches: – literature review. – experiments and field studies. Description: the objective of the second phase of the MasterProc project was twofold: first, to find out how the existing ways of working in each of the participating companies could be improved, and secondly, to find out a solution for creating flexible methods, i.e. methods that companies would be able to adapt to circumstances of a particular project. The weaknesses of the companies’ software development processes that were identified during the first phase of the project were thoroughly studied. Improvements were proposed in terms of guidelines, methods, tools, etc. and discussed in one-day workshops with each participating company. Some of the improvements were tested through experiments and field studies. For example, in one of the companies it was found out that when iterative development was taken the project managers were unable to manage iterations. To resolve this problem a research was conducted to find out how different approaches, specifically agile methods, suggest managing iterations in iterative software development. The solution that we suggested was first tested in our laboratory and after that in a real project performed by the company. Since the feed back was positive the improvement was scheduled for its implementation. The second objective of this phase was to make methods of the companies flexible, so that they could be adjusted to project-specific circumstances. This objective was much more difficult to achieve. At first we conducted literature review to study state of the art in the field of method construction and its deployment. The possible approaches to integrate flexibility into the methods were than presented to chief information officers of the software companies. As it was expected, they all disagree with the possibility to integrate flexibility into their methods by using traditional assembly-based approaches to method construction, as they found them to complex and risky. They consented however to capture their methods (before the project all the companies used informal methods) and to develop appropriate tool support that will help them in adjusting these methods to suit particular projects. As a result of this activity the PCA approach was designed together with its tool-support. The PCA was tested in the laboratory environment and also in real settings provided by the participating companies. The purpose of the evaluation was to find out whether the assumptions on which the PCA was based were correct. For the details see Section 5. Goal 3: to learn what is needed to improve practice and to actually improve practice Activities: – [A5] Conduct a research on improvement strategies. – [A6] For each of the participating companies identify appropriate strategy to make improvements to their software processes and to introduce PCA approach. – [A7] Implement improvements. Research approaches: – literature review. – survey. Description: once current practice is understood and new knowledge is built to support the practice, appropriate measures need to be taken to actually improve the practice. In the MasterProc project a thorough research was conducted to understand how to measure and specifically how to improve software development practices by considering technical and social characteristics of the ways of working in a particular organisation (see e.g. [30]). One of the results of this research is the set of strategies that helps to implement improvements based on the socio-technical characteristics of the problem domain. For example, in the case described above our goal was to improve the company’s practice in the activity that was responsible for managing iterations. The problem with this activity was not that it would not be adopted by the company’s project managers, but it was technically inappropriate, as it leaded into loosely planed iterations. The strategies for implementing improvements to resolve technical deficiencies are much different than strategies for implementing socially-based improvements. The implementation of improvements into the participating companies (including the PCA approach) is still under execution. According to our plans it will be concluded in September 2006.

348

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

3. Related works The research in the software development field has brought about many ideas how to deal with the inflexibility of software development methods. Two of them are a contingency factors approach and, specifically, method engineering. 3.1. Contingency factors approach The contingency factors approach (see e.g. [1,8]) is based on the suggestion that specific factors of the development context are considered and taken into account when selecting an appropriate method from the portfolio of methods. The problem with this approach in practice is that an organisation is expected to be familiar with a range of methods from which they choose the most appropriate one based on the contingencies of the current situation. In practice it is at least unusual for an organisation to have a number of methods available and even more unusual for the developers to be familiar with all of them. This problem has been acknowledged among the researchers who suggested a more pragmatic approach [1]. Instead of selecting from the portfolio of methods the method should be wide enough to encompass all the possible situations. This is the idea that we have followed also in our approach. 3.2. Method engineering Method engineering is another approach to create software development methods that are specifically attuned to organisations or projects. In general, the idea lies in the conceptualisation, construction and adaptation of methods and tools for the development of information systems [5]. Method engineering has a long tradition in the systems development research. An excellent review of the past research can be found in the work edited by Brinkkemper et al [4]. For a review of more recent efforts see the work of Ralyte´ et al [27]. Irrespective of its popularity, the method engineering has never been widely acknowledged or practised by software engineers. In [16], Henderson-Sellers argues that this is because practitioners often (unfairly) view method engineering as having a costly overhead and do not take into account the cost and effort in cases when an ‘out-of-thebox method’3 is used and found as inappropriate to company’s business processes. Whether this is true or not, one thing is certain: method engineering lacks examples of practical application in real development practice [12]. A specific direction in method engineering which shares the same objective as the approach that we advocate in this paper is situational method engineering (SME). In general, SME deals with developing project-specific methods or 3 ‘Out-of-the-box methods’ are presented as ready for immediate use. Sometimes they are also called ‘of-the-shelf methods’ or ‘pre-packaged methods’. RUP (Rational Unified Process) is an example of such a method.

adapting existing ones to specific project situations (e.g. [15,24]). The next section describes various approaches to SME. 3.3. Approaches to SME In the literature, a number of approaches can be found that propose the creation of project-specific methods. One that is probably the most popular is the so-called assemblybased SME, which follows the reuse strategy. In this approach a new method is constructed from the fragments of existing methods. The notion of method fragment was introduced by Harmsen and Brinkkemper [15] who defined it as a reusable part of a method. Fragments can be further categorised into product and process fragments depending on the perspective they cover. Much effort has been put into decomposing existing methods into fragments [4]. Also, different repositories have been proposed for their storage (e.g. [15,27,4]). The method construction using the reuse strategy is, however, far from easy, as the fragments have to be first retrieved from the repository, changed if necessary and than assembled together into one consistent and congruent method. Another approach to SME, known from the literature as the extension-based approach, uses the extension strategy. In this approach, method engineers are provided with extension patterns that help them to identify typical extension situations and provide guidance to perform extensions. In [27], Ralyte´ describes two possible ways to perform extensions: (a) directly through matching extension patterns stored in a library to satisfy the extension requirements, and (b) indirectly through first selecting a meta-pattern corresponding to the extension domain and then guiding the extension applying the patterns suggested by the meta-pattern. Karlsson and ˚ gerfalk have, however, criticised this approach for not conA sidering situations that are actually very frequent in practice, i.e. when a method is both extended in some fragments and reduced in others [22]. As a solution they proposed a new method for SME that uses a combination of the cancellation and extension operators. They named it method for method configuration (MMC). The MMC differs from the aforementioned approaches also in the fact that it does not deal with modular construction of a method but rather with method tailoring taking a particular method as the starting point. From the literature, it is clear that this approach has been somewhat overlooked by the method engineering research in the past. Finally, the approach to SME that seems to be a result of the most recent efforts in the method engineering research is the paradigm-based approach [27] a.k.a. evolution-based approach [2]. This approach is founded on the idea that the new method can be obtained either by abstracting from an existing model or by instantiating a metamodel. A new method is then created by first constructing a product model and then process model while for the construction of both product and process model different strategies are available.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

349

Fig. 1. The timeline of the MasterProc project.

3.4. How the PCA fits into the existing SME research? To position the PCA within the existing SME research we will first describe the characteristics that can be used to compare different SME approaches. There are several aspects in which SME approaches differ, but most notably in: • the point of departure, • the granularity of method fragments used to create situational methods, and • the ability to execute. The point of departure tells what kind of source a SME approach is using to create situational methods. There are at least three points of departure that can be distinguished: (a) a portfolio of methods, (b) a portfolio of fragments from different methods, and (c) a single method. In the first case (a), a portfolio of methods is considered and the method that most matches the project circumstances is selected to be used on the project. An example of the SME approach that uses a portfolio of methods as a point of departure is the Contingency factors approach described in Section 3.1. In the second case (b), situational methods are created from the selection of fragments that are stored in some repository. The fragments belong to different methods and may overlap among each other. The SME approaches that fall into this category are difficult as they require fragments from different methods to be assembled together. The assembly-based SME approaches are examples of this category. In the last case (c), the source is represented by a single method (a.k.a. base method). The method is divided into several fragments which are examined according to the project circumstances and extended, omitted or changed if necessary. An example of this category is the MMC approach. Another distinguishing characteristic of the SME approaches is the granularity layer at which situational

methods are created. The granularity layer of a particular SME approach tells the precision of the elements the approach works with when it creates new or adapts an existing method. The notion of granularity layer has been introduced by Brinkkemper et al. [6] who defined a classification framework for method fragments. Based on this classification, method fragments can be classified according to three dimensions: perspective, abstraction and granularity. The dimension that seems to be the most important and the main discriminating property of method fragments is the granularity layer at which the fragments reside. At one extreme, the granularity layer could be as high as to cover a complete method, while at its lowest layer it would only cover a small piece of a product that is delivered by a small part of the method or the procedure that tells how to produce this product. The granularity layer has been recently debated to be very important SME characteristic, as it influences the complexity of the method engineering, in particular the method assembly (e.g. [22,31]). If method fragments are presented at high level of abstraction the construction of new methods is easier, as the method engineers do not have to worry about the details. Of course, to make this work, the fragments have to satisfy certain criteria (for details see [31]). Another issue worth to be mentioned at this point is that the layer of granularity presents a trade-off between complexity and flexibility. In other words, if the granularity layer of the method building blocks is high, the flexibility in producing new methods is low. In an ideal situation, a SME approach would work with fragments that support the highest possible precision in creating new methods while the consecutive complexity would be resolved by an appropriate tool-support. The last but not least characteristic that can be used to classify various SME approaches is their ability to execute. We use this characteristic to denote how deeply a particular SME approach goes in describing the concepts, data structures and algorithms to create or tailor a new method. For a SME approach it is of utter importance that it assures the

350

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

created methods are consistent and complete and that they suit best to the project circumstances for which they were created. How the consistency is achieved, how the completeness of the created methods is verified, how the fragments are described and retrieved when needed, how the contingency factors are expressed etc. are questions that need answers in order to use the SME approach in real practice. In light of this characteristic, the existing approaches differ a lot. Some of them are described rather superficially while others provide much more details on how the approach actually works. According to the above characteristics the PCA approach can be classified as follows: it uses a single method as the point of departure (base method), it works on a low layer of granularity (thus providing the highest possible flexibility) and provides sufficient details to execute in real practice. For the construction of situational methods it uses a combination of the meta-modelling and extension/reduction based approaches. As such it shares several commonalities with other approaches to SME, but most notably with MMC. Both PCA and MMC suggest configuring an existing method rather then assembling fragments from different methods to construct a new one. As we will show in the next section, the PCA introduces some features that simplifies the creation process and might prove useful also in the combination with other approaches (for details see Section 6). 4. The process configuration approach 4.1. Overview The idea that lies behind the PCA is relatively simple and can be explained as follows (see also Fig. 2): for each individual project a specific process configuration (project-specific method) is created. This is done by selecting components from a method that has been specifically designed for the organisation and thus reflects its actual performance on the projects (base method). The configuration is done by processing the rules that tell in what circum-

stances or project situations it is compulsory, advisable or discouraged to use a particular component. This section is organized as follows: to avoid confusions we will first define the terminology that we use in the paper (Section 4.2). Sections 4.3 and 4.4 will then describe how a base method can be created and represented, and finally, in Section 4.5, algorithms for the PCA will be explained. 4.2. Terminology In our research we try to understand a method also as a social construct. A method is full of philosophy, principles, ideas and points of view of the organisation staff or its users, what explicitly emphasizes its social component. As Cockburn [7] has emphasised, a method is everything we do to achieve a certain result, i.e. a product or services, which are the goal of our work. When talking about systems development this does not merely mean activities that are directly connected to the development (i.e. analysis, planning, etc.) but also patterns of communication, collaboration and coordination, support procedures, means of communication with the parties involved, rules of decision, etc. We also distinguish between methods that are informal and those that are formalised. A method consists on one hand of elements such as procedures, rules, directions, tools, standards, etc. which can be documented either in electronic or classical manuals, and on the other of certain undocumented parts, and above all the knowledge of the organisation members [3]. This is of utter importance for an organiation, because it represents its own values and competitiveness. A method that is used by a particular organisation is typically richer then its’ formalised and documented part. A substantial part of a method is embodied by its users through the knowledge and experience they carry. When such knowledge becomes sufficiently expressible and the tasks in which the knowledge is used sufficiently repeatable, the formalised part of the method can be enriched with new elements (e.g. models, procedures, guidelines, advices, etc.). Besides the fact that informal

Fig. 2. High-level architecture of the PCA.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

methods could never be fully formalised, empirical investigations show that many software development companies do not have any formalised method at all. Or if they have one, they rarely keep it consistent with their actual performance on IT projects (e.g. by enriching it with new knowledge and experiences). The distinction between informal and formalised methods has been clearly acknowledged also by other researchers. Fitzgerald, for instance, emphasises that there is often a wide disparity between the official development process and the actual behaviour of developers in practice [9]. This suggests that in each development project a specific method in-action is created which is shaped by the characteristics of the context, the developers and the information system under development as well as the formalised method and the rational and political roles it plays [11]. The significance of the relationship between formalised method and its background understanding has been discussed also in the work of Introna and Whitley [21]. In their work they actually criticised the idea of method engineering for being based on rather naı¨ve assumption that ‘method is a necessary and sufficient requirement for successful information systems development’. In contrast, they suggest that ‘the successful use of method itself depends on a brother, already present, tacit understanding of the world, an understanding not to be found in any one particular method’. In the PCA we deal with two kinds of methods, which both are formalised: base method and project-specific method. While a base method is specifically attuned to a particular organisation, a project-specific method is adjusted to the needs of a particular project. The distinction between a base method and project-specific method is that the former includes all possible elements that might be useful in performing organisation-specific projects (i.e. a repository of the organisation-specific process components), while the later only consists of those elements that are required for a particular project.

then suggested and discussed with the organisation’s development team. Once a vision for a new method is developed and accepted, a metamodel for the organisation’s base method is designed. A metamodel for the organisation’s base method can be developed from scratch, or by using existing metamodels that have been recently constructed to underpin and help formalise methods. Existing metamodels represent a good source for selecting generic concepts for an adequate representation of the organisation’s base method. When such a metamodel is developed, it is used to help creating the organisation’s base method, i.e. to capture the method fragments. In this process different fragments are captured. Besides fragments that are based on the organisation’s existing practice, previously approved as technically and social appropriate, many new fragments may emerge. These are based on the suggestions for improvements, identified within the analysis of the existing practice. The fragments are first classified according to the underlying metamodel and then described using templates. The templates belong to a metamodel and outline how elements of a certain metamodel type should be described. In the PCA, it is essential that the underlying base method and corresponding rules continuously evolve as a reflection of knowledge and experiences acquired through project performance (see Fig. 3). This means that using the PCA new fragments may emerge as a result of situations that are specific and thus not yet supported by the base method under use. In such cases, additional fragments are captured to describe these new situations, and circumstances are determined in which these new fragments should be used. In practice, it actually takes some time for a base method to become all-inclusive in terms of providing guidelines for all kinds of situations that may happen in projects that a particular organisation is performing. This phase, in which base method rapidly

4.3. Base method Base method is probably the most important prerequisite for the PCA. It is a formal representation of how a particular organisation is performing its projects. The construction of a base method is thus crucial as it presents a foundation for creating project-specific methods. The process that supports the creation of a base method is part of the framework for method reengineering. Since its detailed description would be beyond the scope of this paper, we will provide only a brief description of its main activities. More detailed description can be found in [3]. The construction of a base method is a procedure that has to be done for each organisation individually. It starts with the analysis of the existing practice in an organisation and leads into the identification of the method components (activities, tasks, products, etc.) that are technically and socially sound and those that are in these respects problematical. Possible improvements to the existing practice are

351

Fig. 3. Base method continuous evolution.

352

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

evolves, is called the learning phase. It takes place in the first few projects after the PCA is introduced into an organisation. After the learning phase base method becomes more stable and changes on a large scale less frequent. The idea that it is possible to elicit the ways of working in a particular organisation and to create formalised method (base method) based on the elicited information should not be seen in conflict with the aforementioned findings of Introna and Whithly [21]. What they were arguing was that for an effective use in various development situations one single, pre-developed method will probably not be sufficient. The hypothesis on which our approach is based is not against this opinion. We believe however that if for a particular organisation a formalised method is created directly from its existing practice and knowledge, improved in parts where found inappropriate and than continuously supplemented by new experiences and knowledge that developers gain from the method use, such method might prove very useful for the organisation as it can help to perform projects in more consistent and transparent manner. Moreover, in this way – by using feed-back from method use to enrich formalised part of the method we take care the new knowledge and experiences from the method use are not lost but shared within the organisation’s development community. An in-depth discussion of the needs, rationale for and philosophy of the method evolution that supports our view on the method engineering has been provided by Rossi et al [29]. In their work they suggested how a method rational which they define as a ‘trace of evolutionary method changes and associated method use experiences’ can help to share knowledge of methods between method users and engineers and how

can act as a powerful mechanism to maintain systems development knowledge in an organisation. 4.4. Representation of a base method For the purpose of the PCA we designed a generic data structure that can be used to capture an arbitrary metamodel. The idea of a generic data structure is to allow method engineers to design metamodels according to their perceptions of how a method should be formally represented. Fig. 4 illustrates the main components of the aforementioned generic data structure, base method and project-specific method. The classes representing metamodel are: a metaelement (it can be of two types: content element, such as activity, tool, discipline, role, etc. or process flow element, such as decision node, join and synchronisation) and metalink (links between metaelements). By using such a generic data structure, a base method is represented as a structure of instances of the metaelements and metalinks, and a project-specific method as a selection of the elements and links of the base method. According to [28], the Metamodel represents abstraction level 1, i.e. the Process Metalevel, while both the Base method and Project-specific method represent abstraction level 2, i.e. the Process model. As mentioned before, a base method encompasses various situations that may occur when projects are performed. In other words, it comprises a number of elements and their alternatives which describe several possible ways to perform a particular project (In [4], Hares introduces the notion of project paths to describe different ways of performing a project). The paths and method structure,

Fig. 4. A generic data structure underpinning an arbitrary base method.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

however, are not fixed in the PCA. They are defined by the rules that tell which elements to consider in specific circumstances and consequently which path to take. As depicted in Fig. 4, rules apply directly to the links that bind elements of the method (see the element Condition). As we will see later, these represent just one category of rules that we deal with in the PCA. Fig. 5 shows an example of a part of a base method that has been developed during the evaluation of the PCA in one of the participating companies. It represents how the analysis is performed in that company. More details on the base method content can be found in Section 5.4. The reason the illustration is provided here is to show how process flow and structure rules constrain the links between method elements.

353

Besides the rules that put constraints on the links between elements of the method there are also other types of rules that play important role in the PCA. In general, they can be categorised into constraint rules and facts. Since in performing the PCA rules play essential role we will explain their taxonomy in more detail. 4.4.1. Constraint rules Constraint rules can be seen as assertions that constrain some aspect of the PCA for constructing project-specific methods. They can be decomposed into four subgroups: process flow rules, structure rules, completeness rules and consistency rules. Process flow rules are rules that define conditional transitions among activities in the process view of a method.

Fig. 5. Representation of a base method.

354

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

They define the conditions that have to be met to perform a particular transition. For example, in Fig. 5, the rule R1 defines a conditional transition to the activity Analyse Logical Structure while the rule R2 determines in what circumstances the activity Analyse Logical Structure can be omitted. Similar to process flow rules are rules that belong to the structure rule category. Their distinction is that they can constrain any link between method elements and not just links between activities. In Fig. 5, the rule R4 represents an example of a structure rule. It constrains the link between the activity Develop Prototype of the System and the tool Delphi. Since both process flow rules and structure rules constrain links between method elements they can be specified by providing the link to which they apply and the conditions that they require to be satisfied. Please be aware that such links do not necessarily connect content elements directly (e.g. two activities) but can also connect content elements with process flow elements, such as join or synchronisation (e.g. a link between an activity and process flow element or vice versa) or even two process flow elements (e.g. a link between a decision node and synchronisation point). Structure and process flow rules that belong to a base method of a particular organisation actually define project characteristics that are important at a particular stage of any project performed by the organisation. Their conditional part can be specified using the following template: [C1 LO C2, . . ., LO Cn] where LO is a logical operator and Ci an atomic piece of the condition. Ci corresponds to the following form: [ProjectCharacteristic AO Value] where AO is an arithmetic operator and ProjectCharacteristic a named characteristic of the project. Examples of process flow rules (rules R1, R2 and R3) and structure rules (rule R4 and R5) are provided below.4 • R1: if the process is in the decision node 1 and the scope of the system is large or incremental SDLC is chosen then go to activity Analyse logical structure of the system. • R2: if the process is in decision node 1 and the scope of the system is not large and incremental SDLC is not chosen then go to synchronisation point 2. • R3: if the process is in decision node 2 and problem domain is new or customer requires the prototype of the system then go to activity Develop prototype of the system. • R4: if the process is in activity Develop prototype of the system and the time frame for producing the prototype is more then 1 month then develop the prototype of the system using Delphi tool.

• R5: if the process is in activity Develop prototype of the system and important reports are to be developed then create output artifact Reports as a part of the prototype. The rules above are written in natural language. Their conditional parts are underlined. If we used the template [C1 LO C2 ,. . ., LO Cn] the rule R1, for example, would be rewritten as follows: [scope of the system = large OR SDLC = incremental].5 Project characteristics, such as project length, project risk, project complexity, the scope of the system, the number of parties involved, etc. and their respective domains are defined within the organisation’s base method. However the values that these characteristics receive are project-specific and are thus defined during the PCA execution. As you might noticed, assertions can be very concrete, such as in the rule R5 that asks whether important reports are to be developed. Besides process flow rules and structure rules that both put constraints on associations between elements of a base method the constraint rule category comprises also completeness and consistency rules. The purpose of these two subcategories is to assure that each project-specific method, created from the elements of a base method, is complete and consistent. Completeness rules apply – in contrast to the process flow rules and structure rules – to a metamodel and not to a base method. Their responsibility is to define the conditions that must be met when creating a project-specific method. Completeness rules actually help to check whether a project-specific method that has been created includes all required components. For example, an organisation may decide the following rules have to be followed when creating methods for projects: • R6: each activity except the last one must have at least one successor activity. • R7: each activity must be linked with exactly one role. • R8: each technique must be linked with at least one tool, etc. Completeness rules can be specified in a simple manner using attributes in the metalink class (see Fig. 4). Two attributes are sufficient: one stating the minimal and other the maximal number of connections between the metaelements a particular metalink is connecting. For example, the rule R7 applies to the metalink between the metaelements activity and role specifying that the minimal number of connections is zero and the maximal number of 5

4

The rules are here written in natural language to ensure their understanding. When used in the PCA they need to be translated into the syntax dictated by the supporting tool.

While it may seem very time consuming to specify process flow and structure rules in such a manner, this could be done easily if the tool supports graphical process modelling. In the toolset that we developed in support of the PCA the modelling feature was included and the specification of process flow and structure rules was found rather simple.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

355

that can be combined together in whatever way. Instead, we start from a base method that presents a collection of fragments that are bind together by constraint rules which put limitations on what we can omit from the base method. If captured properly, a base method should consist of several consistent and complete subsets of fragments that correspond to particular development situations. Fig. 6. An example of a potentially conflicting situation.

connections is one. Note that if the rule put a constraint on the number of activities per role, it would apply to a different metalink. Consistency rules are the last category in the group of constraints. They are similar to completeness rules. Their goal is to assure that the selection of fragments comprising a project-specific method is consistent. While completeness rules only apply to elements that are linked together, consistency rules deal with interdependency between any two elements. In other words, for each element e they determine a set of other elements E that need to be included into a project-specific method if e is included. In the example below the rule R9 asserts that the deliverable Business model is dependent on the activity Business modelling. This means that if the deliverable Business model is selected for the inclusion into a project-specific method, the activity Business modelling has to be selected too. While such a dependency may seem trivial it is important as it helps to avoid conflicting situations. Let us consider the situation in Fig. 6. The rules Ra and Rb are both structure rules. Since they define project circumstances independently from each other it might happen that they are in conflict. For example, the rule Ra discourages the production of the Business model artefact while the rule Rb requires it. With the consistency rule R9 such a conflicting situation would be prevented. • R9: the deliverable Business Model depends on the activity Business modelling. Consistency rules can be specified using the following template: [IF e THEN E], where e is an element and E a set of elements that edepends on. The completeness and consistency have been identified already by pioneers of the method engineering field as the two most important criteria for internal or situational-independent quality of a situational method (see [7,18], and [14]). Since in the PCA, situational methods are created by adjusting an existing method (base method), for which we have to take care before that it is complete and consistent, these two qualities should not be too difficult to achieve. The possibility that we end up with an inconsistent or incomplete method should be actually much lower as we do not start from a collection of fragments

4.4.2. Facts Another important group of rules that are considered during the PCA are facts. Facts are assertions that define characteristics of the project for which we create a project-specific method. Depending on how they define project characteristics they can be classified into base facts or derived facts. Base facts define project variables directly while derived facts are derived from base facts using inferences or calculations. In the examples below, the rule R10 is a base fact while the rule R11 is a derived fact. • R10: the project domain is well known. • R11: if the project field is telecommunications or healthcare then the project domain is well known. In the PCA facts are very important as they are checked when structure and process flow rules are processed. For example, a structure rule might state that ‘‘when performing requirements validation there is no need to produce a prototype if the problem domain is well known’’. To be able to perform this rule we must first check the facts about the project domain to find out whether the domain is wellknown. As indicated in the examples of the constraint rule category (see e.g. rules R3 or R5) facts can describe virtually any condition that is important for the project. Furthermore, they are created dynamically during the PCA performance. For example, when an element a is selected to be included into a project-specific method this becomes a fact (a is selected) which could become important latter on in the PCA performance. The metamodel in Fig. 7 shows the relationships between different categories of rules and their components that play important role in performing the PCA. 4.5. The PCA algorithms The algorithm that supports the PCA is relatively simple. It starts with an element in the base method (typically this would be a starting activity) and ends when there is no link that would connect the current element further with any other element. If such links are found they are examined for constraints they might have. When a particular link has no constraints or when constraints exist but are satisfied than the element at the end of that link is processed in the same way using recursion.

356

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

PROCEDURE CreateProjectMethod (pm, e); // pm – project method, e – starting element of the base method BEGIN Find links for the element e For each Link l IF conditions are satisfied for the link l THEN Mark the output element of the Link l as selected for the pm Mark the link l as selected for the pm CreateProjectMethod (output element of the Link l, pm)// recursion END IF NEXT END; When a project-specific method is created using the algorithm above, the elements that have been selected has to be checked for consistency and completeness. The verification algorithms below show how this can be done. PROCEDURE CheckCompletness (pm); // pm – project method BEGIN //completeness verification Select all links from the pm For each Link l //Check the completeness constraint for the link l Count the links that connect the input element of the Link l with the output elements of the same type as is the output element of the Link l IF the number of links is outside the min, max limits then mark the Link l as problematical. END; END; PROCEDURE CheckConsistency (pm, e); // pm – project method, e – starting element or // link of the project-specific method BEGIN //consistency verification Select the set of elements and links D that e is dependent on For each element or link d from D IF d is not selected THEN Mark d as problematical CheckConsistency(pm, d) //recursion.6 END; END;

6 To prevent the algorithm to come into a dead lock, there should be no circular dependency between consistency rules.

To conclude this section it is worth to mention that as a part of the MasterProc project we also developed a tool that supports all the activities of the method reengineering framework, including the activities related to the PCA approach. Since the detailed description of the tool would be out of the scope for this paper, let us just emphasise that in method engineering an appropriate tool-support is indispensable. For the interested reader, a thorough discussion on the existing tools in the method engineering field can be found in [23]. 5. Evaluation of the PCA 5.1. Background As described in Section 2, the PCA has been developed as a part of the MasterProc project. The ultimate goal of the MasterProc was to improve software development practice in Slovenian companies. To reach this goal our idea was to facilitate the companies with a framework and tool-support for reengineering their ways of working, i.e. producing software, so that the gap between their official methods (documented methods they claim to follow) and the ways how they actually develop software (methods in-action) would be as small as possible. The PCA presents an important part in this framework, as it suggests how to incorporate flexibility into formalised or documented methods, so that they could be adjusted to suite best to circumstances of a particular project. Furthermore, the PCA tells how to describe the ways of working in an organisation (organisation’s base method) so that project-specific methods could be than created automatically by using appropriate tool-support. In this section we describe how the evaluation of the first version of the PCA has been performed. Please note that the MasterProc is ongoing project and that the next cycle, in which we expect to receive more valuable feedback from the participating companies is yet to come. 5.2. Evaluation criteria Evaluation of a designed IT artifact requires the definition of appropriate metrics and possibly the gathering and analysis of appropriate data [17]. In our case, the evaluation was performed by testing the PCA in laboratory environment with artificial data, and by observing the use of the PCA in real settings provided by the participating companies. The evaluation in laboratory environment was restricted to functional testing, e.g. to verify how useful is the tool-support for designing and capturing base methods and to create project-specific methods. More important however was the evaluation performed in the participating companies which we describe in this subsection. The aim of this evaluation was to monitor the use of the PCA in multiple projects and to collect data that could show how useful is the PCA in real environment.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

357

Fig. 7. Categories of rules in the PCA.

When selecting quality measures for the evaluation of the PCA, our intention was twofold: first to confirm that the assumptions on which the PCA was based are correct, and second, to show that comparing to traditional assembly-based method engineering approaches which are criticised in the literature for their complexity, the PCA introduces an approach that is simple enough to be absorbed by the software development companies. One of the characteristics that distinguish the PCA from other approaches to method engineering is that in the PCA, project-specific methods are created from the organisation’s base method which documents the ways of working in that organisation. We believe that in each software development organisation, patterns of work could be found that tell how the organisation is developing software. While a large percentage of software companies own some kind of formalised method (typically commercial methods), empirical investigations show that what they really do on

IT projects differs a lot from what is written in the methods they own (e.g. [10,3]). Our assumption in the PCA is that in a typical software organisation the ways of working are sufficiently repeatable to be captured into a formalised method (base method) that would reflect how the organisation actually performs its IT projects. The quality measure that could prove this assumption is the number of changes that we have to do to the base method each time when we want to create a project-specific method. We expect that along with the evolution of the base method this number gradually reduces till it finally reaches the point when it becomes insignificant (see also Section 4.3). Another assumption that was made during the design of the PCA was that by adjusting the base method of an organisation it is possible to create project-specific methods which closely match the methods that are then really used in action. The quality measure that was used to validate this assumption was based on the number of changes that

358

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

were required during the project to make the project-specific method – created from the base method – consistent with the real ways of working on the project. More challenging in the evaluation of the PCA is how to show that the approach is powerful but yet easy enough to be absorbed by software development companies. The quality attribute that would clearly indicate this is the number of companies that successfully implemented the approach in their practice. Since the PCA as described in this paper presents only the first version of the approach for which the most important feedback has not been yet received we left this part of the evaluation to the next period of the MasterProc project. It has to be mentioned however that all the companies that participated in the first learning cycle of the project (see Section 2) agree to remain involved which could be seen as an indication of their belief in the usefulness of the approach. Another issue that we wish to stress at this point is that the measurements that we describe in this section were all conducted in controlled environment, i.e. with the researchers that helped the companies with the creation of the base methods, the usage of the tool, etc. This is important to notice, as it probably had some influence on how the developers and methodologists really acted on the projects. In the further text we provide the results that were obtained by evaluating the PCA in one of the participating companies. 5.3. The company’s profile The company that we selected for the first test is a typical software developing organisation. They develop software based on an object-oriented approach by using their own application framework. It employs over 50 developers and typically takes middle-sized projects that require teams from 3 to 10 developers. A majority of the company’s

developers is well experienced and skilled. A small group of the company’s programmers is dedicated to maintain legacy systems that are still in production. This group works in older development environments and follow traditional structural approach. They are less experienced in using modern information technologies. Before the pilot project was undertaken the company did not have any formal method as such but was developing software based on purely informal agreements and undocumented rules. 5.4. Developing a base method Our first step in the pilot project was to develop a base method for the company using the approach that we briefly described in Section 4.3. By inspecting the existing ways of working in the company we identified the elements that were technically and socially sound and those that were somewhat problematical. Detailed information on measuring socio-technical suitability of a method can be found in [3]. Close examination of the existing practice revealed that there were some similarities between their approach and the method RUP (Rational Unified Process) for which they had been trained before. Based on the conducted analysis several improvements to the existing practice were suggested and discussed with the company’s development team. The main objective of the evaluation at this stage was however to confirm the correctness of the assumptions on which the PCA was designed. To this end, we left the improvements of the existing practice to the next stage of the project which is dedicated to implement the improvements in an appropriate way (see Table 1, Goal 3). Our next step was to design a metamodel that would underpin the company’s base method. Fig. 8 depicts the metamodel elements and relationships that were chosen to adequately represent the company’s method. They represent those parts of the method that the company wanted

Fig. 8. The metamodel underpinning the company’s base method.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

5.5. Creating project-specific methods

Table 2 The number of captured fragments Metaelement Discipline Activity Role Deliverable Technique Guideline Tool Template Example Sum

Number of fragments 10 61 24 52 14 34 20 8 8 221

Table 3 The number of captured rules Rule type Structure rules Process flow rules Derived facts Completeness rules Consistency rules Base facts Sum

359

Number of instances 54 29 27 22 55 0 187

to have in documented form. In the core of the metamodel is the element activity which is in relationship with many other elements. For example, each activity has a role that is responsible for its execution. It is supported by guidelines and can produce or supplement many deliverables. Each deliverable can have a predefined technique which is used to produce the deliverable. Techniques are further supported by tools. Both templates and examples of deliverables could be provided to help to produce them. Activities are further grouped into disciplines. Besides the aforementioned elements which all are of the content type (see also Fig. 5) the metamodel includes elements for supporting the flow of activities. The recursive relationship of the element activity supports simple transitions between activities while for more complex transitions specific control flow elements are required. As depicted in Fig. 8, the abstract element process flow element has three subtypes: join, synchronisation and decision node, which support complex transitions. Note also that cardinalities of the relationships reflect the company’s completeness rules (see Section 4.4) and not associations in general. For instance, in general each technique can be supported by many tools but it is the rule of the company that there should be no more than one tool for each technique. Once the metamodel was approved by the company, the formalisation started. The number of method fragments and rules that were initially captured during the formalisation period is shown in Tables 2 and 3, respectively. Note that base facts were not captured during the formalisation of the base method, as they belong to the definition of the project environment.

After the formalisation the base method was tested in three real projects. In this subsection we will describe how project-specific methods were created from the base method. The projects that were conducted during the test were similar in technical terms and did not deviate much from typical projects the company had taken before. In all three cases the objective was to develop a web based, multi-user system with fat client in java. The company successfully reused its application framework and from this point of view there was no special action that had to be taken during the projects. The projects however differed in size, scope and problem domain. Furthermore, they had different development teams that diverged in the number and level of knowledge their members had about the problem domain. For each of the projects a specific method was created taking into account the project characteristics. This was done in three steps: firstly the method was created from the company’s base method by using the PCA algorithm (see Section 4.5). The result was then verified manually by method engineers and where found incomplete or inappropriate to the project specifics the base method was changed appropriately. The refinement typically included setting the status of selected activities to optional or decomposing deliverables into more detailed artefacts. Accordingly, several rules were changed or newly captured. When the refinement was finally complete the PCA algorithm was restarted. The third step of the method creation was then performed during the project execution. If it was found that some fragment was missing, redundant or inappropriate to the project requirements the necessary changes were made immediately. Finally, when the project was finished, the method engineer took care that the same changes were replicated in the base method. Table 4 shows different data measured during the projects. These are: • Number of fragments in the base method: it tells the number of fragments that were available in the base method before the initial configuration was made. • Number of initially selected fragments: it tells how many fragments were initially selected by the PCA algorithm by taking into account the projects specifics. We named this version of methodology initial methodology. • Number of fragments additionally captured during the refinement: this number tells how many fragments were additionally captured within the refinement of the base method. • Number of fragments selected after the refinement: after the refinement the PCA algorithm was performed again. This number tells how many fragments were selected for inclusion into the project specific method (refined method).

360

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

Table 4 Data collected during the projects

Number of fragments in the base method Number of initially selected fragments (initial method) Number of fragments additionally captured during the refinement Number of fragments selected after the refinement (refined method) Degree of consistency between initial and refined method (Drefined) Number of changes made during the project (final method) Degree of consistency between refined and final method (Dfinal)

• Degree of consistency between initial and refined method: the number tells how coherent were the initial and refined method. The degree was calculated using formula (1), where Nrefined is the number of all fragments in the refined method, Ninitial the number of all fragments in the initial method, and Nchanged the number of fragments in the refined method not included in the initial method. Drefined ¼

N refined  N changed N initial

ð1Þ

• Number of changes made during the project: during the project additional changes were made if the method was found inappropriate. This number tells how many changes were required so that the method was consistent with the real ways of working on the project. The resulted method was named final method. • Degree of consistency between refined and final method: similarly as above, this number tells how coherent were the refined and final method. The degree was calculated using formula (2), where Nfinal is the number of all fragments in the final method, Nrefined the number of all fragments in the refined method and Nchanged the number of fragments that were changed or newly captured during the project execution. Dfinal ¼ 1 

N changed N refined

ð2Þ

5.6. Discussion It can be noticed from Table 4 that the number of fragments in the company’s base method significantly increased after its application in the three projects. The increase was particularly considerable after the first project. The explanation for this lies in the fact that the base method, which was initially captured, was not fragmented enough to serve the PCA. Within the refinement the base method was then improved by adjusting the activities (several activities were added or decomposed into smaller units) and by the decomposition of the deliverables (since the first version of the base method included several composite deliverables, their decomposition contributed a lot to the number of fragments.). This has resulted in higher granularity of the fragments, which by nature brings more options for adaptability. In the next two projects the

Project 1

Project 2

Project 3

221 102 56 92 0, 655 11 0, 880

281 82 14 86 0, 848 5 0, 942

296 91 2 90 0, 920 2 0, 978

increase in the number of fragments was not that significant. This can be supported also by considering the degree of consistency between refined and initial method. The numbers in Table 4, row 6, clearly show that the degree of consistency improved in each project. Similar finding is suggested by the degree of consistency between refined and final method (see Table 4, row 8). As emphasised before, the projects that were conducted during the test were similar in terms of technology that was used to develop the solution. Furthermore, in all three cases object-oriented approach was followed. We anticipate that for this type of projects the learning phase of the base method is nearly finished and that in its future use it will not be necessary to make any major modifications. If taking different projects however, e.g. following structural approach, using different kinds of technologies (without reusing the application framework) etc. the base method would have to go through similar steps. We are aware that the results presented in this section can not add much to the validity of the PCA, as they do not address quality attributes that would imply the PCA is useful in practice or better than existing approaches. The evaluation that will address these issues is yet to come. Our intention at this point was to show that the assumptions on which the PCA is designed and which are rather unique (the PCA is the only approach to method engineering that creates project-specific methods from the method that elaborates patterns of work in a particular organisation) are correct. We believe the results are important for the initial iteration of the MasterProc project and that they suggest the project should continue. 6. Comparing and combining the PCA with other SME approaches The approach that we described in this paper builds on the previous research work on the method engineering field. Its main contribution should be recognised in its ability to execute. In this section we will first describe some of the features that distinguish the PCA from other SME approaches. It is important to stress however that our ambition is more than just to make another method for SME. We believe the PCA can act as a complementary approach and together with other SME approaches form powerful combination. A brief discussion on how the

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

PCA can be used in combination with some other SME approaches will be given in Section 6.2. 6.1. Important features of the PCA As stated above, the PCA shares the main principles with the traditional method engineering. It differs however in some aspects, specifically in those that revealed to be problematical for the application of method engineering in real practice. The PCA introduces several limitations, some of which turned to be its most important features: • It uses organisation-specific method to create project-specific methods: in SME approaches that are based on the reuse strategy, project-specific methods are created from fragments that might come from different methods. In practice this may lead into problems as the users of the created method might not be familiar with all the fragments (they might even resist to use them) and will therefore require additional training. In the PCA, on the other hand, project-specific methods for an organisation are created from its base method, which reflects how the organisation is actually performing its projects. The lack of knowledge of the components that are used to create or configure project-specific methods should therefore never happen as the base method is wellknown and adopted by the development team. In this way the PCA takes into account social aspects of the method construction/configuration which seem to be somewhat overlooked by the other SME approaches. In the framework for reengineering software development methods the social aspects of the method construction have been widely addressed [30]. • It introduces a two-phase method engineering approach to simplify the introduction of method engineering in practice: as described in Section 4, to produce a situational method by using the PCA, two phases are required. In the first phase, we develop a method that is specifically attuned to the needs and requirements of the organisation in question, i.e. it documents its ways of working (organisation-specific method or base method). This is an investment the organisation has to pay in order to use the PCA. Fortunately, software companies seem to understand the need for having formalised methods which come out as the side product of the first phase. We can call this phase method engineering, as we do engineer a new method (by eliciting the knowledge of the organisation’s workers about their ways of working and by improving their software practices). To systemise this work we have developed a framework that tells how to systematically approach to this problem (see [3]). The automation of this phase is however still under construction. The second phase is the development of situational methods. This phase occurs with much higher frequency, i.e. once for each project. It is important however that if the base methodology exists, this phase should not be difficult. In the paper we described in detail

361

how this can be done by using the organisation’s base method as a source. The two-phase method engineering approach is depicted in Fig. 9. • It simplifies the creation of situational methods by not letting the fragments to be changed once selected to be part of a particular situational method: indeed, the PCA does not allow nor require fragments to be changed or refined in order to assemble them together into one consistent method. Instead, the consistency is achieved by the process backbone and rules which are integrated into the organisation’s base method. Of course, the fragments may change as a result of the base methodology evolution, especially during the learning phase (see Section 4.3) but this is not part of the creation of a specific method. • It is easy, yet powerful: the PCA tends to bring the construction/configuration process closer to its users by providing facilities for managing the rules that describe in what circumstances a specific fragment or method component should be used. By using high-level rules method engineers can define virtually any kind of situation that leads either into using or omitting a particular fragment. In the PCA, each method component or fragment is supplemented by a set of rules that define when to use the fragment. In this way the PCA offers very general approach, as it permits to define virtually any situation in which a specific fragment has to be assessed for its inclusion into the method and any action to be taken with that fragment in that situation. The rules that are used as a means for defining how method is configured are atomic and thus independent which allows flexibility to be gradually incorporated into the method. • It focuses on software process improvement: the PCA is actually just a part of the broader framework for reengineering software development methods. Together with other parts of the framework it helps to improve software development practice, hopefully leading into better results in the software development field. 6.2. Using the PCA in combination with other SME approaches As emphasised at the beginning of this section, the PCA can act as a complementary approach, i.e. in combination with other SME approaches. In this subsection we will describe how the PCA could be combined with the assembly based method engineering. Furthermore we will show how the PCA could benefit from the approaches that are capable of packaging low layer fragments into self-contained components at higher layer of granularity. The idea that shows how the combined approach works is illustrated in Fig. 10. As depicted, five activities are performed. The activities that are additional or extended in regard to the stand-alone version of the PCA are: • Improve the base method by using assembly based approach: the first version of the organisation’s base method is rarely found satisfactory in all aspects and

362

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

Fig. 9. Two-phase approach to method engineering.

therefore needs improvements. In our work with companies it happened for example that a whole part of the software development method for which they all agreed was important was missing or was found incomplete. In the stand-alone version of the PCA, the improvements of the first version of the base method are handled manually, by identifying the deficiencies of the method and than by proposing improvements. The first step – identification of the deficiencies – is supported by the model that we have designed to facilitate the identification of the socio-technical deviations [30], while the next step – propose improvements – lacks a formal procedure. In the combined approach this presents an opportunity for the assembly based method engineering. A crucial part of this extension is the repository of fragments, which would have to be filled with fragments from best practices in the software development field. Thus, when some part of the base method would be found deficient, incomplete or missing the assembly based approach would be used to identify appropriate replacements by using the fragments from the repository. • Package fragments into components: One of the issues regarding method engineering that was pointed out as problematical by the method engineering community

and which the PCA seems to ignore is the use of fragments that reside at a low layer of granularity. As mentioned in Section 3.4 the complexity of the process that is used to create situational methods reduces if the fragments are presented at higher level of abstractions. This ˚ gerfalk was taken into consideration by Karsson and A [22], who introduced configuration packages and configuration templates which could present fragments at much higher level of abstraction. In more general way this issue was addressed later on by Wistrand and Karlsson [31], who discussed the use of method components as the building blocks for method engineering. They defined the method component construct as a self-contained part of a system engineering method expressing the process of transforming one or several artifacts into a defined target artifact and the rational for such a transformation. What is interesting with this approach is that method components can be used as black boxes, i.e. we do not need to know their internal details. For the construction of situational methods the method engineer can use interfaces which are available for each method component. Although the creation of situational methods is expected to be fully automated in the PCA, we believe the approach would benefit if worked

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

363

Fig. 10. An extension of the PCA approach.

with components that reside at higher layer of granularity. In the combined approach we therefore propose to package the fragments of the base method into higher level components using one of the aforementioned approaches. Since the PCA algorithms as described in Section 4.5 are not dependent on the granularity layer of the fragments they work with, no changes are required to produce situational methods.

We believe that combining the approaches could be beneficial not only for the PCA (e.g. as shown in this section) but also for other SME approaches, in particular for those that focus on one method when creating situational methods. By representing a base method as a graph of nodes (fragments) and links (relationships between fragments), underlined by a meta model that defines classes of fragments and semantics of their relationships, the PCA

364

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365

provides extremely flexible solution for defining any kind of software development method. Furthermore, for defining how fragments and their relationships are dependent on a particular situation (conditions on relationships between fragments), as well as for specifying their interdependency (consistency of produced methods), the PCA offers a well-considered system of rules which turned to be very useful in the process of creating situational methods as well as for checking the completeness and consistency of the result. We believe this solution could prove useful also for other SME approaches that are based on a singlemethod. In the MMC for example, the authors introduce the notion of configuration packages and configuration templates to present reoccurring development situations in a particular organisation. In this way they facilitate the process of creating situational methods by providing reusable assets. However, it seems that several tasks of the MMC approach need to be performed manually by method engineers who are burdened with a great responsibility. This includes the design of templates and packages, the assurance of consistency, the verification of completeness, etc. If the PCA environment is used to define base method, configuration templates and packages, the MMC could benefit of the analytical power that comes with the PCA environment. 7. Conclusions and further work Software development methods are underused today and one of the reasons for this is their inability to adapt to a particular situation/project. This problem has been acknowledged by the IS community a long time ago. Several solutions have been suggested most notably method engineering. Despite many different, well-considered approaches to create project-specific methods or to configure existing ones, there application in practice is somewhat low. In this paper we presented an approach for creating project-specific software development methods, called Process Configuration. The approach is based on the established method engineering principles, with an important distinction: it tries to avoid the obstacles that hinder the use of the existing approaches in practice. On the other hand, it presents a very useful architecture that can be used to implement any approach to method engineering that is based on the method tailoring. As described in the paper, the approach could be also integrated with the approaches that offer a repository of fragments to create project specific methods as well as with the approaches that are able to package low level fragments into higher level components. Further work on this research will include the implementation of the PCA in the companies that participate in the MasterProc project. The introduction of the PCA that has been done during the evaluation of the PCA (see Section 5) was intended to confirm the assumptions on which the PCA has been designed. Systematic implementation (i.e. following appropriate implementation strategies) of the PCA is however still under progress and is expected

to finish till the end of September 2006. After that, more exactly in the beginning of 2007, we are planning to initiate the next cycle of the MasterProc project. The main goal of the next cycle will be to redesign the PCA according to the feedback received from the companies using the PCA in their every-day practice. Even though the first cycle of the MasterProc has not been yet finished we have already received important initiatives from the participating companies that suggest how to proceed with the MasterProc project in the forthcoming period. Most importantly, there was an initiative to extend the PCA so that it would facilitate not only the description of the method that is appropriate for a certain project but also to support the execution and management of that project (according to the abstraction levels for processes, defined in [28], the PCA would have to be extended to support also the third abstraction level for processes, i.e. development runs). Another ambition for the next cycle of the MasterProc project is to redevelop the tool that supports the PCA to include new features as required by the redesign of the PCA. Moreover, the companies that are already using the PCA have let us known that the current tool-support is deficient and that it needs to be redeveloped in order to be used in every-day practice. It was agreed that the next version of the tool will be developed by a professional software company which will join the group in the next project iteration. Finally, our desire is to combine the PCA with other approaches to method engineering, e.g. as described in Section 6.2. Acknowledgements We thank Professor Brian Henderson-Sellers for his comments to the draft of this paper. We also wish to acknowledge financial support for this project from the Ministry for higher education and research, European Commission and the participating software companies: DataLab Technologies, Marand, IRC, Comland and IPMIT. References [1] D. Avison, A. Wood-Harper, Information systems development research: an exploration of ideas in practice, The Computer Journal 34 (2) (1996) 98–112. [2] M.B. Ayed, J. Ralyte, C. Rolland, Constructing the Lyee method with a method engineering approach, Knowledge-Based Systems 17 (2004) 239–248. [3] M. Bajec, M. Krisper, R. Rupnik, The scenario for constructing flexible, people-focused systems development methodologies, Proceedings of the European Conference on IS, Turku, Finland, 2004. [4] S. Brinkkemper, K. Lyytinen, R.J. Welke, Method engineering: principles of method construction and tool support, in: S. Brinkkemper, K. Lyytinen, R.J. Welke (Eds.), Selected Papers from the International Conference on ‘‘Principles of Method Construction and Tool Support’’. Atlanta, USA, August 1996, Kluwer Academic Publishers, Boston, MA, 1996. [5] S. Brinkkemper, Method engineering: engineering of information systems development methods and tools, Information and Software Technology 38 (4) (1996) 275–280.

M. Bajec et al. / Information and Software Technology 49 (2007) 345–365 [6] S. Brinkkemper, M. Saeki, F. Harmsen, Meta-modelling based assembly techniques for situational method engineering, Information Systems 24 (3) (1999) 209–228. [7] A. Cockburn, Agile Software Development, Pearson Education, Inc., Boston, MA, 2002. [8] G. Davis, Strategies for information requirements determination, IBM Systems Journal 21 (1) (1982) 4–30. [9] B. Fitzgerald, The use of systems development methodologies in practice: a field study, The Information Systems Journal 7 (3) (1997) 201–212. [10] B. Fitzgerald, An empirical investigation into the adoption of systems development methodologies, Information & Management (34) (1998) 317–328. [11] B. Fitzgerald, N.L. Russo, E. Stolterman, Information Systems Development: Methods in Action, McGraw-Hill, Berkshire, UK, 2002. [12] B. Fitzgerald, N.L. Russo, T. O’Kane, Software development method tailoring at Motorola, Communications of the ACM 46 (4) (2003) 65– 70. [13] C.J. Hardy, J.B. Thompson, H.M. Edwards, The use, limitations and customization of structured systems development methods in the United Kingdom, Information and Software Technology 37 (9) (1995) 467–477. [14] A.F. Harmsen, Situational Method Engineering, Doctoral Dissertation, Moret Ernest & Young Management Consultants, Utrecht, The Netherlands, 1997. [15] A.F. Harmsen, S. Brinkkemper, H. Oei, Situational method engineering for information system project approaches, in: A.A. VerrijnStuart, T.W. Olle (Eds.), Methods and Associated Tools for the Information Systems Life Cycle, IFIP Transactions A-55, NorthHolland, Amsterdam, 1994, pp. 169–194. [16] B. Henderson-Sellers, Method Engineering for OO Systems Development, Communications of the ACM 46 (10) (2003) 73–78. [17] A.R. Hevner, S.T. March, J. Park, S. Ram, Design Science in Information Systems Research, MIS Quarterly 28 (1) (2004) 75– 105. [18] R. van de Hoef, F. Harmsen, Quality requirements for situational methods, in: G. Grosz (Ed.), Proceedings of the Sixth Workshop on the Next Generation of CASE Tools, Jyvaskyla, Finland, 1995. [19] M. Huisman, J. Iivari, The individual deployment of systems development methodologiesLecture Notes in Computer Science, 2348, Springer, Berlin, 2002, pp. 134–150.

365

[20] M. Huisman, J. Iivari, The organizational deployment of systems development methodologiesInformation Systems Development: Advances in Methodologies, Components, and Management, Kluwer, Dordrecht, 2003, pp. 87–99. [21] L.D. Introna, E.A. Whitley, Against method-Ism: exploring the limits of method, Information Technology & People 10 (1) (1997) 31–45. ˚ gerfalk, Method configuration: adapting to [22] F. Karlsson, P.J. A situational characteristics while creating reusable assets, Information and Software Technology 46 (9) (2004) 619–633. [23] F. Karlsson, Method Configuration – Method and Computerised Tool Support, Doctoral Dissertation, Linko¨ping University, Linko¨ping, Sweden, 2005. [24] K. Kumar, R.J. Welke, Methodology engineering: a proposal for situation-specific methodology construction, in: W.W. Cotterman, J.A. Senn (Eds.), Challenges and strategies for research in systems development, Wiley, New York, 1992. [25] L. Mathiassen, Collaborative practice research, Information Technology and People 15 (2002) 321–345. [26] P. Middleton, Managing information system development in bureaucracies, Information and Software Technology (41) (1999) 473–482. [27] J. Ralyte´, R. Denecke`re, C. Rolland, Towards a generic model for situational method engineering, Proceedings of the 15th Conference on Advanced Information Systems Engineering, CAISE’03, Velden, Austria, June 2003, pp. 16–20. [28] C. Rolland, Modelling the requirements engineering process, in: H. Kangassalo, H. Jaakkola (Eds.), Third European–Japanese Seminar on Information Modelling and Knowledge Bases, Budapest, Hungary, 1993. [29] M. Rossi, B. Ramesh, K. Lyytinen, J.-P. Tolvanen, Managing evolutionary method engineering by method rationale, Journal of the Association for Information Systems 5 (9) (2004) 356–391. [30] D. Vavpoticˇ, M. Bajec, M. Krisper, Measuring and improving software development methodology value by considering technical and social suitability of its constituent elements, in: O. Vasilecas, J. Zupancˇicˇ (Eds.), Advances in theory, practice and education: Proceedings of the 13th Interernational Conference on IS Development, Vilnius, Lithuania, September 9–11, 2004, Technika, Vilnius, 2004, pp. 228–238. [31] K. Wistrand, F. Karlsson, Method components: rationale revealed, in: Proceedings of 16th International Conference on Advanced Information Systems Engineering (CAiSE*04), Riga, Latvia, June 7–11, 2004, Lecture Notes in Computer Science, Springer Verlag.