A comparative analysis of learning curves: Implications for new technology implementation management

European Journal of Operational Research 200 (2010) 518–528

Contents lists available at ScienceDirect

European Journal of Operational Research journal homepage: www.elsevier.com/locate/ejor

Decision Support

A comparative analysis of learning curves: Implications for new technology implementation management Malgorzata Plaza a, Ojelanki K. Ngwenyama a,b,*, Katrin Rohlf c a

Institute for Innovation and Technology Management, Ryerson University, 55 Dundas Street West, 9 Floor, Room 3-089, Toronto, Ontario, Canada M5B 2K3 Aarhus School of Business, Aarhus University, Denmark c Department of Mathematics Ryerson University, Toronto, Canada b

a r t i c l e

i n f o

Article history: Received 8 September 2007 Accepted 5 January 2009 Available online 20 January 2009 Keywords: Project management Information systems implementation Learning curve models Organization learning

a b s t r a c t New technology implementation projects are notoriously over time and budget resulting in significant financial and strategic organizational consequences. Some argue that inadequate planning and management, misspecification of requirements, team capabilities and learning contribute to cost and schedule over runs. In this paper we examine how learning curve theory could inform better management of new technology implementation projects. Our research makes four important contributions: (1) It presents a comparative analysis of learning curves and proposes how they can be used to help ERP implementation planning and management. (2) Based on empirical data from four ERP implementation projects, it provides illustrations of how managers can apply the curves in different project situations. (3) It provides a theoretical basis for empirical studies of learning and ERP (and other IT) implementations in different organizational settings. (4) It provides empirical justification for the development of learning curve theory in IT implementation. Crown Copyright Ó 2009 Published by Elsevier B.V. All rights reserved.

1. Introduction Cost and schedule overruns during new technology implementation projects are well documented in academic and practitioner journals. Recent studies (Robbins-Gioia, 2002; The Standish Group, 2004) find that more than 70% of ERP implementations are over schedule and budget. Much of the literature aimed at ‘fixing’ the ERP implementation problem focuses on prescribing ‘best practice’ for successful ERP implementation (Davenport, 1998; Markus et al., 2000; Besson and Rowe, 2001). Some of the prescriptions encourage project managers to: (a) adopt standard business processes to fit with the ERP software (Markus et al., 2000; Palaniswamy and Frank, 2000; Sumner, 2000); (b) avoid customizing the software (Parr and Shanks, 2000; Mabert et al., 2001; Murray and Coffin, 2001); and (c) provide appropriate user training (Bingi et al., 1999; Holland and Light, 1999; Al-Mudimigh et al., 2001). Other researchers offer frameworks to assist managers in defining and analyzing critical success factors (Akkermans et al., 1999; Bingi et al., 1999; Holland and Light, 1999; Nah et al., 2001), and project

* Corresponding author. Address: Institute for Innovation and Technology Management, Ryerson University, 55 Dundas Street West, 9 Floor, Room 3-089, Toronto, Ontario, Canada M5B 2K3. Tel.: +1 416 979 5000/4203; fax: +1 416 979 5249. E-mail addresses: [email protected] (M. Plaza), [email protected] (O.K. Ngwenyama), [email protected] (K. Rohlf).

risk factors (Sumner, 2000). Still others suggest the building of intra-organizational coalitions to support ERP implementation projects (Pozzebon and Pinsonneault, 2005). However, there is now growing acknowledgment that team training/learning and improved project management methods are important to ERP implementation success (Bingi et al., 1999; Holland and Light, 1999; Kumar and van Hillegersberg, 2000; Willcocks, 2000; Hong and Kim, 2002; Verville and Halingten, 2002). Recent research has identified a relationship between project team capabilities and IT implementation effectiveness. For example, Chatzoglou and Macaulay (1996) found that project team capabilities are the single most important factor affecting the timelines of software implementation projects. A recent study by Karlsen and Gottschalk (2003) also found that knowledge transfer is significantly related to IT project success. Dixon (2000) found that knowledge transfer among teams enables problem solving on IT projects. Several researchers have also pointed out that inaccurate predictions of team learning performance often lead to consequences of project creep and cost escalation (Keil et al., 1995; Chatzoglou and Macaulay, 1996; Callaway, 1999; Barry et al., 2002; Depledge, 2003; Wiegers, 2003; Wallace and Keil, 2004). ERP implementations projects are especially vulnerable schedule overruns because: (1) ERP software is complex and, in most cases, the managers of organizations implementing these systems often have little or no prior experience with them; (2) the scale of the implementation is often larger in scope than any previous IT implementation in the organization; (3) the organization’s own IT specialists

0377-2217/$ - see front matter Crown Copyright Ó 2009 Published by Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2009.01.010

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

often have little knowledge of the ERP system and project must be augmented by specialized external consultants; (4) the schedule for completion of the ERP implementation is often tight due to competitive pressures on the organization. Seen from this perspective, the management project team learning capabilities is an important dimension of effective project management. Unfortunately, although it is accepted that the team’s learning rate and capabilities influence its performance and productivity, little research has been conducted on this issue (Barry et al., 2002). In this paper we investigate the potential of learning curves as a method for planning and management of ERP implementation projects. This paper extends the work of Ngwenyama et al. (2007) who proposed a learning curve model for managing organizational software upgrades cycles. We focus on three key questions: (1) How can managers achieve effective project team learning that is so critical to the success of ERP implementation? (2) How can managers develop viable project schedules that take into account the learning curve of the project team? and (3) How can managers identify and dynamically adjust project plans and resources in response to contingencies during ERP implementation. Our discussion of these issues unfolds as: in Section 2 we discuss the linkage between learning and project performance, and the challenges of managing ERP projects. In Section 3 we discuss the basic and assumptions of two learning curve models, the S-curve and the simplified exponential curve. In Section 4 we present a comparative analysis of the implications of using both curves using empirical data from four ERP implementation cases from three companies. Finally in Section 5 we conclude with implications for new technology implementation theory and practice. 2. The learning and performance relationship The relationship between learning and project performance was first formally defined by Wright (1936). Since then, several researchers have investigated and modeled this relationship. Towill (1985) studied on-the-job (or industrial) learning and developed a set of parameters for measuring learning and productivity gains in industrial projects. Terwiesch and Bohn (2001) investigated the cost and benefits of learning during ramp-up production of new products. Their primary interests were the cost of speedingup learning and experimentation, and the likelihood and cost of product defects as teams progressed to full production. Although on the surface it might appear that new product manufacturing and ERP implementation projects are different, they share important features. In both cases the project teams must learn new production methods and work with designs of end products of which they may have little knowledge. In both cases the teams need to learn from experience in spite of whatever initial training they may receive. Further, the cost of product defects could be quite significant and threatening to the economic viability of the company. From this perspective, ERP implementation projects can be viewed as limited-time endeavors that can be planned and managed similarly to some new product manufacturing projects. One key difference is that the ERP project team is cross-disciplinary, comprising technical and business experts from various organizational functions, as well as external consultants hired to support the endeavor (Davenport, 1998; Boyer, 2001; Baccarini et al., 2004). Team members of different backgrounds, business disciplines and interests must learn and work together if the implementation project is to succeed (Fedorowicz et al., 1992; Edmondson et al., 2003). Careful selection of the team members, a good project plan, high quality training in the ERP software and implementation practices along with strong top management support are also essential to success (Davenport, 1998; Boyer, 2001; Baccarini et al., 2004).

519

An important factor in successful ERP implementation projects is early team learning. New implementation teams require significant initial classroom training on the ERP software followed by experiential learning on configuring the software in a sandbox setting (Robey et al., 2002). And since team members play different roles on the project, they need to develop cross-functional skills (Boyer, 2001; Rocheleau, 2006). Further, many issues challenge team learning and performance during implementation. Software features may not be well understood; some members leave the team and new ones fill their places (Chambers, 2004; Pendharkar and Subramanian, 2006). A wide range of hygiene factors, such as boredom, fatigue, distractions, can also impede individual and team learning and performance (Eason, 1988; Adler and Clark, 1991). Thus a key problem for project managers is to understand the rate of project team learning in order to effectively plan and develop the project schedule. Predicting the impact of team learning in complex projects is essential to effective project management but it can be quite challenging (Eden et al., 1998; Ellis and Shpielberg, 2003). 3. Modeling learning and performance Since Wright, various learning curve models have emerged, but the two main curves used for cost estimation and productivity assessment are the S-curve and the exponential progress curve1 (Yelle, 1979; Towill, 1985; Argote and Epple, 1990; Pananiswami and Bishop, 1991; Teplitz, 1991; Badiru, 1992; Mosheiov and Sidney, 2003). The S-curve conceptualizes performance improvement as function of practice, with the most dramatic improvements taking place at the beginning of the learning process. According to Arrow (1962), knowledge increases with experience and the process of learning is a product of experience. In an S-curve, either time or a cumulative output can be chosen as an independent variable and influences of experience carry-over, cessation of learning and a start up effect are considered (Howell, 1990). Interestingly, an exponential curve is the one most commonly used to track performance in technology related projects (Butler, 1988; Teplitz, 1991; Teplitz and Amor, 1993; Jovanovic et al., 1995; Jackson, 1998; Chambers, 2004; Dardan et al., 2006; Pendharkar and Subramanian, 2006). The logistic curve is similar to the S-curve but ignores the start up effect. The rationale for using an exponential model is that it adequately describes the performance improvement during the experiential learning phase. The logistic model is robust and has a wide range of applicability. Since in the logistic model output is related to asymptotic performance, the curve coefficient, k, relates changes in performance over time to the performance threshold (when learning is complete) (Yelle, 1979; Edgington and Chen, 2002). In the next section we examine the relationship between the logistic model and the S-curve model, and discuss the range of applications of the logistic model and illustrate how it can be used as an analytical tool to assist project managers in analyzing learning and productivity issues and their implications for IS project scheduling and duration. 3.1. Basic learning curve concepts As stated earlier, learning curve models used on technology and IS projects are often called progress curves or progress functions (Malerba, 1992). Progress curves model practice and performance, where practice is represented by units of time (or the number of times a predefined output is delivered), and performance is measured as a rate, in which a predefined output is produced (Fedorowicz et al., 1992). In classical form, progress curves depict a

1 Space limitation prevents a detailed discussion of learning curve models; the interested reader can refer to Badiru (1992) and Yelle (1979).

520

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

rapid increase of early performance then a decline and diminishment to 0. James (1984) reports that variations in performance relative to practice and experience track the S-curve more closely when hygiene factors that affect performance and learning such as boredom, fatigue and distractions are considered. For our purposes of modeling and analyzing learning and performance in ERP implementation projects we define an initial team training exercise and a two phase implementation cycle, comprising configuration and testing/conversion phases (Markus et al., 2000). Assuming that ERP implementation follows a progress curve, we can model the relationship between the progress curve coefficient k and performance. In this case we define performance as the rate of completion of a task; for example: (1) the number of sessions configured by a single team member in a unit of time, or (2) the number of transactions completed by a single team member in a unit of time. We now define the basic terms and concepts we use in modeling the logistic and S curves:

tation projects will require time extensions (defined as DT) beyond the planned duration. Therefore we will relax Assumption 1 to Ps(t) when project team performance follows the S-curve and Pl(t) when it follows the simplified logistic model. Team performance will always fall below the performance threshold pt, where t is time elapsed on the project. Fig. 1 illustrates the performance outcomes when Assumption 1 is strong and relaxed. When Assumption 1 is valid (strong) the total amount of work required to complete the project is represented RT by the area defined as 0 pt dt. When Assumption 1 is relaxed implementation time is extended by DTl when logistic curve is used, and DTs when S-curve is utilized. However, the total work required remains unchanged as illustrated by the two integrals

Definition of model terms Pl(t), Ps(t) progress functions as functions of time, t performance threshold (constant performance function) pt represents the optimal performance of the fully trained and integrated team operating at their peak performance levels planned project duration is the expected time for compleT0 tion of the implementation. It is estimated in weeks, months or fractions of the above depending on which units is project duration measured in. It is calculated based on the assumption that performance remains predetermined and constant during the progression of the project project durations, calculated in the same units as used for Tl, Ts the planned project duration. They are estimated based on the assumption that performance on the project is represented by either Pl(t) or Ps(t) DTl, DTs extensions of the planned project duration. They are calculated as the difference between T and T0 k progress curve coefficient knowledge absorption capacity coefficient T0k m Team’s initial performance level measured at the beginning of the project and scaled by pt D0(t), D(t) the difference between the logistics and S-curves measured as the difference in areas underneath those curves at time t scaled by pt

3.2.1. Factors affecting team performance Several factors affect team performance including team members’ knowledge, practice and experience gained on implementation projects. These performance changes are measured (the coefficient k), cumulatively in terms of reduction of time required to complete various implementation tasks. Other factors influencing performance improvements during ERP implementations are: (1) team cohesion resulting from storming, forming and norming into the performing stage (Gray and Larson, 2000), (2) internal team members learning the new ERP software, and (3) external team members (consultants) learning the business processes of the organization (Davenport, 1998). Thus for this analysis we make the following assumption:

3.2. Basic assumptions We now outline three basic assumptions upon which our analysis is based; there are planned project duration, team performance levels and forgetting. Planned or expected project duration is a critical project parameter that is usually underestimated by project managers due to uncertainty about (a) the team’s performance capability and (b) the amount of work required to configure, test and install the ERP system. Project duration is often optimistically estimated with an input from the software vendor, who is often biased by their interest in selling the software. Assumption 1 below establishes a key link between planned project duration and the progress curve models.

(RT

P l ðtÞdt ¼ AreaðLogistics curveÞ;

0

P s ðtÞdt ¼ AreaðS curveÞ:

l

Assumption 2. We estimate the progress curve coefficient k (changes in team performance levels) from the average team member’s performance and adjust for the team environment. A key factor contributing to forgetting is the time lag between training and task execution. This is a situation that does not normally exist in ERP implementations. Team members are trained just before the start of the project and the training includes an experiential component in a sandbox. Further, ERP implementation projects are carried out under conditions conducive to learning and performance such as: (1) executive level support and involvement; (2) expert consulting support and guidance; (3) project schedules no longer than 18 months; and (4) performance incentives (Davenport, 1998; Boyer, 2001; Baccarini et al., 2004; Dong, 2004). Conditions which (Fedorowicz et al., 1992) argue, promote learning and overcome boredom, fatigue and forgetting. Thus in our analysis we ignore performance decay due to forgetting:

P l, P s

Training

Performance Threshold

Configuration/ Testing/ Go live Ts

Assumption 1. If the performance of each team member is at pt at all times during the IS project, then the total time required for the implementation will be equal to planned duration T0. This assumption implies a fully trained and integrated project team capable of peak performance. Such a team is seldom available except from the software vendor or specialized consultants who have gained competence from repeated implementation of the software in other firms. In practice, however, most ERP implemen-

0

R Ts

T0

Tl

ΔT ΔT t

0

Progress Function P l: Logistic curve Progress Function P s: S-curve Fig. 1. Illustration of the model parameters.

521

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

Assumption 3. The impact of forgetting is not considered in either model.

the solution to which

(

Pl0 ðtÞ ¼ pt ð1  ekt Þ

if Pl ð0Þ ¼ 0;

ð3:4Þ

ekt Þ if Pl ð0Þ ¼ p0 P l ðtÞ ¼ pt ð1  m1 m

3.3. Formal description of the progress curve models

is again transformed into dimensionless form as We define a progress function P for the average project team member, with k as the progress curve coefficient, and pt the performance threshold. The performance growth rate is thus the derivas expressed by the S-curve equation (3.1) tive dP dt

  dPs Ps : ¼ kP s 1  dt pt

ð3:1Þ

The solution to the Eq. (3.1) is

Ps ðtÞ ¼

pt 1 þ Aekt

ð3:2Þ

;

Pl0 ðtÞ ¼ 1  ekt if Pl ð0Þ ¼ 0; pt Pl ðtÞ m  1 kt ¼1 if Pl ð0Þ ¼ p0 : e pt m

ð3:4:10 Þ ð3:4:20 Þ

The difference between the logistic and S-curves, is defined by the difference in areas underneath those curves at time t scaled by pt (cf. Fig. 1) will be represented as either D0(t) when the initial condition Pl(0) = 0 is applied to the logistic curve (3.4.1)0 , or D(t) when the initial condition Pl(0) = p0 is used.

0 where A ¼ ptpp ¼ pp0t  1 ¼ m  1 and m ¼ pp0t represents the project 0 team’s initial performance (i.e. p0) measured relative to the performance threshold pt. Eq. (3.2) can be further transformed into dimensionless progress function

Theorem 1. for proof see Appendix A. When the team reaches an initial level of performance p0 ¼ pet before the commencement of the project (m equals to constant e) it makes no difference which curve is used to track performance since both the S-curve and logistic curve become identical in relatively short time.

Ps ðtÞ 1 ¼ : pt 1 þ Aekt

Theorem 1 explain why both curves have been used successfully to track performance on ERP projects when team performance reaches the level of p0 equal or close to 30% of the performance threshold prior to the commencement of the project. We will now define Theorem 2 as an extension of Theorem 1 to situations where the project team did not reach the critical level of performance before the commencement of the project. The difference between the logistics and S-curves measured as the difference in areas underneath those curves at time t scaled by pt (Fig. 2) will be represented as D0(t) (Eq. (A.2)0 ).

ð3:20 Þ

Performance growth rate can also be expressed as the logistics equation

dPl ¼ kðpt  Pl Þ dt

ð3:3Þ

Pl, P s

Performance Threshold p t

Progress Functions P s: S-curve

Progress Function P l, Pl(0)=0 Logistic curve

Theorem 2. for proof see Appendix B. When m > e (i.e. p0 does not reach 30% of performance threshold), jD0(t)j depends only on the coefficient k and parameter m ¼ ppt and will reach its maximum as 0 when implementation time approaches infinity t ? 1 or as implementation time equal to critical time t* (Eq. (3.5)).

t ¼ 

0

t

Fig. 2. The difference between the logistic (initial condition of 0) and S-curves measured as the difference in areas underneath those curves at time t for cases when m = e or m > e.

  1 p  2p0 ln t : k pt  p0

ð3:5Þ

The reader should take note that when the team’s initial performance is low (i.e. p0 does not exceed 30% of performance threshold), then the difference between the logistic and S-curves D0(t) is calculated for the initial condition of Pl(0) = 0. Fig. 3 provides a graphic illustration of Theorem 2. On the left plot D0(t) is a function of very short project duration (depicted in weeks) and on the right plot a longer project duration (depicted in months). It

m=3 m=4 m=10 m=14

m=3 m=4 m=10 m=14

k=0.8

k=0.8 2.5

0.5

Δ 0 (t)

Δ0 (t)

2

0

1.5 1 0.5 0

-0.2

1

3

5

7

9

11

13

15

17

19

Implementation time t [months] Fig. 3. The difference between the logistic and S-curves as a function of time: measured for short (plot on the left) and long (plot on the right) implementations.

522

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

can be seen from the plots that jD0(t)j reaches its maximum in less than 6 weeks and becomes smaller as m becoming larger. Fig. 4 illustrates the maximum value of jD0(t)j as a function of m following Theorem 2 when m is dimensionless and D0(t) is expressed in units of measure used to evaluate project duration in relation to productivity threshold pt. We can assume m P 1 since the levels of initial performance can never exceed the performance threshold. Two important observations related to Theorem 2 should be noted: 1. D0(t) will reach minimum if p0 remains close to 30% of the performance threshold (m remains close to constant e), which corresponds again to situation discussed in relation to Theorem 1. 2. When m > 3 (initial performance is in the lower range) the difference between the area under the curves will increase with m. The decision on which curve should be used to track performance changes must be made based on other factors such as team dynamics, amount of consultants, in-house training incorporated into the implementation schedule, familiarity with other team members, consultants, etc. D0(t) will be relatively larger than if m was close to 3 for both short implementations (project with an aggressive schedules and implementation times at 3–4 months) and longer implementations (Fig. 3). We will discuss the selection of the functional form of the progress curve for the case, where initial performance is low (m = 10) in more details in Sections 3 and 4. We will now define Theorem 3, to include situations where the project team exceeded the critical level of performance before the commencement of the project. The difference between the logistics and S-curves measured as the difference in areas underneath those curves at time t scaled by pt (Fig. 5) will now be represented as D(t) (Eq. (A.2)0 ). Theorem 3. When m < e (i.e. p0 is larger than 30% of performance threshold), it makes no difference which curve is used to track performance since both the S-curve and logistic curve become identical in relatively short time. The reader should note that for those high levels of initial performance we use the initial condition of Pl(0) = p0 to calculate the difference between the logistics and S-curves, and the second equation (3.4) represents the logistic curve. D(t) is calculated from

Eq. (A.2)0 . Theorem 3 is depicted in Fig. 6, in which D(t) is plotted as a function of time for various sets parameters: k and m. Following from Theorem 3 and Fig. 6, the S-curve becomes closer to the logistic curve if the project team shows higher levels of initial performance (depicted as plots for m = 1.1 and 1.5). If team’s initial performance level is very high (m < 2) then the situation on the ERP project will be similar to the other industrial projects (in a sense that very little if at all initial training will be provided) and the progress curve will represent performance increases due to team cohesion and integrated work patterns. 3.4. Context for applying the models We will now discuss the context for application of the progress functions. In each project situation the team has the capacity to learn and accomplish a finite amount of work within fixed time period. We call this the knowledge absorption capacity coefficient T0k. The relationship between planned duration T0, the progress curve coefficient k and the knowledge absorption coefficient is defined in (4.1)

T 0k ¼ kT 0 :

ð4:1Þ

Combining the progress coefficient and the project duration into a single variable T0k enables critical analysis of the project situation as Eq. (4.1) also defines the tradeoff between project duration and

Pl, Ps

Performance Threshold p t

Progress Function P l, Pl(0)=p0 Logistic curve Progress Function P s: S-curve

0

t

Fig. 5. The difference between the logistic (initial condition of p0) and S-curves measured as the difference in areas underneath those curves at time t for cases when m < e.

Fig. 4. The maximum value of the difference between the logistic and S-curves as a function of m.

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

523

Fig. 6. The difference between the logistic and S-curves as a function of time: plotted for various sets of parameters k and m.

Table 1 Four categories of IS projects. Project categories

Level of initial performance p0

Amount of initial training prior to implementationa

Amount of consulting support during the implementationb

I II III IV

Moderate (m close to 3) Moderate (m close to 3) Very low (m > 4) Very high (m < 2)

High Medium Low None

Very high Low Medium Very low

a

The following criteria were used for classification: ‘‘low” – 2 weeks or less, ‘‘medium” – more than 2 weeks up to 3 weeks, ‘‘high” – more than 3 weeks. The classification was based on the ratio of consultants to team members: ‘‘low” – 0.25 or less, ‘‘medium” – more than 0.25 but less than 0.5, ‘‘high” – 0.5 and higher but less than 1, ‘‘very high” – 1 and higher. b

Fig. 7. Comparison of logistic and S-curves for Category III projects.

learning. We can analyze the impact of learning on duration (slower learning extends the project duration and faster learning shortens project duration). Further, since k is expressed in [1/‘‘] unit of measure used for project duration”, T0k is dimensionless and can represent planned project duration, if k remains fixed. For the purpose of illustrating the application of the models we define four categories of IS implementation projects (cf. Table 1). Implementation projects that fit categories I and II can be modeled using either the S-curve or logistic curve and to these Theorem 1 is applicable. When the project team is given intensive initial inhouse training (team bonding/cohesion has occurred) and medium to high support from external consultants is available, the project startup effect can be minimized and the logistic curve applicable. However, if the team members received training at different locations and team bonding/cohesion was not achieved the startup effect could be considerable and the S-curve will better represent performance changes. Projects in category III are those in which the team has very little knowledge or experience in the software they are implementing

and only medium external consulting support is available. The initial performance of the category III project team is also much less that 30% of the performance threshold. In such situations Theorem 2 is applicable, and we would suggest the logistic equation with the initial condition Pl(0) = 0, rather than an S-curve. Fig. 7 below presents the implications of using the logistic curve in this situation. We compare the difference between the logistic and S-curves for different levels of initial team performance (m = 1.5, 3, 6, 10, and 14). In Fig. 7 D(t) is scaled by T0 and plotted as a function of kT0 to accommodate the combined effect of project duration and the progress curve coefficient.2 In can be seen that when m = 3 the plot representing the difference between the two curves remains relatively small for all values of kT0. For cases where m < 3, a logistic curve with initial condition of Pl(0) = 0 will not be used. If m > 3, the increased values of D(t) scaled by T0 are present only for very low values of kT0 (implementation times less than 3 months) and even then they are below the level of 0.4 of the performance threshold. For implementations projects within Category IV it is assumed that the project team has a high level of competence obtained from prior experience implementing the same software elsewhere. Consequently, the team members need no training before the commencement of the project and require minimal consulting support during the implementation. In this situation Theorem 3 is applicable and either the logistic curve with initial condition Pl(0) = p0 or the S-curve can be used. Fig. 8 below presents the implications of using the logistic curve in this situation. We compare the difference between the logistic and S-curves for different levels of initial team performance (m = 1.1, 1.5, 2, 3 and 10). In Fig. 8 D(t) is scaled by T0 and plotted as a function of kT0 to accommodate the combined effect of project duration and the progress curve coefficient.

2 Note that DT0 ðtÞ in Figs. 5 and 6 is dimensionless, expressed in the units of measure 0 scaled by performance threshold pt.

524

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

Fig. 8. Comparison of logistic and S-curves for Category IV projects.

4. The case studies The data used to illustrate our learning curve analysis was collected from four ERP implementation projects in three companies. One of the authors was a participant observer on three of the projects and conducted post-implementation data collection on the fourth. All three companies are located in Canada. They vary in size, age and industry. Due to confidential agreements we cannot disclose the names of the companies in our study; we refer to them as Company A, B or C. Company A is manufacturer and processor of vegetable oils and meals. It was established in 1992, has 750 employees and achieves approximately CD$100 million in total sales, 50% of which is from foreign exports. Company B is a member based organization providing services in standards and certifications. It was established over 80 years ago and has total annual sales of CD$150 million. Company C is a global manufacturer of spice ingredients, established in 1919. It is privately owned and has 2500 employees. It supplies spices to various players (supermarket chains, restaurants, and the food industry). The common thread in each of these companies is that they all implemented some type of ERP system and fit the requirements for our analysis. Table 2 provides a summary of basic data about the implementation projects. Complete data including project schedules, budgets and team characteristics and learning activities were collected for each of Projects 1, 2, 3 and 4. 4.1. Details of the projects Project 1 concerns the implementation of several modules of the SAP ERP software over a nine month period. The project team was composed of 20 employees drawn from the company and 20 exter-

nal consultants. Each team member received 15 days of in-class training at various SAP training centers on the various modules that were scheduled to be implemented. They reached a level of performance of 30% before the commencement of the implementation (m = 3). Following our previous discussion we place this implementation project in Category I (Table 1). The following observations were made during the implementation: (1) The ratio of internal staff to consultants during the project was 1:1. The project team had not bonded, so a start up effect was observed during the early stages of the project. (2) Even though extensive consulting support was provided, the team operated just below its performance threshold the first four months of the project. Using the S-curve we were able to estimate k close to 0.8 (see Table 3). (3) To meet the planned implementation schedule excessive overtime was required close to the go live week, and additional post implementation phase work was required. We estimated that approximately five additional weeks (DT observed = 1.25 months), would have been added to the initial project schedule had the manager been able accurately predict the performance of the team. Project 2 concerns the implementation of BAAN ERP software with a Supply Chain Management extension. The project lasted one year (T0 = 12 months). Members of Project Team received inhouse training, during which some level team cohesion was accomplished. A few members attended advanced level classes at various locations. The team reached a 30% level of performance before the commencement of the implementation (m = 3) and fully bonded, so the start up effect was not observed. We classify this implementation project as Category II (cf. Table 1). The following observations were made during the implementation: (1) The consultant to staff ratio during the project was 1:4 ratio. (2) After a little over four months, the team was operating close to their performance threshold. Using the logistic curve we were able to estimate k as close to 0.8 (cf. Table 3). (3) However, the implementation schedule was extended by five additional weeks (DT observed = 1.25 months). Project 3 concerns the first implementation of the BAAN ERP software in one location of Company A. Subsequently, the system was rolled out to three other factories in Ontario and then to factories in Alberta and Quebec. Project 3 took one year (T0 = 12 months). A few members received limited training from the software provider at various locations. We estimated that the team reached a 15% level of performance before the commencement of the project (m = 10). The project team received extensive in-house training during the early stages of the project and some support from external consultants. A start up effect was not observed, and we believe that this was because the team was selected from members of middle management who already had a very good

Table 2 Comparison of the progress curves for the four projects. Project and team characteristics

Project 1 (Company B)

Project 2 (Company C)

Project 3 (Company A)

Project 4 (Company A)

Project duration [months] Participant observation of project Modules implemented

9 7 months

12 9 months

12 Post implementation data collection

9 9 months

MM, FI, CO, Project 20 Over 2 years

Distribution, manufacturing, finance, constraint based planning 12 1–2 years

Distribution, process, manufacturing excluding MPS and MRP, finance 8 1–2 years

Cost accounting, finance, distribution, manufacturing, 8 3–4 years

1 month

1 month

None

2 years

20 1 year 15–20 days

3 1–2 years 15 days

2–4 3 years 10 days

No full time consultants N/A None

Yes

Yes

Yes

Yes

Team size Team avg. IT experience (prior to the project) Team avg. ERP experience (prior to the project) No. Consultants Consultants avg. experience No. Team training days (per member) Was the implementation completed

525

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

curve and was operating close to their performance threshold during the third month of the project. Using the S-curve we were able to estimate k as close to 0.9 (cf. Table 3). Extension to the implementation schedule was not required and post implementation support was estimated as two additional weeks for minor error correction and data clean up (DT observed = 0.5 months).

Table 3 Parameters and progress curve coefficients used in our calculations. Project

1 2 3 4

Parameters m

N0 of months after which performance reached

Team performance levels

3 3 10 1.5

4 4.5 5 2.5

0.97 0.97 0.97 0.97

Equation for calculations of progress curve coefficient k

Progress curve coefficient k

(3.4.2)0 (3.4.1)0 (3.4.1)0 (3.4.2)0

0.8 0.8 0.7 0.9

4.2. Discussion of the analysis The additional time or work required beyond the initial project duration was registered at the end of each project and then converted to months of extension to the original project duration depicted as ‘‘DT observed” in Table 4. The required extensions were also calculated from equations presented in Appendix C, which were derived from the three progress curves. The results are depicted in section ‘‘DT calculated” in columns Pl0, Pl and Ps for two logistic curves and the S-curve correspondingly. In the construction of Table 4 we used one month as a duration unit of measure thus, ‘‘DT observed”, ‘‘DT calculated” and T0 are depicted in months. We present the values of extensions to original project duration (derived from the progress curves) scaled by the values observed on the projects in section ‘‘DT calculated/DT observed”. We also scaled the total time required to complete the four projects when progress is tracked by the three progress curves by the total time observed. We present the results in section ‘‘T calculated/T observed”. The middle line for each project represents the actual parameters as registered or calculated from Eqs. (C.2–C.5). The top and bottom lines are presented to show how sensitive the calculations are in regards misjudgment of k. The columns in grey present the results of using the ‘‘incorrect” progress curve following the argument from our previous discussion. We summarized the analysis of the results depicted in Table 4 into the following observations:

working relationship and were also able to further bond during the in-house training. We classified this project as Category III (cf. Table 1). During the project consultant to staff ratio remained at 1:4 ratio. Extensive consulting support was given only during the early stages of the project to reinforce training. By the sixth month of the project, the team was operating very close to their performance threshold. Using the logistic curve we were able to estimate k as close to 0.7 (cf. Table 3). While the implementation schedule was not extended, excessive overtime was required close to the go-live week and additional post implementation work was required. We estimate that had the manger accurately predicted team performance approximately six more weeks (DT observed = 1.5 months), would have been added to the project schedule. Project 4 concerns the implementation of a heavily customized version of the BAAN ERP software in Company A. This project was completed in nine months (T0 = 9 months), with the team reaching a very high level of performance (m = 1.5) before the commencement of the project. This is due to the fact that several of the team members had participated in the previous implementation. However, although the majority of the team already worked together, the start up effect was observed. We believe that this was due to the fact that no in-house training was provided and team did not have a chance to bond before starting the project. We classify this project as Category IV (cf. Table 1). Consulting support was very minimal and only given on as needed basis to answer team members’ questions. The team demonstrated a very steep progress

1. For the first two projects the results of ‘‘DT calculated/DT observed” based on S-curve are similar to the results based on the logistic curve with the initial conditions of Pl(0) = 0 and very different from the results based on the logistic curve with the initial conditions of Pl(0) = p0. The results remain within ±13%

1

Project Parameters T0

k

M

ΔT observed

PROJECT

Table 4 Comparison of the progress curves for the four projects.

0.7 9

0.8

3

1.25

0.9

2

0.7 12

0.8

3

1.25

0.9

3

0.6 12

0.7

10

1.5

0.8

4

0.8 9

0.9 0.99

1.5

0.5

ΔT calculated

ΔT calculated/ ΔT observed

T calculated/ T observed

ΔT observed/ T0

Pl0

Pl

Ps

Pl0

Pl

Pl0

Pl

[%]

1.42

0.95

1.57

1.14

0.76

1.26

1.02

0.97

1.03

1.25

0.83

1.38

1.00

0.66

1.10

1.00

0.96

1.01

1.1

0.75

1.23

0.88

0.60

0.98

0.99

0.95

1.00

1.45

0.94

1.56

1.16

0.75

1.25

1.02

0.98

1.02

Ps

Ps

1.22

0.84

1.38

0.98

0.67

1.10

1.00

0.97

1.01

1.1

0.74

1.25

0.88

0.59

1.00

0.99

0.96

1.00

1.66

1.5

3.85

1.11

1.00

2.57

1.01

1.00

1.17

1.45

1.28

3.3

0.97

0.85

2.20

1.00

0.98

1.13

1.25

1.15

2.85

0.83

0.77

1.90

0.98

0.97

1.10

1.25

0.42

0.51

2.50

0.84

1.02

1.08

0.99

1.00

1.12

0.38

0.46

2.24

0.76

0.92

1.07

0.99

1.00

1.05

0.35

0.43

2.10

0.70

0.86

1.06

0.98

0.99

13.9

10.4

12.5

5.6

ΔT calculated/T0 [%] Pl0

Pl

Ps

15.8

10.6

17.4

13.9

9.2

15.3

12.2

8.3

13.7

12.1

7.8

13.0

10.2

7.0

11.5

9.2

6.2

10.4

13.8

12.5

32.1

12.1

10.7

27.5

10.4

9.6

23.8

13.9

4.7

5.7

12.4

4.2

5.1

11.7

3.9

4.8

526

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

tolerance levels if k varies between [0.7 and 0.9] and are within [98%, 110%] of the observed extension to the planned project duration. The observations are in good agreement with and confirm the trend depicted in Theorem 1. 2. For the third project the results of ‘‘DT calculated/DT observed” are very similar for both logistic curves and very different from the results based on the S-curve. The results remain within ±14% tolerance levels if k varies between [0.6 and 0.8] and are within [85%, 97%] of the observed extension to the planned project duration. Although the calculated values of extension to initial durations are below the results observed, the results presented in Table 4 confirm the trend depicted in Theorem 2. We would like to point out that the results are very close to what was actually observed if a logistic curve with the initial conditions of Pl(0) = 0 is used for the calculation of DT. 3. For the fourth project the results of ‘‘DT calculated/DT observed” based on S-curve are similar to the results based on the logistic curve with the initial conditions of Pl(0) = p0 and very different from the results based on the logistic curve with the initial conditions of Pl(0) = 0. The results remain within ±16% tolerance levels if k varies between [0.8 and 0.99] and are within [76%, 92%] of the observed extension to the planned project duration. Although the calculated values of extension to initial durations are again below the results observed, the results presented in Table 4 confirm the trend depicted in Theorem 3. We would like to point out that the results are again very close to what was actually observed if a logistic curve with the initial conditions of Pl(0) = p0 is used for calculation of DT, as predicted in our previous discussion. 5. Conclusions Researchers have identified learning and knowledge transfer in ERP implementation projects as a significant problem that requires attention (Bingi et al., 1999; Markus et al., 2000; Robey et al., 2002). However, there has been limited research on modeling and understanding this problem. This research makes four contributions to understanding and managing the learning and knowledge transfer problem in ERP implementation: (1) It presents a comparative analysis of learning curves and proposes how they can be used to help ERP implementation planning and management. (2) Based on empirical data from 4 ERP implementation projects, it provides illustrations of how managers can apply the curves in different project situations. (3) It provides a theoretical basis for empirical studies of learning and ERP (and other IT) implementations in different organizational settings. (4) It provides empirical justification for the development of learning curve theory in IT implementation. In this paper we presented an analysis of the general logistic and S-curves and suggest that these curves can be utilized in different situations of IT implementation. To illustrate how these curves can be utilized to improve planning and management of ERP projects, we presented and analysis using empirical data from four ERP implementation projects in three companies. Based on theoretical explication and our observations, we conclude that a logistic curve is a good approximation of the performance changes on a majority of ERP implementations. The logistic curve with the initial condition of Pl(0) = 0 can be used on ERP implementation projects where the levels of initial performance reaches 30% (i.e. m is equal or greater than 3). The start up effect can be neglected especially if the team goes through the early stages of team integration during the in-house training and is guided by either the instructor or a group of very experienced consultants. The start up effect was, however, observed in a team that achieved the level of initial performance close to 30% but members received individual training and therefore integration was not accomplished during training

phase of the project. Although it was demonstrated that both curves will yield similar results for estimation of total work required on the project an S-curve should be used in such case. We also conclude that in cases where team members already are at a high initial level of performance due to either previous experience with the system or intensive training, both the logistic curve with the initial condition of Pl(0) = p0 and S-curve can be effectively used to track performance. However, when the start up effect is observed (or expected), the S-curve is a better model than logistic curve for tracking team performance. This observation can be explained by the effect of team dynamics, which slows down the performance during the early stages of the project. When team dynamics are not managed by early ERP training and team integration strategies high levels of initial performance will not be achieved and the start up effect will be observed. In this situation the logistic curve with the initial condition of Pl(0) = 0 can be used to track team performance, and is especially effective in situations where team is at a relatively low initial performance level in relation to the performance threshold. As we have shown in our analysis, it is important for ERP implementation managers to understand the impact of the start up effect on project duration. Our analysis offers managers direct insight into both the impact of the start up effect and the impact of intensive in-house training on project planning and management. Our models are easy to implement on ordinary spread programs and as such are immediately useful to managers. Our research also provides a basis for empirical studies in ERP implementation to define learning curve effects in different implementation situations. Such studies can help the development of better management techniques that could lead to improvements in the cost performance of ERP systems. Appendix A In this Appendix we perform a comparative analysis between the progress functions Ps and Pl used in (3.2) and (3.4) respectively. In particular, we compare the areas underneath the two curves for t P 0 and discuss the difference for large times. Defining D(t) as the difference in area underneath the two curves at time t scaled by pt, we have

DðtÞ 

1 pt

Z

t

½Pl ðsÞ  Ps ðsÞds

0

 Z  1 t pt ¼ pt ð1  eks Þ  ds: pt 0 1 þ Aeks

ðA:1Þ

Evaluation of the integral and rearranging gives

8 D0 ðtÞ ¼ 1k ½lnðA þ 1Þ  1 > > > > < þ 1 ekt  1 lnðAekt þ 1Þ k

k

> DðtÞ ¼ 1k ½lnðA þ 1Þ  m1  > m > > : kt 1 m1 kt 1 þ k m e  k lnðAe þ 1Þ

if

Pl ð0Þ ¼ 0;

ðA:2Þ

Pl ð0Þ ¼ p0 :

Since according to (3.2) A = m  1, (A.2) can be rewritten into (A.20 )

(

D0 ðtÞ ¼ 1k ½ln m  1 þ 1k ekt  1k ln½ðm  1Þekt þ 1; DðtÞ ¼ 1k ½ln m  m1  þ 1k m

m1 kt e m

 1k ln½ðm  1Þekt þ 1:

ðA:20 Þ

From this we see that as t ? 1, D has a well-defined limit, and the value of this asymptote is

8 h i > < 1k ½ln m  1 ¼ 1k ln ppt  1 0 h i L  lim DðtÞ ¼ t!1 > 1 ln m  m1 ¼ 1 ln pt  pt p0 : k m k p0 pt

if

Pl ð0Þ ¼ 0; Pl ð0Þ ¼ p0 : ðA:3Þ

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

527

The sign of this limit will depend on the ratio of the productivity threshold to the initial productivity. In cases when we compare the logistic curve under the initial condition of Pl(0) = 0 (condition most commonly used to represent the situation when initial level of performance is low, or p0 6 0.33*pt) to the S-curve, we see that if the productivity threshold is exactly ep0, then both curves become identical in a relatively short time and it makes no difference which curve is used for analysis, as depicted in (A.4).

We solved equations (C.2–C4) numerically for the given choice of parameter values depicted in Table 2. The required extension to the original project duration DT can be then calculated from Eq. (C.5), where T is the numerical solution of either (C.2), (C.3), or (C.4) depending on whether one of the logistic curves or S-curve represents the progress function and T0 is the planned project duration calculated from the constant performance function pt.

8 > < L > 0 if pt > ep0 ; L ¼ 0 if pt ¼ ep0 ; > : L < 0 if pt < ep0 :

DT ¼ T  T 0 : and

ðA:4Þ

Appendix B Here we compute the maximum value of jD(t)j where D(t) is the difference in the area underneath the two curves at time t scaled by pt, whose form is given in (A.2). We will focus again on the situation when the initial level of performance is low, which is represented by the logistic curve under the initial condition of Pl(0) = 0 (first equation in (A.2)). Since D(t) is a continuous function, D(0) = 0 and L  limt!1 DðtÞ is positive, negative or zero depending on the value of pp0t (recall (A.4)) but finite, it follows that the maximum value of jD(t)j will occur either at a critical point for D(t) or as t ? 1. Differentiating D(t) gives

D0 ðtÞ ¼

ekt kt

Ae

þ1

ðA  Aekt  1Þ

ðB:1Þ

from which it follows that D0 (t) < 0 for all time if pt 6 2p0, and has a local minimum (found by setting the derivative equal to zero and solving for t) at

t ¼ 

  1 p  2p0 ln t ; k pt  p0

ðB:2Þ

if and only if pt > 2p0. Note that we used the definition for A given (3.2) to rewrite this result. It follows that the maximum value of jD(t)j is given by

M ¼ max jDðtÞj ¼

jLj if p0 P p2t ; maxfjDðt  Þj; jLjg if p0 < p2t :

ðB:3Þ

Here t* is given in (B.2), D(t) is given in (A.2) and L is given in (A.4). Appendix C In order for the total work required for system implementation to be the same if the progress curve is represented by either of the progress functions: Ps (Eq. (3.2), Pl0 (1st equation from (3.4)), Pl (2nd equation from (3.4)) or constant performance function pt, we require

8RT p ð1  ekt Þdt > P l0 ð0Þ ¼ 0; > < R0 t  T0 T m1 kt p 1  e dt where pt dt ¼ Pl ð0Þ ¼ p0 ; t 0 m > 0 > :RT pt P s ð0Þ ¼ p0 ; dt 0 1þðm1Þekt

Z

ðC:1Þ

which can be integrated and divided by pt to give

kT þ ekt ¼ kT 0 þ 1

if logistic curve and Pl0 ð0Þ ¼ 0;

m  1 kt m1 e ¼ kT 0 þ kT þ m m

ðC:2Þ

if logistic curve and Pl ð0Þ ¼ p0 ; ðC:3Þ

kT þ lnððm  1Þekt þ 1Þ ¼ kT 0 þ ln m if S-curve:

ðC:4Þ

ðC:5Þ

References Adler, P.S., Clark, K.B., 1991. Behind the learning curve: A sketch of the learning process. Management Science 37 (3), 267–288. Akkermans, H., Bogerd, P., Vos, B., 1999. Virtuous and vicious cycles on the road towards international Supply Chain Management. International Journal of Operations and Production Management 19 (5/6), 565–581. Al-Mudimigh, A., Zairi, M., Al-Mashari, M., 2001. ERP software implementation: An integrative framework. European Journal of Information Systems 10, 216–226. Argote, L., Epple, D., 1990. Learning curves in manufacturing. Science 247, 920–924. Arrow, K.J., 1962. The implications of learning by doing. The Review of Economic Studies 29, 155–173. Baccarini, D., Salm, G., Love, P., 2004. Management of risks in information technology projects. Industrial Management and Data Systems 104 (3/4), 286–295. Badiru, A.B., 1992. Computational survey of univariate and multivariate learning curve models. IEEE Transactions on Engineering Management 39 (2), 176–198. Barry, E., Mukhopadhyay, T., Slaughter, S., 2002. Software Project Duration and Effort: An Empirical Study Information Technology and Management 3. Besson, P., Rowe, F., 2001. ERP project dynamics and enacted dialogue: Perceived understanding, perceived leeway, and the nature of task-related conflicts. The DATA BASE for Advances in Information Systems 32 (47–66). Bingi, P., Sharma, M.K., Godla, J.K., 1999. Critical issues affecting an ERP implementation. Information Systems Management 16 (3), 7–14. Boyer, D., 2001. ERP implementation: Managing the final preparation and go-live stages. Government Finance Review 17(6), 41–44. Retrieved on July 20, 2006. . Butler, J.E., 1988. Theories of technological innovations as useful tools for corporate strategy. Strategic Management Journal 9 (1), 15–29. Callaway, E., 1999. Enterprise Resource Planning: Integrating Applications and Business Processes Across the Enterprise. Computer Technology Research Corporation, Charleston, South Carolina. Chambers, C., 2004. Technological advancement, learning and the adoption of new technology. European Journal of Operational Research 152 (1), 226–247. Chatzoglou, P.D., Macaulay, L.A., 1996. A review of existing models for project planning and estimation and the need for a new approach. International Journal of Project Management 14 (3), 173–183. Dardan, S., Busch, D., Sward, D., 2006. An application of the learning curve and the nonconstant-growth dividend model: IT investment valuations at Intel Corporation. Decision Support System 41 (4), 688–697. Davenport, T.H., 1998. Putting the enterprise into enterprise system. Harvard Business Review 76 (4), 121–131. Depledge, G., 2003. Escalation in information systems development projects: The roles of problem recognition and cognitive bias. Ph.D. Dissertation. University of Western Ontario. Dixon, N.M., 2000. Common Knowledge. Harvard Business School Press, Boston. Dong, L., 2004. Management influence on information systems (IS) implementation effectiveness. Ph.D. Dissertation. University of Western Ontario. Eason, K., 1988. Information Technology and Organizational Change. Taylor & Francis, New York. Eden, C., Williams, T., Ackermann, F., 1998. Dismantling the learning curve: The role of disruptions on the planning of development projects. International Journal of Project Management 16 (3), 131–138. Edgington, T.M., Chen, A.N.K., 2002. An economic benefit model for knowledge creation. In: Proceedings of Twenty-Third International Conference on Information Systems. Edmondson, A.C., Winslow, A.B., Bohmer, R., Pisano Gary, P., 2003. Learning how and learning what: Effects of tacit and codified knowledge on performance improvement following technology adoption. Decision Sciences 34 (2). Ellis, S., Shpielberg, N., 2003. Organizational learning mechanisms and managers at perceived uncertainty. Human Relations 56 (10), 1233–1254. Fedorowicz, J., Oz, E., Berger, P.D., 1992. A learning curve analysis of expert systems use. Decision Sciences 23 (4). Gray, C., Larson, E., 2000. Project Management – The Managerial Process. McGrawHill Higher Education Press, Boston, Massachusetts. Holland, C., Light, B., 1999. Critical success factors model for ERP implementation. IEEE Software 16 (3), 30–36. Hong, K.-K., Kim, Y.-G., 2002. The critical success factors for ERP implementation: An organizational fit perspective. Information and Management 40 (1), 25–40. Howell, S.D., 1990. Parameter instability in learning curve models: Invited comments on papers by Towill and by Sharp and Price. International Journal of Forecasting 6 (4), 541.

528

M. Plaza et al. / European Journal of Operational Research 200 (2010) 518–528

Jackson, D., 1998. Technological Change, The Learning Curve and Profitability. Northampton, Mass (distributed by American International Distribution Corporation, Williston, Vt). James, R., 1984. The use of learning curves. Journal of European Industrial Training 8 (7), 13–16. Jovanovic, B., Nyarko, Y., Griliches, Z., 1995. A Bayesian learning model fitted to a variety of empirical learning curves. Brookings Papers on Economic Activity, 247–305. Karlsen, J.T., Gottschalk, P., 2003. An empirical evaluation of knowledge transfer mechanisms for it projects. The Journal of Computer Information Systems 44 (1), 112–119. Keil, M., Mixon, R., Saarinen, T., Tuunainen, V., 1995. Understanding runaway information technology projects: Results from an international research program based on escalation theory. Journal of Management Information Systems 11 (3), 65–85. Kumar, K., van Hillegersberg, J., 2000. Enterprise resource planning: Introduction. Communications of the ACM 43 (4), 22–26. Mabert, V.A., Soni, A., Venkataraman, M.A., 2001. Enterprise resource planning: Measuring value. Production and Inventory Management 42 (3–4), 46–51. Malerba, F., 1992. Learning by firms and incremental technical change. The Economic Journal 102 (413), 845–859. Markus, L.M., Axline, S., Petrie, D., Tanis, C., 2000a. Learning from adopters’ experiences with ERP: Problems encountered and success achieved. Journal of Information Technology 15, 245–265. Markus, M.L., Tannis, C., Zmud, R.W., 2000b. The Enterprise Systems Experience: From Adoption to Success. Framing the Domains of IT Management: Projecting the Future Through the Past. Pinnaflex Education Resources, Inc., Cincinnati, OH. Mosheiov, G., Sidney, J., 2003. Scheduling with general job-dependent learning curves. European Journal of Operational Research 147 (3), 665–670. Murray, M., Coffin, G., 2001. A case study analysis of factors for success in ERP systems implementations. In: Seventh Americas Conference on Information Systems, Boston, Massachusetts. Nah, F.F., Lau, J.L., Kuang, J., 2001. Critical factors of successful implementation of Enterprise Systems. Business Process Management Journal 7 (3), 285–296. Ngwenyama, O., Guergachi, A., McLaren, T., 2007. Using the learning curve to maximize IT productivity: A decision analysis model for timing software upgrades. International Journal of Production Economics 105 (2), 524– 536. Palaniswamy, R., Frank, T., 2000. Enhancing manufacturing performance with ERP systems. Information systems management 17 (3), 43–55.

Pananiswami, S., Bishop, R., 1991. Behavioral implications of the learning curve for production capacity analysis. International Journal of Production Economics 24 (1–2), 157–163. Parr, A.N., Shanks, G., 2000. A model of ERP project implementation. Journal of Information Technology 15 (4), 289–303. Pendharkar, P., Subramanian, G., in press. An empirical study of ICASE learning curves and probability bounds for software development effort. European Journal of Operational Research. Corrected Proof, Available online 26 May 2006. Pozzebon, M., Pinsonneault, A., 2005. Global-local negotiations for implementing configurable packages: The power of initial organizational decisions. Journal of Strategic Information Systems 14, 121–145. Robbins-Gioia, 2002. Biotechnology Survey Reveals Gaps in Project Management. Robbins Gioia Press. Robey, D., Ross, J.W., Boudreau, M.C., 2002. Learning to implement enterprise systems: An exploratory study of the dialectics of changes. Journal of Strategic Information Systems 19 (1), 17–46. Rocheleau, B., 2006. Information Technology, Training, and Organizational Learning. Public Management Information Systems. Idea Group Publishing. Chapter 7. Sumner, M., 2000. Risk factors in enterprise-wide/ERP projects. Journal of Information Technology 15 (4), 317–327. Teplitz, C.J., 1991. The Learning Curve Desk Book: A Reference Guide to Theory, Calculations, and Applications. Quorum Books, New York. Teplitz, C.J., Amor, J.-P., 1993. Improving CPM’s accuracy using learning curves. Project Management Journal 24 (4), 15. Terwiesch, C., Bohn, R., 2001. Learning and process improvement during production ramp-up. International Journal of Production Economics 70 (1), 1–19. The Standish Group, I., 2004. CHAOS Demographics and Project Resolution. Towill, D.R., 1985. The use of learning curve models for prediction of batch production performance. International Journal of Operations and Production Management 5 (2), 13. Verville, J., Halingten, A., 2002. An investigation of the decision process for selecting ERP software: The case of ESC. Management Decision 40 (3), 206–217. Wallace, L., Keil, M., 2004. Software project risks and their effect on outcomes. Communications of the ACM 47 (4). Wiegers, K., 2003. Software Requirements. Redmond, Washington. Willcocks, L., 2000. Enterprise resource planning: The role of the CIO and it function in ERP. Communications of ACM 43 (4), 32–38. Wright, T., 1936. Factors affecting the cost of airplanes. Journal of Aeronautical Science 3 (122–128). Yelle, L.E., 1979. The learning curve: Historical review and comprehensive survey. Decision Sciences 10 (2).