Learning-before-doing in the development of new process technology

research policy ELSEVIER

Research Policy 25 (1996) 1097-1119

Learning-before-doing in the development of new process technology 1 Gary P. Pisano Technology and Operations Management, Harvard Business School, Morgan Hall T97, Soldiers Field, Boston, MA 02163, USA

Final versionreceived April 1996

Abstract The concept of learning-by-doing has featured prominently in the literature on manufacturing improvement and technological innovation. Much of the focus of this literature is on the 'doing' or 'using' that takes place in an actual commercial production or usage environment. But problem-solving can also occur long before a new product or process design is introduced into the factory through computer simulations, laboratory experiments, prototype testing, pilot production runs, and other experiments. These approaches to problem-solving are referred to in this paper as 'learning-before-doing.' Although there is an extensive literature on prototyping and technical problem-solving in R & D, there have been few attempts to determine the type of conditions under which different approaches to learning are most effective. This paper explores the impact of different learning strategies on development performance with detailed data on 23 process development projects from pharmaceuticals and biotechnology. The empirical analysis focuses specifically on how the timing of technology transfer to the factory influences development costs. The results of the analysis indicate learning-bydoing is essential for efficient development in an environment like biotechnology where underlying theoretical and practical knowledge is relatively thin. In contrast, the need for learning-by-doingis far lower in environments like chemical synthesis where underlying theoretical and practical knowledge is deep enough to enable the design of laboratory experiments that effectively model future production experience. The paper concludes with a discussion of the implication of these findings for the management of process development and learning.

1. Introduction The concept of learning-by-doing has featured prominently in the literature on manufacturing im-

This paper is part of a larger study on process developmentin pharmaceuticals. I would like to thank Steven Wheelwrightfor his collaboration on this project, Kim Clark, Robert Hayes, Macro lansiti, Richard Nelson, Richard Rosenbloom, Eric yon Hippel, and the anonymous referees for helpful comments, and Sharon Rossi for excellent research assistance. All errors and omissions remain entirely my responsibility. The financial support of the Harvard Business School Division of Research is gratefully acknowledged.

provement and technological innovation (e.g. Arrow, 1962; Rosenberg, 1982, yon Hippel and Tyre, 1995). There is a large body of evidence documenting the tendency of manufacturing performance to improve with cumulative production experience. 2 Similarly, learning-by-doing has also been identified in detailed studies of the R & D process which show that technical problem-solving generally involves extensive

2 Sec e.g. Wright (1936), Hirsch (1952), Stobaugh and Townsend(1975), and Lieberman(1984), Adler and Clark (1991).

0048-7333/96/$15.00 Copyright © 1996 Elsevier Science B.V. All rights reserved. PII S0048-7333(96)00896-7

1098

G.P. Pisano / Research Policy 25 (1996) 1097-1119

trial-and-error. 3 Although focused on different contexts, studies of manufacturing improvement and R & D processes provide compelling evidence that learning requires some form of 'doing.' More recent work by Adler and Clark (1991) and von Hippel and Tyre (1995) has illuminated some of the critical organizational mechanisms underpinning learningby-doing. While a picture is beginning to emerge of how and why learning-by-doing occurs, there is far less understanding about the different types of experiences involved in learning, and the implications of these differences for problem-solving strategies. The existing literature tends to focus on the 'doing' or 'using' that takes place in an actual commercial production or usage environment. Thus, for example, studies of the learning curve generally take as their starting point the introduction of a new product or process into the full-scale, commercial manufacturing environment. But 'doing' can take many other forms, including computer simulations, laboratory analyses, prototype testing, and pilot production and other experiments that occur before a new process is introduced into the factory. The term 'learning-before-doing' is used in this paper to differentiate these problem-solving strategies from the more conventional notion of learning-by-doing that takes place after a process is transferred to a commercial production environment. Although there is an extensive literature on prototyping and technical problem-solving in R&D, there have been few attempts to compare these methods of 'learning-before-doing' to those associated with learning-by-doing in the actual production environment. 4 Is learning-by-doing (in the factory) always necessary? Under what conditions and for what type of problems can learning-before-doing be an effective problem-solving strategy? These are the central issues explored in this paper. The issue of appropriate learning strategies has important implications for the management of development, and, in particular, the interface between R&D and manufacturing. One of the most challenging, and often most problematic, aspects of product

3 Marples (1961), Allen (1966), Wheelwright and Clark (1992). 4 von Hippel and Tyre (1995), however, do examine the conditions which make learning-by-doing an attractive strategy.

development lies in the development and transfer of new manufacturing process technology into the plant. The literature on development is rife with examples of how technology transfer problems have led to delayed product introductions, development cost overruns, excessive product costs, and quality problems. From the plant's perspective, the problem is often viewed as one of receiving the process technology too late in the development cycle and not having had a chance to 'come down the learning curve' before the start of commercial production. Earlier transfer of process technology to the plant is an approach commonly advocated by manufacturing. Indeed, in recent years, there has been a tendency in the literature on innovation to emphasize the role of the factory in development. 5 The problem can look very different from R&D's perspective, which often sees the problem as one of not having had enough time to develop a fully understood and manufacturable process before the transfer. Lurking beneath this debate are different perspectives on the potential to learn-before-doing versus the need to learn-bydoing. Resolving such debates, and structuring the development process in the most effective manner, requires a deeper understanding of when each of these models of learning is most appropriate. This paper explores the impact of learning strategies on development performance with detailed data on 23 process development projects from pharmaceuticals. This sample includes both traditional chemically synthesized drugs and drugs produced through recombinant biotechnology methods. These data were collected as part of a larger study on the impact of project strategies, organizational structures, and organizational routines on process development performance in the pharmaceutical industry. This paper focuses on how the timing of technology transfer to the plant influences development costs (as measured by scientific and engineering person hours

5 Leonard-Barton (1992), for example, argues that companies should think of their factories as extensions of their R&D laboratories and utilize them more heavily for experimentation and technical development. Her article uses the example of Chaparral Steel, a highly successful 'mini-mill,' and notes that no R&D department exists separately from production. According to the company's chief executive officer, Gordon Forward, "[E]everybody is research and development. The plant is our laboratory."

G.P. Pisano / Research Policy 25 (1996) 1097-1119

invested in the project). 6 The paper is organized as follows: Section 2 develops a framework for analyzing the choice between learning-by-doing and learning-before-doing problem-solving strategies. Section 3 uses the framework to highlight critical differences in the structure of the knowledge bases characterizing biotechnology and chemical processes, and the implications for appropriate learning strategies in each technology. Section 4 provides an empirical analysis of the impact of learning strategies on development performance using data from the process development projects in the sample. Section 5 concludes the paper with a discussion of the implications of the findings for the management of process development and avenues for further research.

1099

over, the challenge in these contexts goes well beyond adopting design-for-manufacturability or simultaneous product and process engineering, s Although these may be critical first steps in achieving better integration between product and process design, they do not address the issue of how process development itself can be made more efficient. The discussion below provides a framework for understanding the drivers of process development efficiency in different kinds of environments.

2.1. Process development and experimentation

Linkages between product and process technology differ substantially across industries, 7 and these differences influence the nature of both the technical and organizational challenges of process development. For instance, in many types of assembled goods industries (automobiles, personal computers, consumer electronics), product functionality and features are not severely constrained or heavily impacted by the process design. At the other end of the spectrum are industries such as pharmaceuticals, chemicals, semiconductors, and advanced materials where product and process designs are highly interdependent and changes in process technology can have a significant impact on product characteristics. (Conversely, minor changes in product design specifications can require the development of completely new process technologies.) In such industries, process development capabilities play an integral role in overall product development performance. More-

Despite differences in the role of process development across industries, there are certain similarities in the nature of problem-solving involved in the development of new process technologies. The starting point for process development is a description of the product, or a product design. When process development starts, of course, the description may be incomplete or in a state of flux. In chemicals, this might be a written description of the molecule, a formula for the set of reactions required to synthesize the molecule, or other data characterizing the molecule. In the model of process development presented here, process developers can be viewed as starting with a set of targets for process performance. 9 These might be framed in terms of unit cost, capacity required, yields, quality levels, critical tolerances, or other operating characteristics. To simplify the exposition, let C represent the set of process performance characteristics when operated under expected commercial conditions (Xc). The performance of the process (C) is determined by choices over a set of process parameters (p). One goal of the process developer is to find a set of process parameters, p, which either optimizes C(p* ]Xc) or at least achieves satisfactory target

6 Another paper in this study has examined the impact of spending on process research activities on overall project lead times. Although both papers draw from a common conceptual framework, the papers differ substantially in terms of both the dependent variable, as well as the key independent variables of interest. 7 See e.g. Abernathy (1978), Pavitt (1984), Utterback (1994).

8 There is a growing body of literature on these and related approaches to product development. See e.g. Whimey (1988), Clark and Fujimoto (1991), Clausing (1993), Ulrich and Eppinger (1995). 9 Because of uncertainty, in practice these targets are often framed as ranges rather than discrete points (e.g. cost should be between $10 and $15 per unit).

2. Framework for process development

1100

G.P. Pisano / Research Policy 25 (1996) 1097-1119

levels. ,0 In a competitive environment, however, development goals should also include finding p * as efficiently and quickly as possible. Finding p* is a technical problem solving process. i, Although their activities and nature may vary across contexts, technical problem solving processes share common characteristics. One is that problem solving is triggered by gaps between desired and actual performance (Newell and Simon, 1972; Iansiti and Clark, 1994). As noted above, in the context of process development, this gap can be framed in terms of differences between what existing process technology can achieve (e.g. yields and costs) and what is required to achieve success in the market. The second characteristic of problem solving is that it generally takes place through iterative cycles of search and selection (Frischmuth and Allen, 1969; Nelson and Winter, 1977; Nelson, 1982). With each cycle, the gap between actual and desired performance becomes progressively more narrow as technical solutions are identified and tested and a subset of solutions is selected. At the heart of this learning process are experiments (both physical and conceptual) that provide feedback about gaps between current and target performance levels. 12 As a result, the quality of feedback from experiments plays a critical role in determining development performance. Experiments can take many forms and be conducted under a variety of conditions. While the traditional image of an experiment is a laboratorybased analysis of product samples or physical prototypes, advances in technology allow some product

Io A controversial issue within the economics of R&D literature is whether it makes sense to model a firm's R&D choices in an optimization framework. This issue is well beyond the scope of the present paper. For convenience, we will use the terms like 'optimal' and 'optimizing' when describing choices. However, for the present purposes, it matters little whether firms are actually optimizing or 'satisfying.' H Viewing product and process development in terms of problem-solving has a long history in the innovation and organization literature. See, for example, Allen (1966), Frischmuth and Allen (1969), Simon (1978), Clark and Fujimoto (1991), Dosi and Marengo (1993), lansiti and Clark (1994), and von Hippel (1994). 12 Wheelwright and Clark (1992) suggest that the role of prototyping is so critical to the development process that it is helpful to view development projects as a series of 'design-build-test' cycles.

The Locus of ~per/mentation and Representativeness

Representativeness of Final Production Environment

Locus of Experimentation

High Full-Scale CommercialFactory

Pilot Plant Located at ProductionSite

Pilot Plant Located at R&D Site

Laboratory

Computer Aided Simulation Low Fig. 1. The locus of experimentation and representativeness.

and process designs to be tested and analyzed using computer-aided simulation. 13 Physical experiments can also be conducted in different ways. In process development, some experiments are conducted in laboratories, others are performed in pilot plants, and still others are run in full-scale commercial plants. One way to distinguish between experimental forms is the extent to which they are conducted under conditions representative of the final use environment. An example of an experiment conducted under highly representative conditions would be testing a process technology in a full-scale commercial facility during normal production hours, utilizing regular production operators. At the other end of the representativeness spectrum would be small-scale laboratory tests and computer simulations (see Fig. 1). Experimental conditions affect experimental outcomes. Thus, experimental outcomes are not always

13 Boeing, for example, in developing its most recent generation of wide-body aircraft, the 777, did not build a single physical prototype but instead relied completely on computer-aidod design and simulation. This practice of using computers to test product designs has been referred to as ' virtual prototyping' (see Garcia et al., 1994).

G.P. Pisano / Research Policy 25 (1996) 1097-1119

representative of the performance that can be expected in some future operating environment. Consider the process developer in search of C(p* ). When a set of process parameters is tested in the laboratory under conditions X L, researchers do not actually observe C(p); instead, they observe laboratory performance, or what might be called L( pl XL)Note, the term 'laboratory conditions' is used broadly to encompass a broad range of venues, from computer simulations through pilot-scale production plants. The vector of experimental conditions might include such things as the scale of the process, the type of equipment (e.g. glass-lined versus stainless steel reactors), the skill of the operators, the cleanliness of the air, and other environmental variables. 14 Experimenters may have different degrees of control over different experimental conditions. For instance, a researcher can choose the type of equipment they use for an experiment, but may have little control over the air cleanliness. Laboratory experiments can be viewed as representations of the future commercial manufacturing process. For instance, a laboratory experiment on a chemical process is qualitatively very different from a commercial production environment. Yet, each element of the experiment has analogies in the factory. The small glass test tubes represent the stainless steel reaction tanks of the factory; the thin glass mixing rods used to stir the reaction simulate the forces of automated steel rotators; the chemist who sets up the experiment and watches over it plays the role of both the future factory operators and the computer-based process control system. Clearly, the conditions of the laboratory experiment are quite different from the future factory (small glass test tubes vs. large stainless steel tanks, a Ph.D. chemist versus factory operators and a computer system, etc.). These differences can lead to a lack of fidelity in experimental results. For example, the chemical process may behave and perform differently in a smallscale versus a large-scale vessel or in a glass versus a stainless steel vessel. The chemists may stir the

a4 In some contexts, the raw materials available in small quantifies for research purposes are slightly different than those available for use in a large-scale production environment.

1 I01

process very differently from the way it would be stirred by large, computer-controlled rotators. For these reasons, the researcher can expect that C( p'l Xc) will not be equal to L( p'[ X L). Yields may be lower; costs may be higher; different impurities might be present. Even tests in a pilot plant can be quite different than what might be expected in a full-scale plant. This is one of the reasons why companies might discover problems with a process technology only after it is transferred and used in the actual commercial environment. Given that experimental conditions affect experimental outcomes, a major challenge of development is to make predictions about C(plXc) based on observations of L ( p [ X L ) . 15

2.2. Learning-by-doing versus learning-before-doing One approach to improving the fidelity of experiments (i.e. reducing the gap between C(plXc) and L(plXL)) is to make test conditions as close as possible to actual operating conditions. At the extreme, test batches might be run in the commercial manufacturing plant, rather than in the laboratory or small-scale pilot facility. If all the conditions of the future manufacturing environment are replicated in the test (the same factory, the same equipment, the same workers, the same suppliers, etc.), X L should be equal to Xc and the test should provide a relatively good indicator of future performance. In reality, given that factory conditions change from dayto-day or even from minute to minute (e.g. people change, equipment wears, the humidity fluctuates, etc.), it is impossible to replicate exactly the future conditions even with an in-factory test. Nevertheless, the in-factory test should provide the most representative experimental environment. The idea that some things can only be learned by running the process in the factory is consistent with the idea of learningby-doing. The power of learning-by-doing as a problem-solving strategy is that it leads to high fidelity experimental results. Learning-by-doing essentially

15 For simplicity, we will repress the added complication that one can have observational errors even in the laboratory because of instrument calibration or other factors. Thus, the true L(p) might differ from the observed L(p).

I 102

G.P. Pisano / Research Policy 25 (1996) 1097-1119

circumvents the problem of predicting C(plXc) based on observations of L(pIXL), by making experimental conditions (X L) as close to commercial conditions (Xc) as possible. Although learning-by-doing leads to high fidelity results, it also creates other costs. Direct costs per experiment are typically higher in a factory setting than in a laboratory setting because of minimum efficient batch sizes and investments in specialized equipment. 16 Factory-based experiments also use manufacturing capacity that might be deployed to make saleable product. 17 A higher cost per experiment means that for any given number of iterations required to converge to the desired performance level, an experimental strategy emphasizing in-factory tests will involve higher development costs. In addition, because factories are complex and sometimes chaotic environments, it can be difficult to identify and control all the relevant intervening variables. Slight but unavoidable batch-to-batch differences in raw materials, workers, equipment, equipment settings, and other process parameters will reduce the signal-to-noise ratio of factory experiments. Furthermore, in some contexts, extremely costly and delicate high precision test and measurement equipment that could be used in laboratory analysis would not be feasible in a production environment. This further compromises the signal-tonoise ratio of moving experimentation from the laboratory to the factory. Doing may be an effective way to learn, but it can also be costly. An alternative, and potentially less costly approach, is to model future manufacturing performance outside the actual production environment using computer simulations, physical laboratory experiments, pilot plant tests, and other forms of simulation. These methods can be viewed as attempts to predict process performance and to identify potential problems before the process is transferred to the

t6 For example, in the automobile case, investments in production tooling for the entire car could cost nearly $100 million. If designs change, the tooling becomes obsolete and must be replaced. 17 In chemical processes, safety issues can also raise costs of full-scale experiments.

factory. Thus, for instance, if a process fails completely in a test tube, then this may be a good indication that this will not work in the factory either. More subtly, researchers might be able to identify impurities or sources of process variance through small-scale models which can be addressed before the process is transferred to the plant. However, although simulations are generally less costly experiments than on-line tests in a factory, they also potentially suffer from the fidelity problems discussed earlier. Given that most product and process development projects use the full spectrum of approaches (from laboratory simulations through 'real life' tests) it suggests that the issue is one of the appropriate balance between approaches, rather than adopting one over the other. Simulations and other forms of learning-before-doing can be used to identify and solve some types of problems. But problems for which a representative experiment can not be designed can only be identified and solved through learning-by-doing. The balance between approaches should be related to the structure of knowledge affecting one's ability to use laboratory experiments and other forms of simulation to predict future performance and identify and solve potential future problems.

2.3. Structure of knowledge and the appropriate learning strategy Over the years, there have been various attempts to characterize how knowledge varies from context to context. Going back to Polanyi (1962), several writers have distinguished between 'tacit' and 'codified' knowledge. 18 While this distinction can be useful, it really only refers to the form in which knowledge is stored. As most writers on the subject have recognized, beneath this distinction is the extent to which underlying cause-effect relationships (or as Polanyi put it, the 'set of rules') have been clearly

is See, for example, Teece (1976) and Kogut and Zander (1992). Winter (1987) developed an expanded taxonomy of knowledge which included additional dimensions: teachability, observable in use, complexity, and links to other components of a system.

G.P. Pisano / Research Policy 25 (1996) 1097-1119

identified and are well understood, t9 For the purpose of determining the appropriate problem-solving strategies, the critical cause-effect relationships relate to the effect of experimental conditions on experimental outcomes. If problem-solvers understand how specific experimental conditions (e.g. scale, equipment, humidity, etc.) affect particular outcomes (e.g. process yield), they can use this knowledge to make predictions about performance under a future set of operating conditions. For instance, if researchers know from either scientific theory or experience that each order of magnitude increase in scale reduces yields by approximately 10%, a small-scale experiment can be used to make predictions about performance at a larger scale. With deeper knowledge of these cause-effect relationships, the researcher can build more complete and accurate models mapping laboratory observations into expected future performance (under some specified set of conditions). In an extreme case, the researcher might have complete knowledge of all the relevant experimental conditions, their impact on process parameters, and the first- and second-order effects of all process parameters on performance. 20 Take the following simple example of a chemical experiment conducted in a 1-1 glass bottle as a simulation for a commercial process to be run in a 5000 gallon stainless steel tank. Assume for simplicity that the only relevant experimental conditions are the scale of the process (1 1 vs. 5000 gallons) and the material composition of the reaction vessel (glass versus stainless steel), that temperature is the only relevant process parameter, and that the researcher is interested only in one performance outcome: yield. If the researcher knows all the relevant experimental conditions and process parameters and the independent and interactive relationships between temperature, scale, vessel materials, and yields, an experiment conducted under one set of conditions (a 1-1 glass bottle) at one specific temperature point can be

19 The Bohn and Jaikumar (1992) 'stages of knowledge' framework provides a useful taxonomy for differentiating knowledge along this dimension. 20 This would essentially correspond to 'Stage 8' in Bohn and Jaikumar's framework.

1103

used to find the temperature that optimizes yield under a different set of conditions (5000 gallons, stainless steel). 21 For any sufficiently complex process, such complete knowledge is probably impossible to obtain. However, to learn-before-doing, the researcher only needs to have a 'global' model of how outcomes observed under some specified set of conditions map into outcomes under another set of conditions. For instance, through years of testing processes in the laboratory and observing actual performance in the factory, researchers may develop a heuristic to predict future performance from laboratory experiments. Prior scientific knowledge or experience provides predictive models of the form: " I f we observe E ( p ' ) in the laboratory, we can expect C ' ( p ' ) in the plant." In this case, the researcher may not be fully aware of how any specific condition affects the process or its performance, but has an overall understanding of how the complete set of experimental conditions jointly influence outcomes. Heuristics may emerge about what to expect based on observations in the laboratory or the pilot plant (e.g. "multiply the yields observed in the laboratory by 0.75 in order to predict future yields in the factory"). In this case, knowledge takes the form C(plXc)= F[( L( plXL ), where the functional form of F could be as simple as a constant or a more complex relationship. When developers talk about a process having a 'linear scale-up,' they mean that what is learned about the process at small scale (and under other laboratory conditions) can be extrapolated to higher scale (and other commercial manufacturing conditions). In any given technological context, opportunities to learn-before-doing should depend on the degree to which critical cause-effect relationships are described by scientific theories or heuristics based on cumulative practical experience. Where underlying basic theoretical knowledge is strong (and where there is relatively complete knowledge of the conditions in the future production environment), one may know enough about critical variables and their behavior to design highly representative laboratory ex-

2t Dimensional analysis can be used to map experimental observations across scales. But to do dimensional analysis, one needs to know the underlying scale effects.

1104

G.P. Pisano / Research Policy 25 (1996) 1097-1119

periments that provide a reasonably accurate prediction of expected commercial performance. 22 Under these conditions, learning-before-doing should play a greater role in the development process (although the need for some learning-by-doing in the factory should still be present). In contrast, where theoretical or practical knowledge of scale and other experimental effects is limited, it will be virtually impossible to predict how the process tested in the laboratory will work when run under actual operating conditions. Subtle and often unknown differences between the laboratory and the factory environment could have major impacts on process performance. Much of what is learned outside the factory could be irrelevant and unless developers have been lucky, rework of the process will likely be required to get it to operate as planned. For this type of technological regime, evaluation of the process needs to be done under conditions as close to actual operating conditions as possible. Transferring the process to the plant when it is still immature, and developing and refining it there (learning-by-doing) should be the more efficient approach. The proposition that the value of learning-bydoing (in terms of development performance) should be higher in environments with a less mature knowledge base is tested with data on 23 process development projects from the biotechnology and chemical segments of the pharmaceutical industry in Section 4. To deepen the discussion of knowledge environments and to motivate the statistical analysis, the next section discusses the nature of process development and process knowledge in both chemical- and biotechnology-based pharmaceuticals.

3. The anatomy of process development in pharmaceuticals

biochemical compounds used in drugs. 23 In the discovery phase of pharmaceutical R&D projects, research scientists develop crude processes for synthesizing relatively small amounts of the molecule under investigation. These laboratory methods of production, however, are almost always completely unsuitable for manufacturing the compound in commercial volumes at required cost and quality levels. Table 1 provides an illustrative comparison of an 'initial discovery process' and a 'final commercial production process' for a representative chemical entity. The two basic process technologies represented in the sample of 23 projects (chemical synthesis and biotechnoiogy) provide an excellent opportunity to examine the proposition that the nature of the knowledge environment influences the optimal learning strategy in development. Each of these technologies is at a very different stage of historical development. Each also lies at a different end of the spectrum in terms of depth of theoretical and practical knowledge. These differences stem from the respective maturities of the underlying scientific fields, the development of relevant scientific theory, and the availability of process engineering heuristics to aid in scale-up. A brief description of the knowledge base and the nature of the technical challenge facing developers in each environment is provided below. 3.1. Chemical synthesis Chemical synthesis is the traditional method for producing the active ingredient found in most drugs. Chemical process development involves determining the sequence of chemical reactions required to construct the desired molecule (the synthetic route), and specifying the reaction conditions, equipment designs, mechanical manipulations, process flows, and

The study focuses on the development of production processes for therapeutically active chemical or

22 On the role of scientific knowledge in supporting efficient R&D, see Nelson (1982).

23 Thus, we have excluded the process of formulating the final drug form (e.g. capsule, tablet, cream, liquid, etc.) taken by patients. For clarification, the term chemical is used to describe small molecules synthesized through traditional organic chemical methods. Biotechnology is used to describe large protein molecules produced from genetically engineered cells.

G.P. Pisano / Research Policy 25 (1996) 1097-1119

1105

Table 1 Comparison of research process and commercial production process for representative chemical entity

Number of chemical steps Equipment Batch size (output) Operators Purity Cost (kg- ' ) Criteria for process design

Initial discovery process

Final commercial production process

25 Test tubes 1-1 flasks Approximately 1 g Ph.D. chemists

7 2000-4000 gallon stainless steel vessels

1-10% Approximately $20000-$50000 Biological activity of molecule Patent issues

operating procedures that optimize performance. Problem-solving draws largely from two bodies of knowledge: chemistry and chemical engineering. Both disciplines have characteristics that influence how problems are framed and the opportunities to use theory, small-scale experimental data, and computer simulations to search for commercially viable solutions. Chemistry and chemical engineering are relatively mature as both academic disciplines and as areas of industrial practice. Fundamental research in chemistry can be traced back to the 18th century. 24 Many of the critical principles of chemical theory were discovered during the late 18th and 19th centuries. Over the course of nearly 2 centuries of basic research, a rich body of laws, principles, and theories has evolved about the way chemical processes work. Application of this fundamental knowledge to industrial production processes dates to the 18th century (the use of chemical synthesis to manufacture drugs began in the mid-19th century). Years of industrial experience, starting initially with the production of synthetic dyes, not only served to validate and refine chemical theory, but it also led to the development of knowledge about how to scale-up chemical processes. Chemical engineering began to emerge as a formal discipline in the United States in the latter part of the 19th century. Through years of cumulative development and scale-up experience, a large

24 On the history of the chemical industry, see Haber (1958).

100-200 kg Technicians Semi-skilled plant workers 99% Approximately $3500 Cost Quality (purity) Conformance to drug and environmental protection regulations Operability

body of chemical engineering heuristics has evolved which is still widely used today to guide process selection, scale-up, and plant design. Thus, although chemistry and chemical engineering are still active research disciplines today, both fields rest on well-established, highly articulated and formalized knowledge bases. In searching for and selecting alternative chemical processes, and in developing a suitable commercial manufacturing process, developers in the chemical environment have at their disposal a wealth of scientific laws (e.g. thermodynamics), principles, and applied models that describe the structure of relationships between different process variables (e.g. pressure, volume, temperature, etc.). Increasingly, this knowledge is being codified in computer models which permit rapid simulation of processes and the impact of specific process parameters and conditions on yields, cost, throughput, and capacity. The knowledge base is not just technical, but also extends to organizational issues. Through cumulative experience, pharmaceutical companies have developed well-established organizational routines and standard operating procedures for quality assurance and process control, production scheduling, changeovers, maintenance, and other production activities. Many of the nuances of the production environment are well understood. Experience with these routines provides concrete starting points for development and guidance about what types of process techniques are feasible within an actual production environment. The structure of knowledge surround-

1106

G.P. Pisano // Research Policy 25 (1996) 1097-1119

ing chemical processes influences problem-solving strategies in very specific ways. The selection and scale-up of a synthetic route is used as an example below.

3.2. Route search, selection, and scale-up Since the route represents the basic architecture of the process, and has a major impact on its commercial performance, its selection and scale-up is a critical element of the development process. Any given molecule can almost always be synthesized from a variety of starting raw materials and different combinations of chemical reactions. Route search and selection involves finding the one with the best properties for commercial manufacturing. Technically feasible synthetic routes are bounded by chemical and physical laws which are very well understood. Thus, process research chemists almost always start their search for a route by generating a list of theoretically feasible alternatives. Options are derived from theory, from clues provided in published literature, or from examining internal databases of processes used for other molecules (with similar structural characteristics). Chemists will sometime talk about conducting 'paper experiments,' meaning that they work out a potential route on paper. Before any physical laboratory experiments are conducted, extensive modeling and analysis are carried out to eliminate processes with serious shortcomings. Two bodies of knowledge interact at this stage: knowledge of the future manufacturing environment, and knowledge of the chemistry. Chemistry tells the researchers something about the properties of the process. How many chemical reactions are required? What raw materials are needed? What intermediate chemicals are created? What are the by-products? How much energy is needed? Knowledge of the manufacturing environment influences the heuristic for route selection. For instance, since total process yields decline multiplicatively with each step, a common heuristic in route selection is that a route with fewer reactions is preferable to one with more reactions. Routes are also commonly screened out if they require scarce raw materials, generate very hazardous by-products, or are likely to require extreme reaction conditions. Conceptual explorations and analysis are useful

for identifying promising leads, or conversely, fatal flaws in a process. But, theory can only narrow down the options so far. While chemical theory gives researchers a good idea about which molecules bond well together, it is less helpful in predicting a reaction's kinetic properties (i.e. how fast the reactions will occur). Thus, the next step is to evaluate the remaining candidates in small-scale experiments to gather physical data needed for thermodynamic and kinetic studies of the process. Multiple iterations are conducted to examine the impact of variations in temperature, concentration, solvents, and other reaction conditions. Thus, problem-solving still requires trial-and-error, but these trials are conducted in the research laboratories. These data can then be plugged into process simulation models (based on chemical engineering heuristics and principles) to get estimates of throughput, capacity requirements, yields, and costs under some set of future operating conditions. Although scale-up is commonly viewed as a critical issue in chemical projects, there are no quantity or scale variables in chemistry. That is, simply scaling up does not, per se, change the chemistry of a process. Scale can, however, have important second-order effects. For example, at larger scale, heat may not be uniformly distributed throughout a reaction vessel. Reactions may fail to take place in those areas where the temperature has fallen below a specific threshold, thus reducing yields or leading to impurities. The challenge of scale-up is to figure out how to operate the process at larger scale, without changing its basic chemistry. This requires having a very precise characterization of the desired chemistry of the process (otherwise, you are not sure what it is you are trying to keep from changing). If the key process variables have been identified in the laboratory, scale effects can be reasonably well modeled using chemical engineering heuristics and dimensional analysis, and confirmed with pilot-scale production tests. Evidence supporting the potential to learn before doing in chemical synthesis development can be gleaned from data on the number, timing, and causes of route changes during development. Table 2 provides a list of the projects (with disguised names), the number of route changes during the entire process development project, and the phase when the

G.P. Pisano / Research Policy 25 (1996) 1097-1119 Table 2 Number and timing of synthetic route change in synthetic chemical projects Project A B C D E F G H I

J K L M

Number of route changes

Development phase when final change implemented a Pilot Pilot Pilot Pilot Pilot Pilot Research Research Pilot Pilot Production Pilot Pilot

a Definition of phases: (1) Research: phase between the start of processs R&D project and the start of the first batch of pilot production. (2) Pilot: phase between start of pilot production and beginning of transfer of process to commercial production site. (3) Production: phase during the transfer and start-up of process in commercial manufacturing facility.

last change was implemented. All of the projects involved at least one route change from the original route used in the discovery of the molecule. As shown, multiple route changes were not uncommon in the sample, although the majority of projects involved only one change. Most strikingly, only one route change occurred during or after the transfer of the process to the commercial production facility. And, in this particular case, there were strong indications from project participants interviewed that the underlying problems with the route (which prompted the change) were well known long before this phase. According to the senior process development chemist involved with this project from its inception through its start-up in production: The last step was the Achilles heel of the process. It didn't work in the research laboratories. It didn't work in the pilot plant, and it didn't work in commercial manufacturing. But each of us thought we could solve the problem once we got to larger scale - that was how we had done it on other projects. An overwhelming fraction of the changes were either finalized or implemented during the pilot production phase of process development. The main

1107

reason for this is that most organizations prefer to run several production batches before committing to or implementing a specific synthetic route change. Through the interviews, an attempt was made to determine whether the technical problems leading to the eventual route change were first identified before or after the start of pilot production. Given that the process development projects in the sample can span multiple years and involve geographically dispersed development departments, tracking and classifying problems ex post involves obvious methodological difficulties and the data must be interpreted cautiously. Of the ten projects involving a route change during the pilot production phase, there were only three clear cases where the problem leading to the eventual change was first discovered during pilot production. 25 In the remaining cases, a problem was initially identified at laboratory scale. Of these cases, about half appeared to be situations wherein route changes were implemented only after extensive attempts to rectify the problem at pilot scale through process refinements failed to achieve desired results. This discussion is not meant to imply that all potential issues can be addressed in research laboratories, and that start-up is trivial. The impact of the subtle idiosyncrasies of the particular production environment (how operators perform certain steps, the lay-out of the factory and the piping, the specific equipment already installed at the plant 26) are difficult to anticipate, and thus there is still a role for learning-by-doing. The issue is one of balance. Given the well-developed state of theoretical and practical knowledge surrounding chemical processes, laboratory and pilot scale search are relatively productive in uncovering problems and in developing the basic architecture of the process. Refining the process and adapting it to the idiosyncrasies of a particular plant are most likely to require learning-by-doing in the factory setting.

25 In some instances, the organization will transfer a process with known problems to the pilot plant only to make sufficient quantities of material to support clinical testing of the drug. 26 Pilot scale facilities generally use highly flexible configurations and equipment in order to accommodate a wide range of potential processes. Commercial manufacturing plants in pharmaceuticals often use equipment that was originally deployed for a specific set of products.

1108

G.P. Pisano / Research Policy 25 (1996) 1097-1119

3.2.1. Biotechnological processes The characteristics of the knowledge base underlying biotechnology process development are quite different from those of chemical synthesis. These differences are largely related to differences in the maturities of the two technologies. In comparison to chemical-based drugs, biotechnology is in its infancy. The major discovery triggering commercial R&D on therapeutic recombinant proteins was only made in 1973.27 The first commercial biotechnoiogy enterprises were founded in the mid-1970s. Although there is extensive basic scientific research in molecular biology, cell biology, biochemistry, protein chemistry, and other relevant scientific disciplines, most of this work has been geared toward the problems of finding and cloning molecules with specific therapeutic effects. Compared with the chemical world, there has been very little basic research done on the problems of engineering larger-scale biotechnology processes. Unlike chemicals, the industry as a whole has accumulated little practical experience with process design and scale-up. The first biotechnologybased pharmaceutical to be manufactured at commercial scale, recombinant insulin, was approved by regulatory authorities in 1982; and since that time, only a total of about 25 biotechnoiogy-based therapeutics have been approved for sale in the United States. Indeed, there was initially skepticism by some observers that recombinantly engineered processes could even be scaled-up. Researchers interviewed during our study generally described the development process as involving 'more art than science.' The critical differences between the structures of knowledge in chemicals and biotechnology fall into three categories: (1) differences in theoretical understanding of the basic processes; (2) ability to precisely fully characterize intermediates and final products; and (3) knowledge of the second-order effects of scale. Each of these is discussed below. 3.2.2. Lack of theory Compared with chemical synthesis, biotechnology process technology is a knowledge regime character-

27In 1973, Stanley Cohen and Herbert Boyer discovered the basic techniqueused to geneticallyengineerbacteria cells so that they could be inducedto producespecificproteins.

ized by a relatively immature theoretical base. The precise principles governing the production of recombinant proteins are dimly understood. There are few general theories or even rules of thumb to guide process search and selection. Researchers interviewed in the study emphasized that 'every protein is different.' Predicting how a potential process might perform based on what worked last time is generally tenuous. One researcher interviewed in the study provided an example that describes deciding whether or not to use fetal calf blood serum in the cell culture media. Because fetal calf serum is extremely expensive and can make purification more complex, the general tendency is to use a 'serum-free' media. But there is no theory to dictate why and when serum might be required. He noted, For one product, we could easily switch to serum-free media. Another product - which uses the exact same host cell - will die completely when switched to serum free. You just can't tell ahead of time what is going to work. Thus, unlike chemical processes, the factors affecting biotechnology processes even at small scale are not well understood. This makes it difficult to predict how a change in the environment (from laboratory to the factory) may affect the process.

3.2.3. Inability to characterize intermediates and final products The protein molecules produced by biotechnology processes have enormously complex structures. Moreover, any given 'molecule' is often really part of a family of related molecules, each of which may differ very slightly from its 'relatives.' Whereas the structure of the relatively small molecules produced through chemical synthesis can be fully characterized, most proteins can not be. As a result, it can be very difficult to trace how a small change in the process might alter the structure of the protein. Moreover, unlike chemical researchers, biotechnology researchers can not isolate, analyze, and characterize each reaction in the process. Such precision is impossible in the biotechnology context where each process is composed of literally thousands of reactions, virtually all of which occur inside a host cell. Although molecular biologists understand the

G.P. Pisano / Research Policy 25 (1996) 1097-1119

mechanisms by which cells synthesize proteins, the precise sequence of intra-cellular reactions for any given product is not well known. Thus, although researchers might observe certain problems in a process (e.g. a sudden drop in yield, a change in the form of the protein), these can not be traced to a specific reaction undertaken in the cell. As a result, it is very difficult to anticipate how a change in some process parameters (e.g. temperature) or operating conditions (e.g. scale) might influence the innerworkings of the process.

3.2.4. Lack of knowledge of second-order scale-up effects As in chemicals, scale can have second-order effects on a biotechnology process, but in the case of biotechnology these effects are not well understood. Given that 'every protein is different,' broadly applicable scale-up heuristics have generally not emerged. It is very difficult to predict commercial process performance from laboratory data. One researcher interviewed during the study recalled the following incident: According to research, clone #53 would express at 30-40 picograms per cell per day (pcd). But once we scaled it up, performance went down to 15 pcd. Compounding the scale-up problem is the lack of deep understanding about how the process operates even at small scale (Section 3.2.2 above). In the example above, researchers could not explain why clone 53 expressed at 30-40 pcd, let alone understand what factors caused it to drop to 15 pcd at larger scale. Researchers generally do not know ahead of time which environmental variables may be relevant, nor can they fully anticipate the impact of those which they have identified. For example, in one project in the sample, low yields after the purification step were discovered during small-scale tests. The purification column (essentially a type of filter) was becoming overloaded, and the active protein was being washed off. The reasons for this occurring were difficult to pinpoint. Removing an earlier process step to reduce the concentration of the active protein in the solution seemed to work at larger scale. However, it was later discovered that it was actually the larger purification column used and not the increased concentration of the solution that solved

1109

the problem. When the process was scaled-up even further in the commercial manufacturing setting, the yield problem re-surfaced because at that scale an even larger purification column was needed. 28 Unfortunately, because the technologies themselves are so different, it is impossible to compare the nature and causes of problems found in biotechnology with those discussed earlier for the chemical sample. For instance, since synthetic routes are not developed in biotechnology processes, we can not compare changes along this dimension. Nevertheless, the qualitative information on process changes and problems across the ten biotechnology projects was very consistent with the examples provided above. In virtually every biotechnology project in the sample, scale-up to both pilot and commercial production was associated with significant process problems and resulting process changes. Given the large number of technical issues associated with each scale-up and transfer, it was impossible to systematically track which problems could have been or were anticipated. However, a general theme from the interviews was that problems associated with changes in scale or production equipment were very difficult, even for an experienced organization to anticipate. Using the framework advanced earlier, we would expect the patterns of effective process development to differ between biotechnology projects and chemical projects. Whereas in the chemical environment, the structure of knowledge permits many major problems to be identified and solved in the laboratory and pilot plants, biotechnology is likely to require a much greater emphasis on learning-by-doing in the factory. This hypothesis is tested below.

2~ The failure of pilot and commercial batches is a common thread in the interviews conducted during the study and in published accounts from firms. In a published interview discussing Genentech's early efforts to seale-up human growth hormone, company co-founder Robert Swanson noted: "Scientifically, we began at square one by taking microorganisms out of the laboratory scale and putting them into much larger stainless steel tanks. You "know, learning how those organisms worked in that environment wasn't easy. The first time we did it, they all dropped dead - they didn't like it." R. Swanson (1986), "'People Make Decisions as Owners," interviewed with Arthur Young High Technology Group, Biotech 86: At the Crossroads: A Survey of an Industry in Evolution (cited in McElvey, 1994).

1110

G.P. Pisam~/ Research Policy 25 (1996) 1097-1119

4. Research design, data, and analysis The data used in the analysis is drawn from a larger study on process development performance in the pharmaceutical industry. Since the type of information required for the analysis is not publicly available, it was necessary to gain the cooperation of pharmaceutical companies willing to participate in the study. Because these data are highly proprietary, the names of the participating firms and details of specific projects, other than aggregate statistics, cannot be disclosed. Data for the present analysis were collected from 23 process development projects. Of these, 13 projects involved the development of traditional chemical processes, while ten involved processes based on new biotechnology-based technology. In total, 11 organizations participated in the study (five established drug companies, five biotechnology firms, and one biotechnology division of an established pharmaceutical company). For each project, data were collected on the history and timing of critical project events, resources expended, and the details of approaches used to identify and solve problems. The data were obtained through a combination of questionnaire surveys, in-depth interviews with key project participants, and proprietary documents. Data from each of the three sources were cross-checked to ensure accuracy and follow-up interviews were used to resolve any discrepancies in the data. In total, the data collection process spanned 2 years, and involved close to 200 interviews with personnel from participating R & D sites and plants in the USA and Europe. The nature of the data collection process is one reason the sample size is relatively small. A second factor limiting the sample size was the population of potential projects. Each process development project in the sample was associated with the development of a new molecular entity, a relatively rare event in the pharmaceutical industry. The largest and most productive pharmaceutical firms rarely launch more than one new molecular entity in any given year, and many companies have gone several years without launching any. The situation for biotechnology is even more constraining given the relatively small number of new drugs developed using this technology. In most instances, the projects in the sample represent each company's most recent development

efforts, as these were the ones for which accurate data and key project participants were most available. There was no reason to suspect that these sampling methods lead to any serious bias in the sample that might influence the results.

4.1. Dependent variable The dependent variable, process development costs, is measured as the total number of scientific and engineering person-hours required for all activities associated with process research, development, scale-up, and transfer into the commercial plant. 29 Process development cost is a particularly important dimension of performance in pharmaceuticals given the high uncertainty surrounding development. Most new drugs in development never make it to market. 30 The capability to carry-out individual projects with fewer resources helps to reduce the risks and total costs of development. The start of a project was marked by the initiation of research designed to find a process that might eventually be suitable for commercial production. This point could be identified from project records and interviews that indicated when process research personnel first became involved in the project. The successful validation of the process at full commercial scale was the event used to mark the end of the project. Because validation has regulatory implications, pharmaceutical companies keep excellent records on when it occurs. The total number of hours invested in process devel-

29 Other potential dependent variables include lead time and manufacturing performance (e.g. yield). An analysis of the determinants of lead time performance is presented in Pisano (1994). An analysis of manufacturing performance was not possible because consistent and comparable data on this dimension were not available across the sample of projects. Pisano (1996) analyzes the quality of the manufacturing process design on a sub-sample of projects and fmds no apparent trade-offs between development lead time, development costs, and the quality of the manufacturing process design. 30 The high rate of project failures does not introduce bias into the present sample. In pharmaceuticals, development project terminations are almost always due to failures of the drug to show safety or efficacy in clinical trials. Since these reasons are completely unrelated to the development of the process technology, they do not cause the project sample to be skewed in ways that might influence the results of this study.

G.P. Pisano/ Research Policy25 (1996) 1097-1119 opment was calibrated using project records (e.g. time sheets) and estimates from personnel involved in the project.

4.2. Independent variables 4.2.1. Timing of technology transfer The main independent variable of interest is the timing of the process technology transfer to the plant. Since the total length of projects varies significantly, it is important to compare the timing of the transfer to the plant relative to the total process development lead time. The variable (TRANSFER) was constructed as follows: Elapsed months from project

1111

D N A processes were included (e.g. monoclonal antibodies were excluded from the sample). 31 An additional control for project level differences in complexity included the scale of the process (measured as the total output of the product in the first full year of commercial production). Because of the enormous range in the sample (from grams to hundreds of thousands of metric tons), this variable was measured logarithmically. The variable appears in the model as Log(SCALE). It was suggested in discussions with study participants that the model control for whether the drug was for an acute or chronic indication. 32 A dummy variable (ACUTE) was included in the model to indicate that the process was being developed for an acute treatment drug. 33

start to technology transfer TRANSFER =

Total process development lead time (months)

The lower this index, the more the organization is presumed to be relying on learning-by-doing.

4.3. Additional variables 4.3.1. Project complexity and content Because there is a fair degree of heterogeneity in the sample of projects, variables to control for these differences and their impact on development productivity are included in the model. One important difference is whether the projects were biotechnology- or chemical-based. A dummy variable (CHEMICALS) is included to indicate whether the project was chemical-based. As an additive dummy, this variable indicates whether or not there are differences in the means of the productivity between the chemical and biotechnology projects. Differences in complexity should also influence the number of hours required. For chemical projects, a measure of complexity is the number of chemical reactions required to synthesize the molecule. This variable (CHEM × REACTIONS) was constructed by multiplying the chemical dummy (CHEMICALS) with the number of reactions required in the synthesis. To help constrain differences in complexity within the biotechnology sample, only biotechnology projects involving the development of recombinant

4.4. Descriptive statistics Table 3 presents means and standard deviations of the variables for both the full sample and the chemical and biotechnology sub-samples. Regarding the strategy for technology transfer, significant differences appear between chemical and biotechnology pharmaceuticals. In the average chemical project, the process technology is transferred to the commercial facility when the project is approximately 79% completed (in calendar months) whereas the typical biotechnology process is transferred after 57.5% com-

31 Preliminary statistical analysis revealed that different types of recombinant process technologies(e.g. bacterial cell versus mammalian cell system) did not influence development hours in a significant way. 32 Chronic diseases are those which require on-going treatment over long periods of time (e.g. high blood pressure). An acute condition is one which requires treatment over a very short time (e.g. a bacterial infection). Whether a drug is being developed for a chronic disease or an acute condition may influence process development efforts. Because they are administered for a relatively short period of time, acute therapies typically require less lengthy clinical trials to demonstrate both safety and efficacy. A shorter expected clinical trial period means there is less time to complete process development. Spending more resources may be viewed as necessary to complete the work in the required time. 33 In preliminary analysis, other control variables pertaining to project content and project strategy were included. Because these other variables did not improve the statistical quality or insight of the models, and since their inclusion did not impact the results, they were dropped from further analysis.

1112

G.P. Pisano / Research Policy 25 (1996) 1097-1119

Table 3 Descriptive statistics means (standard deviation in parenthesis)

DEV.HOURS ACUTE Log(SCALE) TRANSFER

Full sample (n = 23)

Chemicals (n = 13)

Biotechnology (n = 10)

193864 (197243) 0.208 (0.415) 1.700 (2.120) 0.697 (0.238)

181965 ( 177197) 0.231 (0.439) 3.200 (1.390) 0.791 (0.162)

209334 (229707) 0.182 (0.405) - 0.257 (0.981) 0.575 (0.272)

pletion. This would suggest that biotechnology projects, on average, are relying more on a development strategy of 'learning-by-doing' in the actual commercial production environment, whereas in chemical projects more of the learning is occurring before process reaches the plant floor. One might argue that this is partially due to differences in the regulatory guidelines governing biotechnology and chemical-based pharmaceutical production. With biotechnology, food and drug administration (FDA) tends to place greater emphasis on manufacturing products for the final phase of clinical trials in commercial facilities. However, there are two factors that need to be kept in mind. First, differences in regulation influence the timing of technology transfer to the factory relative to the overall product development and regulatory filing schedule. The measure used in this analysis is the timing of technology transfer relative to the time horizon of the process development schedule. Second, there is substantial variation in the timing of technology transfer to the commercial manufacturing facilities within both the chemical and biotechnoiogy samples. In the biotechnology sample, for example, one project was transferred to the plant after only about 15% of the process development work had been completed, while there were three other projects which were transferred after being approximately 90% completed. The range in the chemical project sample spans from 48% to 96%. Differences within technological class of this magnitude suggest that development strategy choices, rather than strict regulatory requirements, are at work. Additional evidence that managerial choice is driving differences in the timing of technology trans-

fer can be gleaned from analyzing projects involving identical or closely matched molecules in the sample. Within the current sample, three pairs of projects involved development of essentially the same molecule. Pair 'A' and Pair 'B' each involved process development for nearly identical biotechnology-based drugs; Pair 'C' involved process development for two nearly identical synthetic chemical drugs. Assuming no significant observation errors, differences in the timing of technology transfer within identical pairings should be due largely to managerial choice, rather than regulatory factors or uncontrolled differences in project complexity. Within each pair, there were relatively large differences in the percentage of the project completed prior to technology transfer to the plant. Within Pair 'A,' one project was transferred after 15% completion while the other was transferred after 60% completion. The corresponding figures for Pair 'B' are 25% and 57%, and for Pair 'C,' they are 66% and 88%. Although three pairs are too few for statistical analysis, they provide a useful 'reality check' against alternative explanations. 4.5. Regression analysis

To analyze the impact of technology transfer strategy on development costs, and then to determine whether this impact differs between biotechnology and chemical process technology projects, the following model was estimated using ordinary least squares: DEV.HOURSi = a 0 +/31CHEM i +/32CHEM x REACTIONS i + fl3ACUTEi + flaLog(Scale)i

+/35TRANSFER i + f l 6 C H E M X TRANSFERi + e i The main coefficients of interest are /35 and /36. /35 is an estimate of the effect of the timing of technology transfer on development costs for the biotechnology projects in the sample. /36, the coefficient on the interactive term, indicates the difference between the effect in biotechnology and the effect in chemicals. From the earlier discussion, /35 is expected to be positive, reflecting the need for learning-by-doing

G.P. Pisano / Research Policy 25 (1996) 1097-1119

in the development of biotechnology processes. If, as hypothesized, the structure of knowledge in the chemical environment requires less learning-by-doing for effective performance, /36 should be negative.

I 113

ing development costs within the closely matched project pairs discussed earlier, however, suggests that the model has done a reasonably good job controlling for such differences. Because the projects within each pair are essentially identical, differences in project content and technical complexity are virtually eliminated. The ratios of development hours consumed with each of the pairs were 5:1, 1.09:1, and 2.23:1 (thus, in the first pair, one project consumed five times as many resources as the other, nearly identical project). The average ratio across these pairs is 2.77, a figure very close to the 2.70 ratio estimated statistically across the entire sample. These data suggest that differences in project performance are not driven solely by content or complexity, and that project strategies and execution capabilities matter. Model 5 shows the effects of the technology transfer on total process development costs across the entire sample. The coefficient is both positive and statistically significant indicating that later transfers to the factory are associated with higher development costs. This result seems to confirm the idea that learning-by-doing in the plant is necessary for the efficient development of process technology.

4.6. Results

Results of the regression analysis are shown in Table 4. Models 1-4 include only the variables controlling for differences in project content and complexity. Given the high degree of project heterogeneity, the explanatory power of these content variables was not surprising. However, after adjusting for differences in project content and complexity, significant development costs differential between the best and worst project in the sample was a factor of 2.70. Differences of approximately this magnitude have also been found in studies of development performance in other industries. One methodological concern with a small sample of heterogeneous project is the possibility of remaining (uncontrolled for) differences in content and complexity. This issue, of course, is not unique to this study. Given the complex nature of development projects, it is virtually impossible to identify all possible variables. Compar-

Table 4 Regression

results process development

Variables

Model

Constant

1

209.3

Model ** *

(56.7) CHEM

- 334.7 × REACT

31.2

2

225.5

* *

- 503.6

** *

(168.9) **

(12.2)

30.2 51.4

Model 4

* *

177.5 " * *

(43.9)

(42.5)

- 363.6

* *

23.3

*

ACUTE

- 296.2

* *

33.9 (80.7)

*" *

(105.3)

(9.3)

(30.4)

Model 5

184.7 * * *

(137.0)

(11.7)

Log(SCALE)

e r r o r s in p a r e n t h e s e s ) Model 3

* **

(54.8)

(142.4) CHEM

hours (standard

- 429.2

23.1 * *

23.3

19.7

22.9 (23.1)

296.6

** *

(81.2)

318.5

** * **

(8.5)

(25.2) ** *

(75.5)

269.1 268.4

(86.3)

- 198.4

30.0

(244.1)

(192.5)

24.7

* *"

** *

256.4

** *

(70.1) * *

436.6

** *

3.30

* P < 0.10, * * P<0.05, t-tests for TRANSFER Coefficients

0.24 "

3.36

0.54 * *

7.49

0.55 * * *

* * * P<0.01. and CHEM

X TRANSFER

and standard errors divided by 103 .

a r e f o r o n e .tail d i s t r i b u t i o n s .

9.99

0.62 * **

8.12

**

8.46

444.5

* **

-545.1

**

(244.1)

0.67 * *

** *

(139.7)

(266.9) 0.17

263.6 (63.3)

(145.3) -513.3

Adj. R 2

* * *

(7.7)

7.9

x TRANSFER

F-value

24.7

(22.8)

(124.9) CHEM

-72.6

(7.9)

(75.2)

TRANSFER

Model 7

-65.32 (91.08)

(128.7)

(92.5)

Model 6

0.68 * **

10.67

* **

G.P. Pisano / Research Policy 25 (1996) 1097-1119

1114

.

,

.

.

,

,

0

300000 "1"

200000

oy
~. 100000

"~

2_

o

< - 100000

- 200000

o

.

0

.

o

. 2

. 3

" 4

,

,

.5

.6

"

. 7

.

. 8

. 9

I

"liming of T e c h n o l o g y Transfer

Fig. 2. Relationship between development hours and technology transfer biotechnolgy sub-sample (n = 10).

Models 6 and 7, however, provide additional insight about the impact of learning-by-doing on development costs across the two technological classes in the sample. In both models, the coefficient on TRANSFER (fls) is positive and statistically significant, indicating that a high degree of learning-bydoing in biotechnology is essential for development efficiency. As hypothesized, the coefficient on the interactive term (CHEM × TRANSFER) is negative (and statistically significant), indicating that learning-by-doing is less important for development efficiency for chemical-based projects. 34 The negative coefficient on the interactive term indicates that development performance in chemicals does not rely as much on learning-by-doing as it does in biotechnology. It does not necessarily mean that learning-by-doing is unimportant in chemicals. The sum of r5 and r6 (which is the overall effect for the chemical sub-sample) is slightly negative in Models 6 and 7. In neither case, however, is the sum statistically significant. This indicates that a chemical-based project that followed a strong learning-bydoing (i.e. early transfer to the plant) may not perform any worse than one in which most of the development work occurred before the transfer. To gain further insights, the relationship between the technology transfer strategy and development hours is plotted separately for the chemical and biotechnology sub-samples (Fig. 2 and Fig. 3). In these

34 Given the relatively small sample size, an analysis was conducted to examine whether any particular outliers were driving the results. The models were re-estimated 23 times, each time with one observation excluded. In no case did the exclusion of a point lead to a significant change in the coefficients of primary interest.

illustrations, development hours is adjusted for differences in project complexity. 35 The differences between these plots are quite striking. In the biotechnology sub-sample, the need for early technology transfer and learning-by-doing is quite clear. The regression for the biotechnology sub-sample is highly significant ( F = 11.37, P < 0.01, R 2 = 0.54). The pattem for the chemical-based project is quite different. There was no correlation between how early or late a process technology was transferred to the factory and development efficiency performance ( F = 0.02, P = 0.89, R 2 < 0.01). Here, the need for learning-by-doing is less clear, and there appears to be no systematic advantage of transferring the process to the plant at an early stage. In chemical synthesis, learning-by-doing is neither necessary nor sufficient for high development efficiency. Moreover, although learning-before-doing is a viable problem-solving strategy in this arena, there appears to be room for a much greater diversity of approaches. To provide additional insights into the role of knowledge and firm-specific competences in development performance in the chemical segment, a pair of contrasting examples is provided briefly below. 4.7. Case examples

The lack of a systematic correlation between timing of technology transfer and the development costs

32 Thus, the y-axis is the residual of the regression equation DEV.HOURSi = ot0 +/3~CHEMi +/32CHEM XREACTIONSi + fl3ACUTE i + fl4Log(Scale)+ ei.

G.P. Pisano/ ResearchPolicy 25 (1996) 1097-1119 Chemical Projects (n=13) 200000

~

. 0

.

.

.

.

.

.

.

.

.

.

.

.

150000

0

100000

""

0 S0000

~"~ ;>

o

o

o~

';'

-50000

o

0

d~ -looooo

0

-150000" ;.~ <

-200000

"

-250000

• o

-300000

" • .45

, .5

-

, .55

•

, .6

Thning

•

, .65

-

, .7

-

, .75

of Technology

-

,

-

.8

, .85

-

~ .9

,

I

,

.95

Transfer

Fig. 3. Relationship between developmenthours and technology transfer chemical sub-sample(n = 13). performance in the synthetic chemical segment suggests that there is room for much greater diversity of approaches in this context. Closer analysis of two contrasting cases suggests that the impact of the timing of technology transfer on development performance may be a function of the locus of capabilities within the organization. 4.7.1. Example one: the problems of early transfer Interestingly, the chemical project with the earliest transfer to the plant had the highest development costs. While one outlier does not constitute a trend, it at least raises the question: can transfer to the plant be too early? In this case, it does appear that the lack of development prior to the transfer contributed to higher overall development costs. Although the company's process research chemists became involved in the project at a very early stage of the project, and indeed helped to invent the initial route used to discover the molecule, very little subsequent work was undertaken to find alternative synthetic routes that might have been more suitable for commercial production. 36 After selecting the synthetic route, the process was then scaled-up in a research pilot plant

36 After only 3 months of additional process research, a slightly modified version of the initial synthetic route was discovered and selected for commercial production. This time period for route selection was extremelyshort in comparison with other chemical projects in the sample(the averagetime for makinga commitment to a final synthetic route was 40 months, and eight of the 13 projects involved the development and pilot testing of multiple synthetic routes).

1115

where it was used over the next year to produce batches of the drug for clinical trials. The research pilot plant was approximately half the scale of commercial production. Transfer of the process from the research pilot plant to the commercial manufacturing site began relatively early in the project for two reasons. First, the development organization on the production site had spare engineering capacity at the time, and thus wanted to get a head start on the project. Second, the process technology had extremely low yields (approximately 1%) and would thus need to be further scaled-up in order to meet clinical supply requirements. The team of chemical engineers responsible for the process at the plant reported that these problems were largely unrelated to scale. Since the product was expected to be produced commercially in relatively small volumes, and commercial scale was only approximately two times the research pilot scale, scale-up was not a significant issue (in contrast, it was typical in the sample for commercial production to involve a ten-fold scale-up). In addition, the few scale-up problems that did occur were, according to the development group at the plant, fully expected. For example, one purification step used chromatography which was known from past experience to be difficult to scale-up. An alternative purification process was implemented for this step. Despite the lack of major scale-up issues, extensive work was required on each of the ten steps of the process during and after the transfer in order to increase yields, throughput times, and manufacturability. Many of these problems were rooted in the chemistry of the process itself and had been previously identified by chemical process research laboratories and during production. For example, two steps of the process had minor stability problems, another used a raw material that was not commercially available, and another generated an intermediate that was difficult to purify. Although the plant had a very strong development organization that employed both chemists and chemical engineers, its experience had largely been in solving scale-up and process equipment issues. Since these problems were rooted in the chemistry of the process rather than its scale or equipment configuration, much of the development at this phase required extensive assistance from the process research group. Eventually, the major prob-

1116

G.P. Pisano / Research Policy 25 (1996) 1097-1119

lems with each step were solved, and additional yield and throughput improvement were implemented in other steps of the process. Participants in the project hailed it as a case of successful collaboration between process research and manufacturing. The data indicate, however, that this collaboration may have come at a very high cost. The early transfer did not appear to surface any additional technical problems with the process. But it did cost the firm its ability to fully exploit the process research laboratory's comparative advantage in solving chemistry problems (which were known at the outset). In addition, by freezing the synthetic route very early on, the organization essentially committed itself to a basic process architecture with known yield and throughput problems. 37 Extensive development resources then needed to be spent to optimize and trouble-shoot individual components of the process, when an alternative synthetic route might have avoided them from the outset.

4.7.2. Example 2: early transfer and the capability for learning-before-doing Counter to the predictions of the framework, the chemical project with the lowest development costs was among those with the earliest transfer to the plant in the chemical sub-sample. In terms of timing of technology transfer, this project's strategy was not all that different from the one described above. However, more detailed investigation revealed that at the time of transfer, the process was considerably more developed than in the example above. Furthermore, this firm was somewhat unusual among the other pharmaceutical companies in the study in having strong process chemistry capabilities located at the manufacturing plant. This meant that even once the process was transferred to the plant, any further improvements of the basic process chemistry could be undertaken on-site and there was no need to send the process back to the process research labs for •rework.' As in the case described above, the company's process research chemists became involved early in

37 Data presented in Pisano (1996) reveals that changing synthetic routes, on average, lead to a 70% reduction in manufacturing costs, whereas optimization and improvement of any specific route lead to average cost reductions of 30%.

the project and worked with discovery chemists in synthesizing the molecule. The initial 'discovery route' was used to make small quantities of the compound for animal tests. However, the process was complex and difficult to use even at laboratory scale, and thus it became clear almost immediately that an alternative route would be needed for commercial production. Thus, unlike the project described above, process research chemists continued searching for alternative synthetic routes that might be more suitable for commercial manufacturing. Drawing from experience with structurally similar molecules, and from published literature, the process research derived (over a period of approximately 10 months) an alternative route that looked, in theory, to have superior characteristics. A final commitment to this route, however, was not made until the route was tested extensively in the laboratory and its practicality demonstrated in the pilot plant. Process researchers involved in the project noted that they continued to look at alternative processes as 'fallbacks' but after a few batches of pilot production, the organization was confident that it had an efficient process, with total yields approaching 87% (approximately 99% per step). Over the next several months, the process was further tested and refined in the pilot plant where, according to a developer interviewed about this project, "95% of all problems with the process were identified." Once transferred to the commercial manufacturing facility, the plant's process chemistry group undertook further evaluations of the chemistry and explored alternative methods of synthesis, and several full-scale pilot tests were undertaken on the process. Although no better synthetic routes were discovered at this time, the plant's process chemistry group worked on ways of combining process steps (in order to improve throughput) and adapting the process to specific idiosyncrasies of the production plant and the environmental regulatory constraints under which it operated. 38 These were considered to be relatively minor process changes and did not

38 For example, one step of the process needed to be modified so that a particular effluent would not be released into the air and another purification step was modified so that it would be more consistent with the equipment and procedures currently in place at the plant.

G.P. Pisano / Research Policy 25 (1996) 1097-1119

consume extensive resources. No major problems (either anticipated or unanticipated) were reported in scaling-up the process, transferring into production, or in achieving the desired levels of commercial performance. Two factors thus seem to distinguish this case and help to explain why early transfer did not result in higher development costs. First, the company's process development strategy focused on selecting high performing synthetic routes that would be feasible in commercial production, rather than trying to optimize (or fix) routes which were less than desirable. A solid process architecture laid the foundation for an earlier and smoother transfer to commercial production. Second, the plant's strong process chemistry capabilities meant that it could continue to develop, refine, and if necessary change the basic chemistry of the process even after transfer and start-up of production. 5. Conclusion

The notion that manufacturing should be a seamless extension of the R&D laboratory, a place where extensive development should occur, has gained credence in recent years. The results presented show clearly that this prescription is likely to be beneficial in certain types of technological environments, but not in all. In environments like chemical pharmaceuticals which are characterized by deep theoretical knowledge and a significant accumulation of practical experience, the role of the plant in development may not be critical because developers can anticipate and respond to manufacturing issues, without actually doing their work in the plant. In emerging technologies, like biotechnology, which are typically characterized by less mature theoretical underpinnings and less accumulated practical knowledge, the plant may be a critical venue for development. In these environments, it is simply impossible for developers to anticipate and respond to manufacturing concerns without actually doing their work in the actual production environment. Thus, the role of the plant in development, and the locus of development competencies within organizations may be technology life-cycle dependent. The reader is cautioned not to misinterpret the results as implying that developers can ignore manu-

1117

facturing concerns or develop un-manufacturable processes with abandon. In all contexts, whether the technology is mature or immature, developers need to understand manufacturing concerns. The findings simply show that in certain environments, process developers may be better able to anticipate and respond to manufacturing issues, without actually performing their work in the plant. In such environments, most of the learning that needs to take place can be done before the process is transferred to the plant. The results highlight some important implications for both development and manufacturing strategy. One relates to vertical integration and the role of in-house manufacturing. In recent years, there has been a tendency for firms, particularly young organizations in emerging technology industries, to utilize outside partners or contractors for manufacturing. This appears to be consistent with the idea of focusing on one's 'core competencies.' However, the results presented here suggest that such a manufacturing strategy could be costly. It is precisely in these emerging technologies that the idiosyncrasies of process technologies are not fully understood, and require development activities to be closely linked to the factory. A second manufacturing strategy issue concerns capacity and facilities planning. In an environment like biotechnology where learning-by-doing is absolutely essential to high development productivity, physical plant and equipment need to be installed long before required to support commercial production requirements and some excess commercial manufacturing capacity may need to be tolerated to support development in the factory. While such a strategy adds to both development costs and risks, it may also yield relatively high returns in an environment like biotechnology. Using multi-purpose or flexible manufacturing plants may help to reduce the capital risks required for early technology transfer. In contrast, in contexts where late transfer to the plant is not costly, there is less risk of using productspecific equipment since the firms can wait longer to design and install it. 39 39 In the study, we found some evidence that flexible facilities entailed significantly higher capital and operating costs than plants designed to produce a specific product.

1118

G.P. Pisano / Research Policy 25 (1996) 1097-1119

An interesting issue for further exploration concerns how firms build their scientific knowledge bases about process technologies. Clearly, in some industries, publicly diffused scientific research plays an important role. However, as suggested in the case examples, the capability to learn before doing may be firm-specific. Even within biotechnology, there were differences in the extent to which individual firms were building scientific knowledge bases for future process development. An interesting dynamic may be at work. To have the capability to develop and simulate processes outside the factory, an R & D laboratory needs deep scientific knowledge about the technology as well as detailed knowledge about how the factory actually operates. Few production environments are stable. As factories operate, learning takes place and the environment changes. To maintain the capability to simulate processes and to anticipate production needs, feedback from production back into R & D is critical. If this proposition is true, it suggests that the value of close contact between R & D and manufacturing is realized in terms of the longer-term enrichment of the firm's R&D knowledge base. Clearly, this issue merits further research as it has significant implications, not only for how organizational capabilities evolve, but also for the management of development projects and processes.

References Abemathy, W.J., 1978, The Productivity Dilemma (The Johns Hopkins University Press, Baltimore, MD). Adler, P. and K. Clark, 1991, Behind the learning curve, Management Science 37(3), 267-281. Allen, T.J., 1966, Studies of the problem-solving process in engineering design, IEEE Transactions on Engineering Management EM-13(2), 72-83. Arrow, K., 1962, The economic implications of learning by doing, Review of Economic Studies 29, 166-170. Bohn, R. and R. Jaikumar, 1992, The structure of technological knowledge in manufacturing, Working Paper (November). Clark, K. and T. Fujimoto, 1991, Product Development Performance: Strategy, Organization, and Management in the World Auto Industry (Harvard Business School Press, Boston). Clausing, D., 1993, Total Quality Development: A Step-by-Step Guide to World Class Concurrent Engineering (ASME Press, New York). Dosi, G. and L. Marengo, 1993, Some elements of an evolution-

ary theory of organizational competencies, in: R.W. England (Editor), EvOlutionary .Concepts in Contemporary Economics (University of Michigan Press, Ann Arbor). Frischmuth, D.S. and T.J. Allen, 1969, A model for the description and evaluation of technical problem solving, IEEE Transactions on Engineering Management EM-16(2), 58-64. Garcia, A.B. and J. Gocke, 1994, Virtual Prototyping: Concept to Production (Defense Systems Management College Press, Fort Belvoir). Haber, L., 1958, The Chemical Industry During the Nineteenth Century (Oxford University Press, Oxford). Hirsch, W., 1952, Manufacturing progress function, Review of Economics and Statistics 34, 143-155. lansiti, M. and K.B. Clark, 1994, Integration and dynamic capability: evidence from product development in automobiles and mainframe computers, Industrial and Corporate Change 3(3), 557-605. Kogut, B. and U. Zander, 1992, Knowledge of firm, combinative capabilities, and the replication of technology, Organization Science 3, 383-397. Leonard-Barton, D., 1992, The factory as a learning laboratory, Sloan Management Review 34(1), 23-38. Lieberman, M., 1984, The learning curve and pricing in the chemical processing industries, Rand Journal of Economics 15(2), 213-228. Marples, D., 1961, The decision of engineering design, IRE Transactions on Engineering Management 2, 55-71. McElvey, M., 1994, Evolutionary Innovation, Ph.D Thesis, Linkopings Universitet, Sweden. Nelson, R., 1982, The role of knowledge in R&D efficiency, Quarterly Journal of Economics (August), 453-470. Nelson, R. and S. Winter, 1977, In search of useful theory of innovation, Research Policy 5, 36-76. Newell, A. and H. Simon, 1972, Human Problem Solving (Prentice Hall, Englewood Cliffs, NS). Pavitt, K., 1984, Sectoral patterns of technical change: towards a taxonomy and a theory, Research Policy 13, 343-373. Pisano, G., 1994, Knowledge, integration, and the locus of learning: an empirical analysis of process development, Strategic Management Journal 15(winter), 85-100. Pisano, G., 1996, The Development Factory: Unlocking the Potential of Process Innovation (Harvard Business School Press, Boston). Polanyi, M., 1962, Personal Knowledge: Toward a Post-Critical Philosophy (Harper Torchbooks, New York). Rosenberg, N., 1982, Inside the Black Box: Technology and Economics (Cambridge University Press, New York). Simon, H.A., 1978, Rationality as process and as product of thought, American Economic Review 69(4), 1-16. Stobaugh, R. and P. Townsend, 1975, Price forecasting and strategic planning: the case of petrochemicals, Journal of Marketing Research 12, 19-29. Teece, D.J., 1976, The Multinational Corporation and the Resource Cost of International Technology Transfer (Ballinger, Cambridge, MA). Ulrich, K. and S. Eppinger, 1995, Product Design and Development (McGraw-Hill, New York).

G.P. Pisano / Research Policy 25 (1996) 1097-1119

Utterback, J.M., 1994, Mastering the Dynamics of Innovations (Harvard Business School Press, Boston, MA). yon Hippel, E., 1994, The impact of 'sticky date' on innovation and problem solving, Management Science 40(4), 429-439. yon Hippel, E. and M. Tyre, 1995, How the 'learning by doing' is done: problem identification in novel process equipment, Research Policy 24(1), 1-12. Wheelwright, S. and K. Clark, 1992, Revolutionizing Product Development (Free Press, New York).

1119

Whitney, D., 1988, Manufacturing by design, Harvard Business Review 66, 83-91. Winter, S.G., 1987, Knowledge and competence as strategic assets, in: D.J. Teece (Editors), The Competitive Challenge (Ballinger Publishing Company, Cambridge, MA). Wright, T.P., 1936, Factors affecting the cost of airplanes, Journal of Aeronautical Science 3, 122-128.