Designing a performance measurement system for the research activities: A reference framework and an empirical study

J. Eng. Technol. Manage. 25 (2008) 213–226

Contents lists available at ScienceDirect

J. Eng. Technol. Manage. journal homepage: www.elsevier.com/locate/jengtecman

Designing a performance measurement system for the research activities: A reference framework and an empirical study Vittorio Chiesa a, Federico Frattini a,*, Valentina Lazzarotti b, Raffaella Manzini b a b

Politecnico di Milano, Dipartimento di Ingegneria Gestionale, Milan, Italy Universita` Carlo Cattaneo – LIUC, Castellanza, Varese, Italy

A R T I C L E I N F O

A B S T R A C T

Article history:

Designing a performance measurement system (PMS) for R&D activities is a very critical but challenging task for supporting decision making and people motivation. Therefore, the subject is widely discussed in literature, but the use of a PMS for R&D is still uncommon among companies. The paper aims at making a step further in the field, elaborating a reference framework that describes the logical steps for the definition of a PMS for R&D. Moreover, the problem of designing an effective PMS is in-depth studied in a real context, a biotech company that operates in the field of pharmaceutical research. ß 2008 Elsevier B.V. All rights reserved.

Available online 8 August 2008 JEL classification: O32 Keywords: Research & development Performance measurement Performance measurement system Biotechnology Pharmaceutical research

1. Introduction Defining and implementing a performance measurement system (PMS) within the company is considered a critical activity for supporting decision making, motivating people, stimulating learning, improving coordination and communication (Shank and Govindarajan, 1993; Schumann et al., 1995; Kerssens-van Drongelen and Bilderbeek, 1999). In other words, the PMS is nowadays considered fundamental for achieving the company’s objectives. As a consequence, all the main activities, processes and/or functions within companies have recently become the object of a PMS: not only the

* Corresponding author at: Politecnico di Milano, Dipartimento di Ingegneria Gestionale, Via Giuseppe Colombo, 40, 20133 Milan, Italy. Tel.: +39 02 2399 2796; fax: +39 02 2399 2720. E-mail address: [email protected] (F. Frattini). 0923-4748/$ – see front matter ß 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.jengtecman.2008.07.002

214

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

primary ones, such as production or logistics, but also the supporting ones, such as all the administrative processes. According to this trend, R&D as well is being considered as a set of activities and processes whose performance should be monitored and measured, particularly because in many competitive contexts technological innovation (and, hence, the results of R&D activities) is the main source of sustainable competitive advantage. However, defining a PMS for R&D activities is recognized in literature as a very difficult task, since effort levels may not be observable in quantitative, measurable terms, success is uncertain (and influenced by uncontrollable factors) and it can be assessed only after long delays (Tipping et al., 1995; Brown and Svenson, 1988; Kerssens-van Drongelen and Bilderbeek, 1999; Loch and Tapper, 2002). As a consequence, in the last years many contributions have been written aimed at discussing the subject and suggesting possible approaches (Pappas and Remer, 1985; Brown and Svenson, 1988; Chiesa and Masella, 1996; Werner and Souder, 1997; Hauser, 1998; Driva and Pawar, 1999; Driva et al., 2000; Poh et al., 2001; Godener and Soderquist, 2004). Such contributions have so far concentrated mainly on performance measurement, i.e. on defining a set of dimensions of performance to be controlled and the metrics (or indicators) to be used for the measurement of such performance. Far less contributions are dedicated to the definition of a whole performance measurement system, i.e. an integrated system not only able to measure a specific set of performance, but also to explain the managerial and organizational meaning of each measure, to suggest the most appropriate use of each measure and to analyze R&D performance with respect to the overall company strategy. The main contributions in this direction come from Kerssens-van Drongelen and Cook (1997) and Kerssens-van Drongelen and Bilderbeek (1999), who applied the concept of Balanced Scorecard (Kaplan and Norton, 1992) to R&D, and from other authors (Bremser and Barsky, 2004) who have adopted more recently a similar approach. This paper aims at making a further step in this field, i.e. in-depth studying the problem of defining a system for performance measurement in R&D units, and not only to identify some metrics and indicators. In particular, the focus is on the R&D units of companies for which technological innovation, and, hence, R&D activities are critical for competition. These companies, in fact, are generally characterized by a very complex and dynamic R&D environment and, hence, represent a challenging field for this study. According to this aim, a reference framework is identified in the paper, that represents a systematization of literature contributions in the field, and describes the logical steps for the definition of the PMS for R&D. The framework, starting from the corporate strategy and the R&D strategy, comes to the definition of:  the dimensions of performance to be measured and the related indicators,  the structure to be defined for the measurement system,  the process aspects to be implemented for the proper operation of the system. In a few words, the suggested framework gives a ‘‘practical’’ guide that should help managers in the definition of all the elements of a PMS, in accordance with the overall company’s strategy. Then, the framework is applied to a specific case study, with the aim of: 1. verifying the actual applicability of the framework in a real context, in which all the ‘‘contingencies’’ are taken into consideration, exploring in detail the problems and difficulties emerging during the application; 2. enriching, if possible, the framework itself and/or modifying it, according to the evidence emerged during the application; 3. giving a very detailed and concrete example of how the huge literature on performance measurement, which gives many suggestions in terms of dimensions of performance, indicators, process aspects, can be integrated within a system, internally coherent and adequate for a specific strategic context; 4. discussing the possible generalization of the framework, i.e. verifying whether and how it can be applied to other contexts than the one analyzed in this paper.

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

215

The case study, as already pointed into evidence, has been selected in a context where technological innovation represents the critical source of competitive advantage. The analyzed company, NiKem Research, is a biotech firm that operates in the field of pharmaceutical research, mainly offering services with a high technical and scientific content and capable of supporting the new drug’s R&D process. This is an interesting case, in which a company almost collapses on its R&D function that represents the 90% of the overall activities. The paper is structured as follows: Section 2 gives an overview of the literature on R&D PMS and presents the proposed framework; Section 3 describes the case study and the application of the proposed framework; finally, Section 4 discusses the results and draws some managerial implications. 2. A literature review and a theoretical framework There are many definitions of technological innovation. Here, we assume the definition given by Freeman (1976), as re-adapted by Chiesa (2001). Technological innovation is a process which includes the technical, design, manufacturing, management and commercial activities involved in the marketing of a new (or improved) product or the first use of a new (or improved) manufacturing process or equipment. Different generations of the innovation process are identified in literature beginning from the 1960s to the present (Chiesa, 2001). In particular, the latter two generations emphasise that technological innovation is not sequential, is cross-functional by nature and often multi-firm. Links with suppliers and customers are very strong along with the whole innovation process. Links with firms take place in a variety of forms (joint ventures, consortia, alliances, contracts, etc.). R&D activities clearly play an important role in the process of innovation. There are many definitions of R&D activities as well. A traditional classification concerns the macro-phases which make up the R&D process (Chiesa, 2001). It is possible to identify: basic research, which is ‘‘an activity aimed to generate knowledge related to the working principles of natural and social science without direct relation to industrial applications (products, services, production processes)’’; applied research, which is ‘‘aimed to the production of knowledge required to define the means to fulfil a specific and explicit need’’; development, which ‘‘consists of the systematic use of knowledge oriented to the development of materials, methods, tools, systems’’. Development is composed of a series of phases: design, prototyping and testing, engineering, installation, maintenance and postcommercialisation service. In parallel with the changed conception of the innovation process (the generations of the innovation process), different generations of R&D activities have been identified. From a concept of R&D with no interaction with the rest of the company and considered as an overhead cost, the importance of integration in several forms has been progressively pointed out: e.g. in terms of links between strategy and R&D activities, cross-functional teams and integration with customers. The last generation emphasises that R&D is ‘‘a part of a total innovation system including competitors, suppliers, customers and distributors’’. If we examine the R&D activities in terms of their accountability, that is the most relevant topic for this work, other significant changes can be pointed out. In fact, R&D was once considered to be a unique, creative and unstructured process that was difficult, if not impossible, to manage and control (Kerssens-van Drongelen and Cook, 1997). Today, R&D consideration has also changed concerning with this aspect, although it is generally recognized that it is challenging to establish accountability for many R&D activities (Tipping et al., 1995; Brown and Svenson, 1988; Kerssens-van Drongelen and Bilderbeek, 1999). Since the 1990s, relevant changes in the business environment (in terms of intensified competition, shortened product life cycles, advanced technology and automation, etc.) have focused managers’ attention on R&D’s contribution to competitive advantage (Kerssens-van Drongelen and Cook, 1997). These changes have brought companies to pay attention to their R&D processes in terms of efficiency, internal and external customer focus, time to market (TTM), innovativeness, etc. (Kumpe and Bolwijn, 1994). In other words, R&D is a key strategic issue that must be aligned with corporate and business strategies and their various expected performance (Pearson et al., 2000; Bremser and Barsky, 2004). In order to encourage such alignment, a performance measurement system is required, composed of the following elements, linked through reciprocal relations (Kaplan and Norton, 1992; Kerssens-van Drongelen and Cook, 1997; Kerssens-van Drongelen and Bilderbeek, 1999; Bremser and Barsky, 2004):

216

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

 the dimensions of performance to be monitored and the related indicators, qualitative and/or quantitative;  the structure of the system, i.e. the articulation in ‘‘controlled objects’’;  the process aspects to be defined for a proper working of the system, i.e. all the norms and rules governing the PMS, the timing and frequency of measurement for each controlled object and for the different dimensions of performance, the role and tasks of people involved in the implementation of the system. The contributions on the contextual approach give many relevant suggestions for the design of a PMS for R&D activities, identifying the contextual factors (for a literature review, see Kerssens-van Drongelen and Cook, 1997) that influence the definition of the PMS’s elements:  the company’s R&D strategy, in terms of long-term objectives (or critical performances) that are coherent with the business strategy, the competitive context (rules of competition and main competitive pressures) and the general environmental features (macroeconomic factors, institutional norms, social and cultural characteristics);  the entities to be monitored (i.e. R&D division or department, sub-department, project, individual) organized in a certain structure (i.e. according to scientific disciplines, typology of activities, product line, project, etc.);  the type of activities to be monitored (i.e. basic research; applied research; development, that are tasks with different degrees of uncertainty);  the PMS’s objectives (i.e. the purpose of the measurement, e.g. motivating people, diagnosing activities, supporting decisions, stimulating learning, improving communication and coordination between R&D and other company’s organizational units);  the resources (time, money, people, competencies) available for the implementation of the PMS. A tentative systematization of all those suggestions is given in the framework in Fig. 1. Such a framework describes the logical entities to be considered in the definition of the PMS for R&D and it can be helpful to practically guide the system design. The framework is made up of two parts:  the contextual factors;  the consequent PMS’s elements; and it enlightens the following aspects:  within the context, those factors that constitute the R&D ‘‘environment’’ (i.e. R&D strategy, R&D organization and management, R&D activities) can be distinguished. The R&D strategy drives

Fig. 1. A reference framework for defining a contextual R&D PMS.

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

217

Box 1. The links between the contingencies and the PMS elements Purposes of the measurement and dimensions of performance If, as it is common especially in highly uncertain and dynamic R&D environments, the measurement system serves the purpose of motivating people, particular cautions are required (Kerssensvan Drongelen and Cook, 1997). In the case of motivational purpose, the assumption is that by feeding back information about their performance, possibly coupled with incentives, people will be motivated to change their behaviour. Two conditions should be satisfied in order to design an effective motivational system. First of all, the system has to include only those factors that can be controlled by the person subjected to the measurement. Furthermore, if an incentive or a reward program is coupled with the measurement (i.e. bonuses, promotions), the system should measure all the important aspects of a work. This is due to the fact that there is a human tendency to direct more effort towards tasks that are being measured and consequently rewarded, giving less attention to tasks that are not. However, especially in research activities, tasks may not be objectively measurable (because of their intrinsic high uncertainty and relevant content of creativity), although they are generally critical in order to obtain appreciable results in the long term. This circumstance, linked to the type of activities contingency, makes it particularly challenging to establish accountability criteria. Purpose of the measurement and process aspects The purpose of the measurement is a contextual factor that is capable of influencing also some PMS process aspects. For example, whether the main PMS’s objective is to diagnose R&D activities, it is necessary to choose standards to measure performance against that give the possibility to objectively judge the value of a specific indicator and to make comparisons over time, thus enlightening eventual improvement in R&D performance. At the same time, the frequency and the timing of the measurement must be chosen so that the PMS is capable of gathering and transferring performance data to R&D and top managers timely and coherently with their informational needs. Type of activities and dimensions of performance In a research context, literature suggests that the system should include dimensions of performance regarding the research process (a dimension of expected performance could be, for example, ‘‘the achievement of professionalism standards’’, and a possible indicator ‘‘the frequency of participation to international conferences’’) rather than the research output, i.e. dimension of performance such as ‘‘the work productivity’’, measured by ‘‘the number of patents’’ indicator (Loch and Tapper, 2002; Hauser, 1998; Chiesa et al., 1996). Rewards should not only be financial, but also emphasise recognition (Loch and Tapper, 2002). Type of activities and process aspects Also the process aspects of the PMS (e.g. standards/norms to measure performance against) should be adapted to the activities features. For example, if the ‘‘number of patents’’ is the indicator for individual performance measurement, two opposite norms could be used to measure performance against: explicitly recorded standards or subjective and implicit norms. The choice should tend to vary with the characteristics of the work. Reasonably, for activities with a high uncertainty, the subjective techniques should be preferred because they take into consideration a wider range of relevant aspects (qualitative aspects and/or environment conditions, that are not under the control of the researcher). R&D strategy and dimensions of performance This relationship enlightens that the choice about the dimensions of performance to be monitored is driven by R&D strategy. In effect, the strategic objectives are the critical performances to be pursued to guarantee company’s competitive success. A PMS is useful if it manages to monitor these critical performances (or, at least, a part of them) that are assumed as dimensions of performance of the system itself. R&D strategy and process aspects The PMS process aspects can be influenced also by the choices the company makes in terms of R&D strategy and objectives. For example, whether the firm means to undertake a time-based competition that requires short time to market (TTM) and technological pioneering behaviours, it is necessary that coherent decisions are made in terms of timing and frequency of the measurement, so that the PMS is capable of supporting R&D and top management decision processes.

218





 

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

the choice of the R&D activities to be internally carried out and their organization in a certain structure; the PMS’s available resources can influence the PMS’s objectives, because they can represent a constraint that limits the informative completeness of the system and thus the achievable purposes (and the dimensions of performance to be monitored); the PMS’s objectives are driven also by the R&D management, e.g. because of a leadership style that stresses more or less the motivational aspects of the system; as far as, the objectives are driven by the type of activities (i.e. a high level of creativity and complexity can require a particular attention to researchers’ motivation and coordination); all the contextual factors influence the design of the PMS’s elements (see Box 1 for some examples of relationships); all the PMS’s elements are interrelated and in close relationship, thus stressing their reciprocal dependence and the systematical nature of the PMS.

Explanations concerning some relations between the contextual factors and the PMS’s elements are illustrated in Box 1. Whether it may appear obvious that the structure of the system should be defined in order to collect information about the performance of all R&D organizational units, the link, for example, between the purpose of the measurement and the PMS’s dimensions of performance is less evident. The explanations in Box 1 try to clarify these links and enlighten further complexities deriving from another contextual factor, i.e. the type of activities that can also influence the definition of the PMS’s process aspects. 3. The case study and the framework application The framework has been first applied in a real context with the aim of guiding the definition of a PMS in a pharmaceutical company, NiKem Research. The problems emerged during the application of the framework have indicated the need to integrate it with a list of key questions, that replicate its overall structure and can be used to guide the PMS design coherently with the framework underlying logic. This has shown to be an important step towards the practical orientation of the model itself. Fig. 2 summarizes the results of this empirical investigation and allows for a preliminary explanation of the reference model underlying logic. After a brief description of the company’s profile, the contextual factors and the associated PMS’s elements are illustrated more in detail. 3.1. Company’s profile NiKem Research is an Italian company founded in 2001 that operates in the field of pharmaceutical research, mainly offering services with a high technical and scientific content and capable of supporting the new drug’s R&D process. Its origins could be traced back to the merger between GlaxoWellcome and SmithKlineBeecham, two big pharmaceutical companies with Italian operative units, respectively set in Verona and Milan. The merger took place in 2000 and brought to the birth of GSK (GlaxoSmithKline); the top management established that all the activities would be aggregated in Verona, thus implying the partial transferring of SmithKlineBeecham’s personnel from Milan to the Venetian city. A part of SmithKlineBeecham’s discovery team, that was going to be totally cancelled, decided not to throw away the excellent competences in medicinal chemistry they have developed during more than 20 years. They thus presented a spin-off project to GSK aimed at the creation of a small, highly specialized company, capable of supporting pharmaceutical firms’ R&D processes with specific services. 3.2. Contextual factors 3.2.1. General environment and competitive context When NiKem Research was born, the features of the pharmaceutical market were particularly propitious for the birth and the success of this kind of ventures. In fact, the increasing complexity,

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

Fig. 2. The framework application.

219

220

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

costs, risks and investments, typical of new drugs’ discovery and development processes, were forcing big pharmaceutical companies to reorganize their R&D structures and to outsource R&D activities to smaller and more flexible firms (Muffatto and Giardina, 2003). In order to understand the characteristics of the service offered by NiKem Research, it could be useful to briefly describe the structure of the research and development process in the pharmaceutical industry (for more details see Muffatto and Giardina, 2003; Chiesa, 2003; Katzung, 2000; Paoletti et al., 2001). The identification of a new drug in the pharmaceutical industry is the result of a complex R&D activity that is basically composed of two steps: the first one relates to the identification of a new compound (drug discovery), the second is concerned with the development of the discovered compound (drug development), followed by approval of the qualified regulatory authority. The drug discovery phase can be split into two fundamental stages: the biological activities and the chemical activities. The former is composed of the target identification and the target validation activities, while the latter includes the lead generation and the lead optimization tasks. The identification of the target is the starting point of the process, and it consists in the pointing out of a particular molecular target (e.g. a gene or a protein), that is responsible for a specific pathology. The next step of the biological activities (target validation) implies a deep investigation into the characteristics of the identified target in order to discover how it acts and how it generates the disease. Once the target has been identified and the process through which it works has been understood, it is necessary to discover, through an intense screening activity, a set of compounds (named leads) that, interacting with the target, may have positive impact on the pathology. This stage of the discovery phase is called lead generation, and it ends with the identification of a limited number of compounds that need to be optimized. The lead optimization phase of the discovery process is a very critical, risky and innovative one that requires excellent competences in the field of medicinal and combinatorial chemistry. It basically consists in the analysis of the generated chemical compounds in order to identify the most balanced chemical entity in terms of potency at the molecular target, selectivity, efficacy, defects, pharmacokinetic and toxicological properties. To evaluate the safety and effectiveness of the optimized pharmaceutical compound before it is destined to the market, it has to be carefully tested, according to very severe and strict procedures. This phase is called drug development, and it is divided into two macro-activities: the pre-clinical and the clinical tests. The former consists of a set of laboratory tests (called in vitro) and tests on animals (named in vivo), with the aim of evaluating the drug’s safety and toxicity level. The clinical tests, on the other hand, are articulated into three sequential steps: phases I, II and III. They are conducted on human volunteers in order to fix the proper dosage and to assess the drug’s effectiveness and tolerability. Once the development stage is ended, the results of the tests are gathered and analyzed, and, eventually, a request for the market approval of the new drug is forwarded to the qualified authority, that may approve its commercialisation or not. Going back to the recent evolutions in R&D processes of the pharmaceutical industry, it should be remembered that, with the ending of the human genome project, all the 25,000 human genes were identified; about 3000 were recognized to be ‘drugable’ or ‘physiologically relevant’ molecular targets. They represented, together with the diffusion of high throughput screening (HTS) techniques, an enormous incentive for pharmaceutical companies to intensify their screening activity; this brought to the identification of a huge number of lead compounds that needed to be optimized. It was especially the large scale to be achieved in the tasks of lead generation and optimization in order to keep competitive, their high level of inherent risk and costs, and the competences they require, that forced more and more big pharmaceutical companies to outsource these activities to highly specialized and flexible organizations, thus externally acquiring the missing skills and critical mass.1 3.2.2. Strategy and objectives In the propitious context just described, the spin-off project was approved by GSK, that decided to transfer, through a management-by-out operation, personnel, laboratories and other facilities to the 1 Due to the fact that NiKem almost collapses on its R&D function, in Fig. 2, that represents the applied framework there is no relevant distinction between business strategy and R&D strategy.

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

221

new-born NiKem Research, and to support the start-up phase of its activity with a research contract of 2 years, capable of covering the 85% of its start-up costs. In 2001, on August 1st, NiKem Research was officially established and started working as service provider, highly specialized in the activities of ‘lead generation’ and, especially, ‘lead optimization’, with a staff of 25 researchers. NiKem Research is thus a typical example of Contract Research Organizations (CRO), where ‘contract research’ is defined as ‘. . . the activity by which a client hires the services of an external organization to carry out a specific piece of R&D’ (Haour, 1992). In 2003, NiKem Research’s top management realized that the company’s business model, completely based on the provision of specific services, would reveal itself too fragile and not capable of assuring high value and steady profits on the long term. This was mainly due to the fact that the pharmaceutical market is traditionally exposed to significant sales fluctuations that negatively reflect on the performances of small CROs like NiKem Research. Another threat was represented by those service providers operating in emerging countries like India, Russia or China; it was thought they would be capable, in 4–5 years, of offering high quality services, comparable to European or American companies’, but at a significantly lower price (from 1/3 to 1/5). This convinced NiKem Research’s top management to modify the firm’s strategy and to dedicate part of the available resources to internal research activities, with the aim of generating novel leads to be internally optimized and developed until they are ready for the clinical stage of the Drug Development process. In order to carry out the pre-clinical tests, pharmacological competences are necessary; this has brought NiKem to collaborate with the ‘‘Istituto Nazionale dei Tumori’’ and with various Italian universities (e.g. Firenze, Ferrara). At that stage of the R&D process, the leads would be licensed to or partnered with big pharmaceutical companies that own the financial resources and the complementary assets necessary to complete the development phase and to introduce the new drugs into the market. The internal research activity assures NiKem Research another advantage, i.e. the possibility to have a buffer of highly specialized human resources that can be used to face sudden and transitory increases in the request for discovery services. Even if the service provision and the internal research activities require that very similar types of scientific challenges are faced and that similar technologies and competencies are acquired (e.g. pharmacokinetic, preliminary toxicology, MTS, additional ADME profiling), they significantly differ when we consider the critical success factors where NiKem Research has to excel in order to gain a competitive advantage. In the business of discovery services, the objectives to be pursued are the quality level of the service (at a cost level aligned to the competitors’ one), the respect of the promised delivery time (punctuality) and the confidentiality in the management of the information concerning the clients’ compounds. On the other hand, these variables do not represent competitive pressures in the strategic area of internal research, where it is far more important to adopt a long-term perspective and to try to gain the leadership in the identification of novel candidates to clinical development. In this area, even if concrete results are obviously necessary, time constraints are less severe due to the fact that specific clients’ pressures are not relevant. After all, a critical objective has been identified at the overall company level, that becomes another specific objective for each business area. This critical dimension consists in the building of an external technology reputation. This factor is very important first of all because it provides contacts with new potential clients in the business of discovery services. In addition, it provides relations with big pharmas and biotechs in order to improve the internal research pipeline. 3.2.3. R&D activities According to the traditional classification concerning the macro-phases which compose the R&D process (see, e.g., Chiesa, 2001), it is possible to classify NiKem Research’s activity as basic and applied research and therefore strongly creative, uncertain and quite unstructured. 3.2.4. R&D organization and management It can be stated that NiKem Research is organized on the basis of a matrix structure. The researchers are grouped in departments, according to the discipline in which they are specialized. The ‘‘departmental dimension’’ of the matrix structure is necessary in order to assure a high level of

222

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

specialization in the company’s core scientific competences (the main departments are: medicinal chemistry, analytical chemistry, computational chemistry, screening). Within each department, a director is appointed. He is responsible for the management of the activities and the resources in the department and for its scientific results. Furthermore, each department includes a series of laboratories. They represent a lower level along the departmental dimension and are defined according to ‘‘scientific specialization’’ criteria. A ‘‘laboratory head’’ manages the activities of this organizational unit. On the other hand, the ‘‘project dimension’’, by means of inter-departmental teams, is important to put together the transversal skills that are necessary to produce the company outputs. Within each project, a team leader is appointed as well; he is, in general, the researcher with the major experience, or charisma, or specialized in the most crucial discipline for the project success. He is responsible for the results achieved by the team. In this matrix structure, however, a major organizational power is assigned to the department structure rather than to the project structure. This power can be in practice appreciated in terms of a wider possibility in the orientation of the researchers’ activities, through the definition and the valuation of their goals and responsibilities. After all, NiKem recognizes that human resources are crucial in order to achieve high quality services. Consequently, the leadership style is surely a participative one with the basic intent of obtaining people collaboration. 3.2.5. PMS objectives The leadership approach and the type of activities (i.e. their high level of creativity and complexity that requires a particular attention to researchers’ motivation and coordination) guide the choice of the PMS’s objectives towards a clear prevailing of the motivational purpose, even if also the aspect of diagnosing activities is not ignored. Particular attention has thus been paid to the alignment of the indicators with the specific goals and responsibilities of the people subjected to the measurement. This means that the system has to include only those factors that can be controlled by the evaluated people. This aspect is practically considered when the PMS’s elements are defined, in particular when the dimensions of performance and the related indicators are assigned to the different objects composing the PMS’s structure. In any case, it has been realized that, in order to enhance the motivational capability of the performance measurement system, the objectives assigned by a chief to a single researcher must be collectively defined (‘‘shared objectives’’). Furthermore, they must be measurable, even if they do not necessarily produce tangible outputs. However, a certain degree of performance ‘‘un-measurability’’ is expected because of the intrinsic nature of the company’s activity, that is basic and applied research, and therefore strongly creative, uncertain and quite unstructured. This circumstance influences the financial incentives (bonuses) that are coupled with the measurement in order to enforce the motivational purposes of the system. In fact, incentives are depending not only on the specific individual performances, but they are linked also to the company’s overall earnings. This link is justified on the basis of the assumption that each researcher (by means of his competences, behaviour, cultural values, etc.), contributes in any case to the company’s success, even if personal, concrete and measurable results are far from being evident. The purpose of diagnosing activities can be recognized in the project-progress monitoring. This type of control is formal and structured, by means of reports and meeting with the clients. The necessity of a formal approach can be appreciated when we consider in particular the provision of lead generation and lead optimization services. These stages of the R&D process are, in fact, very critical for the drug development success and involve customer assets whose intellectual property must be protected. Besides, the likely client’s criticism is increased by the fact that NiKem has relationships with companies in competition. Further complexity is caused by the long duration of the relationships. The formalization of the approach is thus considered a mean to limit the customer’s worries. 3.2.6. PMS resources NiKem Research small dimensions have an inevitable impact on the resources available to design and then implement the PMS. Above all, human resources availability has been identified as the most

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

223

important constraint in the PMS’s design2 as far as few people will undertake the activities necessary to the proper operation of the system. This circumstance has lead to the definition of a not very sophisticated system (i.e. including few dimensions of performance, few indicators, a simple structure), although representing a good starting point for a further deepening. 3.3. PMS elements 3.3.1. Dimensions of performance The choice of the dimensions of performance to be monitored is driven by R&D strategic objectives. In Fig. 2 are reported the selected dimensions of performance (one or more) corresponding to each strategic objective. For example, in the business area of discovery services, the ‘‘service quality level’’ objective has been translated into two dimensions of performance to be monitored in order to assure its achievement (i.e. service effectiveness and capability of acquiring new technologies and competencies), as far as the ‘‘service cost level’’ has been turned into a dimension of internal efficiency (i.e. respect of the planned service costs). On the other hand, in the business area of internal research, the dimensions of performance, coherent with the objective of ‘‘gaining the leadership in the identification of novel candidates to clinical development’’, are basically the quality of identified/ optimized leads and the capability of scouting new collaboration opportunities. 3.3.2. Indicators It is important to choose the correct qualitative and/or quantitative indicators in order to make the dimensions of performance operatively measurable. It would be important to select those performance indicators that allow for a simple and direct measurement and that, even if not questionable, can be collectively discussed. This indicates a clear preference towards quantitative indicators. Anyway, the characteristics of uncertainty, risk and un-measurability of R&D activities, as said above, makes it often necessary to integrate quantitative indicators with non-numeric and qualitative metrics. This tendency can be clearly recognized in the case of NiKem Research, where both these types of indicators are applied, even to measure the same dimension of performance. A joint use of quantitative and qualitative indicators has been established, for example, to measure the capability of acquiring new technologies and competencies, where the numeric indicator based on the ‘‘time needed to acquire new technologies’’ has been integrated with the qualitative appreciation of the international relevance of the technologies/competencies acquired. 3.3.3. Structure The structure of the system reflects the organizational R&D structure as described above. This means that the monitored objects are basically the ‘‘project’’ and the ‘‘department’’. In addition, the focus on the motivational purpose of the system leads to dedicate a particular attention also to the performance of individual researchers. The assignment of the selected indicators to the structure levels is driven by the basic purpose of the performance measurement system, i.e. motivating people. In other words, it is important to make a specific organizational unit responsible only for those performance indicators it can directly and completely influence. In particular, it has been recognized that the controllable factors tend to vary moving across the organizational levels. For example, the individual researcher cannot be held responsible for the success of a specific screening task (i.e. the quality of novel identified leads), due to the fact that the uncertainty of this activity is too high and out of his control (the chances a molecule has of becoming a lead are very low). On the other hand, it can be considered a responsibility of a research project as a whole, seen that the project manager can influence its output, e.g. by improving processes and by allocating resources to a good portfolio of research activities. The match between the indicators and the structure levels is a very critical process because of its motivational implications. Fig. 2 also reports the final output of the assignment choices. 2 Apart from our contribution, a ‘‘design team’’ has been appointed including the Chief Operating Officer, an external consultant and two researchers, respectively employed in the discovery services and in the internal research activities.

224

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

3.3.4. Process aspects It has been observed that the researchers, because of their high level of education, are very autonomous in carrying out their work, whose creative nature probably stresses this behaviour (they often say ‘‘we need to be autonomous in our work’’). The autonomy is typical of both the business areas. The respect of this working condition requires that the measurement system is not too strict and constraining. This need has influenced the choice of the norms that NiKem Research is going to implement. Subjective standards are, in fact, prevalent in order to encourage people initiative also in the case of results that are different from those agreed. For example, an actual number of patents that is lower than the standard number, can be anyway considered a good performance through subjective evaluations by superiors, that can be rightly less severe than a quantitative method objectively applied. Considering the timing of the system, it has been decided to assign the individual objectives at the beginning of the year. The evaluation of the objectives’ progressive achievement occurs about every four months with a final evaluation at the end of the year. 4. Conclusions and managerial implications This paper aims at making a step further in the understanding of the problems to be faced when defining a system for performance measurement in R&D units. According to this aim, a reference framework is identified, that represents a systematization of literature contributions in the field and describes the logical steps for a proper definition of the PMS for R&D, thus giving a ‘‘practical’’ guide that should help managers facing this challenge. The scheme here suggested explicitly establishes relationships between contextual factors (i.e. dimensions of the R&D ‘‘environment’’- R&D strategy, R&D organization and management, R&D activities-, measurement system objectives and resources available for the system design and implementation) and PMS’s elements that are reciprocally interrelated, coherently with the systemic nature of the PMS. Then, the framework has been applied to a company specifically selected in a context where technological innovation represents the critical source of competitive advantage. This is clear when we consider that NiKem Research, the analyzed firm, almost collapses on its R&D function that represents the 90% of the overall activities. Above all, the case study has surely provided an enriching experience regarding the aspects of system definition, thanks to the problems emerged during the application of the suggested approach. In particular, it was clearly perceivable the need to integrate the framework with a list of key questions, that replicate its overall structure and can be used to guide the PMS design coherently with the framework underlying logic. This has shown to be an important step towards the practical orientation of the model itself, allowing the company to deeply understand its own strategic objectives and the consequent dimensions of performance to be monitored. The choice of the specific indicators that allow the measurement of the dimensions of performance revealed itself to be a very critical task, at least as challenging as their assignment to the structure levels in order to consider the accountability and motivational implications. Anyway, the case provides a very detailed and concrete example of how the huge literature on performance measurement, which gives many suggestions in terms of dimensions of performance, indicators, process aspects, can be integrated within a system, internally coherent and adequate for a specific strategic context. In fact, various contextual factors emerge as basic aspects that can influence the measurement system intents. In particular, it has been recognized that dimensions of performance (and related indicators) should reflect the activity portfolio derived from the company’s strategy and its critical long-term objectives. Moreover, the main purpose that has brought to the introduction of the measurement system (i.e. ‘‘motivating people’’) has decisively guided its definition. This explains why particular attention has been paid to the alignment of the indicators with the specific goals and responsibilities of the people subjected to the measurement; the system has to include only those factors that can be controlled by the evaluated people. Furthermore, it has been recognized that the controllable factors tend to vary moving across the organizational levels. This means that the organizational structure has a great importance in the measurement system design. The organizational levels to be considered and matched with the

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

225

selected performance indicators, in fact, directly derive from the ‘‘matrix’’ structure applied in NiKem Research. In any case, the match between the indicators and the structure levels revels itself to be a very critical process because of its motivational implications. The type of activity (basic and applied research) and the peculiarities of the needed human resources (for example, their autonomy) are the other factors that have had some influence on the system design. It has been observed that the researchers, because of their high level of education, are very autonomous in carrying out their work, whose creative nature probably stresses this behaviour. The respect of this working condition requires that the measurement system is not too strict and constraining. This need has thus influenced the choice of the norms that NiKem Research is going to implement. Subjective standards are, in fact, prevalent in order to encourage people initiative also in the case of results that are different from those agreed. Finally, NiKem Research small dimensions have had an inevitable impact on the resources available to design and then implement the PMS. In particular, human resources availability has been identified as the most important constraint in the PMS’s design and implementation tasks. This circumstance has lead to the definition of a not very sophisticated system (i.e. including few dimensions of performance, few indicators and a simple structure), although representing a good starting point for a further deepening. Further investigations are surely necessary in order to appreciate the operation of such a system (‘‘is it accepted by the researchers?’’) and its eventual evolution. From this point of view, the framework proposed is dynamic in its nature: as far as the contingencies and contextual factors change, the PMS characteristics (i.e. the dimensions of performance monitored, the structure of the system, the process aspects) coherently change as well. This is a crucial point, given the dynamic nature of the competitive context and the rapidly changes in the rules of competition. The synthetic framework we have elaborated in this article fits the specific characteristics of NiKem Research, but it represents an important empirical basis for future analysis. First of all, we mean to further investigate whether it can be usefully applied to other Contract Research Organizations working in the field of pharmaceutical research, conducting other case studies on the matter. This will give us the opportunity to improve and correct the proposed scheme and, probably, generalize it, seen the similarities between CROs in terms of type of activities and strategy. Moreover, it would be interesting to study, by means of surveys, the possibility to adapt the framework to innovative companies working in other industries. This will probably require a deep change in the selected dimensions of performance (and related indicators), due to completely different types of activities and business models and, therefore, critical long-term objectives. Another relevant topic for future research concerns the integration of the R&D PMS with the company’s management control system, i.e. with the control of the overall economic, financial and non financial performance of the company. This, in fact, poses interesting problems concerning: the information flows; the style chosen for monitoring performance; the coherence between, on the one hand, the R&D PMS elements, and, on the other, the company’s PMS elements, particularly in terms of structure and process aspects.

Acknowledgement We thank Mr. Giuseppe Giardina, Chief Operating Officer in NiKem Research, for his contribution by means of a deep knowledge of R&D processes and control requirements. References Bremser, W.G., Barsky, N.P., 2004. Utilizing the balanced scorecard for R&D performance measurement. R&D Management 34 (3), 229–237. Brown, M.G., Svenson, R., 1988. Measuring R&D productivity. Research-Technology Management 31 (4), 11–15. Chiesa, V., 2003. La Bioindustria. Strategie competitive e organizzazione industriale nel settore delle biotecnologie farmaceutiche. ETAS, Milano. Chiesa, V., 2001. R&D Strategy and Organisation. Imperial College Press, London.

226

V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226

Chiesa, V., Coughlan, P., Voss, C.A., 1996. Development of a technical innovation audit. Journal of Production Innovation Management 13 (2), 105–136. Chiesa, V., Masella, C., 1996. Searching for an effective measure of R&D performance. Management Decision 34 (7), 49–57. Driva, H., Pawar, K.S., 1999. Performance measurement for product design and development in a manufacturing environment. International Journal of Production Economics 60, 61–68. Driva, H., Pawar, K.S., Menon, U., 2000. Measuring product development performance in manufacturing organisations. International Journal of Production Economics 63 (2), 147–159. Freeman, C., 1976. Economics of Industrial Innovation. Pinter Publisher, London. Godener, A., Soderquist, K.E., 2004. Use and impact of performance measurement results in R&D and NDP: an exploratory study. R&D Management 34 (2), 191–220. Haour, G., 1992. Stretching the knowledge-base of the enterprise through contract research. R&D Management 22 (2), 177–182. Hauser, J.R., 1998. Research, development and engineering metrics. Management Science 44 (12), 1670–1689. Kaplan, R.S., Norton, D.P., 1992. The balanced scorecard measures that drive performance. Harvard Business Review 70 (1), 71–79. Katzung, B.G., 2000. Farmacologia generale e clinica. Piccin, Padova. Kerssens-van Drongelen, I.C., Bilderbeek, J., 1999. R&D performance measurement: more than choosing a set of metrics. R&D Management 29 (1), 35–46. Kerssens-van Drongelen, I.C., Cook, A., 1997. Design principles for the development of measurement systems for research and development processes. R&D Management 27 (4), 345–357. Kumpe, T., Bolwijn, P.T., 1994. Towards the innovative firm-challenge for R&D management. Research-Technology Management 37 (1), 38–44. Loch, C.H., Tapper, S., 2002. Implementing a strategy-driven performance measurement system for an applied research group. The Journal of Product Innovation Management 19 (3), 185–198. Muffatto, M., Giardina, G., 2003. Innovazioni nei Processi di Ricerca in Campo Farmaceutico. Economia & Management 6, 107– 121. Paoletti, R., Nicosia, S., Clementi, F., Fumagalli, G., 2001. Farmacologia generale e molecolare. UTET, Torino. Pappas, R.A., Remer, D.S., 1985. Measuring R&D productivity. Research-Technology Management 28 (3), 15–22. Pearson, A.W., Nixon, W.A., Kerssens-van Drongelen, I.C., 2000. R&D as a business—what are the implications for performance measurement? R&D Management 30 (4), 355–366. Poh, K.L., Ang, B.W., Bai, F., 2001. A comparative analysis of R&D project evaluation methods. R&D Management 31 (1), 63–76. Shank, J.K., Govindarajan, V., 1993. Strategic Cost Management: The New Tool for Competitive Advantage. The Free Press, New York. Schumann, P.A., Ransley, D.L., Prestwood, D.C.L., 1995. Measuring R&D performance. Research-Technology Management 38 (3), 45–54. Tipping, J.W., Zeffren, E., Fusfeld, A.R., 1995. Assessing the value of your technology. Research-Technology Management 38 (5), 22–39. Werner, B.M., Souder, W.E., 1997. Measuring R&D performance – state of the art. Research-Technology Management 40 (2), 34–42.