Indicators and outcomes of Canadian university research: Proxies becoming goals?

Research Policy 35 (2006) 1586–1598

Indicators and outcomes of Canadian university research: Proxies becoming goals? Cooper H. Langford a,b,∗ , Jeremy Hall b , Peter Josty a , Stelvia Matos b , Astrid Jacobson b a

The Centre for Innovation Studies, 125 Alistair Ross Technology Centre, 3553 31st Street NW, Calgary, AB, Canada T2L 2K7 b The University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4 Available online 27 October 2006

Abstract This paper analyzes Canadian Government attempts to encourage and measure commercialization of university knowledge for socio-economic improvement. Universities are regarded as major, insufficiently exploited, repositories of knowledge. Here, paths by which knowledge can be transferred across institutional boundaries, and various input and output/outcome measures are identified. Available Canadian data are evaluated and a detailed quantitative and qualitative study of one institution is presented. Three key issues emerge: (1) current proxies focus on licensing and spin-off, and do not measure several important paths of knowledge flow; (2) most readily available proxies are derived from aggregate data and are inadequate to fully reflect the idiosyncratic and pathdependent nature of innovation; (3) if the goals and incentives of the actors in the ‘triple helix’ are skewed or misinterpreted by indicators, universities and firms may engage in counterproductive activities. We propose additional indicators that might help to prevent one measurable dimension from becoming the policy driver to the detriment of the overall goals. © 2006 Published by Elsevier B.V. Keywords: University–industry knowledge transfer; Indicators; Innovation policy

1. Introduction In 2002, the Government of Canada released an innovation policy report ‘Achieving Excellence’ (Industry Canada, 2002; p. 52) that included a specific initiative with respect to universities and commercialization. “Leverage the commercialization potential of publicly. . .funded academic research. Support academic institutions in identifying intellectual property with commercial potential and forging partnerships with the private sector to commercialize research ∗

Corresponding author at: The University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4. Tel.: +1 403 220 3228; fax: +1 403 289 9488. E-mail address: [email protected] (C.H. Langford). 0048-7333/$ – see front matter © 2006 Published by Elsevier B.V. doi:10.1016/j.respol.2006.09.021

results. . . Academic institutions would be expected to manage the public investment in research as a strategic national asset by developing innovation strategies and reporting on commercialization outcomes. . . An evolving partnership would see universities more aggressively contributing to innovation in Canada, in return for a long-term government commitment to their knowledge infrastructure.” A particular novelty is the expressed expectation that academic institutions would ‘manage’ the public investment for commercialization outcomes. Consequently, the government entered discussion with the Association of Universities and Colleges of Canada (AUCC), leading to a framework agreement on federally funded research (AUCC, 2002). Key points concerning targets, responsibilities, and evaluation include the following:

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

“. . .Canadian universities have committed. . . [to] triple their commercialization performance. . . subject to. . . government investments (p. 1). Universities are responsible for the strategic coordination of the research efforts that will deliver these benefits. . . The parties also agree on. . . universities’ participation in the Statistics Canada survey of university commercialization (p. 2–3). AUCC agrees to produce a periodic public report that demonstrates the collective progress made by universities. . . [in] knowledge transfer including commercialization and innovation (p. 3).” Regarding the commitment to triple the amount of commercialization, the specific indicators implied are those of Statistics Canada’s Survey of University Commercialization (Read, 2003). These include revenues and expenditures related to intellectual property (IP) management, spin-off companies (and equity held in spin-off companies), invention disclosures, patent applications, patents awarded, new licenses and revenues from IP. Details of research funding are also reported (drawn from the agency’s Higher Education Statistics). Policies for university IP ownership, contracting and consulting are surveyed, but outcomes of such policies are not covered. The agreement posits an intriguing form of ‘triple helix’ relationship. The universities’ association and the federal authorities have specified the relationship between universities and industry, assuming that the government should drive university contributions to industry and national interests. Moreover, the core of the relationship between university research and national benefit is presumed to be spin-off firm formation and licensing of patents. Other relationships that include various forms of university–industry collaboration (contracts, consortia, and consulting) and the role of graduates are downplayed. Thus, while high-level policy may implicitly recognize the non-linear nature of innovation, the reality is that measurements are inherently based on a linear model of innovation. Usable statistics are recognized as, at best, proxies for measuring innovation (Freeman and Soete, 1997), yet there remains the danger that proxies become de facto goals and shape the evolution of relationships. The Canadian concerns are not unique. Similar interest in the return on investment of university research is widely expressed and the literature on technology transfer through spin-off and licensing is extensive. Mowery et al. (2004) review much of the US literature. In the UK, the Lambert Report (2003) emphasizes globalization and outsourcing of industrial research as an opportu-

1587

nity for universities. The ‘third stream’ discussion (outcomes that concern the use, communication, application and exploitation of knowledge) examines how government funding can support university–industry interaction. Molas-Gallart et al. (2002) provide a comprehensive review of possible indicators of ‘third stream’ university activity, supporting the ‘top-down’ development of a profile for application. They recommend that such development be carried out cautiously and experimentally. Following these lines of inquiry, this paper first reviews the various dimensions of university–industry interaction and provides a critique of currently available Canadian indicators. These indicators implicitly assume a linear understanding of innovation, capturing at best incompletely the reality of knowledge exploitation. An in-depth quantitative and qualitative study of university–industry relations in one particular institution is then presented. The case study reveals the unavoidable imprecision in standard data collection (e.g. different spin-off and patent counts) and underlines the difficulty of disaggregating the most rewarding exchange or translation activities quantitatively. Finally, we offer a cautious ‘bottom-up’ approach to the short-term improvement of the basket of innovation indicators in Canada. 2. What is known about commercialization and university research There is a substantial literature on innovation, commercialization and university research (e.g. Rosenberg and Nelson, 1994; Mansfield, 1998; Pavitt, 1998; Florida, 1999; Salter and Martin, 2001; Cohen et al., 2002; Pavitt, 2004) and recent comprehensive reviews simplify our summary (e.g. Reamer et al., 2003). We will assume that the objective of the government initiative is to stimulate economic and social benefits and not specifically to raise intellectual property revenues for the universities. Thus, the objective of commercialization involving university research is broadly interpreted as ‘innovation’, the process of bringing novelty into productive use (Schumpeter, 1934) that provides a net social benefit, what Baumol (1990) calls ‘productive entrepreneurship’ versus unproductive, rentseeking, behavior. To understand any science-based innovation, a complex process must be analyzed. It is rare for a single individual to participate in both the discovery of basic science and in the development of commercial products. Individual scientists and engineers may enter and exit the process at any stage and there is much reflexivity in the paths of knowledge communication. As Reamer

1588

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

et al. (2003, p. 15) observe: “In each trajectory of science, technology and product, the process that unfolds is idiosyncratic, entirely dependent on context, individual and organizational capacities, and unique circumstances”. One major study of British industry–university collaboration (Laursen and Salter, 2004) found that there was an over-estimation of universities as a direct knowledge source for innovation: “The interactions between universities and industrial firms remain largely indirect subtle and complex” (p. 1212). They further emphasize that managerial choice, industrial characteristics, and other heterogeneous factors play a major role in how knowledge is accumulated. Agrawal (2001) provides a comprehensive review of literature on university to industry knowledge transfer. Here lies the problem: the aggregate nature of currently available indicators ‘blurs out’ the idiosyncrasies and unique path dependencies. We are not suggesting that indicators are de facto inappropriate, but rather that they should be interpreted with caution; specifically it is necessary to examine any indicator in light of this problem. As will be shown in our case studies, there are significant outcomes that are not found in the aggregate data, and such benefits could be ignored if currently available proxies were allowed to dominate. 2.1. Knowledge migration pathways In this complex environment, how does technology knowledge migrate across institutional boundaries? Reamer et al. (2003) suggest five pathways: (1) Cooperative research and development. (2) Licensing or sale of intellectual property (IP) and spin-off. (3) Technical assistance. (4) Information exchanges. (5) Hiring skilled people. Numerous surveys of senior industrial research managers and their public statements (a current example is Lazaridis (2005), founder of Research in Motion) have advanced the intuitively plausible opinion that hiring skilled graduates is most important. Highly qualified personnel are the principal product of universities, and play a major role in developing absorptive capacities in firms (Cohen and Levinthal, 1990). The importance of talent was reinforced in the study reported below of one university and in recent cluster studies in the same geographical area (Langford et al., 2003, 2005). One of the few comprehensive studies of graduates, their businesses, and quantitative evaluation of the factors influ-

encing their choices is the Bank of Boston study (1997) of MIT graduates over a century. It suggests, in agreement with Florida (1999), that innovative businesses choose to locate near universities mainly for access to talent. The impact of graduates is not yet explored by Statistics Canada (Read, 2003), where the concern is intellectual property commercialization in the higher education sector. The overall aims of the agency’s science and innovation information program, however, include study of ‘linkages’ defined as “the means by which S&T knowledge is transferred among actors” (Read, 2003, p. 5). Formal collaborative activity with universities is significant for industry. Both the largest and start-up business sectors have a high propensity for such collaboration (Cohen et al., 2002). The KNOW study documents the wide distribution and sectoral variation of propensity to collaborate, and the significance of information exchange in Europe (Fontana et al., 2004). Institutionalized collaborative research with universities is well developed in Canada. The Pulp and Paper Research Centre was founded at McGill in 1925 (Pulp and Paper Research Centre, 1988), and has been followed by industry-sponsored centers on campuses across the country. A recent example is the multimillion dollar University of Windsor/Chrysler Canada Ltd. Automotive Research and Development Centre, the only research facility of its kind in North America (AUCC, 2005). Input proxies for some of these activities are embedded in reports of industry investment in university research discussed below. Tripartite university–industry–government consortia also exist, such as TRlabs (2005) in Western Canada. Off-campus labs sponsored by universities, industries, the federal and three provincial governments collaborate with five universities, but their funds are not captured in university funding reports. Their specialties are coordinated with local industry clusters and concentrations. At a next level, federal and provincial granting agencies have sponsored university research chairs in collaboration with individual firms and industrial consortia across the country, ranging from studies on risk, to asphalt technology, to quantum computing. Some chairs have multiple sponsors. Chairs represent a longterm commitment to collaboration that can facilitate ‘translation’ at the university–industry interface. Many university–industry collaborations focus on individual projects. These may form the largest part of the industry research sponsorship reported by universities. Many of these are stimulated by government participation but matching funds are reported separately in government sponsorship.

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

Canada also has a major federal research sponsorship scheme to promote collaborative research in areas of national importance, the ‘Networks of Centres of Excellence’ (NCE, 2004). This program fosters university research networks across the country, with a requirement for active engagement with user partners from industry or the public sector. Some reported university IP arises from research in these networks and much of the investment is embedded in reports of federal funding of university research, separated from industry contributions in the data. This program establishes long-term networks where user partners can provide insight into key problems and affect research agendas. Licensing and sale of IP and spin-off formation is the pathway on which the AUCC–Industry Canada agreement focused. It is also the central concern of the Statistics Canada surveys mentioned above. Output and outcome indicators are available for this area. Time series can be derived on a year-by-year basis from reports made by university IP managers to the Association of University Technology Managers (discussed below). An important aspect from our study confirmed the extent to which IP transfer need not involve the institutional IP managers. This brief review must mention the subtle pathways of information exchange (e.g. use of publications and conferences) and technical assistance (e.g. access to specialized equipment). There is little current and systematic information on the Canadian experience. Recent American data from the Carnegie Mellon Survey on industrial R&D (Cohen et al., 2002, p. 1), documents the importance attached by industrial research managers. Their extensive report is summarized by observations that validate all of Reamer categories: “Contrary to the notion university research largely generates new ideas for industrial R&D projects, . . . public research both suggests new R&D projects and contributes to the completion of existing projects in roughly equal measure. . . The results also indicate that the key channels through which university research impacts industrial R&D include published papers and reports, public conferences and meetings, informal information exchange, and consulting.” Cohen et al. (2002) found that public research is critical to a few industries and important across much of the manufacturing sector. Cluster studies (Langford et al., 2003, 2005) and analyses of patterns in one institution in Canada are consistent with the American data. Key pathways may be outside the areas for which ongoing statistical data are available.

1589

3. A critique of presently available indicators 3.1. Limitations on conceptualization of Canadian indicators Following Jaffe (1998), we categorize performance measurement indicators as being concerned with inputs, outputs and outcomes (impacts1 ), The indicators can operate at three levels of representation: measures of (direct), proxies for (indirect), or correlates of (linked by some statistical hypothesis) the phenomenon under analysis (Jaffe, 1998). Input measures have obvious limitations, particularly because they are concerned with intent rather than success. The awarding of competitive peer reviewed research grants is perhaps a good input proxy, but still a measure of promise and not a guarantee of output. Industry sponsorship investment can be used as an input proxy for industry-driven collaboration, the ‘market pull’ driver of knowledge transfer (Landry et al., 2002). Output and outcome indicators deal with results. Publication, patent, and citation data are widely used and available output proxies. Available outcome (impact) indicators are common proxies for innovation, aligned to the pathway passing through technology licensing and spin-off creation. They suggest the linear model. Combined with input measures of R&D funding, output measures of publications and citations, outcome indicators such as licenses and spin-off formation are clearly suggestive of the linear model, an approach widely discredited (Nightingale, 2004). Unfortunately, this can produce a serious overall distortion. For example a recent review of the literature (Guena et al., 2003) reinforces that university–industry interactions are most important in forms where they require the reflexive interactivity captured by more modern models of innovation such as ‘chain-link’ (Kline and Rosenberg, 1986), Mode 2 (Gibbons et al., 1994), and Fifth Generation (Rothwell, 1992) innovation models. According to Nightingale (1998), the linear model of scientific output leading to commercially viable technology fails to explain key features of innovation such as the importance of tacit knowledge, industrial science, non-science based technology, industrial differences regarding scientific requirements and why sometimes technology precedes the science that explains it. He further notes that the linear approach “reinforces a constricted view of the practical relevance of basic research by concentrating on direct and more

1

Statistics Canada distinguishes outcomes (medium-term) from impacts (longer-tem). See Earl et al. (2004).

1590

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

easily measurable contributions, to the neglect of indirect ones” (p. 691). Yet most outcome proxies are implicitly linear. For example, two of the most common measures, patent and patent citation analysis, are based on the assumption that output proxies of explicit knowledge (patents and papers) lead to increased knowledge that eventually finds commercial benefits. Pavitt (1998) concluded that patents granted to universities provide only a partial and distorted picture of contributions of university research to technical change (although patent citations may be more useful). Note that the critique of available and widely used indicators does not assert that they lack value as proxies or correlates of certain aspects of the innovation process, but that there is a danger in generalizing a partial analysis. Emphasis on output and outcome measures relevant to one path of knowledge exchange can affect incentives, long recognized as influencing how individuals respond to organizational goals (Jensen and Meckling, 1976; Eisenhardt, 1989) and more recently on knowledge transfer (Szulanski, 1996; Osterloh and Frey, 2000; Hall and Sapsed, 2005). Henderson et al. (1998) for example found that during the last few decades, universities have been under increasing pressure to translate their work into privately appropriable knowledge. In response, there has been notable increases in university licensing. They also found that university patenting greatly increased from 1968 to 1988, at a greater rate than university spending. However, during this period the patenting quality may have decreased, begging the question that universities may have been patenting because it became perceived as a measure of success—‘patenting for the sake of patenting’ rather than to protect commercially viable intellectual property. Other proxies of this path such as licensing, spin-off creations, and the establishment of university transfer offices may suffer from similar problems, where these proxies in practice become the goal. Such phenomena have been conceptualized as institutional isomorphism. According to DiMaggio and Powell (1983), organizations engage in isomorphic behavior in response to environmental pressures, with the aim of gaining legitimacy. Coercive isomorphism “results from both formal and informal pressures exerted on organizations by other organizations upon which they are dependent and by cultural expectations in the society within which organizations function” (p. 67). This includes state regulations and laws, rules and standards imposed by non-governmental entities, as well as implicit pressures from stakeholders. ‘Mimetic’ isomorphism occurs under uncertainty: “When organizational technologies

are poorly understood, when goals are ambiguous, or when the environment creates symbolic uncertainty, organizations may model themselves on other organizations” (p. 69). It may occur unintentionally and indirectly, or be explicitly, diffused by organizations. In both cases, organizations mimic other organizations that are perceived as both similar and more legitimate or successful—i.e. benchmarking, ‘herd mentality’, or to use Aldrich and Fiol’s (1994) title ‘fools rush in’. In this case, if enough dogma emphasized the importance of particular measurement proxies, both coercive and mimetic (i.e. response to the inherently uncertain nature of innovation) effects may lead to these proxies becoming the ultimate goal rather than being used for their intended purpose. An ‘efficient market’ perspective would counter the ‘patenting for the sake of patenting’ isomorphism argument by emphasizing that in the long-run industry discourages inappropriate measurement proxies. However, this perspective regards firms as homogeneous, disregarding strategic choice, idiosyncrasies, context, individual and organizational capabilities and unique circumstances, all of which have been identified above as key factors in industry–university collaboration. 3.2. Canadian data sources The data presently available to assess and benchmark Canadian university research have input and output proxy and outcome correlate components that correspond to Jaffe’s (1998) categories. The oftenemphasized input side includes the Canadian Association of University Business Officers (CAUBO) reports, used by Statistics Canada to derive university research expenditures. The university research inputs are broken down into contributions from government granting agencies, business and foundations. Illustrative data for selected leading institutions for 1 year appear in Table 1. It is important to remember that the Canadian funding system is fundamentally based on project grants. The reports do not include general university funding (GUF). Faculty salaries aside, GUF is relatively unimportant in Canadian universities and does not function as a discriminator. Note that input data relates to outputs and outcomes only after time lags. The simplest lags are from funding to publication (4–5 years) and to citations indicating uptake (5–6) years (Crespi and Geuna, 2004). These lags do not take into account the complex interactions among distinct technical elements needed for industrial innovation (Valentin and Jensen, 2002). Thus, a minimum lag for government funded

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

1591

Table 1 Inputs to university research, 1999–2000 (CDN$ 000) University

Income Governmental

Dalhousie McGill McMaster Queen’s University of Toronto Waterloo Western University of Alberta University of Calgary UBC

Non-government donations, grants and contracts

Federal government

Provincial government

Other provinces

33,020 87,010 54,180 36,292 146,709 30,501 39,345 81,017 46,076 79,031

5,973 27,184 14,284 8,813 57,881 15,071 10,509 59,293 35,428 17,959

149 0 0 81 138 300 0 5830 74 1134

Foreign 7,552 7,093 0 946 16,558 223 5,524 446 972 4,220

Total income

Individual

Business enterprises

Foundations

0 4409 0 270 1983 386 5175 959 1377 600

22,980 21,425 14,357 22,367 65,876 12,200 16,931 22,141 20,873 36,913

N.A. 32,616 17,380 4,499 30,448 1,077 6,558 5,994 8,353 7,442

69,674 179,737 100,201 73,268 319,593 59,758 84,042 175,680 113,153 147,299

Source: CAUBO. These data were provided by the Office of Research Services, University of Calgary.

basic university research outcomes must be at least 4–6 years. The lag for work directly sponsored by users will depend on the nature of the sponsor–researcher relationship. At the other end of the spectrum, the only systematic source of output and outcome data directly related to user involvement is the Association of University Technology Managers (AUTM) reports and the parallel, if less frequent, Statistics Canada surveys.2 AUTM reports include the proxies of invention disclosures, patent applications filed, patents awarded, licenses, and the outcome measures of revenue, active licenses, and spin-off firms. A sample of AUTM data from leading research universities in Canada appears in Table 2. The data sources prejudice indicator development in favor of the single path of a university technology patent being licensed to a user or serving as a basis for formation of a spin-off firm. An indicator (not available for Canadian Universities) developed for other jurisdictions is citations of university publications in patents (Jaffe et al., 1993). Another is patterns of co-authorship between university scientists and those in other sectors. A study of the Canadian pattern of co-authorships (Godin et al., 1998) is dominated by university–hospital linkages. This probably reflects the close links of medical faculties and teaching hospitals and consequently

2

Information on university IP policies is available in the Statistics Canada Surveys (Read, 2003). Policies vary across Canada. For example, ownership regimes vary from full university ownership to full creator ownership. However, economic regimes in which technology transfer offices participate commonly arrive at a more or less even split of benefits between creators and the institution.

illuminates only a very special sector. Citation of academic papers in patents might be more revealing (Narin et al., 1997; Pavitt, 1998; Meyer, 2000, 2002), but may be limited because citations appear in patents for many reasons. A significant number of citations are identified only in the patent drafting and application process and have little to do with the origins of the patent. In addition, only a fraction of patents are successfully exploited. The available indicators of outcomes of Canadian university research are strong with respect to the pathway passing from input proxies through technology transfer offices licensing and spin-off creation, the pathway approximating the classic linear model of innovation. Table 2 shows AUTM data for fiscal 1997. It includes license revenues to indicate the innovation successes. Given additional data on the success of spin-off companies, we would have a very strong profile that closely approaches a direct measure of one path, producing a serious overall distortion towards the linear model of innovation. The weakness of indicators of the more subtle phenomena associated with the other four pathways undermines both theoretical and policy analysis. The popular indicator of patent citation to academic publication is at best a correlate. At least in Canada, co-authorship across the university–industry interface is minor. The most direct proxy available is the level of industry sponsorship of university research, an input indicator. However, this surrogate deserves to be unpacked, especially if government policy primarily supportive of inputs to university research is to be linked to increased commercialization outcomes.

1592

Table 2 Data for selected universities from AUTM survey, 1997 (ranked by gross licensing revenue) Year of first 0.5 FTE tech. trans. professional

Total sponsored research (000 US $)

University of Alberta UTI/University of Calgary University of Toronto University of British Columbia University of Waterloo Queen’s University McGill University University of Montreal McMaster University University of Western Ontario

1987 1985

94,676 72,250

1980 1984

Invention disclosures

US patent applications

License and options executed

Gross license revenue (000 US $)

License and options yielding revenue

Legal fees expended (US $)

Legal fees reimbursed. (US $)

US patents issued

85 72

44 13

34 31

3053 2074

41 59

360,899 304,946

102,921 304,419

12 6

205,721 99,009

112 100

60 52

31 19

1606 865

53 59

271,895 686,361

157,249 593,738

7 18

1990

36,261

15

8

14

845

31

89,858

28,027

9

1984 1990 1990

45,630 125,148 126,579

34 60 28

36 33 22

5 15 26

556 311 309

19 30 11

429,783 434,127 125,679

173,995 N.A. 35,132

3 N.A. 5

1987 1995

55,180 50,563

15 64

1 10

5 16

295 46

53 16

0 57,787

0 N.A.

1 5

These data were provided by University Technologies International (UTI), Calgary.

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

Institution

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

4. Quantitative and qualitative study of one institution 4.1. University faculty survey results The University of Calgary is a comprehensive university with doctoral programs in most disciplines and professional faculties including medicine, law, business, architecture, and education. The statistical tables accompanying this paper place it as a middle institution among the ten most research-intensive Canadian universities. The university provided us with access to research and faculty data. A questionnaire was sent to 1314 full time faculty members in 2004 in both hard copy and via email. The response rate was 18.4%, with the margin of error calculated to be 0.057 at 95% confidence. Thus, the data may represent opinions of the faculty overall, but will not be significant for breakdown by individual faculty areas. The lead question was “what is the best way to achieve societal benefit for research?”. One hundred and sixty-nine respondents selected publication in the open literature, 140 collaboration with an organization able to use their ideas, and 25 starting their own new business or venture. (Some marked more than one option.) In contrast, 41 (17%) reported that they had launched a business or other venture generating employment or revenue. It is probably significant that the largest fraction (31/41) came in similar numbers from medicine, science, engineering and social science, with the remainder distributed over six faculties. Consulting and services were the two largest activities with manufacturing making a significant contribution. As expected, distribution of employment levels resembled a power law. Four were in the 10–100 employee category (two service and two manufacturing). Thirty-seven firms were reported to be currently operating, reflecting a high survival rate. An additional 20 responses indicated plans to launch a venture, and 34 respondents reported assisting a venture start-up. Completing projects and exploiting experience were reported significantly as motivating factors. Thirty-one businesses and other ventures formed in the last decade were reported in this sample. AUTM data for ‘spin-off’ creation report 16 firms from Calgary during this period. This underlines that the ‘standard pathway’ via technology transfer agencies is not the unique source of university firm formation as reported elsewhere (e.g. Meyer, 2002, 2003). In Europe, more patents with university contributions appear to be owned by firms than by universities (Crespi et al., 2005). The ‘standard pathway’ would appear to be the minority pathway.

1593

Table 3 Reasons for collaboration with outside organizations Monetary reasons Advancement of science Need to apply knowledge to practical applications Complementary assets Train/support students Need to exploit expertise Identification of market opportunities

20 19 10 9 8 6 4

The nature of economic benefits derived by firms or other organizations from university faculty research or consulting assistance was reported as follows: • Development of a new product or service (20%). • Improvement of an existing product or service (17%). • Design of a new technological process or improvement of an existing one (20%). • Entry into a new market or expansion of an existing market (16%). • Design of a new administrative system or improvement of one (10%). All of the Schumpeterian (1934) categories of innovation are represented except the introduction of a new source of raw material. Faculty contributions are diverse. Respondents identified collaborative relationships with 85 different firms and organizations, 45 businesses and 40 with government and not-for-profits. The motivations reported by faculty members for collaboration with outside firms and organizations are also diverse (Table 3). 4.2. University annual reports The University of Calgary provided aggregate data from the 2003 faculty annual reports submitted by all full time faculty members (1403 reports). These reports identified 81 instances of “technology transfer”. Leaders were: • Engineering: 35 (0.28 per faculty member). • Medicine: 14 (0.03 per faculty member). • Environmental design3 : 7 (0.28 per faculty member). It is commonly assumed that consulting is underreported, but 430 instances of consulting activity were reported. Intensive involvement was:

3

This faculty includes architecture, industrial design, environmental sciences, and planning.

1594

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

• Environmental design: 2.68 per faculty member. • Kinesiology: 0.81 per faculty member. • Engineering: 0.57 per faculty member. Reports of patents, copyright registrations, trademark, and industrial design registrations number 101. Leaders were: • Engineering: 33 (0.28 per faculty member). • Medicine: 37 (0.09 per faculty member). • Science: 30 (0.14 per faculty member). The University’s technology transfer office reported filing 12 patent applications in 2003/2004. The startling discrepancy indicates that formal, AUTM reported, technology transfer administration does not capture all IP protection activities, as is reported in other studies (Meyer, 2002; Crespi et al., 2005). In these, patenting of university related inventions by firms exceeds patenting by universities. Unfortunately, the Calgary reports do not name patent assignees. 4.3. Case study summaries The available indicators for Canada are based on aggregated data that represents either a proxy or a correlate of the process leading to the desired goal. To achieve a more intimate perspective for evaluation of indicators, we conducted fifteen interviews with members of the university community who had been participants in wellknown innovative processes and with several industrial and community members who had interacted with the university. The experiences canvassed included examples related to all five pathways mentioned above (see Table 4). Three major research consortia were identified, two in science and one in engineering, all in pre-competitive

research. The number of sponsoring companies exceeds ten in two of the cases. Research results are available to all sponsors with the exception of a few specific projects sponsored on contracts separate from consortium membership. Two produced outcomes primarily related to processes and the third contributed product technology. The scientists normally develop the research program options, but the sponsors set priorities among options. Knowledge arising is incorporated into internal company processes and it is difficult to disaggregate the contribution of consortium research to commercialization. However, one sponsor representative commented about an oil exploration technology: “one well pays for the entire cost of membership”. Sponsoring companies value the opportunities for contact with research students. The dollar inputs to the consortia form part of the indicator ‘reported industry sponsorship’. The activity identified with the largest economic impact involved an engineering professor holding an endowed research chair. During his career, he worked in industry and briefly in government before joining the university. His university work formed a key part of a process that required large-scale resources for its completion, and thus a large part of the work was carried out in industry. It opened oil reserves of 600 million barrels to exploitation. It is not possible to assign a specific part of the economic return to the work in the university since the success depended on the integrated effort, but the university work was clearly a critical component. It is difficult for statistical proxies to recognize this type of university–industry outcome. A successful case of licensing is medical. The inventor recognized the contribution of the universitytechnology transfer office in structuring the presentation of the invention, but identification of the licensee depended on his important ‘accidental’ contacts. As the invention process developed over the years, the

Table 4 Case study results Knowledge pathway

Type of example identified

Cooperative research and development

Major research consortia Direct participation in a large scale industry project

Licensing or sale of intellectual property (IP) and spin-off

Successful licensing to an established firm Spin-off with a university scientist involved

Technical assistance

Services from a unique facility

Information exchanges

Consulting Contributing to the development of effective practices Working with the infrastructure of a regional cluster

Hiring skilled people

In all cases, some form Recruiting research associates internationally who remain in a local cluster

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

technology transfer office played a critical role in management of the inventor–licensee relationship, facilitating the development of a good working relationship that contributes to knowledge flows that go well beyond the patents. The licensing company is multi-national and local economic benefits are royalties. However, the inventor identifies the greatest social return to his work in the emergence of a ‘Calgary protocol’ that greatly simplified and reduced the costs of diagnosis of the related widespread condition. This benefit was first harvested locally. The successful spin-off studied is a local biotechnology company created through partnership with major multi-national companies that invested and supplied technical and business assistance. The universitytechnology transfer agency played a central role in structuring relationships and management development. The university played a critical role by hosting the incubation stage of technology development, continuing well after company formation. The inventor remains actively involved, and key scientific personnel have been recruited locally. The inventor’s prior experience in industry contributed to recognition of the opportunity. The company currently has a market capitalization of $ 94 million and 48 employees. AUTM data are meaningful correlates of this type of activity. The successful case of technical assistance is based on a unique facility that is valued worldwide in the oil and gas industry, Calgary’s major industrial cluster. Tests in the facility play a role in implementation of one technology of enhanced oil recovery. This has direct impact on feasibility and cost for local and international implementations. The facility is also valued by local industry as a training site. The activity can be represented by data on repeat industrial sponsorship. An important example of consulting centers on development of novel business practices by the holder of an industry–government co-sponsored engineering research chair. The novel practices improved contracting procedures for large-scale activities. In reporting to the government sponsor, one industry user, among many, identified work that produced capital savings of $ 10 million. The approach has been described in a book and widely adopted. The City of Calgary is a representative public sector adopter. The chair is associated with a wellattended professional development program. An engineering department combines a role in the infrastructure of a local cluster with recruitment of highly qualified personnel into the region. The department developed an early strength in an area linked to a local industrial specialization. The departmental leadership undertook a focused campaign to achieve recog-

1595

nition in a major international association that attracted university, industry and government representatives to its conferences. This played a significant role in identifying Calgary as an industry centre, and the use of Calgary manufactured equipment in university research publications has contributed to the region’s reputation. As the reputation of the university program grew, it has been able to attract an international cohort of outstanding research students, with approximately fifty percent remaining in Calgary. Some aspects of this type of case are reflected in data from the Statistics Canada Survey of Doctoral Candidates (Statistics Canada, 2005). Consistent with the Bank of Boston study (1997), all of these examples explicitly emphasized the importance of highly trained personnel. These cases also recognize the importance of the intangible attribute of reputation. The proxies selected in current evaluations, however, do not capture this university–industry outcome. 5. Implications A key Canadian policy agenda is to improve social well-being through commercial innovation. It is assumed that university knowledge can create a net social benefit and productive entrepreneurship (Baumol, 1990). Universities have been identified as a major, yet insufficiently exploited repository of knowledge (Landry et al., 2002). The challenge is how to encourage productive knowledge transfer and accurately measure it. In this paper, we discussed knowledge transfer paths, as well as input and output/outcome measures now available. We identified three key issues regarding the relationship between institutional knowledge transfer and measurement proxies. First, current proxies and correlates do not measure several important paths of knowledge flow, and the Calgary data show that some key metrics may not cover all that they are presumed to report. Critically, a main contribution of universities is qualified personnel diffusing knowledge into the workforce, and increasing absorptive capacity once in the workforce. Second, the most readily available proxies are derived from aggregate data, yet it is well understood that innovation can be idiosyncratic, even within the regularities that arise in particular sectors. It is also recognized that firms are heterogeneous and concerned with appropriating benefits from their innovations. However, the statistical treatment is homogeneous. What are the implications of aggregated, homogeneous proxies for activities that are inherently heterogeneous? Is it essential to add qualitative research to statistical surveys in order to provide nuance to statistical evaluation?

1596

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

Third, goals and incentives determine how individuals respond. If the proxies are skewed or misinterpreted, universities and firms may engage in unproductive activities such as ‘patenting for the sake of patenting’, rather than for appropriate reasons such as disseminating knowledge that is likely to be commercially viable. Furthermore, there are major differences between industrial and academic incentives. Industry typically seeks competitive advantage, generally accepted as being derived from unique, valuable and difficult to imitate assets and capabilities (Barney, 1991). Heterogeneity and appropriability are important (Teece et al., 1997), with the goal of increasing shareholder value. Academics seek peer recognition, and thus may lack interest in appropriability. How policy aims to reconcile these different incentives, and how university assessment procedures will be changed to reflect better institutional knowledge transfer remains unclear (Molas-Gallart et al., 2002). Policy-makers in this context are seeking net social gain through knowledge commercialization. However, policy makers are also constrained to provide policies that are fair and accessible to all firms (i.e. homogeneous). While they may recognize firm heterogeneity, they are required by their mandate to play by homogeneous rules. Although problems with performance measurement are well known, in practice there remains the danger that the proxies ultimately become the outcomes/impacts. In the Introduction, we describe the goals articulated by the Canadian Government, but these are in fact measurement proxies. This problem links with the implicit assumption of a linear model of innovation, appealing only for its simplicity and access to available statistics. Are the broader, yet downplayed forms of university–industry collaboration (contracts, consortia, consulting and role of graduates) being discouraged because linear proxies are driving policy? In the long term, inappropriate government proxies may not influence industry, if one assumes that markets will discourage inefficient behavior. However, institutional isomorphism may occur if the wider knowledge base (in this case policy and academia) propagates the importance of the proxies—i.e. if there is enough dogma, industry may follow suit. This is particularly likely under highly uncertain circumstances, as is the case for much innovative activity. The result is that we may get what we measure, rather than encouraging university knowledge commercialization to maximize net social benefits. 6. Conclusions: improving indicators The immediate need is to find indicators that can be simply implemented and widely accepted and that func-

tion as proxies or correlates for the other four paths of the university–industry knowledge exploitation. Of presently available indicators, an input measure, industry investment in university research, is probably the best proxy for path 1 (university–industry cooperative R&D). The time lag, from expenditure to output, needs to be addressed. This industrial sponsorship indicator could be substantially and readily improved if universities would report industry sponsorship broken down into: (1) consortium and network membership contributions; (2) ongoing contributions (not endowments) to chairs; (3) renewal project sponsorship of the same university group.4 This would identify the strongest university–industry connections (Landry et al., 2002). In fact, these three proposed sub-indicators might also be useful correlates of effective information exchange and effective placement of skilled people. The types of interactions in these categories are the ones that allow development of translation skills, and our interviews repeatedly emphasized the benefit to industrial partners of accessing research students as a part of research collaboration. The creation of these three sub-indicators is appealing as an immediate step to improve the Canadian situation. Further qualitative research is required to validate the suggested correlations and the degree of their comprehensiveness. In 2004, Statistics Canada held a meeting on commercialization, measurements, and indicators (Earl et al., 2004). The discussions recognized that the agency’s survey of earned doctorates (Statistics Canada, 2005) does shed light on the human resource dimension and could be expanded. In addition, enhancement of the existing National Graduates Survey could be illuminating. Initiatives in this direction could add substantially to proxies for knowledge transfer through skilled graduate mobility. The direction is promising. In conclusion, indicators supplementing the study of ‘linear’ technology transfer are feasible in the short term while indicator research continues. These might avoid one measurable dimension becoming the sole driver of policy to the detriment of the overall goals. However, directions for indicator development suggested here are preliminary, leaving at least the pathways of special facilities and consulting still to be addressed. Moreover, the inherent complexity and heterogeneity of innovation 4

cal.

One university research officer regards this sub-division as practi-

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

means that the danger of proxies becoming goals will never be fully mitigated. Acknowledgements The Centre for Innovation Studies conducted research related to this paper for the University of Calgary Office of the Vice-President (Research). We thank the University for support. Clearly, opinions expressed here are the authors’ and not necessarily those of the University. We also acknowledge provision of data from the Academic Annual Reports. We thank University Technologies International for access to AUTM data. We thank the Social Sciences and Humanities Research Council for support of a part of this work. References Agrawal, A., 2001. University to industry knowledge transfer: a review and unanswered questions. International Journal of Management Reviews 3, 285–301. Aldrich, H., Fiol, C., 1994. Fools rush in? The institutional context of industry creation. Academy of Management Review 19, 645–670. AUCC, 2005. Canadian Universities, University of Windsor. http:// www.aucc.ca/can uni/our universities/windsor e.html (accessed 1 April 2005; last updated 3 January 2005). AUCC (The Association of Universities and Colleges of Canada), 2002. Framework of agreed principles on federally funded university research. AUCC, Ottawa. http://www.aucc.ca/ pdf/english/ reports/2002/frame cadre e.pdf (accessed 28 March 2005). Bank of Boston, 1997. MIT: The Impact of Innovation. Bank of Boston Economics Group, Boston, MA. Barney, J.B., 1991. Firm resources and sustained competitive advantage. Journal of Management 17, 99–120. Baumol, W., 1990. Entrepreneurship: productive, unproductive, and destructive. Journal of Political Economy 98 (5), 893–921. Cohen, W., Levinthal, D., 1990. Absorptive capacity: a new perspective on learning and innovation. Administrative Science Quarterly 35 (1), 128–152. Cohen, W., Nelson, R., Walsh, J., 2002. Links and impacts: the influence of public research on industrial R&D. Management Science 48, 1–232. Crespi, G., Geuna, A., 2004. The Productivity of Science: An International Analysis, Prepared for and ©The Office of Science and Technology (UK). SPRU, Sussex, UK. Crespi, G.A., Guena, A., Nesta, L.J.J., 2005. Labour mobility from academia to business. New evidence from a large dataset of EPO inventors. In: Proceedings of the Fifth Triple Helix Conference, Turin, May 18–21 (Paper A180). DiMaggio, P., Powell, W., 1983. The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48, 147–160. Earl, L., Gault, F., Bordt, M., 2004. Summary: Meeting on Commercialization, Measurement, Indicators, Gaps, and Frameworks, Ottawa. Science, Innovation, and Electronic Information Division, Statistics Canada, Ottawa (Catalog #88F0006XIE—No. 007). Eisenhardt, K., 1989. Agency theory: an assessment and review. Academy of Management Review 14 (1), 57–74.

1597

Florida, R., 1999. The role of the university, leveraging talent not technology. Issues on Science and Technology 15 (4), 67–73. Fontana, R., Geuna, A., Matt, M., 2004. Firm size and openness: the driving forces of university–industry collaboration. In: Caloghirou, Y., Constantelou, A., Vonortas, N.S. (Eds.), Knowledge Flows in European Industry: Mechanisms and Policy Implications. Routledge, London. Freeman, C., Soete, L., 1997. The Economics of Industrial Innovation. MIT, Cambridge, MA. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., Trow, M., 1994. The New Production of Knowledge. Sage, London. Godin, B., Gingras, Y., Davignon, L., 1998. Knowledge Flows in Canada as Measured by Bibliometrics. Statistics Canada, Ottawa (ST 98-10). Guena, A., Llerena, P., Matt, M., Savona, M., 2003. Collaboration between a research university and firms and other institutions. Working Paper 108, Science & Technology Policy Research, University of Sussex, Sussex, UK. Hall, J., Sapsed, J., 2005. Influences of knowledge sharing and hoarding in project based firms. In: Love, P., Fong, S., Irani, Z. (Eds.), Management of Knowledge in Project Environments. Elsevier Butterworth Heineman, Oxford. Henderson, R., Jaffe, A., Trajtenberg, M., 1998. Universities as a source of commercial technology: a detailed analysis of university patenting, 1965–1988. Review of Economics and Statistics 80, 119–127. Industry Canada, 2002. Achieving Excellence, Investing in People, Knowledge, and Opportunity. Information Distribution Centre, Industry Canada, Ottawa. Jaffe, A., 1998. Measurement issues. In: Branscomb, L., Keller, J. (Eds.), Investing in Innovation. The MIT Press, Cambridge (Chapter 3). Jaffe, A.B., Trajtenberg, M., Henderson, R., 1993. Geographic localization of knowledge spillovers as evidenced by patent citations. Quarterly Journal of Economics 108, 577–598. Jensen, M., Meckling, W., 1976. Theory of the firm: managerial behavior, agency costs, and ownership structure. Journal of Financial Economics 3, 305–360. Kline, S., Rosenberg, N., 1986. An overview of innovation. In: Landau, R., Rosenberg, N. (Eds.), The Positive Sum Strategy: Harnessing Technology for Economic Growth. National Academy Press, Washington, DC. Lambert, R., 2003. The Lambert Review of University–Business Collaboration. Her Majesty’s Stationary Office, Norwich, UK. Landry, R., Amara, N., Ouimet, M., 2002. Research Transfer in the Natural Sciences and Engineering: Evidence from Canadian Universities. Department of Management, Laval University, Quebec, Canada (accessed 13 April 2005) http://www.kuuc.chair. ulaval.ca/francais/pdf/csrng.pdf. Langford, C.H., Wood, J.R., Ross, T., 2003. The origin and structure of the Calgary wireless cluster. In: Wolfe, D.A. (Ed.), Clusters Old and New. McGill-Queens, Press, Montreal (Chapter 6). Langford, C.H., Wood, J.R., Jacobson, A., 2005. The evolution and structure of the Vancouver wireless cluster. In: Wolfe, D.A., Lucas, M. (Eds.), Global Networks and Local Linkages. McGill-Queen’s Press, Montreal (Chapter 8). Laursen, K., Salter, A., 2004. Searching low and high: what types of firms use universities as a source of innovation? Research Policy 33, 1201–1215. Lazaridis, M., 2005. Commercialization: the system works. University Affairs 46 (2), 18–19. Mansfield, E., 1998. Academic research and industrial innovation: an update of empirical findings. Research Policy 26, 773–776.

1598

C.H. Langford et al. / Research Policy 35 (2006) 1586–1598

Meyer, M., 2000. What is special about patent citations? Differences between scientific and patent citations. Scientometrics 49 (1), 93–123. Meyer, M., 2002. Tracing knowledge flows in innovation systems. Scientometrics 54 (2), 193–212. Meyer, M., 2003. Academic patents as an indicator of useful research? A new approach to measure academic inventiveness. Research Evaluation 12 (1), 17–27. Molas-Gallart, J., Saltere, A., Patel, P., Scott, A., Duran, X., 2002. Measuring third stream activities: final report to the Russell Group Universities. SPRU, University of Sussex, Brighton, UK. http://www.clo.cam.ac.uk/documents/final russell report.pdf (accessed 14 June 2005). Mowery, D.C., Nelson, R.R., Sampat, B.N., Ziedonis, A.A., 2004. Ivory Tower and Industrial Innovation. Stanford Business Books, Stanford University Press, Stanford, CA. Narin, F., Hamilton, K.S., Olivastro, D., 1997. The increasing linkage between US technology and public science. Research Policy 26, 317–330. NCE, 2004. Annual Report 2003–2004. Networks of Centres of Excellence, Ottawa. http://www.nce.gc.ca/annualreport2003 2004/Eng/ index.asp (accessed 14 June 2005). Nightingale, P., 1998. A cognitive model of innovation. Research Policy 27, 689–700. Nightingale, P., 2004. Technological capabilities, invisible infrastructure and the un-social construction of predictability: the overlooked fixed costs of useful research. Research Policy 33 (9), 1259–1284. Osterloh, M., Frey, B., 2000. Motivation, knowledge transfer, and organizational forms. Organization Science 11 (5), 538–550. Pavitt, K., 1998. Do patents reflect the useful research output of universities? Research Evaluation 7 (2), 105–112. Pavitt, K., 2004. Changing patterns of usefulness of university research. In: Grandin, K., Wormbs, N., Widmalm, S. (Eds.), The Science–Industry Nexus: History, Policy, Implications. Nobel

Symposium 124. Walton Publishing, Sagamore Beach, MA, pp. 119–131. Pulp and Paper Research Centre, 1988. Pulp and Paper Research Centre McGill, A partnership. http://www.pprc.mcgill.ca/intro (accessed 1 April 2005). Read, C., 2003. Survey of Intellectual Property Commercialization in the Higher Education Sector, 2001. Science, Innovation, and Electronic Information Division, Statistics Canada, Ottawa (Catalog # 88F0006XIE—No. 012). Reamer, A., Icerman, L., Youtie, J., 2003. Technology Transfer and Commercialization: Their Role in Economic Development. Economic Development Administration, U.S. Department of Commerce, Washington. Rosenberg, N., Nelson, R.R., 1994. American universities and technical advance in industry. Research Policy 23, 323–348. Rothwell, R., 1992. Successful industrial innovation: critical success factors for the 1990s. R&D Management Journal 22 (3), 221–239. Salter, A.J., Martin, B.M., 2001. The economic benefit of publicly funded basic research: a critical review. Research Policy 30, 509–532. Schumpeter, J., 1934. The Theory of Economic Development. Harvard University Press, Cambridge, MA. Statistics Canada, 2005. Survey of Earned Doctorates. http://www. statcan.ca/english/sdds/3126.htm (accessed 21 November 2005). Szulanski, G., 1996. Exploring internal stickiness: impediments to the transfer of best practice within the firm. Strategic Management Journal 17, 27–43. Teece, D., Pisano, G., Shuen, A., 1997. Dynamic capability and strategic management. Strategic Management Journal 18, 507–534. TRlabs, 2005. TRlabs What We Do. http://www.trlabs.ca/whatwedo/ index.html (accessed 1 April 2005; updated 15 March 2005). Valentin, F., Jensen, R.L., 2002. Reaping the fruits of science: comparing exploitations of a scientific breakthrough in European innovation systems. Economic Systems Research 14 (4), 364–388.