- Email: [email protected]
Principles for evaluating knowledge co-production in natural resource management: Incorporating decision-maker values
Journal of Environmental Management 249 (2019) 109392
Contents lists available at ScienceDirect
Journal of Environmental Management journal homepage: www.elsevier.com/locate/jenvman
Research article
Principles for evaluating knowledge co-production in natural resource management: Incorporating decision-maker values
T
Ruth A. O'Connora,g, , Jeanne L. Nelb,c, Dirk J. Rouxb,d, Lilly Lim-Camachoe, Lorrae van Kerkhofff, Joan Leachg ⁎
a
Climate Change Institute, The Australian National University, Canberra, Australia Sustainability Research Unit, Nelson Mandela University, South Africa c Wageningen Environmental Research, Wageningen University, the Netherlands d Scientific Services, South African National Parks, South Africa e CSIRO Agriculture and Food, Brisbane, Australia f Fenner School of Environment and Society, The Australian National University, Canberra, Australia g Centre for the Public Awareness of Science, The Australian National University, Canberra, Australia b
ARTICLE INFO
ABSTRACT
Keywords: Communicative competence Legitimacy Public participation Policy-makers Science communication Science-policy interface
A key reason for undertaking transdisciplinary processes such as knowledge co-production in natural resource management is to access and apply different knowledge systems to complex issues. However, the value of coproduction is predominantly framed by academics. They have focused on research design and outcomes in the form of ‘science informing policy’. In this paper we build a more inclusive and holistic framing of knowledge coproduction incorporating values of non-academic participants, and values related to the participatory process. Specifically, we examine how knowledge is communicated and deliberated upon and the requirements for this to be done effectively. We draw upon empirical data from interviews with natural resource managers who participated in two case studies of knowledge co-production in Australia and South Africa. Their perspectives are captured in eight evaluation principles that build upon existing evaluation frameworks for public participation. Critically, decision-makers valued science-based outputs not just as salient knowledge sources, but to give legitimacy to their decisions. This need for legitimacy necessitates transparency, fairness and inclusivity in knowledge selection, participation and dialogue within knowledge co-production processes. The practice-based knowledge of decision-makers was important for contextualizing and applying science to specific decision contexts. Another key finding is that communicative competence is central to the process of knowledge coproduction because it enables participants to critically explore and understand the knowledge claims of others.
1. Introduction Participatory and transdisciplinary processes are useful mechanisms to achieve complex natural resource management goals such as sustainable development and biodiversity conservation (van Kerkhoff and Lebel, 2006; Hegger et al., 2012; Nel et al., 2016). One reason for this is that these processes enable both academics and non-academics to apply their diverse knowledge and expertise to public decision making. However, the way we currently value these processes is informed predominantly from an academic perspective where success is equated with research impact through the integration of science in policy and management (Fazey et al., 2014; Hansson and Polk, 2018). This focus risks ignoring or subordinating the values and knowledge of non-academic participants. It also risks minimising values inherent in the ⁎
communicative processes associated with co-production, or their outcomes such as social learning and network building (Walter et al., 2007; Roux et al., 2017). These outcomes may have intrinsic value or be necessary pre-cursors to science application. One key non-academic group of participants are decision-makers who we define as individuals situated within organisations who have a socially recognised role to make decisions on behalf of others and who can apply science for the public good. A variety of participatory approaches to natural resource management (NRM) exist, each with slightly different emphases including research (e.g. transdisciplinary research), governance, or communication (e.g. knowledge exchange). All of these approaches are based on collaborative partnerships between academics and non-academics, each with valuable knowledge and expertise to contribute to NRM (Wickson
Corresponding author. Climate Change Institute, The Australian National University, Canberra, ACT, 2601, Australia. E-mail address: [email protected] (R.A. O'Connor).
https://doi.org/10.1016/j.jenvman.2019.109392 Received 17 December 2018; Received in revised form 9 August 2019; Accepted 10 August 2019 0301-4797/ © 2019 Elsevier Ltd. All rights reserved.
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
et al., 2006; Fazey et al., 2013; Wyborn et al., 2019). Application of this knowledge pre-supposes scientists, decision-makers and other knowledgeable participants have communicative competence. That is, all participants have the ability to use language to communicate rationally and create the best possible understandings within what is known through making and defending claims (Habermas, 1984). However, the academic imperative for science to inform policy means the value of communication in participatory processes is outcome oriented in the form of science translation, fostering science legitimacy, and creating commitment to participatory exercises (Mitton et al., 2007; Hegger et al., 2012; Cook et al., 2013). Participants with “strong communication skills” or 3rd party communication specialists such as knowledge brokers can help achieve these goals (Cvitanovic et al., 2016). Communicative competence is a richer concept than translation that frames and values communication as a social process rather than a means to achieve pre-determined outcomes. A focus on communicative competence and how to achieve it, therefore, allows for a broader definition of success in participatory and transdisciplinary activities that can incorporate non-academic values. Our focus is on knowledge co-production as one type of participatory approach in NRM that can stand alone or contribute to other approaches such as transdisciplinary research. We therefore define knowledge co-production according to Armitage et al. (2011, p. 996) as the “collaborative process of bringing a plurality of knowledge sources and types together to address a defined problem and build an integrated or systems-oriented understanding of that problem”. Developing evaluation criteria for knowledge co-production remains a challenge because of its variety of forms, contexts, and participants who may have differing views of what is valuable. Hegger et al. (2012) identified a set of predominantly process-based “success factors” in co-production and the relative merits of evaluating process and outcomes of participatory processes has been discussed (Walter et al., 2007). Advocating a standard set of evaluation criteria is generally inappropriate for complex, value laden endeavours (Fazey et al., 2014). However, broader evaluation principles may be useful for practitioners as prompts to consider and adapt to their context. They can also be useful for promoting reflection about what constitutes good participatory practice and when applied, allow improvement in practice from an empirical base. The aims of this paper are to firstly test and broaden academic framings of knowledge co-production by analysing the perspectives of NRM decision-makers who have participated in these processes. Secondly, we seek to explore the specific values of science communication and what is required to do it effectively as part of knowledge co-production. We see this focus on communicative process as complementing current work on design and impact in participatory processes (Walter et al., 2007; Reed et al., 2018). We start by proposing a set of value propositions for co-production grounded in communication theory and drawn from evaluation frameworks for public participation. We then present the results of two knowledge co-production case studies from Australia and South Africa that test our propositions. Finally, we use the empirical evidence from the cases to modify our propositions into a set of eight evaluation principles for knowledge co-production.
Table 1 Value propositions for knowledge co-production based on evaluation frameworks for public participation (based on; Webler, 1995; Rowe et al., 2004; Chilvers, 2008). Propositions 1. Communication processes should enable participants to make and question claims to create mutual understanding of information presented. 2. Participation by individuals with a diversity of knowledge and experience relevant to the management issue is of value. 3. Co-production requires legitimacy and acceptance of outputs which can be achieved through fairness and transparency in the process. 4. The input of resources in addition to knowledge is of value to enable participation and facilitate communication. 5. Outputs and outcomes of value to participants are necessary to justify their time commitment.
(Bucchi, 2008, p. 73). Current evaluation frameworks are strongly grounded in communication theory and do not explicitly consider decision-makers as participants (or ‘publics’), so we present a set of value propositions in Table 1 about the value of their engagement adapted from five recurring evaluation themes. The first proposition relates to the nature of dialogue and deliberation (Table 1). Specifically, how effectively technical information is communicated to non-academics and how well both academic and nonacademic participants are able to make sense of each other's claims i.e. their communicative competence (Habermas, 1984). The second proposition relates to PES as a mechanism of deliberative democracy, i.e. PES has value in a democratic sense when citizens from a broad range of backgrounds or groups are represented, given a voice and empowered through access to knowledge. Representation is also valued in an epistemic sense when a variety of knowledge and perspectives encourages testing of issue framing or the implications of decisions, thus increasing the robustness of decisions (Russell, 2013). We posit that in knowledge co-production involving decision-makers, the value of diverse representation is primarily epistemic rather than democratic (Table 1). Related to this, our third proposition recognises that the full potential of co-production can only be realised when different sources of knowledge are considered equally and openly. Transparency, fairness and independence of PES processes are also necessary for legitimacy and acceptance of both the process and its outputs. We view legitimacy in co-production as “conformity to recognised principles or accepted rules and standards” (Heink et al., 2015, p2) rather than factual acceptance of decisions. The fourth proposition describes the necessity and value of process inputs such as support for individuals to attend or participate, time for participants to develop competent understandings of the material, and expert facilitation of the process. Finally, proposition 5 (Table 1) relates to the value of outputs and outcomes which for decision-makers may include building trust between people and institutions, or forming new professional partnerships (Roux et al., 2006). 3. Methodology
2. Analytical framework
We used a case study approach to explore the complex social processes of knowledge co-production in depth and in context (Yin, 2014). We focus on a single policy sector—natural resource management (NRM)—recognising that cases in different policy sectors could add unnecessary complexity to the results because of unique contextual issues or variables. With its management imperative, NRM provides an opportunity for exploring how the meanings, implications and application of science are negotiated. Two national knowledge co-production projects (Table 2) were selected to allow iterative refinement of our initial propositions.
We reviewed evaluation frameworks for public participation in public decision making to develop an analytical foundation for data collection and analysis. Public participation broadly refers to activities that seek citizen involvement in decision making processes affecting them (Arnstein, 1969). Public participation or public engagement with science (PES) is also a model of science communication defined as “the diversified set of situations and activities, more or less spontaneous, organised and structured, whereby non-experts become involved, and provide their own input to, agenda setting, decision-making, policy forming and knowledge production processes regarding science” 2
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Table 2 Comparison of the AdaptNRM and National Freshwater Ecosystems Protected Areas (NFEPA) projects.
Project duration Science application Target decision-makers Participants in knowledge co-production mentioned by interviewees Engagement mechanisms & pathways
a
AdaptNRM
NFEPA
2013–2016 A variety of models predicting climate change & changes in species distributions applied to biodiversity & weed management. Planners from regional NRM groups across Australia.
2008–2011 National-scale systematic conservation planning & prioritization system applied to river, wetland & estuary biodiversity.
Target decision-makers plus: Other specialists (e.g. monitoring & evaluation, ecology); Private consultants contracted to NRM groups; Managers. Set scope of project (participants nominated priority adaptation domains for application of climate projections). Product development Written comments on draft guides; Participation in discussions about draft guides (e.g. webinars, teleconferences & workshops); Provision of inputs to guides/co-development of case studies.
Environment & water managers & planners in national, provincial & catchment level agencies across South Africa. Target decision-makers plus: Private consultants working in the environment & water sectors; Boundary spanners. Via project governance; Set scope of project (discussions to align needs across water & environment sectorsa); Input to methods development. Product development Collated, reviewed & refined data inputs individually & in groups; Reviewed & tested products; Provided input on priorities & data through processes of dialogue & co-learning in a variety of workshops at national & catchment scales.
Building on existing cross-sector policy objectives (Nel et al., 2011).
3.1. Case study 1: the AdaptNRM project (Australia)
government officials and consultants. Interviews ranged from 35 to 60 min in duration. The NFEPA case was viewed as a valuable comparison to AdaptNRM being similar in scale and range of communication activities, but a potential contrast in timeframe since completion, regulatory contexts, and science application (Table 2). The five years since project completion had advantages and disadvantages. The key opportunity was to describe policy and other outcomes that may only manifest in the years after project completion (Rowe et al., 2004). The longevity of more immediate outcomes like those described in AdaptNRM could also be explored. These opportunities were considered to outweigh challenges associated with people leaving the sector and loss of detailed recall of specific events over time.
The AdaptNRM project aimed to make existing science about climate adaptation more accessible to regional NRM groups by co-developing a suite of user-targeted guides and data sets (CSIRO, 2014). It involved a team of scientists engaging with regionally-based decisionmakers from across Australia. The mandate of regional NRM groups is to plan and deliver programs that “support healthy and productive country, viable communities and sustainable industries” (Natural Resource Management Regions Australia). In some jurisdictions, NRM groups function as quasi state-based organisations, with the legislative power to implement their own plans. Elsewhere, they function as nongovernment organisations and work with land managers to implement plans through on-ground works. All 56 NRM groups existing at the time were invited to participate in AdaptNRM. The project team had a contact list of 129 individuals and 25 of these representing 23 groups from every state and territory except one were interviewed for the case study. The interviews were conducted in the final stages of the project from September to October 2015 and ranged from 22 to 80 min in duration. Knowledge co-production in AdaptNRM involved a suite of communication activities initiated by the project team which are outlined in Table 2. AdaptNRM was viewed, therefore as a good case of various communication activities involving both decision-makers and science experts for knowledge co-production in NRM.
3.3. Data collection and analysis Qualitative data was collected through semi-structured interviews conducted by the lead author with individual decision-makers. Semistructured interviews allowed the interviewer to simultaneously cover a standard list of topics (the value propositions in Table 1) while also allowing the interviewees to articulate their own values and framings of the two projects (Thomas, 2011). Decision-makers were asked what they valued about their project, their drivers for engagement, enabling and constraining factors. Interviews were conducted face-to-face, via telephone or online and were recorded and transcribed. The transcripts were analysed using qualitative content analysis (Schreier, 2012) which is well suited to exploratory case studies with interview data (Elo and Kyngäs, 2008). It condenses and describes content and the process of categorization provides a means of understanding underlying phenomena. Transcript content was coded using QSR International's NVivo 11 Software. Categories were derived from the data in an inductive process resulting in 22 categories for NFEPA and 21 for AdaptNRM. These categories were clustered into five higher level themes that broadly reflected the value propositions upon which the questions were based. The lead author did all the coding which facilitates consistency. In addition, both coding frames were validated by recoding three transcripts and checking for mismatches. New categories were also added through the validation process. For example, comments about outcomes were more detailed for NFEPA and therefore split into personal, institutional and societal outcomes.
3.2. Case study 2: the National Freshwater Ecosystem Priority Areas project (South Africa) The National Freshwater Ecosystem Priority Areas (NFEPA) project ran as a partnership between research institutions, South African government agencies, and environmental non-government organisations. One goal was to identify freshwater ecosystems such as rivers and wetlands that should be prioritized for biodiversity protection. Another was to promote the adoption of co-produced outputs by relevant agencies (Nel et al., 2011). NFEPA was run by a project team comprising both researchers and managers. They had a contact list of over 350 people from water and environment agencies across the country. Five years after the completion of NFEPA, many of these original participants had either changed role or left their organisations. Eighteen decision-makers from six out of nine provinces were interviewed between May and July 2016 comprising national and provincial 3
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
4. Results
knowledge input was that from the decision-makers themselves, particularly practical knowledge about how to address NRM issues within institutional, regulatory and social contexts. Decision-makers often had science backgrounds as well as: local knowledge (e.g. of species distribution); knowledge about the decision context (e.g. how products could be implemented); and the nature of stakeholders. A new consideration in our second proposition is that elicitation of tacit knowledge (particularly that of non-academics) may be necessary and a valuable contribution of co-production projects: “it really was a fantastic exercise for extracting a lot of data that was otherwise just sitting in people's heads, being lost never having been really extracted or written up” (NFEPA09). Decision-maker knowledge seemed to be necessary to contextualise the available science to a specific decision context, with its particular biogeographic, legislative, and social conditions. The process for deciding which scientists to involve, or which technical approach to take, in knowledge co-production was unclear in both cases and indicates that considerations about process legitimacy also apply to knowledge and method selection. Decision-makers, on the whole, were unaware of how these decisions had been made: “that was in the hands of the scientists” (NFEPA06). While both projects employed collaborative processes in issue-framing, decisions on scope and which scientists to invite were made by the project team (AdaptNRM) and an advisory group (NFEPA). In addition, interviewees in both cases mentioned relevant alternative approaches and knowledge sources that had been either rejected or not considered for reasons unclear to them. These types of opaque decision-making processes may threaten the legitimacy of knowledge co-production and were an unforeseen element of our third proposition on fair and transparent process (Table 1).
We focus here on five themes from the qualitative content analysis that aligned to the value propositions presented in Table 1, but note that other useful insights about the background and motivations of decision-makers emerged. One of the key emergent findings across both cases was that the issue of legitimacy was more complex than originally envisaged. Therefore, in addition to discussing legitimacy as a standalone concept (4.3), we also discuss it under the other three processoriented propositions (4.1, 4.2 and 4.4). We illustrate our results with interviewee quotes that are numbered according to identity. This demonstrates the breadth of responses drawn upon while maintaining confidentiality. 4.1. Decision-makers valued interactive communication Interactive communication processes and their direct outcomes were valued by decision-makers in both cases in a variety of ways that exemplify and expand our first proposition (Table 1). Some participants valued making and questioning claims rather than passively receiving information. For example, in AdaptNRM: “we could amend comments or add additional comments which we might not have thought of before but were triggered by speaking to other people or listening to other people. So that particular side of it I found very useful” (NRM16); and in NFEPA: “We sat around the tables and we talked. And this one said this and that one said no, no remember this and remember that, oh yes I forgot about that!” (NFEPA11). Making and questioning claims appeared to prompt cognitive processing and learning. It also stimulated reflexivity which allowed for re-conceptualisation of issues or developing new ways of thinking: “being involved in NFEPA sort of forced me to take a step back and look at what's happening in the region, what's happening in the country” (NFEPA05). There was also evidence that the inherently social processes of dialogue among colleagues and scientists motivated some participants to critically engage with content, who may not otherwise have done so: “the modules were new information for me and it was easier for me to understand if I was actually being made to interrogate it a bit more instead of just accepting it as the approach we should be using” (NRM01). Another important value of engagement and critical thinking was for decisionmakers to step from understanding information to applying it: “having small group discussions and being able to say: well that won't work for us, how about this?” (NRM23). For example, AdaptNRM decision-makers needed to consider how to take climate models and use them to revise their weed management strategies. In relation to our third proposition regarding legitimacy (Table 1), an important caveat for realising the potential benefits from participation was that the knowledge and experience of all participants be respected: “I think that really helped - it was a transparent open process” (NFEPA08). This expands our proposition regarding the importance of legitimacy suggesting that equality in communicative competence between decision-makers and scientists is necessary for process legitimacy and acceptance.
4.3. Decision-makers valued processes and outputs that legitimised decisions Decision-makers in both cases commonly valued the process and outputs of knowledge co-production as mechanisms to give legitimacy to their decision-making: “one of the ways we can ensure independence is to validate our assessments through the use of credible scientific literature” (NFEPA interviewee, consultant), and “if we were doing things that weren't based in science then we'd lose our credibility. So quickly.” (NRM17). This legitimacy was seen as key to the acceptance of their decisions by stakeholders such as resource users and managers (e.g. landholders, fishers, and mining companies) or other decision-makers (e.g. NRM Boards, departmental heads). These results suggest that for decisionmakers, co-production is not only about solving management problems; it is also about the acceptance of their proposed actions by stakeholders outside the co-production process. While it was clear that most thought basing decisions on science would contribute to legitimacy, the idea that the process by which science informs decisions requires benchmarks of fairness and transparency was less consistent. 4.4. Decision-makers valued support to participate and engage meaningfully Our fourth proposition about the value of project inputs (Table 1) was exemplified in both cases. Support to participate either in person “if all the meetings are up in Johannesburg or Pretoria then you have some limitations to involvement” (NFEPA05) or via video link: “the webinars were excellent.” (NRM17) was valued. The expertise of the AdaptNRM and NFEPA project teams in facilitation of group sessions that enabled communicative competence was also valued: “It was a very professional team. I really think some of the best facilitation on break away groups and special groups discussions” (NFEPA12). This facilitation was partly about decision-maker inputs being heard, valued and responded to. Planning undertaken by project teams who collated and synthesised inputs for well-targeted group sessions allowed decision-makers to engage effectively with minimal effort: “I think the overall way they've run the project has actually been particularly good. They've had structured involvement in the development of the modules” (NRM07). Well organised processes prompted some decision-makers to “make time” to consider new
4.2. Decision-makers valued science and the elicitation of practitioner knowledge Our second proposition foreshadowed the importance of science to decision-makers and this was exemplified in the results. Interviewees in both cases consistently considered scientific knowledge as critical for their work. When asked about the value of science to their work, terms such as “core”, “fundamental”, “critical”, and “cornerstone” were used. One South African interviewee captured a common view: “as a conservation agency, how do you work without science?” (NFEPA05). Coproduction projects can provide decision-makers a means of accessing relevant science and scientists “Probably the major benefit to this project I think is the interaction with scientists” (NRM07). Another critical 4
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Table 3 A summary of outcomes from knowledge co-production described by decision-makers. Outcome level
Types of outcomes emerging from case studies
Individual
Changes in knowledge, understanding and skills Changes in attitude Changes in practice New or improved professional relationships Knowledge and information more accessible New knowledge and information incorporated in policy, strategy and plans Changes in organisational culture Step changes in narrative
Organisation Cross-organisation/Institution
knowledge: “you've got that many other things going on in your world that you turn your attention to this for the space that the project's created for you which is usually a meeting and when you're bussing to the meeting …” (NRM12). The capacity of decision-makers’ host organisations appeared to limit participation in some instances regardless of project inputs. Jurisdictions with the least resources were under-represented in both AdaptNRM and NFEPA (project leader pers. comm.) and we were unable to recruit participants from these same jurisdictions. In South Africa, low levels of engagement with NFEPA were attributed to limited staff numbers and expertise of host organisations (project leader pers. comm.).
(NFEPA10). Another may be changes to how organisations conceptualise and approach issues “NFEPA inspired a lot of this thinking of how can we use mapping to improve our decision-making” (NFEPA08). Multiple participatory processes including NFEPA may have also resulted in re-conceptualisation or re-framing of issues across institutions (Table 3). Some South African decision-makers felt NFEPA had played an important part in a change in the national conservation narrative: “the precept is there [the importance of aquatic ecosystems] so when we go into meetings, we don't have to try and re-sell the idea … we've got traction” (NFEPA16). These findings exemplify our fifth proposition (Table 1) but indicate it may be useful to differentiate between individual, institutional and cross-institutional outcomes.
4.5. Decision-makers valued beneficial project outputs and outcomes
5. Evaluation principles
Learning was one of the most common outcomes mentioned by participants in both case studies: “I've learnt from that. You know apart from the actual content of it, just their approach and how they've tackled the problem” (NRM25). Participants learned about science and from the practice-based experience of peers, learned about the project outputs, and developed skills in the use of products: “the benefit of being involved in the project from its development stage—in the sense of the stakeholder meetings—is that we got to understand a lot more of the project to implement it” (NFEPA13). In this way, individual learning outcomes were sometimes pre-cursors to the use of project outputs. Use of the outputs by participants can be seen as a change in practice. We regard this as one of the outcomes of knowledge co-production manifest at the individual level (Table 3). Decision-makers also valued co-production as a means of building and maintaining professional relationships with colleagues and scientists. This relationship-building was inextricably tied to social, communicative processes: “I actually think it's [the value of the project] more about relationships than accessibility (NRM04). Examples were given of how relationships allowed for easier sharing of knowledge as well as facilitating collaborations in future projects. However, the importance of relationship-building depends on context with evidence that the NRM community of practice was already well established in South Africa so the challenge was more about including newcomers: “a real risk of using networks is to what extent are we pushing the boundaries of those networks and to what extent are we quite comfortable and complacent working with the people that we know” (NFEPA09). Participants also valued better access to the knowledge of both colleagues and scientists: “I think the most use was seeing the diversity of approaches from the different regions and chatting to people afterwards” (NRM23) and “Although I know about specialists, I rarely talk to them” (NFEPA12). Increased accessibility of knowledge and information was through both project outputs and face-to-face interaction. Institutional outcomes were evident mainly from NFEPA which was completed five years before interviews in contrast to AdaptNRM that was in its wrap up phase. A clear institutional outcome was use of project outputs by organisations with 23 specific and generic policy and management instruments mentioned. Changes to organisational processes and culture were harder to identify and attribute but one example may be non-participants using project outputs: “And in our processes we specify, we actually force people to look at the NFEPAs”
5.1. Level of transparency, inclusivity and fairness The case studies indicate that principles of transparency, fairness and inclusivity cut across the different elements of the co-production process and are critical for the legitimacy and acceptance of those processes and their outputs (Fig. 1). Specifically, communication within co-production requires equality in competence (principle 1), the knowledge inputs for co-production require transparent selection of expertise (principle 2) and support for participation needs to promote inclusivity (principle 3). 5.1.1. Principle 1: knowledge co-production should aim for participants with different types of knowledge to have equal communicative competence We argue that inclusive and fair dialogue and deliberation is necessary in knowledge production to provide an equal opportunity for different perspectives to be expressed, understood and questioned. The importance of being listened to and being able to contribute knowledge and experience was often mentioned by interviewees. Achieving this reflects a commitment by projects to knowledge symmetry. We were unable to analyse the dialogue between decision-makers and scientists, but in PES and to a lesser extent transdisciplinary research, the scientific voice can dominate in discussions with non-academics (Davies, 2013; Hansson and Polk, 2018). This means the challenge for co-production may be fair consideration of decision-maker and other nonacademic knowledge and expertise. As in PES, knowledge co-production may not realise its participatory potential if non-academic participation is tokenistic or for purely instrumental goals such as getting “buy-in” for products (Rowe et al., 2004). 5.1.2. Principle 2: knowledge co-production should have transparent, documented processes of determining knowledge inputs and technical approaches Decision-makers commonly felt that decisions based on science would be considered legitimate and accepted by stakeholders outside co-production. One reason that this is a questionable assumption is that complex decision contexts in NRM can draw on multiple sources of science. This means that numerous quite different decisions could be based on science. In both our cases, implicit or explicit choices were 5
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Fig. 1. Eight elements of knowledge co-production that contribute to robust, informed and legitimate natural resource management decisions.
5.1.3. Principle 3: projects should be able to support groups and individuals with relevant knowledge and experience to participate The level of engagement by decision-makers in co-production appeared to vary across organisations. One factor limiting participation appeared to be low capacity of host organisations in terms of staff and resources. Both projects tried to compensate for this by funding travel and using technology that enabled individuals to participate but did not pay for participant time. Whether this type of investment is necessary will depend on the context of knowledge co-production. For example, support for wide participation may be considered necessary if socially robust decisions are important (Hegger et al., 2012), or if participants can increase the relevance of outputs (Reed et al., 2018). In an NRM context, broad representation may be a mechanism to avoid reinforcing disadvantage when organisations lacking the capacity to engage fail to accrue the benefits of co-production and are left further behind (Robins and Dovers, 2007). Our results suggest project support may be insufficient to achieve broad institutional representation, particularly where institutional capacity is limited. We suggest support for participation is evaluated according to how well it enables broad participation across stakeholder organisations and its flexibility to respond to participant needs.
be done effectively have not been unpacked. Decision-makers in our two case studies valued interactive communication as a mechanism for individual learning, developing mutual understanding of different knowledge sources, and the development of relevant, useable outputs. Currently, interactive communication in participatory processes tends to be framed instrumentally as a means to achieve pre-determined endpoints such as the long-term commitment of participants (Hegger et al., 2012). This particular framing also conflates the social value of communication with its core function of producing and negotiating meanings (Wynne, 2014). This function is particularly necessary in co-production where there are different types of knowledge to be considered for application (in our case) to NRM issues. This necessitates enabling both academic and non-academic participants to question the potential application of knowledge in a given decision context which is a step beyond the more passive characterisation of communication as “listening” for learning (Reed et al., 2018). We propose that explicit procedures and resources are needed so participants can meaningfully discuss the knowledge and information available and formulate their own views about it. In public participation involving citizens, communicative competence is evaluated according to established procedures rather than traits of individuals (Webler, 1995). While co-production may be easier among motivated individuals with strong communication skills (Cvitanovic et al., 2016) we argue that the primary criterion for participation in knowledge coproduction (as opposed to implementation) is the knowledge an individual can contribute. Projects can support communicative competence by, for example, employing facilitation experts with strong communication skills (discussed further in section 5.4). Procedures for promoting competence include communicating science in ways that are relevant and accessible to the non-specialist (Chilvers, 2008) and where meanings, assumptions, and uncertainties are clear (Roux et al., 2006). Requirements for the analysis and communication of other knowledge forms are less clear, but should at a minimum involve acknowledging their potential value and providing the opportunity for them to be discussed.
5.2. Interactive communication
5.3. Knowledge inputs
5.2.1. Principle 4: knowledge co-production should establish communicative competence The need for dialogue and deliberation in knowledge co-production is accepted, but its purposes, functions, and the pre-requisites for it to
5.3.1. Principle 5: knowledge co-production processes should identify and elicit non-academic knowledge and explore the idea of ‘relevant knowledge’ A strong theme emerging from both case studies was the importance of decision-maker knowledge in co-production processes. Bureaucratic
made about which expertise to include. It is unlikely a given project can, or should, include all relevant expertise so choices are likely inevitable. Having documented, transparent processes of selection not only promotes trust, it allows for the evaluation of the comprehensiveness of the knowledge engaged with, and possible biases (Fischer et al., 2014). The idea of multiple, and possibly differing, scientific perspectives and how they are chosen has not been a focus in knowledge co-production where greater attention has been given to the quality of research knowledge (Dunn and Laing, 2017). Collaborative consideration of a comprehensive range of knowledge sources and expertise should be considered in co-production because these choices are a significant manifestation of power in participatory processes (Fazey et al., 2013).
6
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
knowledge, defined as the knowledge held by bureaucratic officials (civil servants) about political and administrative procedures and processes (Edelenbos et al., 2011), is a recognised input to co-production at the science-policy interface. This characterisation does not adequately encompass the knowledge of the AdaptNRM and NFEPA decision-makers which also included local knowledge of ecosystems and stakeholders, and practice-based knowledge important for conceptualising how science could be applied in specific decision contexts. Our findings also contrast with a study of marine park managers whose lack of scientific training was seen as a barrier to successful knowledge exchange (Cvitanovic et al., 2016). Studies of research uptake in NRM have also shown that decisionmakers may not always have up to date or comprehensive knowledge to draw on (Walsh et al., 2014). While the nature and importance of decisionmaker knowledge will vary according to the decision context, our study shows that co-production at the science-policy interface needs to acknowledge and consider the knowledge inputs of decision-makers. We need to better articulate what that knowledge might consist of and how to judge its quality (Tengö et al., 2014).
Implementing outputs and addressing complex societal problems is an ideal outcome of transdisciplinary processes (van Kerkhoff and Lebel, 2006; Cook et al., 2013) and was an aim of both AdaptNRM and NFEPA. While decision-makers valued relevant useable outputs, their implementation is a problematic benchmark for evaluating co-production. Use of outputs can depend on factors beyond the scope of the process (Lemos and Morehouse, 2005) as illustrated in AdaptNRM, where a change in federal government took climate change off the political agenda making the implementation of regional plans problematic. There is also a conceptual problem with narrowly equating success in knowledge co-production with science informing policy. Apart from ignoring or subordinating other knowledge inputs, it limits the agency of decision-makers to understand science but decide not to apply it based on their own expert knowledge of the decision context. Instead, we suggest a core outcome of knowledge co-production grounded in mutual understandings of available knowledge gained by participants in the process. 5.5.2. Principle 8: knowledge co-production should result in mutual understandings of available knowledge by participants and flow-on institutional benefits associated with the process and its outputs Co-production processes where participants actively engage with different knowledge sources to produce outputs can be viewed as a form of dialogic engagement. A benchmark for good dialogic engagement is participants gaining a mutual understanding of each other's knowledge (Taylor and Kent, 2014). This understanding can then form the basis for determining and applying the best available knowledge and expertise to policy and management. Decision-makers valued learning about the application of science from other practitioners. This was most striking in AdaptNRM where mechanisms for the implementation of science were less obvious. Such social learning is a recognised outcome of coproduction and particularly valued when situated in wider social or institutional networks (Armitage et al., 2011). Learning at a cross-institutional level was evident in South Africa but not Australia. This may reflect the longer period since project completion but is more likely a product of the longer term investment in multiple transdisciplinary activities over time in South Africa involving some of the key NFEPA participants (Roux and Nel, 2013). Strengthened professional relationships among decision-makers and scientists were also valued because they improved access to relevant information and led to other collaborations where costs and benefits could be shared. The value of strong professional networks has also been described in the context of urban water management where relationships built over many years between researchers and managers resulted in quick adoption of research in a crisis situation (Dunn et al., 2018). These findings broaden current framings equating the value of relationships between scientists/ knowledge brokers and decision-makers with implementation of coproduced outputs (Lemos and Morehouse, 2005; Reed et al., 2018).
5.4. Support to enable participation 5.4.1. Principle 6: knowledge co-production processes should provide resources to enable dialogue and deliberation including: facilitation, planning and 3rd spaces The time and effort of dedicated project teams was valued by decisionmakers in both cases. These teams had substantial knowledge of the science and/or management domain and they performed several functions including promoting critical but respectful dialogue and eliciting tacit knowledge. Facilitation can also occur through governance arrangements (Armitage et al., 2011), professional networks (e.g. Dunn et al., 2018) or specialised individual facilitators. Neutrality and skills in social facilitation, and building collaborative networks are recognised as important qualities of boundary-spanning facilitators (Reed et al., 2018). However, our study shows that in co-production processes where it is important to elicit and deliberate upon knowledge, a key function of facilitators is also enabling the communicative competence of participants. In our cases, synthesising knowledge inputs and planning structured, iterative engagement exercises at key milestones were valued because decision-makers were time constrained and national scale projects had large information inputs. Third spaces or shadow spaces outside of normal work environments were also valued as a “change of scene” conducive to informal and facilitated dialogue. The neutrality of such spaces for dialogue and community building has also been emphasised (CSIRO, 2014; Roux et al., 2017; Hansson and Polk, 2018). 5.5. Outputs and beneficial outcomes 5.5.1. Principle 7: knowledge co-production should deliver knowledge based outputs according to the aims and needs of participants and project sponsors Planned co-production activities such as our two case studies generally require a funding source or sponsor, who will have specific requirements. This can take the form of: transdisciplinary research, e.g. developing sustainable cities (Hansson and Polk, 2018); commissioned research, e.g. urban water management (Dunn et al., 2018); or advisory groups made up of researchers, community and government officials (Edelenbos et al., 2011). Therefore, any evaluation of co-production activities needs to address the stated needs of sponsors as well as participants. Meeting multiple goals requires recognition of potential power imbalances among participants, deliberate effort to avoid entrenching these, and remembering the underlying normative aims of coproduction to support societal change (Wyborn et al., 2019) Decisionmakers in both cases valued co-production because of its knowledgebased outputs such as maps, guidelines and decision-tools. Elsewhere decision-makers have expressed a preference for science packaged in a form that can be used to solve problems associated with their institutional mandate (Dunn and Laing, 2017).
6. Conclusions Considering the perspectives of decision-makers who participated in two cases of knowledge co-production has broadened our conceptualisation of its value beyond science informing policy. While NRM decision-makers value science-based outputs as salient knowledge sources, they also seek legitimacy in contested decision contexts such as water management and climate change adaptation. This means the value of co-production cannot be de-coupled from the political imperative for decisions to be accepted by outside stakeholders. We argue that this legitimacy cannot be achieved purely by basing decisions on science but is built through fairness, transparency and inclusivity in all aspects of co-production processes. Our second aim was to explore the value and requirements for effective communication in co-production. A key finding was that effectively harnessing the knowledge of decision-makers and critically examining the scope of science to meet decision-maker needs requires communicative competence among all 7
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
participants. The knowledge of decision-makers can be critical to the application of science in complex decision contexts. Facilitators and boundary spanners can play a key role in enabling meaningful deliberation that can deliver mutual understanding of knowledge—what we see as a fundamental goal of co-production. A next step in the development of knowledge co-production theory and practice is testing principles such as ours in the evaluation of coproduction activities and critiquing the results. We have demonstrated that non-academic knowledge is a critical consideration but methods for assessing the quality of that knowledge—similar to the accepted principles for good science—are lacking. How communicative competence manifests (or not) and the enabling environments for this should be explored through direct observation and analysis of the interactive processes within co-production. Enabling communicative competence will allow the full potential of co-production to be realised through effectively drawing upon the knowledge of all participants.
332–343. https://doi.org/10.1093/scipol/sct062. Habermas, J., 1984. The Theory of Communicative Action, vol. I Reason and the Rationalization of Society, Beacon, Boston. Hansson, S., Polk, M., 2018. Assessing the impact of transdisciplinary research: the usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact. Res. Eval. 27, 132–144. https://doi.org/10.1093/reseval/ rvy004. Hegger, D., Lamers, M., Van Zeijl-Rozema, A., Dieperink, C., 2012. Conceptualising joint knowledge production in regional climate change adaptation projects: success conditions and levers for action. Environ. Sci. Policy 18, 52–65. https://doi.org/10. 1016/j.envsci.2012.01.002. Heink, U., Marquard, E., Heubach, K., Jax, K., Kugel, C., Neßhöver, C., Neumann, R.K., Paulsch, A., Tilch, S., Timaeus, J., Vandewalle, M., 2015. Conceptualizing credibility, relevance and legitimacy for evaluating the effectiveness of science–policy interfaces: challenges and opportunities. Sci. Public Policy 42, 676–689. https://doi.org/10. 1093/scipol/scu082. Lemos, M.C., Morehouse, B.J., 2005. The co-production of science and policy in integrated climate assessments. Glob. Environ. Chang. 15, 57–68. https://doi.org/10. 1016/j.gloenvcha.2004.09.004. Mitton, C., Adair, C.E., McKenzie, E., Patten, S.B., Perry, B.W., 2007. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 85, 729–768. Natural Resource Management Regions Australia. http://nrmregionsaustralia.com.au/ (accessed 21 October 2015).. Nel, J.L., Murray, K., Maherry, A., Petersen, C.P., Roux, D.J., Driver, A., Hill, L., Van Deventer, H., Funke, N., Swartz, E.R., Smith-Adao, L.B., Mbona, N., Downsborough, L., Nienaber, S., 2011. Technical Report for the Freshwater Ecosystem Priority Areas Project. WRC Report No. 1801/2/11. Water Research Commission, South Africa. Nel, J.L., Roux, D.J., Driver, A., Hill, L., Maherry, A.C., Snaddon, K., Petersen, C.R., SmithAdao, L.B., Van Deventer, H., Reyers, B., 2016. Knowledge co-production and boundary work to promote implementation of conservation plans. Conserv. Biol. 30, 176–188. https://doi.org/10.1111/cobi.12560. Reed, M.S., Vella, S., Challies, E., de Vente, J., Frewer, L., Hohenwallner-Ries, D., Huber, T., Neumann, R.K., Oughton, E.A., del Ceno, J.S., van Helden, H., 2018. A theory of participation: what makes stakeholder and public engagement in environmental management work? Restor. Ecol. 26, S7–S17. https://doi.org/10.1111/rec.12541. Robins, L., Dovers, S., 2007. NRM regions in Australia: the ‘haves’ and the ‘have nots’. Geogr. Res. 45, 273–290. https://doi.org/10.1111/j.1745-5871.2007.00460.x. Roux, D.J., Nel, J.L., 2013. Freshwater conservation planning in South Africa: milestones to date and catalysts for implementation. Water S.A. 39, 151–164. https://doi.org/ 10.4314/wsa.v39i1.15. Roux, D.J., Nel, J.L., Cundill, G., O'Farrell, P., Fabricius, C., 2017. Transdisciplinary research for systemic change: who to learn with, what to learn about and how to learn. Sustain. Sci. 12, 711–726. https://doi.org/10.1007/s11625-017-0446-0. Roux, D.J., Rogers, K.H., Biggs, H.C., Ashton, P.J., Sergeant, A., 2006. Bridging the science-management divide: moving from unidirectional knowledge transfer to knowledge interfacing and sharing. Ecol. Soc. 11. Rowe, G., Marsh, R., Frewer, L.J., 2004. Evaluation of a deliberative conference. Sci. Technol. Hum. Values 29, 88–121. https://doi.org/10.1177/01622439032591. Russell, A.W., 2013. Improving legitimacy in nanotechnology policy development through stakeholder and community engagement: forging new pathways. Rev. Policy Res. 30, 566–587. https://doi.org/10.1111/ropr.12037. Schreier, M., 2012. Qualitative Content Analysis in Practice. Sage Publications, London. Taylor, M., Kent, M.L., 2014. Dialogic engagement: clarifying foundational concepts. J. Public Relat. Res. 26, 384–398. https://doi.org/10.1080/1062726X.2014.956106. Tengö, M., Brondizio, E.S., Elmqvist, T., Malmer, P., Spierenburg, M., 2014. Connecting diverse knowledge systems for enhanced ecosystem governance: the multiple evidence base approach. Ambio 43, 579–591. https://doi.org/10.1007/s13280-0140501-3. Thomas, G., 2011. How to Do Your Case Study – A Guide for Students and Researchers. Sage Publications, Los Angeles, CA. van Kerkhoff, L., Lebel, L., 2006. Linking knowledge and action for sustainable development. Annu. Rev. Environ. Resour. 31, 445–477. https://doi.org/10.1146/ annurev.energy.31.102405.170850. Walsh, J.C., Dicks, L.V., Sutherland, W.J., 2014. The effect of scientific evidence on conservation practitioners' management decisions. Conserv. Biol. 29, 88–98. https:// doi.org/10.1111/cobi.12370. Walter, A.I., Helgenberger, S., Wiek, A., Scholz, R.W., 2007. Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method. Eval. Program Plann. 30, 325–338. https://doi.org/10.1016/j.evalprogplan.2007.08. 002. Webler, T., 1995. “Right” discourse in citizen participation: an evaluative yardstick. In: Renn, O., Webler, T., Wiedemann, P. (Eds.), Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse. Kluwer Academic Publishers, Dordrecht, pp. 35–86. Wickson, F., Carew, A.L., Russell, A.W., 2006. Transdisciplinary research: characteristics, quandaries and quality. Futures 38, 1046–1059. https://doi.org/10.1016/j.futures. 2006.02.011. Wyborn, C., Datta, A., Montana, J., Ryan, M., Leith, P., Chaffin, B., Miller, C., van Kerkhoff, L., 2019. Co-producing sustainability: reordering the governance of science, policy, and practice. Annu. Rev. Environ. Resour. 44, 1–28. https://doi.org/10.1146/ annurev-environ-101718-033103. Wynne, B., 2014. Further disorientation in the hall of mirrors. Public Underst. Sci. 23, 60–70. https://doi.org/10.1177/0963662513505397. Yin, R.K., 2014. Case Study Research: Designs and Methods, fifth ed. Sage Publications, Los Angeles.
Acknowledgements Human ethical research clearance for the Australian interviews was granted on 26/6/2015 by the ethics coordinator at the School of Communication and Arts, University of Queensland. The Chair of the Science and Medical DERC at the Australian National University gave clearance for the South African interviews on 15/04/2016 (protocol 2016/077). This research was supported by an Australian Government Research Training Program (RTP) Scholarship and a CSIRO postgraduate studentship. Declarations of interest: none. References Armitage, D., Berkes, F., Dale, A., Kocho-Schellenberg, E., Patton, E., 2011. Co-management and the co-production of knowledge: learning to adapt in Canada's Arctic. Glob. Environ. Chang. 21, 995–1004. https://doi.org/10.1016/j.gloenvcha.2011.04.006. Arnstein, S.R., 1969. A ladder of citizen participation. Am. Inst. Planners 35, 216–224. Bucchi, M., 2008. Of deficits, deviations and dialogues: theories of public communication of science. In: Bucchi, M., Trench, B. (Eds.), Handbook of Public Communication of Science and Technology. Routledge, Oxon, pp. 57–76. Chilvers, J., 2008. Deliberating competence: theoretical and practitioner perspectives on effective participatory appraisal practice. Sci. Technol. Hum. Values 33, 155–185. https://doi.org/10.1177/0162243907307594. Cook, C.N., Mascia, M.B., Schwartz, M.W., Possingham, H.P., Fuller, R.A., 2013. Achieving conservation science that bridges the knowledge–action boundary. Conserv. Biol. 27, 669–678. https://doi.org/10.1111/cobi.12050. CSIRO, 2014. About AdaptNRM. http://adaptnrm.csiro.au/about-adaptnrm/, Accessed date: 28 November 2016. Cvitanovic, C., McDonald, J., Hobday, A., 2016. From science to action: principles for undertaking environmental research that enables knowledge exchange and evidencebased decision-making. J. Environ. Manag. 183, 864–874. https://doi.org/10.1016/j. jenvman.2016.09.038. Davies, S.R., 2013. The rules of engagement: power and interaction in dialogue events. Public Underst. Sci. 22, 65–79. https://doi.org/10.1177/0963662511399685. Dunn, G., Laing, M., 2017. Policy-makers perspectives on credibility, relevance and legitimacy (CRELE). Environ. Sci. Policy 76, 146–152. https://doi.org/10.1016/j. envsci.2017.07.005. Dunn, G., Bos, J., Brown, R., 2018. Mediating the science-policy interface: insights from the urban water sector in Melbourne, Australia. Environ. Sci. Policy 82, 143–150. https://doi.org/10.1016/j.envsci.2018.02.001. Edelenbos, J., Van Buuren, A., van Schie, N., 2011. Co-producing knowledge: joint knowledge production between experts, bureaucrats and stakeholders in Dutch water management projects. Environ. Sci. Policy 14, 675–684. https://doi.org/10.1016/j. envsci.2011.04.004. Elo, S., Kyngäs, H., 2008. The qualitative content analysis process. J. Adv. Nurs. 62, 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x. Fazey, I., Bunse, L., Msika, J., Pinke, M., Preedy, K., Evely, A.C., Lambert, E., Hastings, E., Morris, S., Reed, M.S., 2014. Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Glob. Environ. Chang. 25, 204–220. https://doi.org/10. 1016/j.gloenvcha.2013.12.012. Fazey, I., Evely, A.C., Reed, M.S., Stringer, L.C., Kruijsen, J., White, P.V.L., Newsham, A., Jin, L., Cortazzi, M., Phillipson, J., Blackstock, K., Entwistle, N., Sheate, W., Armstrong, F., Blackmore, C., Fazey, J., Ingram, J., Gregson, J., Lowe, P., Morton, S., Trevitt, C., 2013. Knowledge exchange: a review and research agenda for environmental management. Environ. Conserv. 40, 19–36. https://doi.org/10.1017/ S037689291200029X. Fischer, A.R.H., Wentholt, M.T.A., Rowe, G., Frewer, L.J., 2014. Expert involvement in policy development: a systematic review of current practice. Sci. Public Policy 41,
8
Contents lists available at ScienceDirect
Journal of Environmental Management journal homepage: www.elsevier.com/locate/jenvman
Research article
Principles for evaluating knowledge co-production in natural resource management: Incorporating decision-maker values
T
Ruth A. O'Connora,g, , Jeanne L. Nelb,c, Dirk J. Rouxb,d, Lilly Lim-Camachoe, Lorrae van Kerkhofff, Joan Leachg ⁎
a
Climate Change Institute, The Australian National University, Canberra, Australia Sustainability Research Unit, Nelson Mandela University, South Africa c Wageningen Environmental Research, Wageningen University, the Netherlands d Scientific Services, South African National Parks, South Africa e CSIRO Agriculture and Food, Brisbane, Australia f Fenner School of Environment and Society, The Australian National University, Canberra, Australia g Centre for the Public Awareness of Science, The Australian National University, Canberra, Australia b
ARTICLE INFO
ABSTRACT
Keywords: Communicative competence Legitimacy Public participation Policy-makers Science communication Science-policy interface
A key reason for undertaking transdisciplinary processes such as knowledge co-production in natural resource management is to access and apply different knowledge systems to complex issues. However, the value of coproduction is predominantly framed by academics. They have focused on research design and outcomes in the form of ‘science informing policy’. In this paper we build a more inclusive and holistic framing of knowledge coproduction incorporating values of non-academic participants, and values related to the participatory process. Specifically, we examine how knowledge is communicated and deliberated upon and the requirements for this to be done effectively. We draw upon empirical data from interviews with natural resource managers who participated in two case studies of knowledge co-production in Australia and South Africa. Their perspectives are captured in eight evaluation principles that build upon existing evaluation frameworks for public participation. Critically, decision-makers valued science-based outputs not just as salient knowledge sources, but to give legitimacy to their decisions. This need for legitimacy necessitates transparency, fairness and inclusivity in knowledge selection, participation and dialogue within knowledge co-production processes. The practice-based knowledge of decision-makers was important for contextualizing and applying science to specific decision contexts. Another key finding is that communicative competence is central to the process of knowledge coproduction because it enables participants to critically explore and understand the knowledge claims of others.
1. Introduction Participatory and transdisciplinary processes are useful mechanisms to achieve complex natural resource management goals such as sustainable development and biodiversity conservation (van Kerkhoff and Lebel, 2006; Hegger et al., 2012; Nel et al., 2016). One reason for this is that these processes enable both academics and non-academics to apply their diverse knowledge and expertise to public decision making. However, the way we currently value these processes is informed predominantly from an academic perspective where success is equated with research impact through the integration of science in policy and management (Fazey et al., 2014; Hansson and Polk, 2018). This focus risks ignoring or subordinating the values and knowledge of non-academic participants. It also risks minimising values inherent in the ⁎
communicative processes associated with co-production, or their outcomes such as social learning and network building (Walter et al., 2007; Roux et al., 2017). These outcomes may have intrinsic value or be necessary pre-cursors to science application. One key non-academic group of participants are decision-makers who we define as individuals situated within organisations who have a socially recognised role to make decisions on behalf of others and who can apply science for the public good. A variety of participatory approaches to natural resource management (NRM) exist, each with slightly different emphases including research (e.g. transdisciplinary research), governance, or communication (e.g. knowledge exchange). All of these approaches are based on collaborative partnerships between academics and non-academics, each with valuable knowledge and expertise to contribute to NRM (Wickson
Corresponding author. Climate Change Institute, The Australian National University, Canberra, ACT, 2601, Australia. E-mail address: [email protected] (R.A. O'Connor).
https://doi.org/10.1016/j.jenvman.2019.109392 Received 17 December 2018; Received in revised form 9 August 2019; Accepted 10 August 2019 0301-4797/ © 2019 Elsevier Ltd. All rights reserved.
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
et al., 2006; Fazey et al., 2013; Wyborn et al., 2019). Application of this knowledge pre-supposes scientists, decision-makers and other knowledgeable participants have communicative competence. That is, all participants have the ability to use language to communicate rationally and create the best possible understandings within what is known through making and defending claims (Habermas, 1984). However, the academic imperative for science to inform policy means the value of communication in participatory processes is outcome oriented in the form of science translation, fostering science legitimacy, and creating commitment to participatory exercises (Mitton et al., 2007; Hegger et al., 2012; Cook et al., 2013). Participants with “strong communication skills” or 3rd party communication specialists such as knowledge brokers can help achieve these goals (Cvitanovic et al., 2016). Communicative competence is a richer concept than translation that frames and values communication as a social process rather than a means to achieve pre-determined outcomes. A focus on communicative competence and how to achieve it, therefore, allows for a broader definition of success in participatory and transdisciplinary activities that can incorporate non-academic values. Our focus is on knowledge co-production as one type of participatory approach in NRM that can stand alone or contribute to other approaches such as transdisciplinary research. We therefore define knowledge co-production according to Armitage et al. (2011, p. 996) as the “collaborative process of bringing a plurality of knowledge sources and types together to address a defined problem and build an integrated or systems-oriented understanding of that problem”. Developing evaluation criteria for knowledge co-production remains a challenge because of its variety of forms, contexts, and participants who may have differing views of what is valuable. Hegger et al. (2012) identified a set of predominantly process-based “success factors” in co-production and the relative merits of evaluating process and outcomes of participatory processes has been discussed (Walter et al., 2007). Advocating a standard set of evaluation criteria is generally inappropriate for complex, value laden endeavours (Fazey et al., 2014). However, broader evaluation principles may be useful for practitioners as prompts to consider and adapt to their context. They can also be useful for promoting reflection about what constitutes good participatory practice and when applied, allow improvement in practice from an empirical base. The aims of this paper are to firstly test and broaden academic framings of knowledge co-production by analysing the perspectives of NRM decision-makers who have participated in these processes. Secondly, we seek to explore the specific values of science communication and what is required to do it effectively as part of knowledge co-production. We see this focus on communicative process as complementing current work on design and impact in participatory processes (Walter et al., 2007; Reed et al., 2018). We start by proposing a set of value propositions for co-production grounded in communication theory and drawn from evaluation frameworks for public participation. We then present the results of two knowledge co-production case studies from Australia and South Africa that test our propositions. Finally, we use the empirical evidence from the cases to modify our propositions into a set of eight evaluation principles for knowledge co-production.
Table 1 Value propositions for knowledge co-production based on evaluation frameworks for public participation (based on; Webler, 1995; Rowe et al., 2004; Chilvers, 2008). Propositions 1. Communication processes should enable participants to make and question claims to create mutual understanding of information presented. 2. Participation by individuals with a diversity of knowledge and experience relevant to the management issue is of value. 3. Co-production requires legitimacy and acceptance of outputs which can be achieved through fairness and transparency in the process. 4. The input of resources in addition to knowledge is of value to enable participation and facilitate communication. 5. Outputs and outcomes of value to participants are necessary to justify their time commitment.
(Bucchi, 2008, p. 73). Current evaluation frameworks are strongly grounded in communication theory and do not explicitly consider decision-makers as participants (or ‘publics’), so we present a set of value propositions in Table 1 about the value of their engagement adapted from five recurring evaluation themes. The first proposition relates to the nature of dialogue and deliberation (Table 1). Specifically, how effectively technical information is communicated to non-academics and how well both academic and nonacademic participants are able to make sense of each other's claims i.e. their communicative competence (Habermas, 1984). The second proposition relates to PES as a mechanism of deliberative democracy, i.e. PES has value in a democratic sense when citizens from a broad range of backgrounds or groups are represented, given a voice and empowered through access to knowledge. Representation is also valued in an epistemic sense when a variety of knowledge and perspectives encourages testing of issue framing or the implications of decisions, thus increasing the robustness of decisions (Russell, 2013). We posit that in knowledge co-production involving decision-makers, the value of diverse representation is primarily epistemic rather than democratic (Table 1). Related to this, our third proposition recognises that the full potential of co-production can only be realised when different sources of knowledge are considered equally and openly. Transparency, fairness and independence of PES processes are also necessary for legitimacy and acceptance of both the process and its outputs. We view legitimacy in co-production as “conformity to recognised principles or accepted rules and standards” (Heink et al., 2015, p2) rather than factual acceptance of decisions. The fourth proposition describes the necessity and value of process inputs such as support for individuals to attend or participate, time for participants to develop competent understandings of the material, and expert facilitation of the process. Finally, proposition 5 (Table 1) relates to the value of outputs and outcomes which for decision-makers may include building trust between people and institutions, or forming new professional partnerships (Roux et al., 2006). 3. Methodology
2. Analytical framework
We used a case study approach to explore the complex social processes of knowledge co-production in depth and in context (Yin, 2014). We focus on a single policy sector—natural resource management (NRM)—recognising that cases in different policy sectors could add unnecessary complexity to the results because of unique contextual issues or variables. With its management imperative, NRM provides an opportunity for exploring how the meanings, implications and application of science are negotiated. Two national knowledge co-production projects (Table 2) were selected to allow iterative refinement of our initial propositions.
We reviewed evaluation frameworks for public participation in public decision making to develop an analytical foundation for data collection and analysis. Public participation broadly refers to activities that seek citizen involvement in decision making processes affecting them (Arnstein, 1969). Public participation or public engagement with science (PES) is also a model of science communication defined as “the diversified set of situations and activities, more or less spontaneous, organised and structured, whereby non-experts become involved, and provide their own input to, agenda setting, decision-making, policy forming and knowledge production processes regarding science” 2
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Table 2 Comparison of the AdaptNRM and National Freshwater Ecosystems Protected Areas (NFEPA) projects.
Project duration Science application Target decision-makers Participants in knowledge co-production mentioned by interviewees Engagement mechanisms & pathways
a
AdaptNRM
NFEPA
2013–2016 A variety of models predicting climate change & changes in species distributions applied to biodiversity & weed management. Planners from regional NRM groups across Australia.
2008–2011 National-scale systematic conservation planning & prioritization system applied to river, wetland & estuary biodiversity.
Target decision-makers plus: Other specialists (e.g. monitoring & evaluation, ecology); Private consultants contracted to NRM groups; Managers. Set scope of project (participants nominated priority adaptation domains for application of climate projections). Product development Written comments on draft guides; Participation in discussions about draft guides (e.g. webinars, teleconferences & workshops); Provision of inputs to guides/co-development of case studies.
Environment & water managers & planners in national, provincial & catchment level agencies across South Africa. Target decision-makers plus: Private consultants working in the environment & water sectors; Boundary spanners. Via project governance; Set scope of project (discussions to align needs across water & environment sectorsa); Input to methods development. Product development Collated, reviewed & refined data inputs individually & in groups; Reviewed & tested products; Provided input on priorities & data through processes of dialogue & co-learning in a variety of workshops at national & catchment scales.
Building on existing cross-sector policy objectives (Nel et al., 2011).
3.1. Case study 1: the AdaptNRM project (Australia)
government officials and consultants. Interviews ranged from 35 to 60 min in duration. The NFEPA case was viewed as a valuable comparison to AdaptNRM being similar in scale and range of communication activities, but a potential contrast in timeframe since completion, regulatory contexts, and science application (Table 2). The five years since project completion had advantages and disadvantages. The key opportunity was to describe policy and other outcomes that may only manifest in the years after project completion (Rowe et al., 2004). The longevity of more immediate outcomes like those described in AdaptNRM could also be explored. These opportunities were considered to outweigh challenges associated with people leaving the sector and loss of detailed recall of specific events over time.
The AdaptNRM project aimed to make existing science about climate adaptation more accessible to regional NRM groups by co-developing a suite of user-targeted guides and data sets (CSIRO, 2014). It involved a team of scientists engaging with regionally-based decisionmakers from across Australia. The mandate of regional NRM groups is to plan and deliver programs that “support healthy and productive country, viable communities and sustainable industries” (Natural Resource Management Regions Australia). In some jurisdictions, NRM groups function as quasi state-based organisations, with the legislative power to implement their own plans. Elsewhere, they function as nongovernment organisations and work with land managers to implement plans through on-ground works. All 56 NRM groups existing at the time were invited to participate in AdaptNRM. The project team had a contact list of 129 individuals and 25 of these representing 23 groups from every state and territory except one were interviewed for the case study. The interviews were conducted in the final stages of the project from September to October 2015 and ranged from 22 to 80 min in duration. Knowledge co-production in AdaptNRM involved a suite of communication activities initiated by the project team which are outlined in Table 2. AdaptNRM was viewed, therefore as a good case of various communication activities involving both decision-makers and science experts for knowledge co-production in NRM.
3.3. Data collection and analysis Qualitative data was collected through semi-structured interviews conducted by the lead author with individual decision-makers. Semistructured interviews allowed the interviewer to simultaneously cover a standard list of topics (the value propositions in Table 1) while also allowing the interviewees to articulate their own values and framings of the two projects (Thomas, 2011). Decision-makers were asked what they valued about their project, their drivers for engagement, enabling and constraining factors. Interviews were conducted face-to-face, via telephone or online and were recorded and transcribed. The transcripts were analysed using qualitative content analysis (Schreier, 2012) which is well suited to exploratory case studies with interview data (Elo and Kyngäs, 2008). It condenses and describes content and the process of categorization provides a means of understanding underlying phenomena. Transcript content was coded using QSR International's NVivo 11 Software. Categories were derived from the data in an inductive process resulting in 22 categories for NFEPA and 21 for AdaptNRM. These categories were clustered into five higher level themes that broadly reflected the value propositions upon which the questions were based. The lead author did all the coding which facilitates consistency. In addition, both coding frames were validated by recoding three transcripts and checking for mismatches. New categories were also added through the validation process. For example, comments about outcomes were more detailed for NFEPA and therefore split into personal, institutional and societal outcomes.
3.2. Case study 2: the National Freshwater Ecosystem Priority Areas project (South Africa) The National Freshwater Ecosystem Priority Areas (NFEPA) project ran as a partnership between research institutions, South African government agencies, and environmental non-government organisations. One goal was to identify freshwater ecosystems such as rivers and wetlands that should be prioritized for biodiversity protection. Another was to promote the adoption of co-produced outputs by relevant agencies (Nel et al., 2011). NFEPA was run by a project team comprising both researchers and managers. They had a contact list of over 350 people from water and environment agencies across the country. Five years after the completion of NFEPA, many of these original participants had either changed role or left their organisations. Eighteen decision-makers from six out of nine provinces were interviewed between May and July 2016 comprising national and provincial 3
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
4. Results
knowledge input was that from the decision-makers themselves, particularly practical knowledge about how to address NRM issues within institutional, regulatory and social contexts. Decision-makers often had science backgrounds as well as: local knowledge (e.g. of species distribution); knowledge about the decision context (e.g. how products could be implemented); and the nature of stakeholders. A new consideration in our second proposition is that elicitation of tacit knowledge (particularly that of non-academics) may be necessary and a valuable contribution of co-production projects: “it really was a fantastic exercise for extracting a lot of data that was otherwise just sitting in people's heads, being lost never having been really extracted or written up” (NFEPA09). Decision-maker knowledge seemed to be necessary to contextualise the available science to a specific decision context, with its particular biogeographic, legislative, and social conditions. The process for deciding which scientists to involve, or which technical approach to take, in knowledge co-production was unclear in both cases and indicates that considerations about process legitimacy also apply to knowledge and method selection. Decision-makers, on the whole, were unaware of how these decisions had been made: “that was in the hands of the scientists” (NFEPA06). While both projects employed collaborative processes in issue-framing, decisions on scope and which scientists to invite were made by the project team (AdaptNRM) and an advisory group (NFEPA). In addition, interviewees in both cases mentioned relevant alternative approaches and knowledge sources that had been either rejected or not considered for reasons unclear to them. These types of opaque decision-making processes may threaten the legitimacy of knowledge co-production and were an unforeseen element of our third proposition on fair and transparent process (Table 1).
We focus here on five themes from the qualitative content analysis that aligned to the value propositions presented in Table 1, but note that other useful insights about the background and motivations of decision-makers emerged. One of the key emergent findings across both cases was that the issue of legitimacy was more complex than originally envisaged. Therefore, in addition to discussing legitimacy as a standalone concept (4.3), we also discuss it under the other three processoriented propositions (4.1, 4.2 and 4.4). We illustrate our results with interviewee quotes that are numbered according to identity. This demonstrates the breadth of responses drawn upon while maintaining confidentiality. 4.1. Decision-makers valued interactive communication Interactive communication processes and their direct outcomes were valued by decision-makers in both cases in a variety of ways that exemplify and expand our first proposition (Table 1). Some participants valued making and questioning claims rather than passively receiving information. For example, in AdaptNRM: “we could amend comments or add additional comments which we might not have thought of before but were triggered by speaking to other people or listening to other people. So that particular side of it I found very useful” (NRM16); and in NFEPA: “We sat around the tables and we talked. And this one said this and that one said no, no remember this and remember that, oh yes I forgot about that!” (NFEPA11). Making and questioning claims appeared to prompt cognitive processing and learning. It also stimulated reflexivity which allowed for re-conceptualisation of issues or developing new ways of thinking: “being involved in NFEPA sort of forced me to take a step back and look at what's happening in the region, what's happening in the country” (NFEPA05). There was also evidence that the inherently social processes of dialogue among colleagues and scientists motivated some participants to critically engage with content, who may not otherwise have done so: “the modules were new information for me and it was easier for me to understand if I was actually being made to interrogate it a bit more instead of just accepting it as the approach we should be using” (NRM01). Another important value of engagement and critical thinking was for decisionmakers to step from understanding information to applying it: “having small group discussions and being able to say: well that won't work for us, how about this?” (NRM23). For example, AdaptNRM decision-makers needed to consider how to take climate models and use them to revise their weed management strategies. In relation to our third proposition regarding legitimacy (Table 1), an important caveat for realising the potential benefits from participation was that the knowledge and experience of all participants be respected: “I think that really helped - it was a transparent open process” (NFEPA08). This expands our proposition regarding the importance of legitimacy suggesting that equality in communicative competence between decision-makers and scientists is necessary for process legitimacy and acceptance.
4.3. Decision-makers valued processes and outputs that legitimised decisions Decision-makers in both cases commonly valued the process and outputs of knowledge co-production as mechanisms to give legitimacy to their decision-making: “one of the ways we can ensure independence is to validate our assessments through the use of credible scientific literature” (NFEPA interviewee, consultant), and “if we were doing things that weren't based in science then we'd lose our credibility. So quickly.” (NRM17). This legitimacy was seen as key to the acceptance of their decisions by stakeholders such as resource users and managers (e.g. landholders, fishers, and mining companies) or other decision-makers (e.g. NRM Boards, departmental heads). These results suggest that for decisionmakers, co-production is not only about solving management problems; it is also about the acceptance of their proposed actions by stakeholders outside the co-production process. While it was clear that most thought basing decisions on science would contribute to legitimacy, the idea that the process by which science informs decisions requires benchmarks of fairness and transparency was less consistent. 4.4. Decision-makers valued support to participate and engage meaningfully Our fourth proposition about the value of project inputs (Table 1) was exemplified in both cases. Support to participate either in person “if all the meetings are up in Johannesburg or Pretoria then you have some limitations to involvement” (NFEPA05) or via video link: “the webinars were excellent.” (NRM17) was valued. The expertise of the AdaptNRM and NFEPA project teams in facilitation of group sessions that enabled communicative competence was also valued: “It was a very professional team. I really think some of the best facilitation on break away groups and special groups discussions” (NFEPA12). This facilitation was partly about decision-maker inputs being heard, valued and responded to. Planning undertaken by project teams who collated and synthesised inputs for well-targeted group sessions allowed decision-makers to engage effectively with minimal effort: “I think the overall way they've run the project has actually been particularly good. They've had structured involvement in the development of the modules” (NRM07). Well organised processes prompted some decision-makers to “make time” to consider new
4.2. Decision-makers valued science and the elicitation of practitioner knowledge Our second proposition foreshadowed the importance of science to decision-makers and this was exemplified in the results. Interviewees in both cases consistently considered scientific knowledge as critical for their work. When asked about the value of science to their work, terms such as “core”, “fundamental”, “critical”, and “cornerstone” were used. One South African interviewee captured a common view: “as a conservation agency, how do you work without science?” (NFEPA05). Coproduction projects can provide decision-makers a means of accessing relevant science and scientists “Probably the major benefit to this project I think is the interaction with scientists” (NRM07). Another critical 4
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Table 3 A summary of outcomes from knowledge co-production described by decision-makers. Outcome level
Types of outcomes emerging from case studies
Individual
Changes in knowledge, understanding and skills Changes in attitude Changes in practice New or improved professional relationships Knowledge and information more accessible New knowledge and information incorporated in policy, strategy and plans Changes in organisational culture Step changes in narrative
Organisation Cross-organisation/Institution
knowledge: “you've got that many other things going on in your world that you turn your attention to this for the space that the project's created for you which is usually a meeting and when you're bussing to the meeting …” (NRM12). The capacity of decision-makers’ host organisations appeared to limit participation in some instances regardless of project inputs. Jurisdictions with the least resources were under-represented in both AdaptNRM and NFEPA (project leader pers. comm.) and we were unable to recruit participants from these same jurisdictions. In South Africa, low levels of engagement with NFEPA were attributed to limited staff numbers and expertise of host organisations (project leader pers. comm.).
(NFEPA10). Another may be changes to how organisations conceptualise and approach issues “NFEPA inspired a lot of this thinking of how can we use mapping to improve our decision-making” (NFEPA08). Multiple participatory processes including NFEPA may have also resulted in re-conceptualisation or re-framing of issues across institutions (Table 3). Some South African decision-makers felt NFEPA had played an important part in a change in the national conservation narrative: “the precept is there [the importance of aquatic ecosystems] so when we go into meetings, we don't have to try and re-sell the idea … we've got traction” (NFEPA16). These findings exemplify our fifth proposition (Table 1) but indicate it may be useful to differentiate between individual, institutional and cross-institutional outcomes.
4.5. Decision-makers valued beneficial project outputs and outcomes
5. Evaluation principles
Learning was one of the most common outcomes mentioned by participants in both case studies: “I've learnt from that. You know apart from the actual content of it, just their approach and how they've tackled the problem” (NRM25). Participants learned about science and from the practice-based experience of peers, learned about the project outputs, and developed skills in the use of products: “the benefit of being involved in the project from its development stage—in the sense of the stakeholder meetings—is that we got to understand a lot more of the project to implement it” (NFEPA13). In this way, individual learning outcomes were sometimes pre-cursors to the use of project outputs. Use of the outputs by participants can be seen as a change in practice. We regard this as one of the outcomes of knowledge co-production manifest at the individual level (Table 3). Decision-makers also valued co-production as a means of building and maintaining professional relationships with colleagues and scientists. This relationship-building was inextricably tied to social, communicative processes: “I actually think it's [the value of the project] more about relationships than accessibility (NRM04). Examples were given of how relationships allowed for easier sharing of knowledge as well as facilitating collaborations in future projects. However, the importance of relationship-building depends on context with evidence that the NRM community of practice was already well established in South Africa so the challenge was more about including newcomers: “a real risk of using networks is to what extent are we pushing the boundaries of those networks and to what extent are we quite comfortable and complacent working with the people that we know” (NFEPA09). Participants also valued better access to the knowledge of both colleagues and scientists: “I think the most use was seeing the diversity of approaches from the different regions and chatting to people afterwards” (NRM23) and “Although I know about specialists, I rarely talk to them” (NFEPA12). Increased accessibility of knowledge and information was through both project outputs and face-to-face interaction. Institutional outcomes were evident mainly from NFEPA which was completed five years before interviews in contrast to AdaptNRM that was in its wrap up phase. A clear institutional outcome was use of project outputs by organisations with 23 specific and generic policy and management instruments mentioned. Changes to organisational processes and culture were harder to identify and attribute but one example may be non-participants using project outputs: “And in our processes we specify, we actually force people to look at the NFEPAs”
5.1. Level of transparency, inclusivity and fairness The case studies indicate that principles of transparency, fairness and inclusivity cut across the different elements of the co-production process and are critical for the legitimacy and acceptance of those processes and their outputs (Fig. 1). Specifically, communication within co-production requires equality in competence (principle 1), the knowledge inputs for co-production require transparent selection of expertise (principle 2) and support for participation needs to promote inclusivity (principle 3). 5.1.1. Principle 1: knowledge co-production should aim for participants with different types of knowledge to have equal communicative competence We argue that inclusive and fair dialogue and deliberation is necessary in knowledge production to provide an equal opportunity for different perspectives to be expressed, understood and questioned. The importance of being listened to and being able to contribute knowledge and experience was often mentioned by interviewees. Achieving this reflects a commitment by projects to knowledge symmetry. We were unable to analyse the dialogue between decision-makers and scientists, but in PES and to a lesser extent transdisciplinary research, the scientific voice can dominate in discussions with non-academics (Davies, 2013; Hansson and Polk, 2018). This means the challenge for co-production may be fair consideration of decision-maker and other nonacademic knowledge and expertise. As in PES, knowledge co-production may not realise its participatory potential if non-academic participation is tokenistic or for purely instrumental goals such as getting “buy-in” for products (Rowe et al., 2004). 5.1.2. Principle 2: knowledge co-production should have transparent, documented processes of determining knowledge inputs and technical approaches Decision-makers commonly felt that decisions based on science would be considered legitimate and accepted by stakeholders outside co-production. One reason that this is a questionable assumption is that complex decision contexts in NRM can draw on multiple sources of science. This means that numerous quite different decisions could be based on science. In both our cases, implicit or explicit choices were 5
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
Fig. 1. Eight elements of knowledge co-production that contribute to robust, informed and legitimate natural resource management decisions.
5.1.3. Principle 3: projects should be able to support groups and individuals with relevant knowledge and experience to participate The level of engagement by decision-makers in co-production appeared to vary across organisations. One factor limiting participation appeared to be low capacity of host organisations in terms of staff and resources. Both projects tried to compensate for this by funding travel and using technology that enabled individuals to participate but did not pay for participant time. Whether this type of investment is necessary will depend on the context of knowledge co-production. For example, support for wide participation may be considered necessary if socially robust decisions are important (Hegger et al., 2012), or if participants can increase the relevance of outputs (Reed et al., 2018). In an NRM context, broad representation may be a mechanism to avoid reinforcing disadvantage when organisations lacking the capacity to engage fail to accrue the benefits of co-production and are left further behind (Robins and Dovers, 2007). Our results suggest project support may be insufficient to achieve broad institutional representation, particularly where institutional capacity is limited. We suggest support for participation is evaluated according to how well it enables broad participation across stakeholder organisations and its flexibility to respond to participant needs.
be done effectively have not been unpacked. Decision-makers in our two case studies valued interactive communication as a mechanism for individual learning, developing mutual understanding of different knowledge sources, and the development of relevant, useable outputs. Currently, interactive communication in participatory processes tends to be framed instrumentally as a means to achieve pre-determined endpoints such as the long-term commitment of participants (Hegger et al., 2012). This particular framing also conflates the social value of communication with its core function of producing and negotiating meanings (Wynne, 2014). This function is particularly necessary in co-production where there are different types of knowledge to be considered for application (in our case) to NRM issues. This necessitates enabling both academic and non-academic participants to question the potential application of knowledge in a given decision context which is a step beyond the more passive characterisation of communication as “listening” for learning (Reed et al., 2018). We propose that explicit procedures and resources are needed so participants can meaningfully discuss the knowledge and information available and formulate their own views about it. In public participation involving citizens, communicative competence is evaluated according to established procedures rather than traits of individuals (Webler, 1995). While co-production may be easier among motivated individuals with strong communication skills (Cvitanovic et al., 2016) we argue that the primary criterion for participation in knowledge coproduction (as opposed to implementation) is the knowledge an individual can contribute. Projects can support communicative competence by, for example, employing facilitation experts with strong communication skills (discussed further in section 5.4). Procedures for promoting competence include communicating science in ways that are relevant and accessible to the non-specialist (Chilvers, 2008) and where meanings, assumptions, and uncertainties are clear (Roux et al., 2006). Requirements for the analysis and communication of other knowledge forms are less clear, but should at a minimum involve acknowledging their potential value and providing the opportunity for them to be discussed.
5.2. Interactive communication
5.3. Knowledge inputs
5.2.1. Principle 4: knowledge co-production should establish communicative competence The need for dialogue and deliberation in knowledge co-production is accepted, but its purposes, functions, and the pre-requisites for it to
5.3.1. Principle 5: knowledge co-production processes should identify and elicit non-academic knowledge and explore the idea of ‘relevant knowledge’ A strong theme emerging from both case studies was the importance of decision-maker knowledge in co-production processes. Bureaucratic
made about which expertise to include. It is unlikely a given project can, or should, include all relevant expertise so choices are likely inevitable. Having documented, transparent processes of selection not only promotes trust, it allows for the evaluation of the comprehensiveness of the knowledge engaged with, and possible biases (Fischer et al., 2014). The idea of multiple, and possibly differing, scientific perspectives and how they are chosen has not been a focus in knowledge co-production where greater attention has been given to the quality of research knowledge (Dunn and Laing, 2017). Collaborative consideration of a comprehensive range of knowledge sources and expertise should be considered in co-production because these choices are a significant manifestation of power in participatory processes (Fazey et al., 2013).
6
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
knowledge, defined as the knowledge held by bureaucratic officials (civil servants) about political and administrative procedures and processes (Edelenbos et al., 2011), is a recognised input to co-production at the science-policy interface. This characterisation does not adequately encompass the knowledge of the AdaptNRM and NFEPA decision-makers which also included local knowledge of ecosystems and stakeholders, and practice-based knowledge important for conceptualising how science could be applied in specific decision contexts. Our findings also contrast with a study of marine park managers whose lack of scientific training was seen as a barrier to successful knowledge exchange (Cvitanovic et al., 2016). Studies of research uptake in NRM have also shown that decisionmakers may not always have up to date or comprehensive knowledge to draw on (Walsh et al., 2014). While the nature and importance of decisionmaker knowledge will vary according to the decision context, our study shows that co-production at the science-policy interface needs to acknowledge and consider the knowledge inputs of decision-makers. We need to better articulate what that knowledge might consist of and how to judge its quality (Tengö et al., 2014).
Implementing outputs and addressing complex societal problems is an ideal outcome of transdisciplinary processes (van Kerkhoff and Lebel, 2006; Cook et al., 2013) and was an aim of both AdaptNRM and NFEPA. While decision-makers valued relevant useable outputs, their implementation is a problematic benchmark for evaluating co-production. Use of outputs can depend on factors beyond the scope of the process (Lemos and Morehouse, 2005) as illustrated in AdaptNRM, where a change in federal government took climate change off the political agenda making the implementation of regional plans problematic. There is also a conceptual problem with narrowly equating success in knowledge co-production with science informing policy. Apart from ignoring or subordinating other knowledge inputs, it limits the agency of decision-makers to understand science but decide not to apply it based on their own expert knowledge of the decision context. Instead, we suggest a core outcome of knowledge co-production grounded in mutual understandings of available knowledge gained by participants in the process. 5.5.2. Principle 8: knowledge co-production should result in mutual understandings of available knowledge by participants and flow-on institutional benefits associated with the process and its outputs Co-production processes where participants actively engage with different knowledge sources to produce outputs can be viewed as a form of dialogic engagement. A benchmark for good dialogic engagement is participants gaining a mutual understanding of each other's knowledge (Taylor and Kent, 2014). This understanding can then form the basis for determining and applying the best available knowledge and expertise to policy and management. Decision-makers valued learning about the application of science from other practitioners. This was most striking in AdaptNRM where mechanisms for the implementation of science were less obvious. Such social learning is a recognised outcome of coproduction and particularly valued when situated in wider social or institutional networks (Armitage et al., 2011). Learning at a cross-institutional level was evident in South Africa but not Australia. This may reflect the longer period since project completion but is more likely a product of the longer term investment in multiple transdisciplinary activities over time in South Africa involving some of the key NFEPA participants (Roux and Nel, 2013). Strengthened professional relationships among decision-makers and scientists were also valued because they improved access to relevant information and led to other collaborations where costs and benefits could be shared. The value of strong professional networks has also been described in the context of urban water management where relationships built over many years between researchers and managers resulted in quick adoption of research in a crisis situation (Dunn et al., 2018). These findings broaden current framings equating the value of relationships between scientists/ knowledge brokers and decision-makers with implementation of coproduced outputs (Lemos and Morehouse, 2005; Reed et al., 2018).
5.4. Support to enable participation 5.4.1. Principle 6: knowledge co-production processes should provide resources to enable dialogue and deliberation including: facilitation, planning and 3rd spaces The time and effort of dedicated project teams was valued by decisionmakers in both cases. These teams had substantial knowledge of the science and/or management domain and they performed several functions including promoting critical but respectful dialogue and eliciting tacit knowledge. Facilitation can also occur through governance arrangements (Armitage et al., 2011), professional networks (e.g. Dunn et al., 2018) or specialised individual facilitators. Neutrality and skills in social facilitation, and building collaborative networks are recognised as important qualities of boundary-spanning facilitators (Reed et al., 2018). However, our study shows that in co-production processes where it is important to elicit and deliberate upon knowledge, a key function of facilitators is also enabling the communicative competence of participants. In our cases, synthesising knowledge inputs and planning structured, iterative engagement exercises at key milestones were valued because decision-makers were time constrained and national scale projects had large information inputs. Third spaces or shadow spaces outside of normal work environments were also valued as a “change of scene” conducive to informal and facilitated dialogue. The neutrality of such spaces for dialogue and community building has also been emphasised (CSIRO, 2014; Roux et al., 2017; Hansson and Polk, 2018). 5.5. Outputs and beneficial outcomes 5.5.1. Principle 7: knowledge co-production should deliver knowledge based outputs according to the aims and needs of participants and project sponsors Planned co-production activities such as our two case studies generally require a funding source or sponsor, who will have specific requirements. This can take the form of: transdisciplinary research, e.g. developing sustainable cities (Hansson and Polk, 2018); commissioned research, e.g. urban water management (Dunn et al., 2018); or advisory groups made up of researchers, community and government officials (Edelenbos et al., 2011). Therefore, any evaluation of co-production activities needs to address the stated needs of sponsors as well as participants. Meeting multiple goals requires recognition of potential power imbalances among participants, deliberate effort to avoid entrenching these, and remembering the underlying normative aims of coproduction to support societal change (Wyborn et al., 2019) Decisionmakers in both cases valued co-production because of its knowledgebased outputs such as maps, guidelines and decision-tools. Elsewhere decision-makers have expressed a preference for science packaged in a form that can be used to solve problems associated with their institutional mandate (Dunn and Laing, 2017).
6. Conclusions Considering the perspectives of decision-makers who participated in two cases of knowledge co-production has broadened our conceptualisation of its value beyond science informing policy. While NRM decision-makers value science-based outputs as salient knowledge sources, they also seek legitimacy in contested decision contexts such as water management and climate change adaptation. This means the value of co-production cannot be de-coupled from the political imperative for decisions to be accepted by outside stakeholders. We argue that this legitimacy cannot be achieved purely by basing decisions on science but is built through fairness, transparency and inclusivity in all aspects of co-production processes. Our second aim was to explore the value and requirements for effective communication in co-production. A key finding was that effectively harnessing the knowledge of decision-makers and critically examining the scope of science to meet decision-maker needs requires communicative competence among all 7
Journal of Environmental Management 249 (2019) 109392
R.A. O'Connor, et al.
participants. The knowledge of decision-makers can be critical to the application of science in complex decision contexts. Facilitators and boundary spanners can play a key role in enabling meaningful deliberation that can deliver mutual understanding of knowledge—what we see as a fundamental goal of co-production. A next step in the development of knowledge co-production theory and practice is testing principles such as ours in the evaluation of coproduction activities and critiquing the results. We have demonstrated that non-academic knowledge is a critical consideration but methods for assessing the quality of that knowledge—similar to the accepted principles for good science—are lacking. How communicative competence manifests (or not) and the enabling environments for this should be explored through direct observation and analysis of the interactive processes within co-production. Enabling communicative competence will allow the full potential of co-production to be realised through effectively drawing upon the knowledge of all participants.
332–343. https://doi.org/10.1093/scipol/sct062. Habermas, J., 1984. The Theory of Communicative Action, vol. I Reason and the Rationalization of Society, Beacon, Boston. Hansson, S., Polk, M., 2018. Assessing the impact of transdisciplinary research: the usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact. Res. Eval. 27, 132–144. https://doi.org/10.1093/reseval/ rvy004. Hegger, D., Lamers, M., Van Zeijl-Rozema, A., Dieperink, C., 2012. Conceptualising joint knowledge production in regional climate change adaptation projects: success conditions and levers for action. Environ. Sci. Policy 18, 52–65. https://doi.org/10. 1016/j.envsci.2012.01.002. Heink, U., Marquard, E., Heubach, K., Jax, K., Kugel, C., Neßhöver, C., Neumann, R.K., Paulsch, A., Tilch, S., Timaeus, J., Vandewalle, M., 2015. Conceptualizing credibility, relevance and legitimacy for evaluating the effectiveness of science–policy interfaces: challenges and opportunities. Sci. Public Policy 42, 676–689. https://doi.org/10. 1093/scipol/scu082. Lemos, M.C., Morehouse, B.J., 2005. The co-production of science and policy in integrated climate assessments. Glob. Environ. Chang. 15, 57–68. https://doi.org/10. 1016/j.gloenvcha.2004.09.004. Mitton, C., Adair, C.E., McKenzie, E., Patten, S.B., Perry, B.W., 2007. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 85, 729–768. Natural Resource Management Regions Australia. http://nrmregionsaustralia.com.au/ (accessed 21 October 2015).. Nel, J.L., Murray, K., Maherry, A., Petersen, C.P., Roux, D.J., Driver, A., Hill, L., Van Deventer, H., Funke, N., Swartz, E.R., Smith-Adao, L.B., Mbona, N., Downsborough, L., Nienaber, S., 2011. Technical Report for the Freshwater Ecosystem Priority Areas Project. WRC Report No. 1801/2/11. Water Research Commission, South Africa. Nel, J.L., Roux, D.J., Driver, A., Hill, L., Maherry, A.C., Snaddon, K., Petersen, C.R., SmithAdao, L.B., Van Deventer, H., Reyers, B., 2016. Knowledge co-production and boundary work to promote implementation of conservation plans. Conserv. Biol. 30, 176–188. https://doi.org/10.1111/cobi.12560. Reed, M.S., Vella, S., Challies, E., de Vente, J., Frewer, L., Hohenwallner-Ries, D., Huber, T., Neumann, R.K., Oughton, E.A., del Ceno, J.S., van Helden, H., 2018. A theory of participation: what makes stakeholder and public engagement in environmental management work? Restor. Ecol. 26, S7–S17. https://doi.org/10.1111/rec.12541. Robins, L., Dovers, S., 2007. NRM regions in Australia: the ‘haves’ and the ‘have nots’. Geogr. Res. 45, 273–290. https://doi.org/10.1111/j.1745-5871.2007.00460.x. Roux, D.J., Nel, J.L., 2013. Freshwater conservation planning in South Africa: milestones to date and catalysts for implementation. Water S.A. 39, 151–164. https://doi.org/ 10.4314/wsa.v39i1.15. Roux, D.J., Nel, J.L., Cundill, G., O'Farrell, P., Fabricius, C., 2017. Transdisciplinary research for systemic change: who to learn with, what to learn about and how to learn. Sustain. Sci. 12, 711–726. https://doi.org/10.1007/s11625-017-0446-0. Roux, D.J., Rogers, K.H., Biggs, H.C., Ashton, P.J., Sergeant, A., 2006. Bridging the science-management divide: moving from unidirectional knowledge transfer to knowledge interfacing and sharing. Ecol. Soc. 11. Rowe, G., Marsh, R., Frewer, L.J., 2004. Evaluation of a deliberative conference. Sci. Technol. Hum. Values 29, 88–121. https://doi.org/10.1177/01622439032591. Russell, A.W., 2013. Improving legitimacy in nanotechnology policy development through stakeholder and community engagement: forging new pathways. Rev. Policy Res. 30, 566–587. https://doi.org/10.1111/ropr.12037. Schreier, M., 2012. Qualitative Content Analysis in Practice. Sage Publications, London. Taylor, M., Kent, M.L., 2014. Dialogic engagement: clarifying foundational concepts. J. Public Relat. Res. 26, 384–398. https://doi.org/10.1080/1062726X.2014.956106. Tengö, M., Brondizio, E.S., Elmqvist, T., Malmer, P., Spierenburg, M., 2014. Connecting diverse knowledge systems for enhanced ecosystem governance: the multiple evidence base approach. Ambio 43, 579–591. https://doi.org/10.1007/s13280-0140501-3. Thomas, G., 2011. How to Do Your Case Study – A Guide for Students and Researchers. Sage Publications, Los Angeles, CA. van Kerkhoff, L., Lebel, L., 2006. Linking knowledge and action for sustainable development. Annu. Rev. Environ. Resour. 31, 445–477. https://doi.org/10.1146/ annurev.energy.31.102405.170850. Walsh, J.C., Dicks, L.V., Sutherland, W.J., 2014. The effect of scientific evidence on conservation practitioners' management decisions. Conserv. Biol. 29, 88–98. https:// doi.org/10.1111/cobi.12370. Walter, A.I., Helgenberger, S., Wiek, A., Scholz, R.W., 2007. Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method. Eval. Program Plann. 30, 325–338. https://doi.org/10.1016/j.evalprogplan.2007.08. 002. Webler, T., 1995. “Right” discourse in citizen participation: an evaluative yardstick. In: Renn, O., Webler, T., Wiedemann, P. (Eds.), Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse. Kluwer Academic Publishers, Dordrecht, pp. 35–86. Wickson, F., Carew, A.L., Russell, A.W., 2006. Transdisciplinary research: characteristics, quandaries and quality. Futures 38, 1046–1059. https://doi.org/10.1016/j.futures. 2006.02.011. Wyborn, C., Datta, A., Montana, J., Ryan, M., Leith, P., Chaffin, B., Miller, C., van Kerkhoff, L., 2019. Co-producing sustainability: reordering the governance of science, policy, and practice. Annu. Rev. Environ. Resour. 44, 1–28. https://doi.org/10.1146/ annurev-environ-101718-033103. Wynne, B., 2014. Further disorientation in the hall of mirrors. Public Underst. Sci. 23, 60–70. https://doi.org/10.1177/0963662513505397. Yin, R.K., 2014. Case Study Research: Designs and Methods, fifth ed. Sage Publications, Los Angeles.
Acknowledgements Human ethical research clearance for the Australian interviews was granted on 26/6/2015 by the ethics coordinator at the School of Communication and Arts, University of Queensland. The Chair of the Science and Medical DERC at the Australian National University gave clearance for the South African interviews on 15/04/2016 (protocol 2016/077). This research was supported by an Australian Government Research Training Program (RTP) Scholarship and a CSIRO postgraduate studentship. Declarations of interest: none. References Armitage, D., Berkes, F., Dale, A., Kocho-Schellenberg, E., Patton, E., 2011. Co-management and the co-production of knowledge: learning to adapt in Canada's Arctic. Glob. Environ. Chang. 21, 995–1004. https://doi.org/10.1016/j.gloenvcha.2011.04.006. Arnstein, S.R., 1969. A ladder of citizen participation. Am. Inst. Planners 35, 216–224. Bucchi, M., 2008. Of deficits, deviations and dialogues: theories of public communication of science. In: Bucchi, M., Trench, B. (Eds.), Handbook of Public Communication of Science and Technology. Routledge, Oxon, pp. 57–76. Chilvers, J., 2008. Deliberating competence: theoretical and practitioner perspectives on effective participatory appraisal practice. Sci. Technol. Hum. Values 33, 155–185. https://doi.org/10.1177/0162243907307594. Cook, C.N., Mascia, M.B., Schwartz, M.W., Possingham, H.P., Fuller, R.A., 2013. Achieving conservation science that bridges the knowledge–action boundary. Conserv. Biol. 27, 669–678. https://doi.org/10.1111/cobi.12050. CSIRO, 2014. About AdaptNRM. http://adaptnrm.csiro.au/about-adaptnrm/, Accessed date: 28 November 2016. Cvitanovic, C., McDonald, J., Hobday, A., 2016. From science to action: principles for undertaking environmental research that enables knowledge exchange and evidencebased decision-making. J. Environ. Manag. 183, 864–874. https://doi.org/10.1016/j. jenvman.2016.09.038. Davies, S.R., 2013. The rules of engagement: power and interaction in dialogue events. Public Underst. Sci. 22, 65–79. https://doi.org/10.1177/0963662511399685. Dunn, G., Laing, M., 2017. Policy-makers perspectives on credibility, relevance and legitimacy (CRELE). Environ. Sci. Policy 76, 146–152. https://doi.org/10.1016/j. envsci.2017.07.005. Dunn, G., Bos, J., Brown, R., 2018. Mediating the science-policy interface: insights from the urban water sector in Melbourne, Australia. Environ. Sci. Policy 82, 143–150. https://doi.org/10.1016/j.envsci.2018.02.001. Edelenbos, J., Van Buuren, A., van Schie, N., 2011. Co-producing knowledge: joint knowledge production between experts, bureaucrats and stakeholders in Dutch water management projects. Environ. Sci. Policy 14, 675–684. https://doi.org/10.1016/j. envsci.2011.04.004. Elo, S., Kyngäs, H., 2008. The qualitative content analysis process. J. Adv. Nurs. 62, 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x. Fazey, I., Bunse, L., Msika, J., Pinke, M., Preedy, K., Evely, A.C., Lambert, E., Hastings, E., Morris, S., Reed, M.S., 2014. Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Glob. Environ. Chang. 25, 204–220. https://doi.org/10. 1016/j.gloenvcha.2013.12.012. Fazey, I., Evely, A.C., Reed, M.S., Stringer, L.C., Kruijsen, J., White, P.V.L., Newsham, A., Jin, L., Cortazzi, M., Phillipson, J., Blackstock, K., Entwistle, N., Sheate, W., Armstrong, F., Blackmore, C., Fazey, J., Ingram, J., Gregson, J., Lowe, P., Morton, S., Trevitt, C., 2013. Knowledge exchange: a review and research agenda for environmental management. Environ. Conserv. 40, 19–36. https://doi.org/10.1017/ S037689291200029X. Fischer, A.R.H., Wentholt, M.T.A., Rowe, G., Frewer, L.J., 2014. Expert involvement in policy development: a systematic review of current practice. Sci. Public Policy 41,
8