An evaluation of systems for the integrated assessment of capture fisheries

ARTICLE IN PRESS

Marine Policy 31 (2007) 458–469 www.elsevier.com/locate/marpol

An evaluation of systems for the integrated assessment of capture fisheries Duncan Leadbittera, Trevor J. Wardb, a

Asia Pacific Regional Office, Marine Stewardship Council, 10/46-48 Urunga Parade, Miranda, Sydney, Australia Institute for Regional Development M004, University of Western Australia, Stirling Highway, Nedlands, Perth, Australia

b

Received 18 November 2006; accepted 7 December 2006

Abstract Integrated fishery assessments are intended to take an integrated and wholistic view of fisheries management, to provide comprehensive systems-based evaluations of the sustainability of individual fisheries and their products. Using guidance derived from decision theory and management science, we develop 18 criteria to evaluate the likely effectiveness of nine integrated fishery assessment systems. The assessment systems do not consistently consider four key criteria: the socio-economic impacts of fisheries, the provision of food security, the local relevance of the assessment to the fishery under assessment, and the independent peer review of assessment outcomes. Other important areas of weakness include the lack of appropriate consultation processes relating to both the design of the assessment system and the outcomes of fishery assessments, and limitations in the quality of data and information used in assessments. Improvements will be needed in these areas across all the systems we evaluated if fishers, consumers, managers and other stakeholders are to be provided with consistently reliable assessment outcomes, and an assurance of the sustainability of capture fisheries. r 2007 Elsevier Ltd. All rights reserved. Keywords: Fisheries sustainability; Assessment criteria; Decision theory

1. Introduction The diversity and availability of information for the management of fisheries has long been an issue of concern for managers and stakeholders. However, the need for quality information for fisheries management has grown in importance in recent years because of the perilous state of many stocks [1], concern over the impacts of fishing on non-target species and habitats [2], the increasing number and diversity of fisheries stakeholders [3], the increasing prevalence of cost-recovery policies in fisheries management [4], and the failure of fisheries management to achieve effective integrated solutions to the complexities of managing wild fish stocks [5]. The past decade has seen a shift away from traditional, single species management approaches towards more integrated management approaches that purport to address the interactions between fisheries and the wider environCorresponding author.

E-mail address: [email protected] (T.J. Ward). 0308-597X/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.marpol.2006.12.008

ment and the concerns of stakeholders [4,6]. These shifts have occurred in response to the increasing recognition that fisheries management requires much more than management of the stocks [7–9]. This has also been expressed as a pressing need for a paradigm shift [5] from stock management to integrated fishery system management using the concepts of management science [10]. The new paradigms for fisheries management include the socioeconomic aspects of fisheries, the involvement of stakeholders in decision-making, and the environmental impacts of fishing beyond the impact on the target stocks. Attempts to evaluate, in an explicit and transparent way, the socioeconomic and environmental aspects of fisheries management have also emerged because managers, industry and other stakeholders seek to fully address the wider expectations of Ecologically Sustainable Development (ESD), as articulated by the World Commission on Environment and Development [11]. In Australia, these wider expectations have been expressed in both policy and legislation at both the State and Federal levels. In 1991, the Ecologically Sustainable

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

Development Fisheries Working Group recommended that fisheries be managed from an ecosystem perspective [12]. Since that time almost all the fisheries management legislation in Australia has been rewritten and ESD is now an explicit requirement in many of the new laws and policies. In the state of New South Wales, following a 2000 court judgment, detailed and explicit environmental impact assessments of commercial fisheries were prepared. Whereas there has been significant progress on the commercial fisheries there has been little regard given to including the recreational, sport and indigenous fisheries in the same framework, despite evidence of the high level of recreational catches and impacts on non-target species in some of these fisheries [13] and the impacts of some indigenous fisheries (http://www.deh.gov.au/coasts/species/ turtles/pubs/national-approach.pdf). These shifts in management approach in Australia have been paralleled, to some extent, in other parts of the world. For example, in New Zealand the government is currently preparing an Oceans Policy as well as an environmental management strategy for that nation’s commercial fisheries (http://www.mfe.govt.nz/issues/oceans/). Canada, too, has been one of the world leaders in the development of ocean policy and law in relation to fisheries management [14]. As the demand for, and availability of, information for better fisheries management has increased, so too has the range of expected applications for that information. In fisheries terms, for example, simple catch data and landed catch values are no longer sufficient to satisfy the interests of various parties with legitimate interests and concerns for the welfare of a capture fishery. Information on stock status, take of threatened species, impact of the fishery on habitats, area fished and the relative economic benefits of differing fishery sectors are but some of the many areas where stakeholders expect to have current information when making an assessment of a fishery. Frameworks for capturing, manipulating and presenting the amount and types of information now required for this breadth of fisheries management are relatively new, but are proving to be essential if stakeholders are to have access to information about the key parameters of a fishery and its supporting environment in a timely and accessible manner [15]. As the demand for more complex systems to manage fisheries has increased, so has the need for assessments that can evaluate and report on the comprehensiveness and effectiveness of such fishery management systems for the purpose of making reliable assessments of sustainability. Here, we identify a number of fishery assessment systems that are broad in their scope, and can be considered to attempt to take an integrated approach to assessment of a fishery management system through consideration of stock, environmental and socio-economic aspects of management. In this brief report, we consider several of the main integrated assessment systems that are used worldwide, and a number of regional systems used to assess and report on fishery management systems. We

459

summarise the basic elements and approach of each assessment system, and using a purpose-built criteria set, make a preliminary analysis of the role and relative utility of each assessment system to determine if they hold any major weaknesses in common. 2. Integrated fishery assessment systems We define an integrated fishery assessment system to be a cohesive and comprehensive set of principles, criteria and assessment approaches to determine the effectiveness of a system that is used for managing a wild capture fishery. To be ‘integrated’, the assessment systems should consider the stock, the environment where the fishery operates, the social and economic impacts of the fishery, and the nature and context of the management system under which the fishery must operate all within a single-linked assessment system. The assessment system should have a repeatable quantitative basis that underpins its decisions, although we accept that this may not always be essential if a robust qualitative approach is used. One of the major factors that determine the comprehensiveness of a fishery assessment system is the purpose of the system. Here, we distinguish between assessment systems that are designed to mainly assess the biophysical elements of the fishery (such as the fish stock) from those that are designed to assess the fishery as a whole, including such matters as the social, cultural and economic impacts of fishing. The former, of which there are many examples may be part of the latter, but it is the latter that is our focus in this analysis. Assessment systems may be subjected to close scrutiny because commercial decisions depend on them, they influence consumer choices in the marketplace, or because politically sensitive species are involved (such as those that may be fished in more than one national or regional jurisdiction). However, in all cases, assessment systems require a high degree of rigour and robustness so that they can reasonably and credibly meet the sustainability reporting purposes for which they have been established. Here, we consider nine integrated fishery assessment systems (Table 1). These systems include some that have been prepared by environment groups to provide information to consumers about which fish they should or should not eat, some prepared by scientists to provide fishery managers and scientists with technical procedures to assess fisheries, some by regulators to satisfy a need for policydriven changes in fishery practices, and one prepared by an accreditation and labelling body to provide market-based incentives for improved fishery practices. The primary purpose of each of these assessment systems is broadly similar: to assess and report on the effectiveness and performance of the management system for a fishery to maintain or improve sustainability. However, the approach and information base they each use is different. Some, such as RapFish, may compare one fishery against another, a single fishery over time, or a real fishery against

ARTICLE IN PRESS 460

D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

Table 1 Integrated fishery assessment systems Program (alphabetic)

Source

Coverage

Blue Ocean Institute (BOI) Environment Protection & Biodiversity Conservation Act; Guidelines for sustainable fisheries (EPBC) Friend of the Sea (FoS) Good Food Guide Marine Stewardship Council (MSC) New Zealand Forest and Bird, Best Fish Guide (NZFB) RapFish Unilever Traffic Lights Victoria National Parks Association (VNPA)

Environment NGO Australian Department of Environment and Heritage

United States Australia

NGO Eco-labelling Program Marine Conservation Society, UK NGO Eco-labelling Program Environment NGO University of British Columbia Unilever Environment NGO

Italy United Kingdom Global New Zealand Global Global Australia

a theoretically ideal fishery, in order to establish a relative performance level for the fishery being assessed. Others, such as the assessment system encoded into Australia’s Environmental Protection & Biodiversity Conservation Act (EPBC), the Marine Stewardship Council (MSC), and the colour-coded Unilever Traffic Light systems simply evaluate a fishery on its merits against an externally established set of criteria that represent a sustainability standard. The standards and benchmarks against which a fishery is compared can also vary, and in the systems we review, standard setting includes those that are derived internally by comparison with other fisheries of the same type, those set by reference to external independent authorities, or those set from theory or idealised practices. The various assessment systems use different types, quality and quantity of data and information, and this leads to differences in the amount of technical detail involved in both the decisions and reporting of each assessment system. In ‘traffic light’ systems, fish are allocated to either red, green or yellow categories according to whether the suppliers of the advice want consumers (or buyers in the case of the Unilever system) to respectively avoid, eat, or select these fish with caution. Others (such as RapFish and the MSC) are aimed at scientists, managers and well-informed stakeholders, they assume a substantial technical knowledge and experience in natural resource management within the target community, and they provide considerably more data and analysis in support of their assessment outcomes. The increasing number of assessment systems of different types, especially those with eco-labels directly advising consumers at the point of retail sale, may create confusion amongst consumers, and there are issues of technical credibility related to the outcomes of the different systems. Because there are financial and environmental issues at stake, and the livelihoods of fishing families and related businesses can be affected, we consider that it is important to establish the transparency and effectiveness of the various forms of integrated fishery assessment systems that purport to provide advice to purchasers about fishery sustainability issues.

Here, we summarise the elements of the nine selected fishery assessment systems and evaluate their likely effectiveness taking account of the various assumptions and models underpinning each system, and make a preliminary evaluation of the relative merits of each system against our criteria. We selected these specific systems both because they represent the range of types of systems in current use, and because relevant program details were readily available for our criteria-based evaluation. Our evaluation framework is based on the principles of a systems approach to the evaluation of natural resource management initiatives [16]. 3. Fishery assessment system attributes The overall result of the assessment of a particular fishery can vary according to which assessment system is used. This may be due to a variety of factors (see below), but the diversity of outcomes may not only be confusing for the users of the information derived from the assessments but could also conceivably have significant consequences for the fishery participants. For example, an assessment system that has deliberate weightings or unintended design flaws could cause severe economic hardship for a fishery if the outcomes were effective in adversely influencing consumer behaviour. Equally, a system that is designed to ‘greenwash’ (i.e., a system that uses an environmentally weak benchmark or standard), or a system that is internally biased towards an industry perspective (such as may arise in industry self-assessment) could find a fishery to be consistent with the weak standard, or not assess some important issues, and hence support an environmentally damaging fishery that may have little or no prospects for improvement. Whilst no assessment system will be perfect, we consider that some level of protection against unwarranted outcomes can be provided by ensuring that the design of the system is public and transparent, and that the assessment mechanism is open to scrutiny so that interested parties have the opportunity to make appropriate inputs and responses and to determine how any specific assessment outcomes were derived.

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

The ‘fisheries-management science’ approach to fishery management has been proposed as an effective way to deal with the issues of managing a fishery in an integrated way, incorporating the many inherent uncertainties and complexities [10]. The principles of this approach are derived from management science—using a scientific decision approach to issue management and problem solving. This involves gathering relevant available information, developing and evaluating alternatives against specific objectives, and monitoring the progress of the decision outcomes in the fishery system. This is achieved by having the management system directly address the complexity, organisation and context of the fishery. The ‘complexity’ of a fishery refers to the need for data and information on all aspects of stock, ecosystems, by-catch issues, institutional arrangements, and external matters that relate to, or may affect, the fishery. The ‘organisation’ of a fishery refers to the breadth of participants in the fishery management process, including fishers themselves, as well as government managers, fisheries scientists and economists, conservation interests, local communities and the range of other stakeholders and institutions. The ‘context’ of the fishery refers to the need for all participants to agree on the range of decision alternatives available to manage the fishery, including objectives related to all relevant aspects of the fishery. From these three principles of ‘fisheries-management science’ [10], we derive three attributes that are vital for effective and efficient sustainable fishery management, and are the basis for our evaluation of the effectiveness of integrated fishery assessment systems: comprehensiveness, transparency and accountability, and the nature and quality of data and information. 4. Evaluation criteria set For the purposes of our comparative evaluation of integrated fishery assessment systems, we establish a set of evaluation criteria that we consider should be implemented in making a robust and defendable assessment of a fishery. These criteria directly relate to the desirable attributes of a fishery management system as discussed above. Here, we use these criteria as the basis for comparing the likely relative utility of each selected assessment system and to search for any criteria weaknesses the systems may hold in common. 4.1. Comprehensiveness The level of comprehensiveness and flexibility relates to both the fishery management system and to the assessment system. In a fishery management system, we consider comprehensiveness to include coverage of environmental issues and stakeholder participation in management, both key characteristics of an effective fishery management system. The range of issues evaluated by an assessment system will depend to a degree on the intended audience and purpose, but flexibility and comprehensiveness is

461

required to incorporate local differences in cultures, fishery objectives and ecological situations. ESD, as defined in Australian legislation, is a commonly adopted framework designed to manage Australian fisheries so that they achieve sustainability. The various expressions of ESD used in Australia contain explicit reference to the social and economic aspects of resource management. However, none of the fishery assessment systems we examined, other than RapFish, cover the full range of social and economic matters relevant to fisheries management and sustainability. Such variability amongst assessment systems in the way in which they deal with socio-economic issues reflects to some extent the difficulty that managers and stakeholders have experienced in identifying measurable indicators and the related performance benchmarks for social factors and for some environmental factors. Fishery management systems have found the establishment of standards and benchmarks for social issues to be a particular difficult area. For example, making judgements about what is good or bad in terms of a specific social issue, such as income distribution, is difficult for a fishery management system. In the absence of agreed standards and benchmarks, an integrated fishery assessment system may impose a cultural or political basis that may lead to substantial disagreement over the outcome of an assessment, so it is unsurprising that both fishery management and fishery assessment systems do not deal comprehensively with social issues. As well as having a unique approach to assessment, different systems will normally use a different set of standards or benchmarks, reflecting their pedigree, and their different purposes and intended audiences. However, since all integrated assessment systems have a common purpose in promoting the improved sustainability of fisheries, and since modern fishery management systems need to be comprehensive in their scope, we consider that fishery assessment systems must also assess comprehensiveness. We identify five key aspects of ‘comprehensiveness’ as important criteria for assessing a fishery management system (Table 2). 4.2. Transparency and accountability The transparency and accountability of an assessment system is important if the system is to be open to objective criticism and unbiased review, and if the findings of any specific fishery assessment are to be acceptable to the broadest range of stakeholders, including the fishers themselves. The imperative for transparency also extends to any assumptions, implicit and explicit models, and data and information used as the basis for judgements in any specific fishery assessment. Transparency is important at a number of levels and places in a fishery management system and also within an assessment system. For example, we consider that: (a) The assessment system design and content should be clear and explicit, including the purpose of the system,

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

462

Table 2 The criteria set for evaluation of the integrated fishery assessment systems Principle Comprehensiveness

Criterion 1. 2. 3. 4. 5.

Stock condition and performance Environmental impacts of the fishery Social and economic impacts of the fishery Food security at local, national and regional levels Scope of the management system

Transparency and accountability

6. 7. 8. 9. 10. 11. 12. 13. 14.

Wide consultation over system design Wide consultation over assessment of individual fisheries Specific details of the standards and standard setting process are publicly available Specific details of the assessment system are publicly available Specific details of the process and outcomes of any specific fishery assessment are publicly available There is a capacity for localization without undue bias Internal consistency and repeatability of the assessment process Independent peer review of the decision processes and outcomes The assessment system has been trialed and adapted after field-testing in real-world fisheries

Nature, use and quality of data

15. 16. 17. 18.

Data, models and assumptions are scientifically robust Sources of data and information are explicit, and are independently verifiable Treatment of data is transparent There is internal consistency and repeatability of any data analysis process

(b)

(c)

(d)

(e)

the range of issues subject to assessment, and the weightings allocated to factors of concern/interest; also the assessment methodology must be clear and produce outcomes that are readily interpretable by those interested in the results. A number of the systems we considered (e.g. EPBC and MSC), were subject to wide consultation with a range of stakeholders in their design phase, which we consider assisted in refining their clarity and focus. Individual fishery assessments: not all the information available for assessment of a fishery is derived from publicly available scientific assessments. An assessment system that facilitates the capture and documentation of information in the unpublished (‘grey’) literature, anecdotal information and various points of view will help to enhance the understanding of the fishery issues, but this should be made available to stakeholders for verification. Specific fishery performance indicators: the assessment system should determine the extent to which the basis for all important decisions in the fishery is transparent. Parameters (preferably expressed as measurable indicators) used in a fishery management system should be clear and explicit. Standards: the mechanism for specifying the standards and interpreting them into benchmarks that are used to assess performance of the fishery against each indicator must be assessed and reported in the assessment. Peer review: peer review has a vital role in ensuring there is independent technical credibility (and thus the wider acceptability) of the decisions arising from fishery assessments. It provides an added check on the interpretations of experts and helps to ensure that important aspects are not overlooked and balance is maintained in the interpretation of information.

(f) Public access to information: interested members of the public should have access to the information used by the assessment in reaching its outcomes. This information should be available in a way that encourages easy access and use. The results of assessments are also sensitive to a variety of design factors, including how much emphasis is placed on a particular element being considered. Mechanisms for varying the emphasis, for example, can include: (a) Giving unequal weight to assessment elements: if there are a large number of elements (indicators) being considered then the influence of any particular one may become insignificant. This can be a problem for an assessment when serious fishery impacts, such as major impacts on endangered species, are not highlighted and given some primacy, but are ‘averaged out’ over a number of other indicators. Weighting systems can overcome this problem, but they are difficult to develop and apply in an unbiased manner. (b) The location of an assessment element in a hierarchical system: for systems which have issues of primary consideration and subordinate issues then the choice of whether a particular issue is considered of primary importance or not can be highly influential. An issue can be artificially given importance by giving it prominence in a hierarchy, such as, for example, assuming that the longevity of a species is a surrogate for the potential for overfishing. (c) Input indicators versus output/outcome indicators: the primary purpose of fishery assessment systems is to provide an assessment of fishery performance. Data documenting performance are not always available and

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

surrogates are commonly used. For example, gear type is often used as a surrogate for environmental impact, with the use of benthic trawls and dredges being used as a surrogate for high impact and lines as a surrogate for lower impact. Whilst such surrogacy relationships may be commonly true, they do not always hold, and the assessment system needs to give priority to data on outcomes rather than assumptions about impacts derived from cause–effect models developed in other fisheries or ecological situations. We consider that there are nine important criteria for evaluation of the transparency and accountability of a fishery assessment system (Table 2). 4.3. Nature, use and quality of data It would be desirable to make an assessment of a fishery always based on well designed, peer reviewed, independent scientific data. There is not likely to be any fishery for which such an assessment could be conducted solely using data from such sources. Grey literature, coupled with unpublished data, and anecdotal information and experience from local managers and fishers is crucial in providing the context and information about a fishery. The skills and experience of the assessors are therefore vital to be able to suitably assess and incorporate such knowledge. It also means that the assessment system needs to be explicit about the sources of data and information and how they are treated during the assessment process, particularly if any computer decision-support systems are used, such as for weighting or aggregating scores. We consider that there are four relevant criteria for evaluating the data and knowledge used in an assessment system (Table 2). 5. Evaluation methods We base our evaluation of the design of assessment systems on the principles of decision theory and management science in fisheries management [10]. We also make use, where access was possible, of publicly available information about the systems to determine their basic attributes, and to determine if their outcomes are likely to be robust in providing consumers, managers, fishers and other stakeholders with reliable information about the integrated management of a fishery. We evaluate each assessment system using the criteria set developed above for this purpose. Each criterion is constructed to reflect each of the elements of an integrated fishery management system that we consider to be important. The evaluation criteria used here do not attempt to assess the merit of each benchmark (standard) that is applied within each assessment system. This means that our process of evaluation will not derive the ‘best’ assessment system, since we do not attempt to assess the appropriateness of a standard that may be adopted within an

463

assessment system. We consider that the specific performance standards that a fishery is assessed against must be a feature of the assessment system itself, and relate directly to the purpose of the assessment. For this reason, we consider that an assessment system should have either a clear and explicit set of standards aligned to each assessment criterion (such as an established and well accepted benchmark practice), or define the process through which the relevant benchmark will be established for the fishery being assessed. The assessment system should therefore either specify a set of appropriate benchmarks that encode the standard for guiding performance assessments, or provide explicit protocols about the establishment of such standard within the fishery being assessed. Such benchmarks may be determined flexibly, by expert judgement using broad external guidance (as in the MSC program), while others may use imposed or international benchmarks (as in RapFish), or otherwise derive their own benchmarks through a technical review and analysis process. This may mean that a fishery is favourably assessed by one system, yet reaches an unfavourable outcome within another. Nonetheless, the specification of benchmarks that encode the standard into a form of measurable or observable parameter is an attribute of the assessment system itself. Here, we evaluate whether the fishery assessment system uses benchmarks or not, and if it has an appropriate process for deriving them in relation to the specified purpose for the assessment. We make no judgement about the merit of any specific benchmark or standard used in a fishery because we consider this to be a complex area of science that falls beyond the scope of this initial empirical review of the systems. To make our evaluation of the nine assessment systems, we review and assess the publicly available details about each assessment system to determine if they met each of the 18 criteria listed above. The main features of each system are briefly described below. For the purposes of an internally consistent comparative evaluation we have assigned each system a score out of 5, based on the following scale: 0—does not appear to consider; 5—appears to fully consider. The median scores are used to report on the relative potential of each system to be able to make an effective assessment of a fishery management system, and to identify which criteria are the weakest across all systems. The variability within each criterion and each assessment system is represented by the 75% quartile of the assigned scores. 6. Brief description of selected integrated fishery assessment systems 6.1. Blue Ocean Institute The Blue Ocean Institute (BOI) has created a quantitative evaluation system for assessing the main seafood species sold in the US [17]. Based on the methodology used in the US-based Audubon Society’s Seafood Lovers Almanac [18], the system provides the basis for advice to

ARTICLE IN PRESS 464

D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

consumers as to which fish they should buy or avoid. Scores are allocated to a wide series of parameters that provide a measurement of the performance of the fishery under scrutiny. Whilst the BOI makes use of publicly available information it does not make available its determinations about the status of species, beyond a recommendation in a seafood guide. 6.2. Environment Protection & Biodiversity Conservation Act; Guidelines for sustainable fisheries (EPBC)

For each of the 20 species the Guide recommends ‘to avoid’ there is a reason provided to justify its presence on the list. However, these reasons seem somewhat arbitrary and in some cases seem incorrect (e.g., yellowfin tuna is claimed to be overfished). The sources of information for making the judgments are generally not provided and some groups of species (e.g. groupers) are lumped together making specific consumer choices difficult, and fishery improvements hard to specify. 6.5. Marine Stewardship Council (MSC)

The Australian federal Environment Protection & Biodiversity Act, 1999, requires fisheries that fall into certain categories (in particular, those that export seafood products from wild harvest fisheries) to undergo comprehensive assessments of the capacity of the fisheries management regime to deliver ecologically sustainable fisheries. The assessments are prepared by the relevant (federal or state) fishery management agency against a series of principles and criteria (http://www.deh.gov.au/ coasts/fisheries/index.html) and then submitted to the federal Department of Environment and Heritage for evaluation and approval. Assessments of nearly all of Australia’s exported species have been carried out and the government has approved for export all the species for which an approval has been sought.

The Friend of the Sea (FoS) program is a seafood advisory system used by an Italy-based retailer, Co-op Italia, as a source of advice on sustainable seafood (http:// www.friendofthesea.org/). In many respects, the system is modelled closely on the Marine Stewardship Council’s system. However, there are no third party, independent certification bodies involved and the system essentially involves a self-assessment by candidate fisheries prior to approval by the FoS Board. The Friend of the Sea program is part of the Earth Island Project Network and covers both farmed and wild caught fish and shellfish products.

The Marine Stewardship Council is an international accreditation and labelling body that has established a standard for well-managed and sustainable fisheries. The Council accredits independent certifiers to evaluate fisheries against this standard and, if they comply, then the fishery is awarded the right to carry the MSC eco-label, and can advertise this to consumers (see www.msc.org, [20]). The standard, known as the principles and criteria, contains three elements relating to stock status, fishing impacts on the wider environment and the efficacy of the fishery management system. The standard does not provide any explicit evaluation of the social and economic aspects of a fishery, although such aspects were proposed to be separate principles when the MSC was first created. However, worldwide consultation could find no agreement on how a fishery should perform in socio-economic terms. The fishery assessment system is based on scores on a set of performance indicators derived by the accredited certifier to best represent that MSC principles and criteria for the fishery under assessment. The standard is therefore flexible, because the MSC’s system makes use of expert judgment to devise these indicators on a case-by-case basis. All fisheries that have been assessed have passed certification, subject to conditions. The MSC system provides for appeals by stakeholders, which can force a detailed review of the assessment decisions. This enables specific weaknesses to be revealed, and ultimately to be corrected.

6.4. Marine Conservation Society—The Good Fish Guide

6.6. New Zealand Best Fish Guide

The Good Fish Guide is a fishery assessment system aimed at advising consumers of seafood in the UK about which species of fish and invertebrates are better choices than others. It was prepared by the UK-based Marine Conservation Society [19]. The Good Fish Guide is based on a non-quantitative evaluation system, and provides a large amount of information about fish, fisheries (methods, management, etc.), fish products and other salient issues. It provides a list of issues to be addressed when making judgments about the sustainability of a species and also a list of 20 species to avoid. However, there is no clear linkage between all this information, and there is no clear description of the technical basis for the Guide.

The New Zealand Best Fish Guide has been developed by the New Zealand-based Royal Forest and Bird Protection Society (RFBPS, http://www.forestandbird.org.nz/bestfishguide/gettheguide.asp). The RFBPS has assessed all of New Zealand’s commercial fisheries and allocated them to one of two (out of three possible) categories of sustainability. The guide does not identify any NZ fisheries as acceptably managed with low levels of habitat damage and/or bycatch. The remaining fisheries are classified as either ‘amber’ (31 species—choose with caution) or ‘avoid’ (36 species—do not choose). The guide is based on a detailed set of criteria and assessment procedures developed by the society specifically for the New Zealand fisheries context. The guide classifies

6.3. Friend of the Sea

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

fishery products based on six criteria: fish catch, impacts of fishing on bycatch and habitat, biology/natural history of the species and the risk of overfishing, effectiveness of management and research, impacts on protected species, and the effectiveness of the management unit (multiple vs. single quota species). Each fishery is assessed using a semiquantitative system of ranking of performance using five fixed categories of performance established for each criterion. The criterion for fish catches and impacts on protected species are weighted to be the most important contributors to the outcome of the assessment for each species. The assessments appear to be conducted by staff of the RFBPS and findings are not open to stakeholder input or challenge. The assessment is based on high-level standards for sustainability, providing a highly conservative set of assessment outcomes. 6.7. RapFish RapFish is a sophisticated, quantitative fishery assessment system created by the Fisheries Centre at Canada’s University of British Columbia. It uses Multi Dimensional Scaling (MDS) data analysis to evaluate a fishery (or smaller unit, e.g., gear type) using sets of attributes which can be varied to suit the circumstances. RapFish creates reference points to set boundaries within which the fishery attributes are scored. These boundaries define the best and worst fishery on a scale of 0–100%. Constructed reference fisheries are created using randomly allocated scores for each of the attributes to assist the definition of differences amongst fisheries (or other units). The system has been well tested, both in a theoretical sense and using real world fisheries. There have been a number of published papers outlining how the system works and how subject fisheries perform (e.g. [21]). 6.8. Unilever Traffic Lights Unilever is one of the largest handlers of fish in the world and supplies bulk products such as pollock, hoki, cod and hake. The Unilever Traffic Lights system is an internal company fishery assessment system designed to implement the company’s commitment to sustainable fisheries by encouraging a focus on supporting sustainable and wellmanaged fisheries (http://www.unilever.com/Images/ Unilevers%20Fish%20Sustainability%20Initiative_tcm139157.pdf). Company scientists evaluate each fish product proposed for purchase by the Unilever group of companies, to ascribe them to one of three colours (red, orange or green) signifying levels of sustainability. Products classified as red are actively avoided by the Unilever purchasing systems, and products classified as green are considered to be the most sustainable, and possibly suitable for MSC assessment. The Traffic Lights criteria involve a consideration of whether the fishery makes use of fisheries research, has a

465

quota system, has regulatory tools and a control system and whether there is a long-term management plan in place. There appears to be no peer review system, or process for public review of the data and decisions. 6.9. Victorian National Parks Association (VNPA) Bathgate [22] prepared a structured evaluation of the status of important commercial fishery species from southern Australia for a consumer advisory document distributed by the Victorian National Parks Association (VNPA). The methodology is well documented and publicly available but it is not known what trials were put in place, or how sensitive the system is to significant changes in scores. The evaluation system is quantitative in nature—it allocates scores and rankings to attributes deemed important for assessing the sustainability of an exploited species. The system uses four main criteria as a basis for the evaluation: status and sustainable yield; fishing method (e.g. amount of bycatch, impact on habitat, use of mitigation measures); management and research (e.g. input and output controls, management plans and stock assessment); and biology and risk of overfishing. 7. Results and discussion In order for an integrated fisheries assessment system to properly reflect contemporary views of fishery sustainability, and to adequately assess the sustainability of a fishery, we consider that the system should be able to demonstrate that it adequately assesses each of the three important principles of fishery-management science: complexity, organisation and context. To achieve this, the assessment system itself should be comprehensive, covering all important matters relating to the sustainability of a fishery (such as stock condition and trends, and environmental, social and economic impacts of the fishery). Second, the form and processes of the assessment system should be robust and implemented in a fair and transparent manner taking account of all reasonable risks, costs and benefits, so that managers, fishers, consumers and other stakeholders can be confident in the fidelity and probity of the assessment system. Third, the nature, quality and use of data and information used within the assessment system should be clear and open to analysis and verification. We identified 18 specific attributes of an integrated fishery assessment system that are important if the system is to be effective, efficient and accountable in the eyes of consumers, fishers, managers and other stakeholders. These attributes, representing the principles of fisherymanagement science, form the assessment criteria we use here to evaluate each integrated fishery assessment system. The range in median scores and the variability of scores across the criteria (Fig. 1) shows that there are four main weaknesses of the assessment systems revealed in our evaluation (criteria that achieved a median score of 1 or

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

466

less across all systems evaluated). Also, the scores suggest that some systems will be likely to perform better than others in the sense of providing more robust and defendable assessment outcomes. The criterion weaknesses are:

   

the extent to which the socio-economic attributes and impacts of the fishery are assessed, the extent to which the fishery contributes to food security at a range of scales, the capacity for localisation of the assessment to adapt to local issues and unique aspects of the fishery under assessment, without undue bias, the extent to which assessment decisions and outcomes are subjected to independent peer review.

Two further criterion weaknesses (median scores of 2 across all criteria) are also evident and of great concern for the credibility of an integrated assessment system: the limited extent of consultation processes about the design of the assessment system, and the limited extent of consultation over the assessment outcomes for individual fisheries. The weakness in assessing the socio-economic attributes of fishery management flows directly from a lack of attention to socio-economic issues in most fishery management systems. There are consistent calls for much greater attention to this aspect, particularly such matters as stakeholder consultation, co-management and collective decision-making [5], but as yet these matters are not well implemented in most fisheries management systems.

The matter of fisheries providing for food security at local and regional scales is not well-assessed in most of the systems we evaluated. This is perhaps surprising, given the range of scales of fishing activities and the level of international interest, but this gap probably reflects the dominance of large-scale commercial fishing over localscale fishing, and the issues of wealth generation rather than food security. Nonetheless, we consider food security to be an important matter, both at the global level and in the sense of providing for local and regional stakeholder engagement with fisheries management. It is an important parameter of sustainability and integrated fishery management, but at this stage is not well addressed in either fisheries management [23] or in assessment systems. Consultation on system design and assessment of individual fisheries can provide greater credibility, and we would expect to have direct implications for the long-term resilience and effectiveness of such systems. Some systems, such as that created by the NZ Forest and Bird Protection Society apparently involved no consultation with industry or government stakeholders, and some, such as the Friend of the Sea system, make no provision for comment or input into the fishery assessments (which are provided by the applicants for endorsement). By contrast, the Australian EPBC fishery assessments conducted for the Australian Department of Environment and Heritage are advertised for public comment before a decision is finalised. The MSC process goes further and seeks both comments on the indicators proposed to be used to assess the fishery status (known as the assessment tree) prior to the formal fishery

12

10

8

6

4

2

C IO FO O D SC O PE SY ST EM AS SE SS ST AN D PU BL IC O U TC O M LO C AL C O N SI S PE ER TR IA L M O D EL SO U R C E D AT A R EP EA T

R

O

SO

VI EN

ST

O

C K

0

Assessment Criteria Fig. 1. The median scores (filled bars) and the 75% quartile values (open) assigned to each of the 18 assessment criteria for the nine assessment systems.

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

assessment being undertaken, and then provides a draft assessment report for stakeholder review and input before the final decision is taken about a fishery. Such consultation processes are likely to result in more appropriate assessment systems and individual assessments through appropriate ‘localisation’ of the criteria and weighting systems. The various systems generally make use of expert judgement to a greater or lesser degree so as to enable decisions to be made in the absence of fully comprehensive, peer reviewed, scientific evidence. The extent to which anecdotal information and ‘grey’ literature influences the final decision should be made clear, especially where systems rely on self-assessment or where consultation is absent. Nonetheless, the diversity of scores on these criteria in our evaluation shows that this is an area where assessment systems need to make substantive improvements in securing high-quality peer reviews to provide a measure of further independent verification of the interpretations and outcomes of individual fishery assessments. Also, although data quality has not been assessed in detail in this review, there is likely to be a strong link between the ways in which data and information are treated and the performance of the assessment system, and this matter was found to be highly variable across the nine systems. The role of expert judgement and peer review is crucial in considering explicit and implicit weighting in the assessment system. The overt weighting that is given to a particular aspect of a fishery may reflect a specific interest of the assessment system, and this needs to be made clear and explicit to consumers. Some systems put particular emphasis on issues of interest to specific stakeholders (such as industry or environmental groups). For example, the BOI and VNPA systems explicitly seek information on the age of the fish at a high level in the evaluation, as they consider that long-lived fish are more vulnerable to overfishing. Whereas this may commonly be true, it is also the case that good management may reduce such a risk to an acceptable level, and this parameter may therefore not always be a good surrogate for impact. However, where an input (such as age) is used at a high level in an assessment, this effectively applies a high weighting to this information, and can result in a distorted assessment outcome if other important and directly related parameters and sources of information (such as stock status) are over-ridden by this weighting. This is especially a risk in highly structured and strongly hierarchical systems, such as that operated by the MSC, or in the inflexible RapFish system. In the absence of public and peer scrutiny and oversight or guidance there could be a possibility that vital issues are dealt with at lower levels in the evaluation than is warranted, and thus have little influence on the overall result. Both MSC and RapFish impose particular views about how fisheries should be managed, through the structure of their assessment systems. For example, the RapFish system dictates (via pre-determination of the scoring scale) the form of socio-economic considerations, such as income

467

distribution, into the assessment result. Such considerations may have no influence on whether the stock is sustainable or the level of fishing impacts, but may be important if the fishery being assessed has a restricted access management system and so directs benefits to a narrow section of the stakeholder community (e.g. [24]). A more flexible approach to this matter of the form of income distribution, and other socio-economic considerations, would be to cater for different cultural and fishery management circumstances, and allow the acceptable form of benefit distributions to be determined within the assessment system. The MSC imposes a data-rich assessment system on fisheries, requiring for example a detailed analysis and models of stock status, and generally, only larger fisheries can afford to submit to MSC assessment. Assessments of fishing impacts using risk-based systems may require less data, but are likely to provide more highly uncertain assessment outcomes because of the difficulties of correctly assigning consequences without an appropriate knowledge base to estimate the structure and extent of possible risks. Fishery sustainability is a function of social and economic as well as ecological factors, and while there may well be a case for imposing preferred fishery management models because they are considered to be effective in certain cultural circumstances, the more effective assessment approach is likely to be to permit flexibility in the design and standard of the assessment system to best match the fishery circumstances. This flexibility should encompass and explicitly recognise the purpose of the assessment system, rather than externally dictate the form of preferred socio-economic benefit or management structure. This is not to argue that the sustainability standard should only match the local aspirations of the fishery, but that in assessing the form and structure of the management, the assessment system needs to be flexible enough to provide for alternative ways to achieve a sustainable fishery, consistent with the standard that is defined within the assessment system. Each fishery assessment system has been created for a particular purpose and this influences how the system is constructed. For example, the system used by Unilever group of companies is designed to provide a rapid scan of a fishery for non-technical company staff to enable them to make decisions about purchases of a product. By contrast, the MSC system is much more comprehensive and robust as the results of any given assessment have to withstand technical comment and review in the marketplace. Differences in purpose such as these are probably largely responsible for the diversity we observe across assessment systems in the scores on our criteria (Fig. 2), and despite this range in scores in our evaluation, they do not necessarily relate to ‘best’ or ‘worst’ assessment systems. Our aim was to reveal any widely spread weaknesses in assessment systems, not evaluate their specific relative performance, which would need to be conducted using a number of real fishery examples (or possibly the use of

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

468 11 10 9 8 7 6 5 4 3 2 1 0 BOI

EPBC

FoTS

MCSGFG

MSC

NZF&B

Rapfish

Unilever

VPA

Assessment Systems

Fig. 2. The median scores (filled bars) and the 75% quartile values (open) assigned to each of the nine assessment systems for the assessment criteria.

theoretical optimum fisheries). However, if assessment systems do not make clear their purpose, design and procedures, consumers may not be able to discriminate between the outcomes of different systems, particularly when outcomes are conveyed in simple terms (such as pass/fail). The output from a fishery assessment system (such as a recommendation to consumers to choose or avoid a specific seafood product) can be a powerful tool, with the capacity to influence management decisions, fishery economics and/or product performance in the marketplace [20]. The flow-on effects to fishing communities and the wider marine environment could well be enormous, either positively or negatively. In the case of government systems (such as the EPBC), there is also the potential for drastically reducing the market for specific products, because of the direct feedback to regulatory systems. In the case of assessment systems operated by nongovernment organisations, concern over the potential trade and commercial impacts have been expressed by industry and governments worldwide. For example, there has been much global discussion about the speculated impact of the MSC on international trade [25], and the FAO continues to discuss its response to not only the MSC but also to other seafood eco-labelling initiatives, and has released a set of guidelines for eco-labeling of fish products [26]. In all these cases (and in other systems, such as dolphin-safe tuna [27]), it is clear that the issues of transparency and accountability are crucial to the likely success of assessment systems, and this is clearly linked to the credibility of seafood labels,

claims and product incentives in the marketplace. Where assessment systems fail to meet specific criteria, this needs to be addressed, and at the very least communicated to the end-users of the assessment outcomes, usually seafood consumers. The evaluation we report here is one contribution to that process of improving the transparency and accountability in integrated fishery assessment systems. 8. Conclusions The integrated fishery assessment systems evaluated here are highly diverse in their application of key criteria, and hence the robustness of their assessments are likely to be equally as diverse. We identify four specific areas of criterion weakness across most of the systems, and where improvements should be urgently developed. Without further development in those systems that have been established to issue eco-labels and other related forms of buying recommendations, particularly for those that are global in scale, there is a risk that the overall effectiveness of all market-based seafood eco-labelling systems will be degraded. Consumers may become disengaged and disillusioned with a proliferation of assessment systems that make competing claims about the sustainability of fishery products, or alternatively, the majority of products will carry one or more forms of sustainability label thus eliminating the market-incentive that drives the voluntary system of fishery assessment. This risk can be reduced through improvement of assessment criteria in the areas of weakness we have identified here. Specifically, as national

ARTICLE IN PRESS D. Leadbitter, T.J. Ward / Marine Policy 31 (2007) 458–469

jurisdictions begin to implement the FAO procedures for eco-labelling of seafood products, and more broadly adopt voluntary market-based incentive systems, the design and implementation of a robust set of assessment criteria such as those we have applied here may be able to make an important contribution to the global success of marketbased incentives for fishery sustainability. References [1] Pauly D, Christensen V, Gue´nette S, Pitcher TJ, Sumaila UR, Walters CJ, et al. Towards sustainability in world fisheries. Nature 2002;418:689–95. [2] Tillin HM, Hiddink JG, Jennings S, Kaiser MJ. Chronic bottom trawling alters the functional composition of benthic invertebrate communities on a sea-basin scale. Marine Ecology Progress Series 2006;318:31–45. [3] Kaplan IM, McCay BJ. Cooperative research, co-management and the social dimension of fisheries science and management. Marine Policy 2004;28:257–8. [4] FAO Reykjavik conference on responsible fisheries in the marine ecosystem. Reykjavik, Iceland, 1–4 October 2001, Rome: FAO. [5] Caddy JF, Seij JC. This is more difficult than we thought! The responsibility of scientists, managers and stakeholders to mitigate the unsustainability of marine fisheries. Philosophical Transactions of the Royal Society B—Biological Sciences 2005;360:59–75. [6] Cochrane KL. Reconciling sustainability, economic efficiency and equity in fisheries: the one that got away? Fish and Fisheries 2000;1: 3–21. [7] Caddy J. Fisheries management in the twenty-first century: will new paradigms apply? Reviews in Fish Biology and Fisheries 1999;9:1–43. [8] Charles AT. Sustainable fisheries systems. London, UK: Blackwell Science; 2001. [9] Gallagher AD, Johnson G, Glegg G, Trier C. Constructs of sustainability in coastal management. Marine Policy 2004;28:249–55. [10] Lane DE, Stephenson RL. Fisheries Management Science: a framework for the implementation of fisheries management systems. ICES Journal of Marine Science 1999;56:1059–66. [11] WCED. Our common future: the Bruntland report. New York: Oxford University Press from the World Commission on Environment and Development; 1987.

469

[12] ESD Fisheries Working Group. Ecologically sustainable development working groups. Final report—fisheries. Canberra: Australian Government Publishing Service; 1992. [13] McPhee D, Leadbitter D, Skilleter GA. Swallowing the bait: is recreational fishing ecologically sustainable? Pacific Conservation Biology 2002;8:40–51. [14] Foster E, Haward M, Coffen-Smout S. Implementing integrated oceans management: Australia’s south east regional marine plan (SERMP) and Canada’s eastern Scotian shelf integrated management (ESSIM) initiative. Marine Policy 2005;29:391–405. [15] Liu WH, Ou CH, Ting KH. Sustainable coastal fishery development indicator system: a case of Gungliau, Taiwan. Marine Policy 2005; 29:199–210. [16] Bellamy J, Walker D, McDonald G, Syme G. A systems approach to the evaluation of natural resource management initiatives. Journal of Environmental Management 2001;63:407–23. [17] Bronstein C, Lee M, Safina C. Harnessing consumer power for ocean conservation. Conservation in Practice, Summer 2003;4(4). [18] Lee M, editor. Seafood Lover’s Almanac. New York: National Audubon Society; 2000. [19] Clarke B. The Good Fish Guide: the ultimate consumer guide to eating eco-friendly fish. UK: Marine Conservation Society; 2002. [20] May B, Leadbitter D, Sutton M, Weber M. The marine stewardship council. In: Phillips B, Ward T, Chaffee C, editors. Eco-labeling in fisheries: what is it all about? UK: Blackwell Science; 2003. [21] Pitcher TJ, Preikshot DB. Rapfish: a rapid appraisal technique to evaluate the sustainability status of fisheries. Fisheries Research 2001; 49(3):255–70. [22] Bathgate R. The ecological sustainability of marine fisheries: an overview of southeastern Australian fisheries. Report for the Victorian National Parks Association, 1999. [23] Nielsen J, Hara M. Transformation of South African industrial fisheries. Marine Policy 2006;30:43–50. [24] Crosoera D, van Sittert L, Ponte S. The integration of South African fisheries into the global economy: past, present and future. Marine Policy 2006;30:18–29. [25] Leadbitter D, Gomez G, McGilvray FM. Sustainable fisheries and East Asia Seas: can the private sector play a role? Ocean and Coastal Management 2006;49(9-10):662–75. [26] FAO. Guidelines for the ecolabelling of fish and fishery products from marine capture fisheries. Rome: FAO; 2005. [27] Brown J. An account of the dolphin-safe tuna issue in the UK. Marine Policy 2005;29:39–46.