Environmental sustainability indicators for cash-crop farms in Quebec, Canada: A participatory approach

Ecological Indicators 45 (2014) 677–686

Contents lists available at ScienceDirect

Ecological Indicators journal homepage: www.elsevier.com/locate/ecolind

Environmental sustainability indicators for cash-crop farms in Quebec, Canada: A participatory approach Marie-Noëlle Thivierge a , Diane Parent a , Valérie Bélanger a , Denis A. Angers b , Guy Allard a , Doris Pellerin a , Anne Vanasse a,∗ a b

Faculté des sciences de l’agriculture et de l’alimentation, Université Laval, Québec, QC, Canada G1V 0A6 Soils and Crops Research and Development Centre, Agriculture and Agri-Food Canada, 2560 Hochelaga Boulevard, Québec, QC, Canada G1V 2J3

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 17 June 2013 Received in revised form 22 April 2014 Accepted 11 May 2014

On-farm environmental assessment, with consideration to the specificity of the farming system and the geographic zone, can enable farmers to include the environmental aspect in their management decisions. In the province of Quebec, Canada, 45% of the cultivated land is dedicated to grain production and among the 13,800 farms that sell grains, 3975 are specialized in this production. Cereal-based systems have their own constraints and realities and could benefit from a specific tool to assess their environmental sustainability. The objective of this research was to adapt and further develop a set of indicators of environmental sustainability at the farm level for cash-crop farms of the province of Quebec, in order to provide a self-assessment and decision-aid tool to farmers. Using a methodology based on focus groups of experts (researchers, stakeholders, and farmers), several indicators developed for dairy farms were adapted to cash-crop farms. Then the set of indicators was tested on cash-crop farms across the province through interviews with 31 farmers. The indicators were weighted according to their contribution to four sub-objectives of environmental sustainability (soil, water, air, and biodiversity conservation). A new type of chart was designed to help farmers understand and interpret the scores obtained from the set of indicators. Finally, a questionnaire was sent to the 31 farmers for end-use validation. A total of 16 indicators emerged from this research. The weighting reveals that, out of a total of 177 points, the indicators that contribute the most to environmental sustainability of cash-crop farms are “integrated pest management” (21 points), “crop diversity” (19 points), “riparian buffer strip” (18 points), and “incorporation of manure into the soil” (16 points). In comparison with a radar chart and a conventional bar chart, a new bar chart revealed to be a better decision aid tool, allowing the majority of farmers to identify the sustainability weaknesses of a fictive farm. However, the graphic design of this chart could be improved for easier understanding. The end-use validation confirmed the interest of farmers in this decision-aid tool. © 2014 Elsevier Ltd. All rights reserved.

Keywords: Sustainable agriculture On-farm assessment Decision-aid tool Cropping system Bar chart End-use validation

1. Introduction Environmental sustainability can tenance of natural capital, which providing sink and source functions 1995; Van Cauwenbergh et al., 2007).

be defined as the maincomprises the resources in ecosystems (Goodland, Many attempts to address

∗ Corresponding author. Tel.: +1 418 656 2131x12262; fax: +1 418 656 7856. E-mail addresses: [email protected] (M.-N. Thivierge), [email protected] (D. Parent), [email protected] (V. Bélanger), [email protected] (D.A. Angers), [email protected] (G. Allard), [email protected] (D. Pellerin), [email protected] (A. Vanasse). http://dx.doi.org/10.1016/j.ecolind.2014.05.024 1470-160X/© 2014 Elsevier Ltd. All rights reserved.

sustainability have been made since the Rio Earth Summit of 1992, through efforts from several countries to establish indicators for measuring progress (Rigby et al., 2001). Indicators are variables that provide information on other variables that are less available (Gras et al., 1989). They simplify the information (Andersen et al., 2013; Girardin et al., 1999; Mitchell et al., 1995; Rigby et al., 2001; Singh et al., 2012) and serve as a benchmark to make a decision (Gras et al., 1989) or to quantify the degree of compliance with environmental objectives (Van der Werf et al., 2007). In agriculture, on-farm assessment is essential to guide farmers with their management decisions (Häni et al., 2003; Pacini et al., 2003; Van Cauwenbergh et al., 2007). The use of a set of indicators constitutes a holistic approach that takes into account all agricultural practices within the system (Bockstaller et al., 1997).

678

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

One of the first sets of indicators at the farm level was the Farmer sustainability index of Taylor et al. (1993), with 33 weighted indicators designed for cabbage farmers in Malaysia. Their methodology included a panel of experts as well as interviews with farmers. Bockstaller et al. (1997) and Girardin et al. (2000) went one step further by linking their indicators with sustainability sub-objectives or components. Their AGRO*ECO method for cash-crop farms was validated in France and Germany, and the results were presented to farmers using a radar chart. Since its first edition in 2000, the IDEA method (Vilain et al., 2008) moved the focus to the educational aspect of assessing the sustainability at the farm level. The EVAD method (Rey-Valette et al., 2008), inspired by IDEA’s principles, improved and documented the methodology to help farmer groups to construct their own set of indicators using a participatory methodology. The MOTIFS method (Meul et al., 2008) contributed to the user-friendliness of this kind of tool with an improved version of the radar chart. Other indicators and methods for on-farm sustainability assessment were developed within the European Union, as the Common Agricultural Policy (CAP) reform under Agenda 2000 made of sustainable development a priority (Commission of the European Communities, 1999). Recently, in the province of Quebec, Canada, Bélanger et al. (2012) developed agri-environmental indicators to specifically assess the sustainability of dairy farms. Indicators at the local production level have to reflect sitespecific characteristics (Sattler et al., 2010), including the climatic and natural conditions of the site (Commission of the European Communities, 1999), and the particularities of the farming system under study (Meul et al., 2008). The climatic and natural conditions prevailing in Quebec differ from those of Europe, mostly regarding the length of the growing season, the water regime, and the nature of arable soil. As those factors have a strong influence on crop production, it appears relevant to offer farmers a tool adapted to their specific conditions. Moreover, in Quebec, grain production has increased by 25% between 1998 and 2007 (BPR, 2008), and 47% of the cultivated land is now dedicated to this production (ISQ and MAPAQ, 2013). Cash crops in Quebec mostly include grain maize (corn), wheat, oats, barley, canola (colza), and soybeans. Among the 13,800 farms that sold grains in 2010, there were 3975 for which it accounted for more than half of the farm income (ISQ and MAPAQ, 2013; Statistics Canada, 2012). Therefore, this specific farming system deserves some attention. The objective of this research was to adapt and further develop a set of farm-level indicators of environmental sustainability for Quebec cash-crop farms, in order to provide a self-assessment and decision-aid tool to farmers. Complementary objectives were to improve the methodology to allocate weights to such indicators, and to design a new type of chart leading to a better interpretation of the scores resulting from the sustainability assessment. 2. Methodology The conceptual framework of the methodology is illustrated in Fig. 1 and will be detailed in Sections 2.1–2.5. The steps in the construction of indicators are interactive: the results from one step could lead to some modifications in previous ones (Rey-Valette et al., 2008). Those feedbacks are illustrated by the arrows in Fig. 1. Furthermore, this methodology can be described as adaptive and iterative (Meul et al., 2009; Rey-Valette et al., 2008). 2.1. Adaptation of indicators from dairy farms to cash-crop farms The original set of indicators from Bélanger et al. (2012) had been developed using the Delphi method (Delbecq et al., 1975) to inquire 25 experts through anonymous individual questionnaires,

Fig. 1. Conceptual framework of the methodology developed to adapt a set of indicators of environmental sustainability to cash-crop farms of the province of Quebec. The arrows illustrate the many feedbacks, making it an adaptive and iterative process.

for several rounds of questions. Thereafter, 12 experts (researchers, stakeholders, and farmers) were gathered to discuss the results in a panel, also referred to as a focus group. See Bélanger et al. (2012) for the detailed methodology regarding the Delphi method and the focus group. This participatory approach is named co-construction of indicators (Rey-Valette et al., 2008) or bottom-up approach (Fraser et al., 2006; King et al., 2000; Singh et al., 2012). According to Rey-Valette et al. (2008), it is important to bring together different stakeholders, including farmers, in the process of indicator construction. The inputs of farmers, often neglected in such processes, increase the likelihood of the indicators being accepted by the users (Dalal et al., 1999; Fraser et al., 2006; King et al., 2000). Thus, to adapt the dairy farm indicators from Bélanger et al. (2012) to the reality and constraints of cash-crop farms, the same type of methodology based on the consultation with experts was chosen, though with a smaller panel of eight experts (researchers, stakeholders, and farmers). The evaluation criteria described by Bélanger et al. (2012) were being sought during the adaptation process and must be seen as guidelines. Thereby, selected indicators should aim at being: (1) easy to implement, (2) immediately understandable, (3) reproducible, (4) sensitive to variations, (5) adapted to the objectives, and (6) relevant for users (see Bélanger et al., 2012, for a detailed description of these evaluation criteria). The discussions among experts were recorded for future references. 2.2. Testing of the indicators on cash-crop farms After a first focus group with the panel of experts, the selected indicators were tested on 31 cash-crop farms across eight areas of the province of Quebec (Table 1). A cash-crop farm can be defined as a farm where cash crops production accounts for 50% or more of its income (Statistics Canada, 2012). The objectives of these tests were to validate the calculations for each indicator, verify if the indicators fulfilled some of the criteria (criteria 1, 3, and 4 of Section 2.1), establish their suitability for all cropping systems, and determine whether the questions were understandable to all farmers. The farms were recruited with the help from several AgriEnvironmental Advisory Clubs across the province. For each farm, a one-to-one interview with the farmer was conducted. During this 2-h interview, a questionnaire was filled with the farmer. The agri-environmental fertilization plan of the farm (AOR, 2002) was also used as a data source. To be easy to implement, on-farm indicators must take advantage of the information already available that is credible (Bockstaller et al., 1997; Halberg, 1999; Meul et al., 2009; Mitchell et al., 1995; Rigby et al., 2001). Feedbacks from farmers were collected to improve the indicators.

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

679

Table 1 Characteristics and geographical distribution of the 31 cash-crop farms used for the testing of the indicators. Geographic areas

Montérégie-Ouest Montérégie-Est Centre du Québec Estrie Chaudière-Appalaches Bas-St-Laurent Mauricie Saguenay/Lac St-Jean

Number of farms

6 6 2 4 2 5 2 4 31

Average land area (ha/farm)

275.9 183.6 348.2 317.3 41.0 266.6 196.0 164.3 231.8

Type of production

Type of soil tillage

Organic production

Conventional production

Conservation tillage only

Conventional tillage only

Mix of both types

2 2 0 2 0 0 1 1 8

4 4 2 2 2 5 1 3 23

2 4 1 0 1 3 0 2 13

0 0 0 2 1 0 0 2 5

4 2 1 2 0 2 2 0 13

2.3. Weighting of the indicators The weight is the measure of the relative importance of an item in an ensemble (Morris, 1992). Here, the importance of an item is seen in regard of environmental sustainability, which can be defined as the maintenance of natural resources that are provided by the ecosystem (Goodland, 1995; Van Cauwenbergh et al., 2007). To facilitate a structured discussion about weighting, the concept of environmental sustainability was subdivided into four subobjectives. These sub-objectives were based on the four challenges ensuring a sustainable agriculture (Lefebvre et al., 2005; Michaud et al., 2006) and are: (1) soil quality conservation (comprising physical, chemical, and biological aspects), (2) water quality conservation (regarding pollution by fertilizers, pesticides, suspended solids, and pathogens), (3) air quality conservation (regarding greenhouse gases, ammonia, and pesticides), and (4) aboveground biodiversity conservation. The aim of these subdivisions is not to oversimplify the relationships existing in this ecosystem, but to structure the discussions among experts. According to Lefebvre et al. (2005), biodiversity comprises the indigenous species which, if reduced, will disrupt the ecosystem functions. To avoid doubleweighting of some indicators, it was decided that belowground biodiversity (microorganisms, fungi, etc.) would be comprised in the biological aspect of soil quality, rather than as part of biodiversity. Although farmers were consulted for the adaptation (Section 2.1), testing (Section 2.2), and validation of the indicators (Sections 2.4 and 2.5), they were not convened for the weighting. This second panel gathered seven experts (researchers and stakeholders). As reported by King et al. (2000), the bottom-up approach, which favors the creation of indicators by the users (farmers), is often criticized because it implies that scientific knowledge has less value than that of farmers. We argue that the weighting of indicators should be based on scientific references in order to reflect the contribution of each indicator to environmental sustainability. The indicators were weighted according to their contribution to each of the sub-objectives. A scale was built from 0, for a nil contribution of the indicator to a sub-objective, to 5 points, for a major contribution of the indicator to a sub-objective. The points awarded to indicators act as units of sustainability. Care was taken to be consistent both horizontally (between sub-objectives for the same indicator) and vertically (between indicators for the same sub-objective), which required several adjustments (Table 2). The points of sustainability of an indicator do not constitute an absolute value, but rather an indication of its relative importance (Rigby et al., 2001) compared to others in the achievement of environmental sustainability as defined in this study. The total contribution of an indicator to the overall environmental sustainability was taken as the sum of the individual contributions of this indicator to the conservation of soil, water, air, and biodiversity (Table 2). As an

implicit premise behind this methodology, the four sub-objectives (soil, water, air, and biodiversity conservation) were assumed of equal importance in environmental sustainability. 2.4. Global expert validation According to Girardin et al. (1999), to validate indicators is to verify if they meet the objectives for which they were created. This corresponds to the “accuracy evaluation” described by Meul et al. (2009). As an indicator is by definition a variable that provides information on other less accessible variables (Gras et al., 1989), it is often impossible to validate it by comparing with real data (Bockstaller and Girardin, 2003; Girardin et al., 1999; Rigby et al., 2001; Vilain et al., 2008). Furthermore, there is rarely a linear relationship between an indicator and a given measure (Bockstaller and Girardin, 2003; Girardin et al., 1999). The participation of experts and the constant reference to scientific literature were considered to be an a priori validation of the set of indicators, called design validation (Bockstaller and Girardin, 2003; Meul et al., 2009). Moreover, after the weighting of the indicators by the panel of experts, the final product was sent to all the experts involved at this point for a global expert validation or output validation (Bockstaller and Girardin, 2003; Rigby et al., 2001; Vilain et al., 2008). All the experts (researchers, stakeholders, and farmers) had the opportunity to provide comments about the calculation or the weighting, which were used to make small adjustments to the set of indicators. We consider that this validation is still in progress: publishing the results in peer-reviewed scientific journals is part of it, as independent experts will express their comments (Meul et al., 2009; Vilain et al., 2008). Furthermore, as this set of indicators will be implemented on farms, more feedbacks and comments from farmers and advisors will be taken into consideration. 2.5. End-use validation The end-use validation, or usefulness test, is a feedback process between users (the farmers to whom the indicators are dedicated) and designers of the indicators (Bockstaller and Girardin, 2003; Bockstaller et al., 1997; Girardin et al., 1999; Mitchell et al., 1995). It corresponds to the “credibility evaluation” described by Meul et al. (2009). It can be performed through a survey among users (Girardin et al., 1999) to verify their satisfaction with the proposed tool (Vilain et al., 2008), their understanding of the results (Bockstaller and Girardin, 2003), and the usefulness of the indicators as a decisionaid tool (Meul et al., 2008). Thereby, a questionnaire was sent to the 31 farmers who participated in the interviews, and it was filled and returned by 16 of them. The questionnaire was designed to inquire about: (i) the perceptions of farmers about the proposed approach, (ii) their opinion about the scores of their farm, (iii) their willingness to use this set

680

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

Table 2 Example of the weighting of three indicators according to their contribution to environmental sustainability sub-objectives. Scale 0 (nil contribution) to 5 points of sustainability (major contribution). Indicator

Soil organic matter content (#1) Soil phosphorus saturation (#2) Deep and surface drainage (#3)

Contribution to environmental sustainability sub-objectives

Contribution to sustainability

Soil quality

Water quality

Air quality

Biodiversity

Maximum weight of the indicator

5 1 4

3 4 2

2 0 4

2 3 0

12 8 10

of indicators, and (iv) the most effective chart to illustrate their scores. It allowed the authors to verify if the indicators fulfilled some of the selection criteria (criteria 2, 5 and 6 of Section 2.1). The questionnaire, which could be filled out in 20–30 min, included multiple-choice questions and spaces for feedbacks.

3. Results and discussion 3.1. Selected indicators The adaptation of the dairy farm indicators of Bélanger et al. (2012) by the panel of experts led to 16 indicators for cash-crop farms, grouped into four components (Table 3). As mentioned in the methodology, the original indicators from Bélanger et al. (2012) had been developed using the Delphi method and focus groups. Despite its high value to structure group communication when dealing with complex problems, the Delphi method has some limitations that must be recognized (Linstone and Turoff, 2002). In the literature, most of the common pitfalls reported are associated with unclear objectives (too specific or too vague), a lack of criteria in the selection of experts, bias introduced by the main researcher’s influential position, the desire for excessive simplicity that leads to a reductionist approach of the studied system, the loss of interest from the participants over time, the creation of an artificial consensus by ignoring some dissident opinions, and finally, the risk to conclude that consensus always means righteousness (Hasson and Keeney, 2011; Keeney et al., 2001; Linstone and Turoff, 2002; Vernon, 2009). Many of these pitfalls are not unique to the Delphi method, but apply to many consensus techniques in action research. Taking appropriate precautions and documenting explicitly the communication process, as the authors endeavored to do, can reduce the vulnerability of the method to criticism (Linstone and Turoff, 2002; Vernon, 2009). As stated by Vernon (2009), the Delphi method “will only be as robust as the researchers’ justification for their protocol.” Indicators #4 to #16 (Table 3) are means-based indicators, also called action variables or management indicators (Payraudeau and Van der Werf, 2005; Von Wirén-Lehr, 2001). These indicators are generally based on methods or means of practicing agriculture (Bockstaller et al., 2008), which arise from decisions taken by the farmer (e.g. to incorporate or not the manure into the soil). It is therefore an indirect assessment, or a prediction, of the environmental impact (Bockstaller et al., 2008; Rigby et al., 2001). Indicators #1 to #3 are state indicators, or effect-based indicators (Payraudeau and Van der Werf, 2005; Von Wirén-Lehr, 2001), calculated from measures taken directly into the field (e.g. soil phosphorus saturation). They identify the state of the environment at a given time (Gras et al., 1989) but without identifying the cause–effect relationship (Bockstaller et al., 2008). This set of indicators being intended as an educational tool, it was considered relevant to combine means-based and state indicators. Moreover, Quebec’s regulation (AOR, 2002) provides requirements covering both agricultural practices and state of the environment. According to Heink and Kowarik (2010) and Rey-Valette et al. (2008), it

is possible to use different types of indicators within the same set, provided that the method of construction and calculation of each indicator is specified. In an effort of transparency, the three state indicators were then identified as such in the component state of the soil resource (Table 3). Among the many differences from the dairy farm indicators of Bélanger et al. (2012) is the addition of an indicator referring to energy consumption. On cash-crop farms of Quebec, 52% of the energy consumption can be attributed to the use of diesel, mainly for motorized vehicles (AGECO, 2006). The diesel consumption is difficult to quantify at the farm level because diesel is used for personal purposes as well as for farm work (Bélanger et al., 2012). Furthermore, the intensity of soil tillage, which is linked to the fuel consumption, is already assessed with indicator #4 (Table 3). Hence, diesel consumption was not retained as a potential indicator. As second in order, the propane gas accounts for 23% of the total energy consumption of cash-crop farms and is mainly used to dry maize grains (AGECO, 2006; La Financière agricole du Québec, 2010). For this reason, the drying of maize was considered relevant as an energy means-based indicator (#8, Table 3). The other differences from Bélanger et al. (2012) include the addition of indicators about split nitrogen applications (#11, Table 3), presence of annual legume crops in the rotation (#6C), seed treatment (#7C) and implementation of refuges along with insect-resistant transgenic crops (#7D), and the removal of the indicator about manure storage. Also, major changes have been made in the calculation of indicators #1, 3, 5, 6A, 6B, 12, and 13. Most of the indicators (Table 3) are expressed with interval classes (0–20–40–60–80–100%) rather than a continuous scale (0–100%). This allows some consistency between quantitative and qualitative indicators and avoids putting too much emphasis on the accuracy of the measurements but rather on the diversity of the selected indicators (Rey-Valette et al., 2008).

3.2. Modifications following on-farm testing The testing on 31 farms led to improvements for some indicators in regard with their understanding by farmers. For erosion in slopping fields (indicator #13), farmers were first asked to identify areas with erosion issues by coloring their farm map. Large differences were observed in the way farmers responded. Many of them did not have the expertise to diagnose soil erosion problems. Gomez et al. (1996) faced a similar problem and used cover crops as an indirect assessment of the risk of soil erosion. The objective behind indicator #13 is not to evaluate the farmer’s ability to identify problems of erosion, but rather the susceptibility of farmlands to erosion (Joel Aubin, 2009, pers. comm.). In such a case, one solution is to ask a succession of yes or no questions to the farmer regarding concrete and objective key factors that play a role in erosion (e.g. slopes, plowing, cover crops, and riparian buffer strip). By aggregating the answers to these questions, it is possible to estimate more objectively the susceptibility of farmlands to erosion. It was decided to present these successive yes or no questions in the form of a decision tree (Fig. 2),

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

681

Table 3 Definition of the 16 indicators for the assessment of environmental sustainability on cash-crop farms in the province of Quebec. Component

Indicator (number)

State of soil resource

Soil organic matter content (#1) Soil phosphorus saturation (#2) Deep and surface drainage (#3) Conservation tillage (#4)

Cropping practices

Sub-indicator

Proportion of the cultivated area with soil organic matter content greater or equal to 3%

Cover crops (#5)

Crop diversity (#6)

Sequence of crops (6A) Perennial crops (6B)

Integrated pest management (IPM) (#7)

Annual legume crops (Fabaceae) (6C) Herbicide use (7A) Insecticide and fungicide use (7B) Seed treatment (7C) Refuges along with insect-resistant transgenic crops (7D)

Drying of maize (#8) Fertilization management

Phosphorus balance (#9) Nitrogen balance (#10) Split nitrogen applications (#11) Incorporation of manure into the soil (#12)

Erosion in sloping fields (#13) Riparian buffer strip (#14)

Windbreaks (#15) On-farm woodlot (#16)

Proportion of the cultivated area with soil phosphorus saturation under the maximum level allowed according to the province of Quebec regulation (<7.6% for clay soils; <13.1% for other soils) Proportion of the cultivated area with deep or surface drainage problems (0% of area with problems = score of 100%; 1–5% = score of 80%; 6–10% = score of 60%; 11–15% = score of 40%; 16–20% = score of 20%; >21% = score of 0%) Proportion of the cultivated area under different types of conservation tillage (areas under no-till = score of 100%; ridge-till = score of 80%; other reduced-till = score of 50%; conventional till = score of 0%) Proportion of annual crops area sown with or followed by green manure or cover crop (areas with green manure incorporated into the soil after winter = score of 100%; with green manure incorporated before winter = score of 70%; with crop remaining during winter (e.g. winter wheat) = score of 40%; without green manure or cover crop = score of 0%) The main crop rotation: (a) includes 3 different crops and (b) does not include the same annual crop more than 2 years in a row (a and b = score of 100%; only a or only b = score of 70%; none of those = score of 0%) Proportion of the cultivated area with perennial crops (≥15% of the cultivated area = score of 100%; 12–14.9% = score of 80%; 9–11.9% = score of 60%; 6–8.9% = score of 40%; 3–5.9% = score of 20%; 0–2.9% = score of 0%) Proportion of annual crops area cultivated with annual legume crops (≥30% of annual crops area = score of 100%; 25–29.9% = score of 80%; 20–24.9% = score of 60%; 15–19.9% = score of 40%; 10–14.9% = score of 20%; 0–9.9% = score of 0%) Proportion of the cultivated area without herbicide (score of 100%), with herbicides according to an IPM approach (score of 70%), with herbicides without IPM approach (score of 0%) Proportion of the cultivated area without insecticide and fungicide (score of 100%), with insecticide and fungicide according to an IPM approach (score of 70%), with insecticide and fungicide without IPM approach (score of 0%) Proportion of annual crops area without seed treatment (insecticide and fungicide)

Solid manure (12A)

Along watercourses (14A) Along agricultural ditches (14B)

Weight 12

8

10

9

10

7

8

4

6

8

4

The appropriate refuges are implemented along with an insect-resistant transgenic crop when required (yes = score of 100%; no = 0%)

3

Ratio of the volume (liters) of propane gas used to dry maize by the total quantity (tons) of maize harvested (≤23.9 L/t = score of 100%; 24–27.9 L/t = score of 80%; 28–31.9 L/t = score of 60%; 32–35.9 L/t = score of 40%; 36–39.9 L/t = score of 20%; ≥40 L/t = score of 0%) Proportion of the cultivated area where the phosphorus added to the soil does not exceed the needs of the crop by more than 10 kg P2 O5 ha−1

5

Proportion of the cultivated area where the nitrogen added to the soil does not exceed the needs of the crop by more than 10 kg N ha−1 Proportion of areas cultivated with maize, spring wheat or colza with split nitrogen applications

Liquid manure (12B)

Farmland management

Indicator definition

Proportion of total amount of solid manure (≥15% DM) or solid waste fertilizer managed under different categories (incorporated within 12 h = score of 100%; applied on a growing crop and incorporated within 12–48 h = score of 80%; applied on a bare soil and incorporated within 12–48 h = score of 60%; applied on a growing crop and non incorporated = score of 40%; applied on a bare soil and non incorporated = score of 0%) Proportion of total amount of liquid manure (<15% DM) managed under different categories (incorporated within 3 h = score of 100%; applied on a growing crop and incorporated within 3–24 h = score of 80%; applied on a bare soil and incorporated within 3–24 h = score of 60%; applied on a growing crop and non incorporated = score of 40%; applied on a bare soil and non incorporated = score of 0%) Limitation of erosion in sloping fields with preventive practices (see decision tree, Fig. 2)

Respect of a riparian strip of 3 m width, without fertilization, without annual crops and without tillage, along all water courses (>2 m2 of flow area) (yes = score of 100%; no = score of 0%) Respect of a riparian strip of 1 m width, without fertilization, without annual crops and without tillage, along all agricultural ditches (≤2 m2 of flow area) (yes = score of 100%; no = score of 0%) Ratio of the total length (m) of windbreaks on the farm by the cultivated area (ha) (>80 m ha−1 = score of 100%; 60–80 m ha−1 = score of 75%; 40–60 m ha−1 = score of 50%; 20–40 m ha−1 = score of 25%; <20 m ha−1 = score of 0%) Presence of on-farm wood lot of a minimum area of 5 hectares (yes = score of 100%; no = 0%) Maximum score

5

9 8

7

9

10

10

8

9

8 177

682

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

Fig. 2. Decision tree for indicator #13, “erosion in sloping fields”.

which facilitates the aggregation of the answers to the several questions. 3.3. Weight of the 16 indicators Fig. 3 illustrates the results of the weighting as established by the panel of experts. The points awarded act as units of sustainability. The contribution of one indicator to the overall environmental sustainability is the sum of the contributions of this indicator to the four sub-objectives, each represented by a different color/texture in Fig. 3. The total weight of this set of indicators is 177. Considering that the objective is to provide a self-assessment and decision-aid tool to farmers, the focus should be on the identification of the indicators needing improvements, rather than on the total score. Aggregating indicators to a unique score leads to a loss of information (Von Wirén-Lehr, 2001). Some indicators show a very large contribution to the overall environmental sustainability of cash-crop farms, e.g. the indicator #7 “integrated pest management” (21 points out of 177), while others show a lower contribution, e.g. indicator #8 “drying of maize” (5 points out of 177) (Fig. 3). The weighting allows the farmer to compare the risks associated with different agricultural practices (Rigby et al., 2001). Fig. 3 also reveals which indicators contribute in a special way to specific sub-objectives. “Crop diversity” (#6) is the indicator that contributes the most to soil quality conservation, “riparian buffer strip” (#14) to water quality, “incorporation of manure into the soil” (#12) to air quality, and “integrated pest management” (#7) to biodiversity conservation.

Fig. 3. The 16 indicators, grouped into four components, and their contribution to environmental sustainability after weighting. The different colors/textures show the contribution of the indicators to the four sub-objectives (biodiversity, air quality, water quality, and soil quality).

For many reasons, some authors chose not to allocate weight to indicators. However, as highlighted by Rey-Valette et al. (2008), the deliberate choice of not weighting indicators is the equivalent to agree that they all have an equal value, which is a form of weighting. This tool is seeking an educational purpose with farmers. According to our research team, to claim that all the indicators have the same value in terms of sustainability would be worse than to attempt allocating weights to them. The consistency of the methodology, coupled with the quality of experts convened, contributes to make the weighting as objective as possible. As stated by King et al. (2000), it is the rigor of the focus group that ensures the validity of the results. In this type of approach, it is decided to accept the subjectivity associated with the decision-making process within the group (Roy, 1992, cited in Bockstaller et al., 1997). The weighting should not be read as the quantitative influence of an agricultural practice on the ecosystem, but rather as a relative contribution to sustainability based upon an understanding of how each agricultural practice will affect the physical, chemical, and biological processes, and then influence the entire system (Rigby et al., 2001). 3.4. Scores of the 31 farms The farm score for a given indicator is calculated by multiplying the result of the farm by the total weight attributed to this indicator (Table 3). For example, for the indicator #8 “drying of maize”, a farm that uses 30 liters of propane per ton of maize will have a score of 3 points (the category 28–31.9 L/t is worth 60% of the total weight of 5 points). The scores of the 31 farms are shown in Table 4, as well as the scores of the leading group as a reference value. This group includes the 25% best-performing farms for the overall environmental sustainability assessment. Thus, the score of the leading group is the average result of those farms for each indicator. The reference value allows the user to compare his score and to assess its value (Bockstaller et al., 2008; Piorr, 2003). According to Halberg (1999), the best way to interpret results at the farm level is to compare with other farms: farmers commonly compare themselves to their peers (Gomez et al., 1996). When asked about this in the end-user validation questionnaire, 15 out of 16 farmers appreciated the option to compare their score with the leading group while all of them liked a comparison to the average of all farms. The authors believe that the use of a leading group can raise the standard, while providing a reference value more stable than an average. It is worth mentioning that the 31 participating farms in the study were chosen to depict the main cultivated areas within the province of Quebec, as well as the diversity of cropping practices for cash-crop farms. The Agro-Environmental Advisory Clubs suggested farms that are members of their organization and, presumably, the ones that tend to be interested in this kind of project. For this reason, the authors believe that most of the 31 farms were a priori more sustainable than the average cash-crop farm in Quebec. However, this test was not meant to establish the environmental sustainability of the farms of the province of Quebec, but rather to verify if the indicators fulfilled the selection criteria, and if they were suitable for all cropping systems. Descriptive statistics are presented in Table 4. For some indicators, even the leading group gets a poor score, as with indicator #5, “cover crops”, where the leading group gets 3.8 out of 10 while the average is 2.3. However, this is consistent with the official data from the province of Quebec, where the areas declared as green manure barely reach 6% of the cultivated lands (BPR, 2008). This example demonstrates that getting a score higher than the leading group does not guarantee a sustainable practice (Bockstaller et al., 2008). Experts agreed that the cover-crop

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

683

Table 4 Results of the testing of the 16 indicators on 31 farms across the province of Quebec. Indicators

Weight

Scores of the 31 farms Median

Average

Scores of the leading group Standard deviation

Median

Average

Soil organic matter content (#1) Soil phosphorus saturation (#2) Deep and surface drainage (#3)

12.0 8.0 10.0

12.0 8.0 8.0

10.6 7.3 7.5

±2.76 ±1.31 ±3.35

12.0 7.9 10.0

11.9 6.9 9.0

Conservation tillage (#4) Cover crops (#5) Crop diversity (#6) Integrated pest management (#7) Drying of maize (#8)

9.0 10.0 19.0 21.0 5.0

4.8 2.1 11.0 16.4 5.0

5.1 2.3 12.4 16.3 3.7

±2.87 ±2.28 ±3.52 ±3.39 ±1.77

4.7 3.9 12.6 19.2 5.0

4.6 3.8 13.2 18.7 4.4

Phosphorus balance (#9) Nitrogen balance (#10) Split nitrogen applications (#11) Incorporation of manure into the soil (#12)

5.0 9.0 8.0 16.0

4.7 8.8 8.0 11.8

4.2 7.8 6.6 11.0

±0.96 ±1.76 ±2.83 ±4.06

4.7 9.0 8.0 14.4

4.4 8.2 8.0 13.9

Erosion in sloping fields (#13) Riparian buffer strip (#14) Windbreaks (#15) On-farm woodlot (#16)

10.0 18.0 9.0 8.0

10.0 18.0 4.5 8.0

8.5 15.4 4.4 7.7

±2.47 ±5.35 ±3.18 ±1.44

10.0 18.0 6.8 8.0

9.3 18.0 6.5 8.0

177.0

130.0

130.8

±15.36

147.9

148.7

Total

practice is quite applicable but simply not a custom for farmers, who sometimes lack knowledge on how to integrate those crops in their rotation. It therefore appeared appropriate to keep this indicator as it is, so that farmers can realize its importance in the sustainability of their farm. Similarly, an indicator for which all the farms have high scores will also be preserved. This is the case for indicators #2 “soil phosphorus saturation” and #16 “onfarm woodlot”, for which we obtain the averages of 7.3 and 7.7 respectively, out of 8. As reported by Singh et al. (2012), some authors use bivariate or multi-variate analyses to verify the correlation for pairs of indicators, or between an indicator and the total score. A sensitivity analysis can also be performed (Andersen et al., 2013). Like Rigby et al. (2001), we found those verifications to be unnecessary. Indeed, each indicator was deemed relevant by the experts and thus carries a message that needs to be delivered in an authentic way to farmers. Although a special effort was made to reduce the redundancy of information in the selection and the weighting of indicators, it is impossible to avoid it completely in a system as complex as a farm. In the case of an educational tool, the redundancy between some indicators may even be conscious and voluntary (Vilain et al., 2008). During the end-use validation, farmers were asked if their scores corresponded to the idea they had of the environmental situation of their own farm. Thirteen out of 15 farmers agreed with their scores. Ten farmers out of 13 (some farmers did not answer this question) claimed that if this set of indicators was available, they would consider using it regularly (every 2 or 3 years). These feedbacks from farmers are crucial, since “a good indicator is an indicator that is used” (Rey-Valette et al., 2008). In addition, 13 out of 13 farmers consider that this tool could help all cash-crop farmers to improve their agricultural practices in order to move towards a better environmental sustainability.

conventional bar chart, and a new bar chart) were presented to the farmers. For a given graph, the scores of a fictive case-study farm were shown. Farmers were asked to identify the three indicators that this fictive farm could use to most improve the farm’s environmental sustainability. The results for the three types of charts are presented below, with emphasis on the comparison between the radar chart (Section 3.5.1) and the new bar chart (Section 3.5.2). 3.5.1. Radar chart Fig. 4 shows the scores of one fictive farm with a radar chart, the most common way to express indicators (Bockstaller et al., 1997; Gomez et al., 1996; Rigby et al., 2001; Von Wirén-Lehr, 2001). Each axis represents an indicator. The results are expressed in percentages. For each indicator, the center is the lowest score (0%), while the outer ring corresponds to the highest score (100%). The weight of each indicator is written besides its number. This type of chart is renowned for facilitating the comparison of results (Bockstaller et al., 1997; Gomez et al., 1996; Vilain et al., 2008), for instance, between a farm and a leading group (Fig. 4).

3.5. Visual presentation As this set of indicators is intended to be a decision-aid tool, the results should be easily understood by farmers. Charts allow presenting the score for each indicator without aggregating them into an overall score (Rigby et al., 2001). The choice of the chart type “is crucial in terms of communication and use” and must be tested (Rey-Valette et al., 2008). Hence, in the end-use validation questionnaire sent to the 31 farmers, different charts (radar chart,

Fig. 4. Scores of the 16 indicators for a fictive case-study farm (solid line) and comparison with the leading group (dotted line) using a radar chart.

684

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

Table 5 Comparison of the three charts proposed to the farmers in the end-use validation questionnaire to illustrate the scores of fictive case-study farms. Number of answers for this question Number of farmers able to identify two out of three indicators to improve Is this chart easy to understand? Which chart do you prefer? (select only one)

Radar chart

Conventional bar chart

New bar chart

16

5

3

10

16

12

10

10

15

6

6

3

Indeed, a visual comparison of two values for the same indicator (on the same axis) is convenient. For example, for the indicator #5, the case-study farm is more sustainable than the leading group (Fig. 4). Nevertheless, a concern raised by Bockstaller et al. (1997) is that the radar chart makes it extremely difficult to compare one indicator to another (on different axes), and therefore, to identify the strengths or weaknesses of the farm. In fact, the weight of each indicator is not illustrated in this chart, although it is written beside the axis. For example, in Fig. 4, the first indicator that the farm could benefit from improving seems to be #8, because its value is very close to the center of the chart. However, this indicator is only worth five points. In terms of sustainability, the farm would gain much more by improving indicator #14, with a possibility of 13 points (units of sustainability). The necessity to have in mind the respective weight of each indicator complicates the decisionmaking process. In the end-use validation questionnaire, only five out of 16 farmers (Table 5) correctly identified at least two out of the three indicators that would most benefit sustainability. Misinterpretations related to the comparison of the indicators are such that this chart does not show the full potential of a set of weighted indicators used as a decision-aid tool. Although this type of chart seems to be the least understood by cash-crop farmers, it is also the one that the largest number of them (12 out of 16) identified as being easy to understand (Table 5). This may be because many farmers in the province of Quebec are familiar with this type of chart through financial or management counseling services. It can be noted that with indicators that are not weighted, this problem of comparison does not occur.

Again, the points awarded to indicators act as units of sustainability. The central axis separates the sustainability units already acquired (colored/textured portion of the bar on the right side of the chart) from those that remained to be earned (white portion on the left side). The indicators that contribute the most to current sustainability of the farm are those with the longest colored/textured bars on the right side of the chart, while the ones that the farm would benefit the most to improve are those with the longest white bars on the left side. With the improvement of agricultural practices over time, longer portions of the bars will be found on the right side of the chart. The dots represent the scores of the leading group. In comparison with the radar chart, it is then more obvious that, from an environmental standpoint, the case-study farm would benefit more by improving indicator #14 than #8 (Fig. 5). Another advantage of this new bar chart, as opposed to the radar chart, is that it allows the comparison of an endless number of indicators without compromising clarity. During the end-use validation, 10 out of 16 farmers (Table 5) could correctly identify at least two out of the three indicators that would most improve environmental sustainability. In addition, 10 out of 14 farmers believe the new bar chart is easy to understand, although it was selected by only three farmers as their favorite (Table 5). The authors believe that the demonstrated greater understanding allowed the farmers to draw accurate conclusions from their scores with the new bar chart, thus justifying its choice as a self-assessment and decision-aid tool.

3.5.2. New bar chart To help reveal all the potential of a weighted set of indicators as a decision-aid tool, another type of chart has been designed (Fig. 5). Each horizontal bar represents one indicator, and the respective weights of the indicators are illustrated by the length of each bar.

Despite the fact that the results obtained with the new bar chart are better than with other charts, it is still disappointing that only 10 out of 16 farmers reached accurate conclusions (Table 5). First, these results need to be validated with more farmers. Secondly, this raises a concern with the objective of a self-assessment tool. According to Bockstaller et al. (1997) and Von Wirén-Lehr (2001), the scores must be supported by advice. The end-use validation questionnaire showed that 14 out of 16 farmers would like written recommendations with the charts. These results are consistent with those of Halberg (1999) and Meul et al. (2009), in which farmers asked for guidance in interpreting the scores. The authors think that the presence of an advisor for the interpretation of the scores could enhance the educational purpose of such a tool. When questioned about this, 12 out of 16 farmers said they would appreciate this presence. A farm advisor aware of the reality of the farm may be able to help the farmer to interpret its scores and to provide advices and additional information (Meul et al., 2008, 2009) taking into account other factors regarding economic and social aspects of sustainability. Several of the participating farmers mentioned not being comfortable with computer technology. A strictly online tool would restrict the access to many farmers. Finally, 13 out of 15 farmers wished not to spend more than 2 h for a sustainability assessment. This corroborates the findings of Rey-Valette et al. (2008). The presence of an advisor could help with the technology as well as with the duration.

Fig. 5. Scores of the 16 indicators for a fictive case-study farm and comparison with the leading group using the new bar chart.

3.6. Is self-assessment possible

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

3.7. Weaknesses 3.7.1. Representativity of the first panel of experts Because this study was meant to adapt existing dairy farm indicators to cash-crop farms, only 10 experts were convened for the first focus group. The withdrawal of two of them brought the size of the discussion panel to eight experts, including two farmers specialized in cash crops from farms certified as organic. Although this had no influence on the weighting, since farmers did not participate in that step, the discussions of the first focus group could have been enriched by the presence of one or two farmers from conventional production. 3.7.2. Evaluation of indicators regarding the selection criteria Mitchell et al. (1995) and Piorr (2003) mentioned the difficulty of meeting all the criteria for each indicator. According to Dalal et al. (1999), an evaluation of the indicators in this regard can be done through discussions or constructive debates held between the experts. In the present study, the selection criteria listed in Section 2.1 were explained to the experts at the first focus group and then discussed more deeply after tests on farms (for criteria 1, 3 and 4 of Section 2.1) and end-use validation (for criteria 2, 5 and 6). However, there was no posteriori evaluation of the criteria. A threshold could have been established, as in Bélanger et al. (2012), where indicators that met less than four out of six criteria were removed. At least, as highlighted by Mitchell et al. (1995), the indicators which do not respond, or only partially, to selection criteria should be identified as such and revised periodically. 4. Conclusions It is essential to advocate an agricultural development that does not harm the rights of future generations to grow (Vilain et al., 2008). The present set of indicators is based upon current knowledge of sustainability and cropping practices on cash-crop farms in the province of Quebec. By the means of focus groups and interviews with farmers, 16 indicators have been adapted to cash-crop production and weighted according to their contribution to four sub-objectives of environmental sustainability (conservation of soil, water, air, and biodiversity). A new type of chart was designed to help farmers interpret their sustainability assessment scores. Finally, an end-use validation questionnaire was used for farmer feedback. Those results need to be validated at a broader scale. Precision agriculture and energetic self-sufficiency of farms are among the elements that should be further explored in subsequent studies. Moreover, special attention must be paid to score transmission to farmers; it should not be forgotten that a set of indicators is a communicative tool, and that the participation of farmers in the construction and validation of sustainability indicators is fundamental. Acknowledgements The authors would like to thank all experts who were involved in this project, as well as all farmers who allowed to test the indicators on their farm and who shared their feedbacks. The fruitful discuss¨ Aubin, from UMR SAS, INRA, ions with Hayo van der Werf and Joel Rennes, France, are warmly acknowledged. The authors are grateful to the Fonds de recherche du Québec – Nature et technologies (FQRNT) for financing this research. References AGECO, 2006. Profil de consommation d’énergie à la ferme dans six des principaux secteurs de production agricole du Québec. Rapport no 1.

685

Agricultural Operations Regulation (AOR), 2002. Environment Quality Act, Q-2, r.26. Minister of Sustainable Development, Environment, Wildlife and Parks, www.mddep.gouv.qc.ca/publications/index.htm (accessed 18.04.14). Andersen, P.S., Vejre, H., Dalgaard, T., Brandt, J., 2013. An indicator-based method for quantifying farm multifunctionality. Ecol. Indic. 25, 166–179. Bélanger, V., Vanasse, A., Parent, D., Allard, G., Pellerin, D., 2012. Development of agrienvironmental indicators to assess dairy farm sustainability in Quebec, Eastern Canada. Ecol. Indic. 23, 421–430. Bockstaller, C., Girardin, P., 2003. How to validate environmental indicators. Agric. Syst. 76, 639–653. Bockstaller, C., Girardin, P., Van der Werf, H.M.G., 1997. Use of agro-ecological indicators for the evaluation of farming systems. Eur. J. Agron. 7, 261–270. Bockstaller, C., Guichard, L., Makowski, D., Aveline, A., Girardin, P., Plantureux, S., 2008. Agri-environmental indicators to assess cropping and farming systems. A review. Agron. Sustain. Dev. 28, 139–149. BPR, 2008. Suivi 2007 du Portrait agroenvironnemental des fermes du Québec. In: Rapport présenté au ministère de l’Agriculture, des Pêcheries et de l’Alimentation du Québec, à l’Union des producteurs agricoles et à Agriculture et Agroalimentaire Canada, Ottawa. Commission of the European Communities, 1999. Directions Toward Sustainable Agriculture. Communication from the Commission, no 22. European Commission, Brussels. Dalal, R.C., Lawrence, P., Walker, J., Shaw, R.J., Lawrence, G., Yule, D., Doughton, J.A., Bourne, A., Duivenvoorden, L., Choy, S., Moloney, D., Turner, L., King, C., Dale, A., 1999. A framework to monitor sustainability in the grains industry. Aust. J. Exp. Agric. 39, 605–620. Delbecq, A.L., Van de Ven, A.H., Gustafson, D.H., 1975. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Scott, Foresman, Glenview, IL. Fraser, E.D.G., Dougill, A.J., Mabee, W.E., Reed, M., McAlpine, P., 2006. Bottom up and top down: analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. J. Environ. Manage. 78, 114–127. Girardin, P., Bockstaller, C., Van der Werf, H.M.G., 1999. Indicators: tools to evaluate the environmental impacts of farming systems. J. Sustain. Agric. 13, 5–21. Girardin, P., Bockstaller, C., Van der Werf, H.M.G., 2000. Assessment of potential impacts of agricultural practices on the environment: the AGRO*ECO method. Environ. Impact Assess. Rev. 20, 227–239. Gomez, A.A., Swete Kelly, D.E., Syers, J.K., Coughlan, K.J., 1996. Measuring sustainability of agricultural systems at the farm level. Methods for assessing soil quality. Soil Sci. Soc. Am. J. 49, 401–410. Goodland, R., 1995. The concept of environmental sustainability. Annu. Rev. Ecol. Syst. 26, 1–24. Gras, R., Benoit, M., Deffontaines, J.P., Duru, M., Lafarge, M., Langlet, A., Osty, P.L., 1989. Le fait technique en agronomie. In: Activité agricole, Concepts et Méthodes d’Étude. Institut National de Recherche Agronomique, Paris. Halberg, N., 1999. Indicators of resource use and environmental impact for use in a decision aid for Danish livestock farmers. Agric. Ecosyst. Environ. 76, 17–30. Häni, F., Braga, F., Stämpfli, A., Keller, T., Fischer, M., Porsche, H., 2003. RISE, a tool for holistic sustainability assessment at the farm level. Int. Food Agribusiness Manage. Rev. 6, 78–90. Hasson, F., Keeney, S., 2011. Enhancing rigour in the Delphi technique research. Technol. Forecast Soc. 78, 1695–1704. Heink, U., Kowarik, I., 2010. What are indicators? On the definition of indicators in ecology and environmental planning. Ecol. Indic. 10, 584–593. Institut de la statistique du Québec (ISQ), Ministère de l’Agriculture, des Pêcheries et de l’Alimentation du Québec (MAPAQ), 2013. Profil sectoriel de l’industrie bioalimentaire du Québec 2012, www.stat.gouv.qc.ca/ statistiques/agriculture/index an.html (accessed 18.04.14). Keeney, S., Hasson, F., McKenna, H.P., 2001. A critical review of the Delphi technique as a research methodology for nursing. Int. J. Nur. Stud. 38, 195–200. King, C., Gunton, J., Freebairn, D., Coutts, J., Webb, I., 2000. The sustainability indicator industry: where to from here? A focus group study to explore the potential of farmer participation in the development of indicators. Aust. J. Exp. Agric. 40, 631–642. La Financière agricole du Québec, 2010. Assurance stabilisation. Coût de production, revenu stabilisé et caractéristiques techniques, 2010, www.fadq.qc.ca (accessed 10.06.13). Lefebvre, A., Eilers, W., Chunn, B., 2005. Environmental Sustainability of Canadian Agriculture: Agri-Environmental Indicator Report Series-Report 2. Agriculture and Agri-Food Canada, Ottawa. Linstone, A., Turoff, M., 2002. The Delphi Method: Techniques and Applications. NJIT, New Jersey. Meul, M., Van Passel, S., Nevens, F., Dessein, J., Rogge, E., Mulier, A., Van Hauwermeiren, A., 2008. MOTIFS: a monitoring tool for integrated farm sustainability. Agron. Sustain. Dev. 28, 321–332. Meul, M., Nevens, F., Reheul, D., 2009. Validating sustainability indicators: focus on ecological aspects of Flemish dairy farms. Ecol. Indic. 9, 284–295. Michaud, A., Martel, S., Seydoux, S., Beaudin, I., 2006. Évaluation des effets combinés des principales pratiques de gestion bénéfiques (PGB). In: Initiative Nationale d’Élaboration de Normes Agroenvironnementales. Institut de recherche et de développement en agroenvironnement, Québec. Mitchell, G., May, A., McDonald, A., 1995. PICABUE: a methodological framework for the development of indicators of sustainable development. Int. J. Sustain. Dev. World 2, 104–123.

686

M.-N. Thivierge et al. / Ecological Indicators 45 (2014) 677–686

Morris, C. (Ed.), 1992. Academic Press Dictionary of Science and Technology. Academic Press, California. Pacini, C., Wossink, A., Giesen, G., Vazzana, C., Huirne, R., 2003. Evaluation of sustainability of organic, integrated and conventional farming systems: a farm and field-scale analysis. Agric. Ecosyst. Environ. 95, 273–288. Payraudeau, S., Van der Werf, H.M.G., 2005. Environmental impact assessment for a farming region: a review of methods. Agric. Ecosyst. Environ. 107, 1–19. Piorr, H.-P., 2003. Environmental policy, agri-environmental indicators and landscape indicators. Agric. Ecosyst. Environ. 98, 17–33. Rey-Valette, H., Clément, O., Aubin, J., Mathé, S., Chia, E., Legendre, M., Caruso, D., Mikolasek, O., Blancheton, J.-P., Slembrouck, J., Baruthio, A., René, F., Levang, P., Morissens, P., Lazard, J., 2008. Guide to the Co-construction of Sustainable Development Indicators in Aquaculture. EVAD, Montpellier. Rigby, D., Woodhouse, P., Young, T., Burton, M., 2001. Constructing a farm level indicator of sustainable agricultural practice. Ecol. Econ. 39, 463–478. Singh, R.K., Murty, H.R., Gupta, S.K., Dikshit, A.K., 2012. An overview of sustainability assessment methodologies. Ecol. Indic. 15, 281–299. Sattler, C., Nagel, U.J., Werner, A., Zander, P., 2010. Integrated assessment of agricultural production practices to enhance sustainable development in agricultural landscapes. Ecol. Indic. 10, 49–61.

Statistics Canada, 2012. Statistics on revenues and expenses of farms 2010. Catalog no. 21-208-X. Minister of Industry, Ottawa. Taylor, D.C., Mohamed, Z.A., Shamsudin, M.N., Mohayidin, M.G., Chiew, E.F.C., 1993. Creating a farmer sustainability index: a Malaysian case study. Am. J. Alternative Agric. 8, 175–184. Van Cauwenbergh, N., Biala, K., Bielders, C., Brouckaert, V., Franchois, L., Cidad, V.G., Hermy, M., Mathijs, E., Muys, B., Reijnders, J., Sauvenier, X., Valckx, J., Vanclooster, M., Van der Veken, B., Wauters, E., Peeters, A., 2007. SAFE – a hierarchical framework for assessing the sustainability of agricultural systems. Agric. Ecosyst. Environ. 120, 229–242. Van der Werf, H.M.G., Tzilivakis, J., Lewis, K., Basset-Mens, C., 2007. Environmental impacts of farm scenarios according to five assessment methods. Agric. Ecosyst. Environ. 118, 327–338. Vernon, W., 2009. The Delphi technique: a review. Int. J. Ther. Rehabil. 16 (2), 69–76. Vilain, L., Boisset, K., Girardin, P., Guillaumin, A., Mouchet, C., Viaux, P., Zahm, F., 2008. La méthode IDEA, indicateurs de durabilité des exploitations agricoles, 3e édn. Guide d’utilisation. Educagri Éditions, Dijon. Von Wirén-Lehr, S., 2001. Sustainability in agriculture-an evaluation of principal goal-oriented concepts to close the gap between theory and practice. Agric. Ecosyst. Environ. 84, 115–129.