- Email: [email protected]
Exploring fishing dependence in gulf coast communities
Marine Policy 34 (2010) 1307–1314
Contents lists available at ScienceDirect
Marine Policy journal homepage: www.elsevier.com/locate/marpol
Exploring fishing dependence in gulf coast communities Steve Jacob a,1, Priscilla Weeks b,n, Benjamin G. Blount c,2, Michael Jepson d,3 a
York College of Pennsylvania, 205 Life Sciences Annex, York, PA 17405, USA Houston Advanced Research Center, 4800 Research Forest Drive, The Woodlands, TX 77381, USA SocioEcological Informatics, 13239 Spring Run, Helotes, TX 78023, USA d National Marine Fisheries Service, NOAA, Southeast Regional Office, Social Sciences Branch, 263, 13th Avenue South, St. Petersburg, FL 33701, USA b c
a r t i c l e in fo
abstract
Article history: Received 14 May 2010 Accepted 5 June 2010
Two unrelated data sources (quantitative secondary data and qualitative primary data) and mixed methodologies (statistical analysis and ethnography) are used to define the concept of, and develop indicators for, fishing dependence. Techniques for integrating differing data sources are developed. Comparisons of the qualitative rankings with the quantitative rankings were, overall, positive and statistically significant. The process used thus confirmed that the indicators were reliable measures for fishing dependence. In terms of external validity and triangulation, the process used was more rigorous than using ethnography ‘‘after-the-fact’’ to ground-truth the quantitative indicators. & 2010 Elsevier Ltd. All rights reserved.
Keywords: Fishing dependence Social indicators Ethnography Evaluation
1. Introduction 1.1. Fishing community dependency and social impact assessment Regulations can have an array of social and economic impacts on fishers and the communities in which they live and work [1–4]. According to the Magnuson–Stevens Fishery Conservation and Management Act, National Standard 8, fishery management plans must identify and consider the impacts of fisheries management on fishing communities [5]. This mandate is based on the recognition that management efforts affect not only the individual harvester or processor but also an array of fishery related businesses such as boatyards, ice suppliers and tackle shops. The Act defines a fishing-dependent community as ‘‘a community which is substantially dependent on or substantially engaged in the harvest or processing of fishery resources to meet social and economic needs, and includes fishing vessel owners, operators, and crew and United States fish processors that are based in such a community’’ [5]. Consequently, Federal law mandates social impact assessment of fisheries regulations including allocations, reallocations, closures, restrictions, limited entry schemes, or any other policy change that might adversely impact fishing-dependent communities [3,6]. This has traditionally
n
Corresponding author. Tel.: +1 281 364 6049; fax: + 1 281 363 7935. E-mail addresses: [email protected] (S. Jacob), [email protected] (P. Weeks), [email protected] (B.G. Blount), [email protected] (M. Jepson). 1 Tel.: + 1 717 815 6412. 2 Tel.: + 1 210 372 0724. 3 Tel.: + 1 727 551 5756. 0308-597X/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.marpol.2010.06.003
been accomplished using community profiles and ethnographic assessment [7]. The purpose of the project on which this paper is based was to develop and evaluate social indicators based on secondary quantitative data to measure the concepts of dependence, gentrification, vulnerability, and resiliency. A mixed methods design was employed. Because an important goal of the project was to evaluate the external validity of the indicators constructed using secondary data, they were compared to independently conducted ethnographic assessments in the same communities. In this paper, the method is described by focusing on the indicators constructed for fishing dependence. There is a tension between qualitative and quantitative research in regard to issues of validity and reliability. Quantitative research tends to be more reliable in that it uses repeated measures in a consistent way. This is less true of qualitative research, which tends to build constructs from emergent findings. Qualitative research, on the other hand, tends to have greater external validity. That is to say, the construct is grounded firmly in the real world. Quantitative research tends to simplify or reduce constructs into easily measured pieces that may not accurately reflect real world conditions. Both types of methods were employed in order to assess how the quantitative and qualitative results vary. Thus, the social indicators were developed concurrent with, but independently of, fieldwork. It should be noted that the fieldwork represents a process of discovery of the concept of dependence as it relates to each study site, as opposed to ground-truthing which takes concepts as defined and verifies them in the study sites. In terms of external validity and triangulating the concepts, the process used was more rigorous than ground-truthing.
1308
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
2. Methods and data 2.1. Secondary quantitative data Three separate data sources were used to build the data set used in the study. The primary source for population information was the US Bureau of the Census, 2000 Decennial Census Summary Tape File 3. The main source for information about fishery landings, permits, and value was provided by NMFS Southeast Fishery Science Center (SEFSC) and Southeast Regional Office (SERO). Lastly, data for marinas and related business were downloaded from the 2002 Economic Census using the US Census Bureau’s ‘‘American FactFinder’’ web page. Two groupings of communities in the Gulf of Mexico, primarily based on the estuaries where they were located, were selected for the study: 5 communities on or near Galveston Bay and 4 communities on or near San Antonio Bay. These bay systems were chosen because they are at different stages of urbanization. The Galveston Bay region has been undergoing rapid and profound change for the past 30 years. In contrast, land use and population changes around the San Antonio–Matagorda Bay complexes are more recent. Such disparate communities were chosen because it was theorized that, given the traditional fishermen strategy of seeking jobs outside the fishery when fishing is bad or the season is closed [8], those communities closest to urban centers with diverse economies would be less dependent on fisheries than those further from urban centers due to the availability of nonfishing jobs close by. Because nine communities would not provide sufficient variation in the data for reliable index development, it was decided to include all communities in the county and adjacent counties. This resulted in a data set with 122 different communities and provided sufficient variation for index development. The results of the community rankings on each variable are reported, however, only for the nine communities as identified. 2.2. Primary qualitative data An inductive grounded approach was used to independently test the characterizations of the selected communities provided by the large, secondary data sets, especially as they relate to the importance of the commercial and recreational fishing sectors. This assessment consisted of two major components: (1) ethnographic field research within the nine communities, involving interviews with fishers, individuals whose businesses were related to fishing, and community officials and leaders about the place and importance of fishing within the communities, and (2) compilation of historical and contextual background information to assess socioeconomic dependence of the community on their fisheries. Both components produced data that allowed for relative ranking of the nine communities. Interview data were based on culturally significant categories provided by the interviewees themselves. Rankings were also constructed from the historical/contextual information. The rankings from the two types of data (interview and contextual) were highly similar. 2.3. Social indicator indices development strategy Three steps were taken to develop the social indices of fisheries dependence. First, correlation coefficients were examined to find underlying patterns of variation. Second, the variables that were most highly inter-correlated and reflected the range of ideas of interest were placed in a principal components analysis, where these variables were determined to be reliable indices. Last, the variables were standardized and weighted for their
effects in the model. Index factor scores were used. Factor scores are similar to composite scores, with the exception that the items are standardized and weighted in regard to their factor loadings. The factor loadings are a rough indication of correlation of the domain concept’s latent structure to the single variable. Therefore, items that are most important in an index receive a higher weighting than a less important item. In principal components, factor loadings less than 0.350 are generally not considered to be significant and in most cases should be removed from a factor scale. One advantage of factor scaling is that negative relationships do not have to be reverse-coded before scaling. This means that negative factor loadings work to reduce the overall score and the absolute number conveys the strength of relationship regardless of being negative or positive. The interpretation of a negative factor loading is similar to a negative Pearson’s r bivariate correlation. The factor scores were standardized with a mean of zero and the scores reflecting standard deviations from that mean. Scales were subsequently tested for internal consistency by using Armor’s [9] theta reliability for factor scales. The theta coefficient is interpreted similar to Cronbach’s alpha and is used for factor scales because it does not assume that all items are weighted equally in the scale. Theta is calculated as y ¼[p/(p 1)] [1 (1/l)], where p is the number of items in the scale and l denotes the largest eigenvalue from the principal components analysis. 2.4. Social indicator indices components and internal reliability To establish internal reliability, multiple indicators for each concept are necessary. At a minimum it is necessary to include enough variables to fully cover the range of the concept, while maintaining unidimensionality (only measuring one central concept). In general, multiple measures are preferred and do increase internal validity when the items are significantly intercorrelated. However as more variables are added to the index it is harder to maintain unidimensionality. Unidimensionality in part is established by principal components analysis. In a principal components analysis a single factor solution provides evidence that the various index items only measure a single concept. The indices in this study range from a low of four items to a high of seven items. Indices with three or fewer items are generally thought to be insufficient to establish internal validity through Cronbach’s alpha or Armor’s theta. Here you will find a description of the components of each index, the principal components analysis and factor loadings, and measures of internal validity including the eigenvalue, percentage explained variation, and Armor’s theta reliability.
3. Results 3.1. Commercial fishing dependence index The commercial fishing dependence index (Table 1) consists of five variables: (1) percentage of labor force employed in agriculture, fishing, and hunting in 2000 (range 1.24–20%), (2) pounds of landings per 1000 population in 2007 (range 7727–2,428,921 pounds per 1000 population), (3) commercial fishing permits per 1000 population in 2007 (range 0.01–15.38 per 1000 population), (4) value of landings per 1000 population (range $11,222–4,845,564), and (5) number of permitted dealers with landings per 1000 persons (range 0.11–4.86). The principal components analysis produced a single factor solution with an eigenvalue of 3.512 with an explained variation of 70.202%. Armor’s theta reliability coefficient was 0.894 reflecting a
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
1309
Table 1 The commercial dependence index. Community
Percent employed in ag., fishing, and hunting
Pounds of landings per 1000 persons
Commercial fishing permits per 1000 population
Value of landings per 1000 population ($)
Dealers with landings per 1000 persons
Commercial dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
3.61 17.70 20.00 10.05 2.45 5.12 1.53 1.24 2.37
9555 941,393 124,196 2,428,921 243,443 749,085 102,372 16,933 14,632
3.23 1.40 15.38 11.88 0.35 0.20 1.05 0.02 0.01
34,638 1,424,031 350,978 4,845,564 211,624 1,835,321 222,314 42,757 11,222
0.26 2.1 4.86 2.38 0.44 1.82 0.25 0.11 0.13
0.498 0.624 1.575 2.057 0.543 0.149 0.644 0.76 0.716
5 3 2 1 6 4 7 9 8
0.765
0.871
0.77
0.873
PC components Factor loading 0.902 Theta reliability: 0.894 Eigenvalue: 3.512 Percentage explained variation: 70.202
Single factor solution
High ranking ¼ more dependent Low ranking ¼less dependent
Table 2 The recreational dependence index. Community
Charter permits per 1000 population
Marinas and related businesses per 1000 population
Marinas and related businesses jobs per 1000 population
Marinas and related businesses gross earnings per 1000 population ($)
Boat launches per 1000 population
Recreational dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
0.52 0.00 26.72 0.00 0.00 0.81 0.74 0.00 0.00
0.44 0.00 2.43 0.40 1.15 0.81 0.19 0.20 0.00
2.01 0.00 8.10 0.99 3.92 3.64 1.62 0.79 0.00
71,921 0 449,392 59,382 182,879 384,304 131,717 31,520 0
0.44 2.10 4.05 0.20 0.26 0.81 0.21 0.09 0.25
0.355 0.517 2.763 0.516 0.188 0.449 0.401 0.656 0.816
4 7 1 6 3 2 5 8 9
0.966
0.940
0.880
0.816
PC components Factor loading 0.933 Theta reliability: 0.947 Eigenvalue: 4.128 Percentage explained variation: 82.551
Single factor solution
high level of internal consistency. The factor loadings ranged from 0.765 to 0.902. The strongest factor loadings in the analysis were for the variables percentage of labor force employed in agriculture, fishing, and hunting in 2000 (0.902), number of permitted dealers with landings per 1000 persons (0.873), and commercial fishing permits per 1000 population in 2007 (0.871). All the factor loadings were above 0.350 and so were included in the index. Variable 1 was from the 2000 US Decennial Census Summary Tape File 3. Variables 2–5 were from a custom data set generated by NMFS SEFSC landings data and SERO permit data for this research. Variables 2–5 were standardized by taking the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000.
3.2. Recreational fishing dependence index The recreational fishing dependence index (Table 2) consists of five variables: (1) charter permits per 1000 population in 2007
High ranking ¼ more dependent Low ranking ¼ less dependent
(range 0–26.72 per 1000 population), (2) marinas and related businesses per 1000 population (range 0–2.43), (3) marinas and related businesses jobs per 1000 population (0–8.1 per 1000 population), (4) marinas and related businesses gross earnings per 1000 population (range $0–449,392), and (5) boat launches per 1000 population (range 0.09–4.05 per 1000 population). The principal components analysis produced a single factor solution with an eigenvalue of 4.128 and an explained variation of 82.551%. Armor’s theta reliability coefficient for this index is 0.947 reflecting very high levels of internal consistency. The factor loadings for this index range from 0.816 to 0.966. The strongest factor loadings in the analysis were for the variables marinas and related businesses per 1000 population (0.966), marinas and related businesses jobs per 1000 population (0.940), and charter permits per 1000 population in 2007 (0.933). All the factor loadings were above 0.350 and so were included in the index. Variable 1 is from SEFSC landings data. Variables 2–4 are from the 2002 Economic Census available from the ‘‘American FactFinder’’ website. All the variables were standardized by taking
1310
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000. 3.3. Social fishing dependence index The social fishing dependence index (Table 3) consists of five variables: (1) percentage of water cover in the municipal boundary (range 0–77.8%), (2) boat launches per 1000 population (range 0.09–4.05 per 1000 population), (3) percentage of labor force employed in agriculture, fishing, and hunting in 2000 (range 1.24–20%), (4) marinas and related businesses per 1000 population (range 0–2.43), and (5) number of permitted dealers with landings per 1000 persons (range 0.11–4.86). The principal components analysis produced a single factor solution with an eigenvalue of 3.438 and an explained variation of 68.780%. Armor’s theta reliability coefficient was 0.886 for this index reflecting high levels of internal consistency. The factor loadings for this index ranged from 0.355 to 0.982. The strongest factor loadings were for the variables number of permitted dealers with landings per 1000 persons (0.982), boat launches per 1000 population (0.942), and percentage of labor force employed in agriculture, fishing, and hunting in 2000 (0.938). All the factor loadings were above 0.350 and so were included in the index. Variables 1 and 3 are from the 2000 Decennial Census Summary Tape File 3. Variables 2, 4, and 5 are from a custom NOAA NMFS database. Variable 1 was derived from Census data by dividing the total land area in the municipality into the percentage water cover. Variables 2, 4, and 5 were standardized by taking the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000.
over 2 million, and the metropolitan population is approximately 4.5 million. The southeastern city limit is only a few miles from the shoreline. The communities around Galveston Bay are no longer geographically and socially distinct entities, even if they are politically separate. To illustrate how dense the communities are, one encounters six different municipalities along an approximately 10 mile strip of road without seeing any distinct changes between communities. In addition to being one of the largest metropolitan areas in the United States, the area also is heavily industrialized. Large petrochemical complexes and oil fields are situated on and to the northeast of the bay. The southern portion of the research area consists of the communities on the shores of a series of connected bays. The landscape surrounding these bays differs significantly from that of the Galveston Bay Complex. The largest metropolitan entity is Victoria, a city of approximately 60,000 populations, situated about an hour to the northwest. Other towns in the region have populations of less than 12,000, most below 3000 and are geographically bounded by producing farms and ranches. Although the communities in this region are relatively small towns, they differ considerably in socioeconomic characteristics and in terms of their historic and current dependence on fisheries. The ethnographic analysis began with observations based upon open-ended interviews with key informants. Interviewees were asked a series of questions related to historical and current commercial and recreational fishing in their communities. The existence and condition of fishing infrastructure, such as commercial and recreational docks, processing plants, ice houses, bait camps, net shops, tackle shops, etc., were noted. Infrastructure (e.g. statues and fishing themed buildings), events (e.g. fishing festivals or tournaments), and the presence of fishing organizations were also noted. These observations were turned into summaries and then generalizations that were collapsed into ratings of high-, medium-, and low-dependence (see Table 4).
3.4. Ethnographic field research 4. Discussion: differing processes with a converging reality Fieldwork was conducted during the summer of 2008. As previously mentioned, the bay systems chosen are differentially impacted by urbanization and population density differs dramatically. The Galveston Bay Complex is situated southeast of Houston, the 4th largest city in the nation with a population of
4.1. Inter-rater agreement A key concept in the measurement of constructs, such as fishing dependence, is the interchangeability of indicators. Any
Table 3 The social fishing dependence index. Community
Percent water cover
Boat launches per 1000 population
Percent employed in ag., fishing, and hunting
Marinas and related businesses per 1000 population
Dealers with landings per 1000 persons
Social dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
28.70 0.00 25.00 3.70 73.40 5.80 77.80 62.80 0.00
0.44 2.10 4.05 0.20 0.26 0.81 0.21 0.09 0.25
3.61 17.70 20.00 10.05 2.45 5.12 1.53 1.24 2.37
0.44 0.00 2.43 0.40 1.15 0.81 0.19 0.20 0.00
0.26 2.1 4.86 2.38 0.44 1.82 0.25 0.11 0.13
0.420 0.695 2.623 0.381 0.374 0.197 0.805 0.801 0.586
6 2 1 3 5 4 9 8 7
0.778
0.982
PC components Factor loading 0.355 Theta reliability: 0.886 Eigenvalue: 3.438 Percentage explained variation: 68.780
0.942
Single factor solution
0.938
High ranking ¼more reliant Low ranking ¼less reliant
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
1311
Table 4 Economic and social dependence on commercial and recreational fishing by community. Community
Rating and comments
Bacliff
Commercial dependence: Low. There are no commercial facilities in Bacliff. Recreational dependence: Low. A few private docks are attached to houses for recreational fishing or boating. Social dependence: Low. There are no significant fishing social activities.
Galveston
Commercial dependence: Low. Galveston has a public dock for a commercial fleet of finfish and shrimp boats. There are also wholesale buyers and shippers. Commercial fishing is neither a major employer nor is it a major source of revenue. Recreational dependence: Medium. Before Ike, there were numerous bait camps, two long public fishing piers, beaches open to wade-fishing, charter, and head boats for hire. The city is dependent on nature based tourism, and recreational fishing is part of that tourism but is not highly promoted. Social dependence: Low. Galveston tourism is based on two major strands: its Victorian past and cultivation of an ‘island feel’. The Victorian past model promotes architectural reconstruction and festivals such as ‘Dickens on the Strand’. The visual manifestation of ‘beach culture’ has been promoted, however, by planting palm trees near the beach and by selling beach type merchandise. There are numerous seafood restaurants that use a fishing motif, but the city as a whole does not celebrate fishing culture in festivals or visually.
San Leon
Commercial dependence: High. Pre-Ike San Leon was home to two oyster processing plants, two large oyster leaseholders, the Vietnamese crab fishery and processing plant, and two Vietnamese shrimp docks with marinas. Recreational dependence: High. The community also has a large recreational fishing center where both recreational and shrimp boats dock. Social dependence: High. The town’s slogan is ‘‘A small drinking community with a large fishing problem.’’ A local landmark restaurant, written up in out of state tourist articles, is a favorite gathering place for locals and tourists.
Texas City
Commercial dependence: Low. There is one wholesale shrimp and finfish dealer in Texas City and an oyster processor. They existed there only because the city annexed the unincorporated area in which they were located. Previously, this area was part of San Leon and socially that is still the case. Several commercial fishermen interviewed lived in Texas City but docked on Port Bolivar. Recreational dependence: Low. Pre-Ike, the Texas City Dike was home to five bait camps and boat ramps. It also had some slips for shrimp boats. The dike was an important recreational venue for family fishing. There was one abandoned bait camp. There were also boat ramps on Dollar Bay. Despite Texas City being a destination for recreational fishing, fishing did not appear to have a significant economic impact on the community as a whole due to the concentration of the petrochemical industry. Social dependence: Low. Texas City is known as an industrial town, not a fishing town. The Texas City Dike, however, was once a symbol of the town and of family oriented recreational fishing. After the destruction of the bait shops on the Dike after Ike, there were many reminiscences about days spent fishing from the Dike on the local newspaper’s website.
Seadrift
Commercial dependence: High. The Vietnamese crabbing community is located on Seadrift as well as a shrimp fleet, but they have decreased substantially in recent years. There is a commercial docking facility with two buyers located. Recreational dependence: Low. Recreational fishing on Seadrift is relatively low, due to essentially no infrastructure for tourism. Social dependence: Medium. At one time, social dependence was high. Historically there was a large Vietnamese crabbing community, but this is now in decline. The community also hosts a Shrimpfest, and houses are decorated with fishing paraphernalia. Except for the remaining Vietnamese, the sense of community appears to be in decline.
Palacios
Commercial dependence: High. Palacios is home to a large gulf shrimp fleet, The Marine Education Center, and TPWD Research Center. Commercial fishing has been the major economic engine for the community. Recreational dependence: Low/medium. The community hosts fishing tournaments and fishing related festivals, and there is potential for growth of recreational fishing. The relatively undeveloped infrastructure for tourism, however, limits the importance of recreational fishing. Social dependence: High. Palacios hosts the ‘‘Shrimp o Ree,’’ Texas Seafood, and Blessing of the Fleet festivals. The shrimp fleet is networked through a few major families who own fleets and provide loans and other support to local fishers. The Vietnamese community is networked through a local church.
Port O’Connor
Commercial dependence: Low. Port O’Connor had a bay and bait fishery, but they are in rapid decline. Recreational dependence: High. The community is highly dependent on the sport fishery. It hosts several large fishing tournaments. One of the founders of CCA has his summer home there. There are a number of bait camps, boat sheds, and ramps. New resort style residential compounds are being built that include boat ramps and slips. Social dependence: High. Recreational fishing is promoted in various ways throughout the community, specifically fishing tournaments and merchandise in shops (for sale even in the ice cream shop). There is also a Fisherman’s Chapel. The major hotel/restaurant complex is now being remodeled to attract tourists for recreational fishing.
reasonable indicator of a construct should be correlated to other indicators of that same construct. In other words if they are both measuring the same thing they should also strongly relate to one another in the same directional pattern. The degree that two differing indicators of the same construct relate to reality is usually referred to as construct validity. To establish construct validity, ideally the quantitative data in our project should be highly correlated with the qualitative data, which best reflects the objective conditions in the community. Inter-rater reliability is the degree to which independent observers evaluate the characteristics of a subject and reach the same conclusion [10]. High level of agreement in ratings generally reflects the reliability of the standards and process. This is true especially if two different raters are applying the same criteria and reaching the same results. However in the case described here, there are two completely different sets of criteria and
processes and a high level of agreement reflects convergence of a construct with reality. In other words, rather than being a reflection of reliability (receiving the same results from repeated measures using the same criteria) it is a reflection of both construct and external validities (the link between a construct and observed reality). A more accurate term to describe our results, therefore, would be in terms of inter-rater agreement [10] instead of inter-rater reliability. There are some widely reported measures used to assess interrater agreement. They are percentage agreement, correlations based indicators such as Pearson’s r or Spearman’s rho, and Cohen’s kappa. Each of these measures has some significant advantages and drawbacks but taken together they allow for a more complete assessment of inter-rater agreement. Percentage agreement is easily understood and has a straightforward interpretation but can be misleading. The percentage agreement
1312
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
is often inflated because a portion of agreement could be directly due to random matching. This is especially true with constructs with relatively few categories. For example, with three categories up to 11.1% (0.333 0.333 or 1 in 3 1 in 3) of agreement could be due to random matching. Correlational techniques like Pearson’s r or Spearman’s rho measure covariation but not to the extent in which there is identical agreement in the categories. The above correlations are generally interpreted in analysis to be substantial above 0.6. The last statistic described is Cohen’s kappa that measures inter-rater agreement and ranges from 1.0 to 1.0. Cohen’s kappa counts only exact matches, then adjusts for random matching, and as such, it is a very conservative measure of agreement. The closer the number is to 1.0, the greater the agreement is above and beyond random matching. If the number is approaching zero, then the level of agreement is close to what would be expected by chance. If the number is below zero and approaching 1, then the agreement is less than what would be expected by just random matching. Cohen’s kappa is calculated by taking the percentage of agreement [Pr(a)] and subtracting the probability of random agreement [Pr(b)], divided by one minus the probability of random agreement [Pr(a) Pr(e)/(1 Pr(e))]. The probability of random agreement is calculated by dividing 1 by the number of categories for rater one and multiplying it by 1 divided by the number of categories [Pr(e) ¼1/k 1/k, where k is the number of categories for the rater]. Cohen’s kappa is generally interpreted with the following framework from Landis and Koch [11]: less than zero ¼no agreement; 0–0.20¼slight agreement; 0.21–0.40¼ fair agreement; 0.41–0.60 ¼moderate agreement; 0.61–0.80 ¼ substantial agreement; and 0.81–1.0¼almost perfect agreement. A t value can be calculated for kappa by dividing the kappa value by the asymptotic standard error when the null hypothesis is true (the true value is 0). This t value has an associated statistical probability that is often reported. While both percentages and correlational techniques tend to be liberal and over-assess levels of agreement, Cohen’s kappa is considered a very conservative measure and underestimates the strength of agreement. This is because it only includes exact matches as agreement, when often misses are only a category off and the raters are actually in relative agreement. It is possible to use a weighted kappa statistic to account for close misses but it is not commonly done and the statistic is not included in any major software packages. To ensure content validity with the construct of dependence, multiple indicators were developed: (1) commercial, (2) recreational,
and (3) social. To evaluate the agreement of the social indicators with the ethnographic research it was necessary to code the indices into the same categories employed in the qualitative analysis. These categories were: (1) low, (2) medium, and (3) high. Each separate community (N ¼122) was coded into one of the three (low, medium, or high) based on the index factor score, so the response categories within the nine communities are not evenly distributed (e.g. 3 lows, 3 mediums, and 3 highs). For the dependence indices of commercial, recreational, and social, direct comparisons of the ratings for the social indicators and the ethnographic data could be made. This is because in the ethnographic coding process the domains of commercial, recreational, and social dependence emerged from the content analysis providing a check of content validity.
4.2. Inter-rater agreement results Table 5 presents the results of the inter-rater agreement analysis for commercial, recreational, and social fishing dependence. The quantitative and ethnographic assessments matched in 66.67% of the communities for the commercial fishing dependence measure. This produced Spearman’s rho of 0.589; however, it was not statistically significant due to the small N size (9 cases). Cohen’s kappa was 0.500 and was statistically significant, reflecting a moderate level of agreement between the two techniques. In two of the three mismatches the categories were only off by one category. However, in Port O’Connor the mismatch was off by two full categories. The quantitative results indicated high commercial dependence based on the 2007 value and pounds of landings and numbers of commercial licenses and dealers. The ethnographic research was conducted in 2009. According to interviewees, the conditions of the commercial fishing industry in Port O’Connor changed greatly in the two-year delay in data reporting. It is also possible that the ethnographic results were underestimated due to selection bias in key informants. In either case in this community there was a large difference in ratings. Continuing with Table 5, but examining the results for recreational dependence, complete agreement was found between the quantitative and qualitative results. They matched on all nine communities producing 100% agreement, Spearman’s rho of 1.0, and Cohen’s kappa of 1.0. In the case of recreational dependence the techniques were in perfect agreement. Also in Table 5 the inter-rater agreement results for social dependence matched on seven of nine communities (77.78%).
Table 5 Commercial, recreational, and social dependence. Community
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
Commercial dependence
Recreational dependence
Quantitative index
Ethnographic assessment
Differing classification
Quantitative index
Ethnographic assessment
Medium High High High Medium Medium Low Low Low
Low High Low High Medium High Low Low Low
n
Medium Low High Medium Medium High Medium Low Low
Medium Low High Medium Medium High Medium Low Low
Matched on 6 of 9 communities 66.67% matched Spearman’s rho¼ 0.589, P¼ 0.095 Kappa ¼0.500, P ¼0.022
n
n
Matched on 9 of 9 communities 100% matched Spearman’s rho¼1.0, P ¼ 0.000 Kappa ¼1.00, P ¼0.000
Social dependence Differing classification
Quantitative index
Ethnograpic assessment
Medium High High High Medium High Low Low Low
Medium Medium High High Low High Low Low Low
Differing classification
n
n
Matched on 7 of 9 communities 77.78% matched Spearman’s rho¼0.900, P ¼0.000 Kappa¼ 0.660, P ¼0.005
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
Spearman’s rho for the two techniques was 0.900 and this was statistically significant. Cohen’s kappa was 0.660 and was statistically significant and reflects substantial agreement between the raters. In the two mismatched communities the categories were off by only one category, with the quantitative ratings higher than qualitative.
5. Conclusions This project, specifically the evaluation of the agreement between ethnographic research and social indicators developed from secondary data illustrates some advantages of the latter. The advantages relate to time and cost savings, community definition, reliability, and comparability of results across communities. The disadvantages relate to the loss of local context and external validity. Assembling a database, creating indices, and profiling communities through secondary data are techniques that are much faster than conducting traditional community profiling through ethnography. In this study, four full-time fieldworkers collected data in nine community case study sites for a little over 2 months. In addition, many weeks were spent coding and analyzing the data. Cost and time are especially relevant given the budgetary constraints of management agencies. It is not always easy to differentiate among communities on the ground. In the Galveston Bay region, for example, it was very difficult for the field research team to tell where community boundaries began and ended. It was also difficult untangling the interviewees’ references to various places as most referred to the general area where they traveled to meet their daily needs, not to specific municipal boundaries. The secondary data were compiled at the minor civil division level and were obviously discrete and pertained only to that place. Using minor civil division-level data would help solve some of the lingering issues surrounding the definition of community in fisheries research in a simple and reasonable fashion. The disadvantage of such an approach is that communities, defined as a place where one can meet daily needs, are often functionally larger than a minor civil division and often revolve around the human ecological concept of central place [4]. Quantitative indicators based on secondary data allowed for a reliable process of measurement and comparability across the nine study communities because the concept being measured (i.e. dependence) was defined identically across all of the study sites using a deductive process. Conversely, the ethnographic process was emergent and to a large extent dependent on the perceptions of interviewees. Additionally, when using different interviewers, there is always the chance that some areas of the interview will be stressed over others. Because the results of ethnographic research are descriptive, it is very difficult to meaningfully compare different communities in terms of intensity. Secondary quantitative data offer ease of analysis and interpretation along with reliable repeatable indicators and can be incorporated into other forms of social modeling that are not possible with ethnography. The disadvantage of relying on such data is the loss of the local context and external validity. Ethnography is advantageous when looking to define constructs from the perspective of the interviewees in an inductive grounded process but is not as effective in a deductive normative process. Though the construction of social indicators using secondary quantitative data offers some significant advantages over ethnography, it would mean nothing if the results were not representative of community conditions. However, in this research very high levels of agreement between the social indicators approach and ethnography were found. Ultimately
1313
the ratings varied little by research technique. Given this circumstance, the advantages of social indicators based on secondary data far outweighed the disadvantages.
5.1. Robustness of the social indicators There were several concerns with the reliance on secondary data to construct indicators that were proven to be unfounded. In fact, the indicators were shown to be extremely robust in regard to indirect measures, time obsolescence, estimated data, and varying data sources. Reliance on indirect indicators requires ‘‘ground-truthing’’ to ensure external validity and they must be linked to actual conditions in the community. The indirect measures used in this study related very well to community conditions that were identified in ethnography. Obsolescence of the data can also be an issue of concern. Given that the last census was nearly a full decade ago, the logical question was: could data that was 10 years old still reflect actual conditions in the community? In addition, the most current NOAA NMFS data available also lagged by several years. However, this lag seemed to make very little difference in the assessment given the high levels of agreement between the techniques. This is at least partially due to the fact that the process ranked communities arraying them among the 122 places. This ranking is relative to the other places and is likely to be very stable across time because most change is incremental. As such, it is not that impacted by data obsolescence. The exception would be boomtown or bust scenarios. This may have been captured in this research in Port O’Connor in regard to the commercial fishing dependency. So with the caveat of rapidly changing situations as seen in Port O’Connor, relying on interim census estimates provided by the Bureau of the Census did not seem to substantially impact the external validity of the social indicators. Another concern was creating indices that were compiled of variables from different data sources. In all cases the data sources were examined for reliability in data collection and reporting to remove those concerns. Any issues with differing units of measure were resolved by the use of principal components analysis to produce a factor score, which is standardized with a mean of zero and a standard deviation of 1. Last, the problem of concatenating the differing data sets is actually simple as the data can be matched by the geographic identifier (FIPS Code) that has become standardized mostly because of GIS analyses.
5.2. Recommendations for the next steps Though social indicators analysis can greatly enhance and streamline community profiling and social impact assessment in fisheries management, ethnography remains an important component in assuring the external validity of the social indicators. In addition, in-depth ethnography should be conducted when specific communities will be extraordinarily impacted by changes in fisheries regulations because the social indicators based on secondary data may be too insensitive to analyze rapidly changing or soon to be changing situations. In this circumstance the social indicators can be used for identifying the places that require indepth ethnographic research. In addition communities should be randomly sampled for in-depth ethnography so evaluations of inter-rater agreement can be made. This is a critical check for external validity. Yet these indicators can provide valuable information that provides context to fishery management regulation, and when built over time, may be able to describe important trends for impact assessment.
1314
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
Acknowledgements Funding for the research upon which the article is based was provided by a Saltonstall-Kennedy NOAA Grant Number: NA08NMF4270412 granted to the Gulf & South Atlantic Fisheries Foundation Inc. Opinions and conclusions expressed or implied are solely those of the authors and do not necessarily reflect the views or policy of the National Marine Fisheries Service, NOAA. The authors would like to recognize the contributions of the fieldwork team: Lovette Miller, Elizabeth Croucher, Meredith Marchioni and Jerry Lord. They especially thank members of the coastal communities interviewed.
References [1] Smith S, Jepson M. Big fish, little fish: politics and power in the regulation of Florida’s marine resources. Social Problems 1993;40:39–49. [2] Smith SD, Jacob S, Jepson M. The stress process in Florida’s commercial fishing families. Society and Natural Resources 2003;16:39–59.
[3] Clay PM, Olson J. Defining ‘‘Fishing Communities’’: vulnerability and the Magnuson–Stevens Fishery Conservation and Management Act. Human Ecology Review 2008;15:143–60. [4] Tuler S, Agyeman J, Pinto de Silva P, LoRusso KR, Kay R. Assessing vulnerabilities: integrating information about driving forces that affect risks and resilience in fishing communities. Human Ecology 2008;15:171–84. [5] Magnuson–Stevens Fishery Conservation and Management Act (1996). U.S. Department of Commerce. NOAA Technical Memorandum NMFSF/SPO-23, December 1996. [6] Jacob S, Farmer FL, Jepson M, Adams C. Landing a definition of fishing dependent communities: potential social science contributions to meeting National Standard 8. Fisheries 2001;26:16–22. [7] Jacob S, Jepson M, Pomeroy C, Mulkey D, Adams C, Smith S. Identifying fishing dependent communities: development and confirmation of a protocol. Marine Fisheries [MARFIN] Project Report NA87FF0433. St. Petersburg: NOAA NMFS; 2002. [8] Davis DL. Occupational community and fishermen’s wives in a Newfoundland fishing village. Anthropological Quarterly 1986;59:129–42. URL: /http:// www.jstor.org/stable/3317199S (accessed 12/05/2010, 18:23). [9] Armor DJ. Theta reliability and factor scaling. In: Costner Herbert L, editor. Sociological methodology. San Francisco: Josey-Bass; 1974. [10] Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Human Communication Research 2002;28:587–604. [11] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.
Contents lists available at ScienceDirect
Marine Policy journal homepage: www.elsevier.com/locate/marpol
Exploring fishing dependence in gulf coast communities Steve Jacob a,1, Priscilla Weeks b,n, Benjamin G. Blount c,2, Michael Jepson d,3 a
York College of Pennsylvania, 205 Life Sciences Annex, York, PA 17405, USA Houston Advanced Research Center, 4800 Research Forest Drive, The Woodlands, TX 77381, USA SocioEcological Informatics, 13239 Spring Run, Helotes, TX 78023, USA d National Marine Fisheries Service, NOAA, Southeast Regional Office, Social Sciences Branch, 263, 13th Avenue South, St. Petersburg, FL 33701, USA b c
a r t i c l e in fo
abstract
Article history: Received 14 May 2010 Accepted 5 June 2010
Two unrelated data sources (quantitative secondary data and qualitative primary data) and mixed methodologies (statistical analysis and ethnography) are used to define the concept of, and develop indicators for, fishing dependence. Techniques for integrating differing data sources are developed. Comparisons of the qualitative rankings with the quantitative rankings were, overall, positive and statistically significant. The process used thus confirmed that the indicators were reliable measures for fishing dependence. In terms of external validity and triangulation, the process used was more rigorous than using ethnography ‘‘after-the-fact’’ to ground-truth the quantitative indicators. & 2010 Elsevier Ltd. All rights reserved.
Keywords: Fishing dependence Social indicators Ethnography Evaluation
1. Introduction 1.1. Fishing community dependency and social impact assessment Regulations can have an array of social and economic impacts on fishers and the communities in which they live and work [1–4]. According to the Magnuson–Stevens Fishery Conservation and Management Act, National Standard 8, fishery management plans must identify and consider the impacts of fisheries management on fishing communities [5]. This mandate is based on the recognition that management efforts affect not only the individual harvester or processor but also an array of fishery related businesses such as boatyards, ice suppliers and tackle shops. The Act defines a fishing-dependent community as ‘‘a community which is substantially dependent on or substantially engaged in the harvest or processing of fishery resources to meet social and economic needs, and includes fishing vessel owners, operators, and crew and United States fish processors that are based in such a community’’ [5]. Consequently, Federal law mandates social impact assessment of fisheries regulations including allocations, reallocations, closures, restrictions, limited entry schemes, or any other policy change that might adversely impact fishing-dependent communities [3,6]. This has traditionally
n
Corresponding author. Tel.: +1 281 364 6049; fax: + 1 281 363 7935. E-mail addresses: [email protected] (S. Jacob), [email protected] (P. Weeks), [email protected] (B.G. Blount), [email protected] (M. Jepson). 1 Tel.: + 1 717 815 6412. 2 Tel.: + 1 210 372 0724. 3 Tel.: + 1 727 551 5756. 0308-597X/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.marpol.2010.06.003
been accomplished using community profiles and ethnographic assessment [7]. The purpose of the project on which this paper is based was to develop and evaluate social indicators based on secondary quantitative data to measure the concepts of dependence, gentrification, vulnerability, and resiliency. A mixed methods design was employed. Because an important goal of the project was to evaluate the external validity of the indicators constructed using secondary data, they were compared to independently conducted ethnographic assessments in the same communities. In this paper, the method is described by focusing on the indicators constructed for fishing dependence. There is a tension between qualitative and quantitative research in regard to issues of validity and reliability. Quantitative research tends to be more reliable in that it uses repeated measures in a consistent way. This is less true of qualitative research, which tends to build constructs from emergent findings. Qualitative research, on the other hand, tends to have greater external validity. That is to say, the construct is grounded firmly in the real world. Quantitative research tends to simplify or reduce constructs into easily measured pieces that may not accurately reflect real world conditions. Both types of methods were employed in order to assess how the quantitative and qualitative results vary. Thus, the social indicators were developed concurrent with, but independently of, fieldwork. It should be noted that the fieldwork represents a process of discovery of the concept of dependence as it relates to each study site, as opposed to ground-truthing which takes concepts as defined and verifies them in the study sites. In terms of external validity and triangulating the concepts, the process used was more rigorous than ground-truthing.
1308
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
2. Methods and data 2.1. Secondary quantitative data Three separate data sources were used to build the data set used in the study. The primary source for population information was the US Bureau of the Census, 2000 Decennial Census Summary Tape File 3. The main source for information about fishery landings, permits, and value was provided by NMFS Southeast Fishery Science Center (SEFSC) and Southeast Regional Office (SERO). Lastly, data for marinas and related business were downloaded from the 2002 Economic Census using the US Census Bureau’s ‘‘American FactFinder’’ web page. Two groupings of communities in the Gulf of Mexico, primarily based on the estuaries where they were located, were selected for the study: 5 communities on or near Galveston Bay and 4 communities on or near San Antonio Bay. These bay systems were chosen because they are at different stages of urbanization. The Galveston Bay region has been undergoing rapid and profound change for the past 30 years. In contrast, land use and population changes around the San Antonio–Matagorda Bay complexes are more recent. Such disparate communities were chosen because it was theorized that, given the traditional fishermen strategy of seeking jobs outside the fishery when fishing is bad or the season is closed [8], those communities closest to urban centers with diverse economies would be less dependent on fisheries than those further from urban centers due to the availability of nonfishing jobs close by. Because nine communities would not provide sufficient variation in the data for reliable index development, it was decided to include all communities in the county and adjacent counties. This resulted in a data set with 122 different communities and provided sufficient variation for index development. The results of the community rankings on each variable are reported, however, only for the nine communities as identified. 2.2. Primary qualitative data An inductive grounded approach was used to independently test the characterizations of the selected communities provided by the large, secondary data sets, especially as they relate to the importance of the commercial and recreational fishing sectors. This assessment consisted of two major components: (1) ethnographic field research within the nine communities, involving interviews with fishers, individuals whose businesses were related to fishing, and community officials and leaders about the place and importance of fishing within the communities, and (2) compilation of historical and contextual background information to assess socioeconomic dependence of the community on their fisheries. Both components produced data that allowed for relative ranking of the nine communities. Interview data were based on culturally significant categories provided by the interviewees themselves. Rankings were also constructed from the historical/contextual information. The rankings from the two types of data (interview and contextual) were highly similar. 2.3. Social indicator indices development strategy Three steps were taken to develop the social indices of fisheries dependence. First, correlation coefficients were examined to find underlying patterns of variation. Second, the variables that were most highly inter-correlated and reflected the range of ideas of interest were placed in a principal components analysis, where these variables were determined to be reliable indices. Last, the variables were standardized and weighted for their
effects in the model. Index factor scores were used. Factor scores are similar to composite scores, with the exception that the items are standardized and weighted in regard to their factor loadings. The factor loadings are a rough indication of correlation of the domain concept’s latent structure to the single variable. Therefore, items that are most important in an index receive a higher weighting than a less important item. In principal components, factor loadings less than 0.350 are generally not considered to be significant and in most cases should be removed from a factor scale. One advantage of factor scaling is that negative relationships do not have to be reverse-coded before scaling. This means that negative factor loadings work to reduce the overall score and the absolute number conveys the strength of relationship regardless of being negative or positive. The interpretation of a negative factor loading is similar to a negative Pearson’s r bivariate correlation. The factor scores were standardized with a mean of zero and the scores reflecting standard deviations from that mean. Scales were subsequently tested for internal consistency by using Armor’s [9] theta reliability for factor scales. The theta coefficient is interpreted similar to Cronbach’s alpha and is used for factor scales because it does not assume that all items are weighted equally in the scale. Theta is calculated as y ¼[p/(p 1)] [1 (1/l)], where p is the number of items in the scale and l denotes the largest eigenvalue from the principal components analysis. 2.4. Social indicator indices components and internal reliability To establish internal reliability, multiple indicators for each concept are necessary. At a minimum it is necessary to include enough variables to fully cover the range of the concept, while maintaining unidimensionality (only measuring one central concept). In general, multiple measures are preferred and do increase internal validity when the items are significantly intercorrelated. However as more variables are added to the index it is harder to maintain unidimensionality. Unidimensionality in part is established by principal components analysis. In a principal components analysis a single factor solution provides evidence that the various index items only measure a single concept. The indices in this study range from a low of four items to a high of seven items. Indices with three or fewer items are generally thought to be insufficient to establish internal validity through Cronbach’s alpha or Armor’s theta. Here you will find a description of the components of each index, the principal components analysis and factor loadings, and measures of internal validity including the eigenvalue, percentage explained variation, and Armor’s theta reliability.
3. Results 3.1. Commercial fishing dependence index The commercial fishing dependence index (Table 1) consists of five variables: (1) percentage of labor force employed in agriculture, fishing, and hunting in 2000 (range 1.24–20%), (2) pounds of landings per 1000 population in 2007 (range 7727–2,428,921 pounds per 1000 population), (3) commercial fishing permits per 1000 population in 2007 (range 0.01–15.38 per 1000 population), (4) value of landings per 1000 population (range $11,222–4,845,564), and (5) number of permitted dealers with landings per 1000 persons (range 0.11–4.86). The principal components analysis produced a single factor solution with an eigenvalue of 3.512 with an explained variation of 70.202%. Armor’s theta reliability coefficient was 0.894 reflecting a
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
1309
Table 1 The commercial dependence index. Community
Percent employed in ag., fishing, and hunting
Pounds of landings per 1000 persons
Commercial fishing permits per 1000 population
Value of landings per 1000 population ($)
Dealers with landings per 1000 persons
Commercial dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
3.61 17.70 20.00 10.05 2.45 5.12 1.53 1.24 2.37
9555 941,393 124,196 2,428,921 243,443 749,085 102,372 16,933 14,632
3.23 1.40 15.38 11.88 0.35 0.20 1.05 0.02 0.01
34,638 1,424,031 350,978 4,845,564 211,624 1,835,321 222,314 42,757 11,222
0.26 2.1 4.86 2.38 0.44 1.82 0.25 0.11 0.13
0.498 0.624 1.575 2.057 0.543 0.149 0.644 0.76 0.716
5 3 2 1 6 4 7 9 8
0.765
0.871
0.77
0.873
PC components Factor loading 0.902 Theta reliability: 0.894 Eigenvalue: 3.512 Percentage explained variation: 70.202
Single factor solution
High ranking ¼ more dependent Low ranking ¼less dependent
Table 2 The recreational dependence index. Community
Charter permits per 1000 population
Marinas and related businesses per 1000 population
Marinas and related businesses jobs per 1000 population
Marinas and related businesses gross earnings per 1000 population ($)
Boat launches per 1000 population
Recreational dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
0.52 0.00 26.72 0.00 0.00 0.81 0.74 0.00 0.00
0.44 0.00 2.43 0.40 1.15 0.81 0.19 0.20 0.00
2.01 0.00 8.10 0.99 3.92 3.64 1.62 0.79 0.00
71,921 0 449,392 59,382 182,879 384,304 131,717 31,520 0
0.44 2.10 4.05 0.20 0.26 0.81 0.21 0.09 0.25
0.355 0.517 2.763 0.516 0.188 0.449 0.401 0.656 0.816
4 7 1 6 3 2 5 8 9
0.966
0.940
0.880
0.816
PC components Factor loading 0.933 Theta reliability: 0.947 Eigenvalue: 4.128 Percentage explained variation: 82.551
Single factor solution
high level of internal consistency. The factor loadings ranged from 0.765 to 0.902. The strongest factor loadings in the analysis were for the variables percentage of labor force employed in agriculture, fishing, and hunting in 2000 (0.902), number of permitted dealers with landings per 1000 persons (0.873), and commercial fishing permits per 1000 population in 2007 (0.871). All the factor loadings were above 0.350 and so were included in the index. Variable 1 was from the 2000 US Decennial Census Summary Tape File 3. Variables 2–5 were from a custom data set generated by NMFS SEFSC landings data and SERO permit data for this research. Variables 2–5 were standardized by taking the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000.
3.2. Recreational fishing dependence index The recreational fishing dependence index (Table 2) consists of five variables: (1) charter permits per 1000 population in 2007
High ranking ¼ more dependent Low ranking ¼ less dependent
(range 0–26.72 per 1000 population), (2) marinas and related businesses per 1000 population (range 0–2.43), (3) marinas and related businesses jobs per 1000 population (0–8.1 per 1000 population), (4) marinas and related businesses gross earnings per 1000 population (range $0–449,392), and (5) boat launches per 1000 population (range 0.09–4.05 per 1000 population). The principal components analysis produced a single factor solution with an eigenvalue of 4.128 and an explained variation of 82.551%. Armor’s theta reliability coefficient for this index is 0.947 reflecting very high levels of internal consistency. The factor loadings for this index range from 0.816 to 0.966. The strongest factor loadings in the analysis were for the variables marinas and related businesses per 1000 population (0.966), marinas and related businesses jobs per 1000 population (0.940), and charter permits per 1000 population in 2007 (0.933). All the factor loadings were above 0.350 and so were included in the index. Variable 1 is from SEFSC landings data. Variables 2–4 are from the 2002 Economic Census available from the ‘‘American FactFinder’’ website. All the variables were standardized by taking
1310
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000. 3.3. Social fishing dependence index The social fishing dependence index (Table 3) consists of five variables: (1) percentage of water cover in the municipal boundary (range 0–77.8%), (2) boat launches per 1000 population (range 0.09–4.05 per 1000 population), (3) percentage of labor force employed in agriculture, fishing, and hunting in 2000 (range 1.24–20%), (4) marinas and related businesses per 1000 population (range 0–2.43), and (5) number of permitted dealers with landings per 1000 persons (range 0.11–4.86). The principal components analysis produced a single factor solution with an eigenvalue of 3.438 and an explained variation of 68.780%. Armor’s theta reliability coefficient was 0.886 for this index reflecting high levels of internal consistency. The factor loadings for this index ranged from 0.355 to 0.982. The strongest factor loadings were for the variables number of permitted dealers with landings per 1000 persons (0.982), boat launches per 1000 population (0.942), and percentage of labor force employed in agriculture, fishing, and hunting in 2000 (0.938). All the factor loadings were above 0.350 and so were included in the index. Variables 1 and 3 are from the 2000 Decennial Census Summary Tape File 3. Variables 2, 4, and 5 are from a custom NOAA NMFS database. Variable 1 was derived from Census data by dividing the total land area in the municipality into the percentage water cover. Variables 2, 4, and 5 were standardized by taking the absolute occurrence for each variable in a community and dividing by the total population and multiplying the result by 1000.
over 2 million, and the metropolitan population is approximately 4.5 million. The southeastern city limit is only a few miles from the shoreline. The communities around Galveston Bay are no longer geographically and socially distinct entities, even if they are politically separate. To illustrate how dense the communities are, one encounters six different municipalities along an approximately 10 mile strip of road without seeing any distinct changes between communities. In addition to being one of the largest metropolitan areas in the United States, the area also is heavily industrialized. Large petrochemical complexes and oil fields are situated on and to the northeast of the bay. The southern portion of the research area consists of the communities on the shores of a series of connected bays. The landscape surrounding these bays differs significantly from that of the Galveston Bay Complex. The largest metropolitan entity is Victoria, a city of approximately 60,000 populations, situated about an hour to the northwest. Other towns in the region have populations of less than 12,000, most below 3000 and are geographically bounded by producing farms and ranches. Although the communities in this region are relatively small towns, they differ considerably in socioeconomic characteristics and in terms of their historic and current dependence on fisheries. The ethnographic analysis began with observations based upon open-ended interviews with key informants. Interviewees were asked a series of questions related to historical and current commercial and recreational fishing in their communities. The existence and condition of fishing infrastructure, such as commercial and recreational docks, processing plants, ice houses, bait camps, net shops, tackle shops, etc., were noted. Infrastructure (e.g. statues and fishing themed buildings), events (e.g. fishing festivals or tournaments), and the presence of fishing organizations were also noted. These observations were turned into summaries and then generalizations that were collapsed into ratings of high-, medium-, and low-dependence (see Table 4).
3.4. Ethnographic field research 4. Discussion: differing processes with a converging reality Fieldwork was conducted during the summer of 2008. As previously mentioned, the bay systems chosen are differentially impacted by urbanization and population density differs dramatically. The Galveston Bay Complex is situated southeast of Houston, the 4th largest city in the nation with a population of
4.1. Inter-rater agreement A key concept in the measurement of constructs, such as fishing dependence, is the interchangeability of indicators. Any
Table 3 The social fishing dependence index. Community
Percent water cover
Boat launches per 1000 population
Percent employed in ag., fishing, and hunting
Marinas and related businesses per 1000 population
Dealers with landings per 1000 persons
Social dependence index
Ranking
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
28.70 0.00 25.00 3.70 73.40 5.80 77.80 62.80 0.00
0.44 2.10 4.05 0.20 0.26 0.81 0.21 0.09 0.25
3.61 17.70 20.00 10.05 2.45 5.12 1.53 1.24 2.37
0.44 0.00 2.43 0.40 1.15 0.81 0.19 0.20 0.00
0.26 2.1 4.86 2.38 0.44 1.82 0.25 0.11 0.13
0.420 0.695 2.623 0.381 0.374 0.197 0.805 0.801 0.586
6 2 1 3 5 4 9 8 7
0.778
0.982
PC components Factor loading 0.355 Theta reliability: 0.886 Eigenvalue: 3.438 Percentage explained variation: 68.780
0.942
Single factor solution
0.938
High ranking ¼more reliant Low ranking ¼less reliant
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
1311
Table 4 Economic and social dependence on commercial and recreational fishing by community. Community
Rating and comments
Bacliff
Commercial dependence: Low. There are no commercial facilities in Bacliff. Recreational dependence: Low. A few private docks are attached to houses for recreational fishing or boating. Social dependence: Low. There are no significant fishing social activities.
Galveston
Commercial dependence: Low. Galveston has a public dock for a commercial fleet of finfish and shrimp boats. There are also wholesale buyers and shippers. Commercial fishing is neither a major employer nor is it a major source of revenue. Recreational dependence: Medium. Before Ike, there were numerous bait camps, two long public fishing piers, beaches open to wade-fishing, charter, and head boats for hire. The city is dependent on nature based tourism, and recreational fishing is part of that tourism but is not highly promoted. Social dependence: Low. Galveston tourism is based on two major strands: its Victorian past and cultivation of an ‘island feel’. The Victorian past model promotes architectural reconstruction and festivals such as ‘Dickens on the Strand’. The visual manifestation of ‘beach culture’ has been promoted, however, by planting palm trees near the beach and by selling beach type merchandise. There are numerous seafood restaurants that use a fishing motif, but the city as a whole does not celebrate fishing culture in festivals or visually.
San Leon
Commercial dependence: High. Pre-Ike San Leon was home to two oyster processing plants, two large oyster leaseholders, the Vietnamese crab fishery and processing plant, and two Vietnamese shrimp docks with marinas. Recreational dependence: High. The community also has a large recreational fishing center where both recreational and shrimp boats dock. Social dependence: High. The town’s slogan is ‘‘A small drinking community with a large fishing problem.’’ A local landmark restaurant, written up in out of state tourist articles, is a favorite gathering place for locals and tourists.
Texas City
Commercial dependence: Low. There is one wholesale shrimp and finfish dealer in Texas City and an oyster processor. They existed there only because the city annexed the unincorporated area in which they were located. Previously, this area was part of San Leon and socially that is still the case. Several commercial fishermen interviewed lived in Texas City but docked on Port Bolivar. Recreational dependence: Low. Pre-Ike, the Texas City Dike was home to five bait camps and boat ramps. It also had some slips for shrimp boats. The dike was an important recreational venue for family fishing. There was one abandoned bait camp. There were also boat ramps on Dollar Bay. Despite Texas City being a destination for recreational fishing, fishing did not appear to have a significant economic impact on the community as a whole due to the concentration of the petrochemical industry. Social dependence: Low. Texas City is known as an industrial town, not a fishing town. The Texas City Dike, however, was once a symbol of the town and of family oriented recreational fishing. After the destruction of the bait shops on the Dike after Ike, there were many reminiscences about days spent fishing from the Dike on the local newspaper’s website.
Seadrift
Commercial dependence: High. The Vietnamese crabbing community is located on Seadrift as well as a shrimp fleet, but they have decreased substantially in recent years. There is a commercial docking facility with two buyers located. Recreational dependence: Low. Recreational fishing on Seadrift is relatively low, due to essentially no infrastructure for tourism. Social dependence: Medium. At one time, social dependence was high. Historically there was a large Vietnamese crabbing community, but this is now in decline. The community also hosts a Shrimpfest, and houses are decorated with fishing paraphernalia. Except for the remaining Vietnamese, the sense of community appears to be in decline.
Palacios
Commercial dependence: High. Palacios is home to a large gulf shrimp fleet, The Marine Education Center, and TPWD Research Center. Commercial fishing has been the major economic engine for the community. Recreational dependence: Low/medium. The community hosts fishing tournaments and fishing related festivals, and there is potential for growth of recreational fishing. The relatively undeveloped infrastructure for tourism, however, limits the importance of recreational fishing. Social dependence: High. Palacios hosts the ‘‘Shrimp o Ree,’’ Texas Seafood, and Blessing of the Fleet festivals. The shrimp fleet is networked through a few major families who own fleets and provide loans and other support to local fishers. The Vietnamese community is networked through a local church.
Port O’Connor
Commercial dependence: Low. Port O’Connor had a bay and bait fishery, but they are in rapid decline. Recreational dependence: High. The community is highly dependent on the sport fishery. It hosts several large fishing tournaments. One of the founders of CCA has his summer home there. There are a number of bait camps, boat sheds, and ramps. New resort style residential compounds are being built that include boat ramps and slips. Social dependence: High. Recreational fishing is promoted in various ways throughout the community, specifically fishing tournaments and merchandise in shops (for sale even in the ice cream shop). There is also a Fisherman’s Chapel. The major hotel/restaurant complex is now being remodeled to attract tourists for recreational fishing.
reasonable indicator of a construct should be correlated to other indicators of that same construct. In other words if they are both measuring the same thing they should also strongly relate to one another in the same directional pattern. The degree that two differing indicators of the same construct relate to reality is usually referred to as construct validity. To establish construct validity, ideally the quantitative data in our project should be highly correlated with the qualitative data, which best reflects the objective conditions in the community. Inter-rater reliability is the degree to which independent observers evaluate the characteristics of a subject and reach the same conclusion [10]. High level of agreement in ratings generally reflects the reliability of the standards and process. This is true especially if two different raters are applying the same criteria and reaching the same results. However in the case described here, there are two completely different sets of criteria and
processes and a high level of agreement reflects convergence of a construct with reality. In other words, rather than being a reflection of reliability (receiving the same results from repeated measures using the same criteria) it is a reflection of both construct and external validities (the link between a construct and observed reality). A more accurate term to describe our results, therefore, would be in terms of inter-rater agreement [10] instead of inter-rater reliability. There are some widely reported measures used to assess interrater agreement. They are percentage agreement, correlations based indicators such as Pearson’s r or Spearman’s rho, and Cohen’s kappa. Each of these measures has some significant advantages and drawbacks but taken together they allow for a more complete assessment of inter-rater agreement. Percentage agreement is easily understood and has a straightforward interpretation but can be misleading. The percentage agreement
1312
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
is often inflated because a portion of agreement could be directly due to random matching. This is especially true with constructs with relatively few categories. For example, with three categories up to 11.1% (0.333 0.333 or 1 in 3 1 in 3) of agreement could be due to random matching. Correlational techniques like Pearson’s r or Spearman’s rho measure covariation but not to the extent in which there is identical agreement in the categories. The above correlations are generally interpreted in analysis to be substantial above 0.6. The last statistic described is Cohen’s kappa that measures inter-rater agreement and ranges from 1.0 to 1.0. Cohen’s kappa counts only exact matches, then adjusts for random matching, and as such, it is a very conservative measure of agreement. The closer the number is to 1.0, the greater the agreement is above and beyond random matching. If the number is approaching zero, then the level of agreement is close to what would be expected by chance. If the number is below zero and approaching 1, then the agreement is less than what would be expected by just random matching. Cohen’s kappa is calculated by taking the percentage of agreement [Pr(a)] and subtracting the probability of random agreement [Pr(b)], divided by one minus the probability of random agreement [Pr(a) Pr(e)/(1 Pr(e))]. The probability of random agreement is calculated by dividing 1 by the number of categories for rater one and multiplying it by 1 divided by the number of categories [Pr(e) ¼1/k 1/k, where k is the number of categories for the rater]. Cohen’s kappa is generally interpreted with the following framework from Landis and Koch [11]: less than zero ¼no agreement; 0–0.20¼slight agreement; 0.21–0.40¼ fair agreement; 0.41–0.60 ¼moderate agreement; 0.61–0.80 ¼ substantial agreement; and 0.81–1.0¼almost perfect agreement. A t value can be calculated for kappa by dividing the kappa value by the asymptotic standard error when the null hypothesis is true (the true value is 0). This t value has an associated statistical probability that is often reported. While both percentages and correlational techniques tend to be liberal and over-assess levels of agreement, Cohen’s kappa is considered a very conservative measure and underestimates the strength of agreement. This is because it only includes exact matches as agreement, when often misses are only a category off and the raters are actually in relative agreement. It is possible to use a weighted kappa statistic to account for close misses but it is not commonly done and the statistic is not included in any major software packages. To ensure content validity with the construct of dependence, multiple indicators were developed: (1) commercial, (2) recreational,
and (3) social. To evaluate the agreement of the social indicators with the ethnographic research it was necessary to code the indices into the same categories employed in the qualitative analysis. These categories were: (1) low, (2) medium, and (3) high. Each separate community (N ¼122) was coded into one of the three (low, medium, or high) based on the index factor score, so the response categories within the nine communities are not evenly distributed (e.g. 3 lows, 3 mediums, and 3 highs). For the dependence indices of commercial, recreational, and social, direct comparisons of the ratings for the social indicators and the ethnographic data could be made. This is because in the ethnographic coding process the domains of commercial, recreational, and social dependence emerged from the content analysis providing a check of content validity.
4.2. Inter-rater agreement results Table 5 presents the results of the inter-rater agreement analysis for commercial, recreational, and social fishing dependence. The quantitative and ethnographic assessments matched in 66.67% of the communities for the commercial fishing dependence measure. This produced Spearman’s rho of 0.589; however, it was not statistically significant due to the small N size (9 cases). Cohen’s kappa was 0.500 and was statistically significant, reflecting a moderate level of agreement between the two techniques. In two of the three mismatches the categories were only off by one category. However, in Port O’Connor the mismatch was off by two full categories. The quantitative results indicated high commercial dependence based on the 2007 value and pounds of landings and numbers of commercial licenses and dealers. The ethnographic research was conducted in 2009. According to interviewees, the conditions of the commercial fishing industry in Port O’Connor changed greatly in the two-year delay in data reporting. It is also possible that the ethnographic results were underestimated due to selection bias in key informants. In either case in this community there was a large difference in ratings. Continuing with Table 5, but examining the results for recreational dependence, complete agreement was found between the quantitative and qualitative results. They matched on all nine communities producing 100% agreement, Spearman’s rho of 1.0, and Cohen’s kappa of 1.0. In the case of recreational dependence the techniques were in perfect agreement. Also in Table 5 the inter-rater agreement results for social dependence matched on seven of nine communities (77.78%).
Table 5 Commercial, recreational, and social dependence. Community
Port Lavaca Seadrift Port O’Connor Palacios Seabrook San Leon Galveston Texas City Bacliff
Commercial dependence
Recreational dependence
Quantitative index
Ethnographic assessment
Differing classification
Quantitative index
Ethnographic assessment
Medium High High High Medium Medium Low Low Low
Low High Low High Medium High Low Low Low
n
Medium Low High Medium Medium High Medium Low Low
Medium Low High Medium Medium High Medium Low Low
Matched on 6 of 9 communities 66.67% matched Spearman’s rho¼ 0.589, P¼ 0.095 Kappa ¼0.500, P ¼0.022
n
n
Matched on 9 of 9 communities 100% matched Spearman’s rho¼1.0, P ¼ 0.000 Kappa ¼1.00, P ¼0.000
Social dependence Differing classification
Quantitative index
Ethnograpic assessment
Medium High High High Medium High Low Low Low
Medium Medium High High Low High Low Low Low
Differing classification
n
n
Matched on 7 of 9 communities 77.78% matched Spearman’s rho¼0.900, P ¼0.000 Kappa¼ 0.660, P ¼0.005
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
Spearman’s rho for the two techniques was 0.900 and this was statistically significant. Cohen’s kappa was 0.660 and was statistically significant and reflects substantial agreement between the raters. In the two mismatched communities the categories were off by only one category, with the quantitative ratings higher than qualitative.
5. Conclusions This project, specifically the evaluation of the agreement between ethnographic research and social indicators developed from secondary data illustrates some advantages of the latter. The advantages relate to time and cost savings, community definition, reliability, and comparability of results across communities. The disadvantages relate to the loss of local context and external validity. Assembling a database, creating indices, and profiling communities through secondary data are techniques that are much faster than conducting traditional community profiling through ethnography. In this study, four full-time fieldworkers collected data in nine community case study sites for a little over 2 months. In addition, many weeks were spent coding and analyzing the data. Cost and time are especially relevant given the budgetary constraints of management agencies. It is not always easy to differentiate among communities on the ground. In the Galveston Bay region, for example, it was very difficult for the field research team to tell where community boundaries began and ended. It was also difficult untangling the interviewees’ references to various places as most referred to the general area where they traveled to meet their daily needs, not to specific municipal boundaries. The secondary data were compiled at the minor civil division level and were obviously discrete and pertained only to that place. Using minor civil division-level data would help solve some of the lingering issues surrounding the definition of community in fisheries research in a simple and reasonable fashion. The disadvantage of such an approach is that communities, defined as a place where one can meet daily needs, are often functionally larger than a minor civil division and often revolve around the human ecological concept of central place [4]. Quantitative indicators based on secondary data allowed for a reliable process of measurement and comparability across the nine study communities because the concept being measured (i.e. dependence) was defined identically across all of the study sites using a deductive process. Conversely, the ethnographic process was emergent and to a large extent dependent on the perceptions of interviewees. Additionally, when using different interviewers, there is always the chance that some areas of the interview will be stressed over others. Because the results of ethnographic research are descriptive, it is very difficult to meaningfully compare different communities in terms of intensity. Secondary quantitative data offer ease of analysis and interpretation along with reliable repeatable indicators and can be incorporated into other forms of social modeling that are not possible with ethnography. The disadvantage of relying on such data is the loss of the local context and external validity. Ethnography is advantageous when looking to define constructs from the perspective of the interviewees in an inductive grounded process but is not as effective in a deductive normative process. Though the construction of social indicators using secondary quantitative data offers some significant advantages over ethnography, it would mean nothing if the results were not representative of community conditions. However, in this research very high levels of agreement between the social indicators approach and ethnography were found. Ultimately
1313
the ratings varied little by research technique. Given this circumstance, the advantages of social indicators based on secondary data far outweighed the disadvantages.
5.1. Robustness of the social indicators There were several concerns with the reliance on secondary data to construct indicators that were proven to be unfounded. In fact, the indicators were shown to be extremely robust in regard to indirect measures, time obsolescence, estimated data, and varying data sources. Reliance on indirect indicators requires ‘‘ground-truthing’’ to ensure external validity and they must be linked to actual conditions in the community. The indirect measures used in this study related very well to community conditions that were identified in ethnography. Obsolescence of the data can also be an issue of concern. Given that the last census was nearly a full decade ago, the logical question was: could data that was 10 years old still reflect actual conditions in the community? In addition, the most current NOAA NMFS data available also lagged by several years. However, this lag seemed to make very little difference in the assessment given the high levels of agreement between the techniques. This is at least partially due to the fact that the process ranked communities arraying them among the 122 places. This ranking is relative to the other places and is likely to be very stable across time because most change is incremental. As such, it is not that impacted by data obsolescence. The exception would be boomtown or bust scenarios. This may have been captured in this research in Port O’Connor in regard to the commercial fishing dependency. So with the caveat of rapidly changing situations as seen in Port O’Connor, relying on interim census estimates provided by the Bureau of the Census did not seem to substantially impact the external validity of the social indicators. Another concern was creating indices that were compiled of variables from different data sources. In all cases the data sources were examined for reliability in data collection and reporting to remove those concerns. Any issues with differing units of measure were resolved by the use of principal components analysis to produce a factor score, which is standardized with a mean of zero and a standard deviation of 1. Last, the problem of concatenating the differing data sets is actually simple as the data can be matched by the geographic identifier (FIPS Code) that has become standardized mostly because of GIS analyses.
5.2. Recommendations for the next steps Though social indicators analysis can greatly enhance and streamline community profiling and social impact assessment in fisheries management, ethnography remains an important component in assuring the external validity of the social indicators. In addition, in-depth ethnography should be conducted when specific communities will be extraordinarily impacted by changes in fisheries regulations because the social indicators based on secondary data may be too insensitive to analyze rapidly changing or soon to be changing situations. In this circumstance the social indicators can be used for identifying the places that require indepth ethnographic research. In addition communities should be randomly sampled for in-depth ethnography so evaluations of inter-rater agreement can be made. This is a critical check for external validity. Yet these indicators can provide valuable information that provides context to fishery management regulation, and when built over time, may be able to describe important trends for impact assessment.
1314
S. Jacob et al. / Marine Policy 34 (2010) 1307–1314
Acknowledgements Funding for the research upon which the article is based was provided by a Saltonstall-Kennedy NOAA Grant Number: NA08NMF4270412 granted to the Gulf & South Atlantic Fisheries Foundation Inc. Opinions and conclusions expressed or implied are solely those of the authors and do not necessarily reflect the views or policy of the National Marine Fisheries Service, NOAA. The authors would like to recognize the contributions of the fieldwork team: Lovette Miller, Elizabeth Croucher, Meredith Marchioni and Jerry Lord. They especially thank members of the coastal communities interviewed.
References [1] Smith S, Jepson M. Big fish, little fish: politics and power in the regulation of Florida’s marine resources. Social Problems 1993;40:39–49. [2] Smith SD, Jacob S, Jepson M. The stress process in Florida’s commercial fishing families. Society and Natural Resources 2003;16:39–59.
[3] Clay PM, Olson J. Defining ‘‘Fishing Communities’’: vulnerability and the Magnuson–Stevens Fishery Conservation and Management Act. Human Ecology Review 2008;15:143–60. [4] Tuler S, Agyeman J, Pinto de Silva P, LoRusso KR, Kay R. Assessing vulnerabilities: integrating information about driving forces that affect risks and resilience in fishing communities. Human Ecology 2008;15:171–84. [5] Magnuson–Stevens Fishery Conservation and Management Act (1996). U.S. Department of Commerce. NOAA Technical Memorandum NMFSF/SPO-23, December 1996. [6] Jacob S, Farmer FL, Jepson M, Adams C. Landing a definition of fishing dependent communities: potential social science contributions to meeting National Standard 8. Fisheries 2001;26:16–22. [7] Jacob S, Jepson M, Pomeroy C, Mulkey D, Adams C, Smith S. Identifying fishing dependent communities: development and confirmation of a protocol. Marine Fisheries [MARFIN] Project Report NA87FF0433. St. Petersburg: NOAA NMFS; 2002. [8] Davis DL. Occupational community and fishermen’s wives in a Newfoundland fishing village. Anthropological Quarterly 1986;59:129–42. URL: /http:// www.jstor.org/stable/3317199S (accessed 12/05/2010, 18:23). [9] Armor DJ. Theta reliability and factor scaling. In: Costner Herbert L, editor. Sociological methodology. San Francisco: Josey-Bass; 1974. [10] Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Human Communication Research 2002;28:587–604. [11] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.