Environmental sustainability measurement in the Travel & Tourism Competitiveness Index: An empirical analysis of its reliability

Ecological Indicators 73 (2017) 589–596

Contents lists available at ScienceDirect

Ecological Indicators journal homepage: www.elsevier.com/locate/ecolind

Environmental sustainability measurement in the Travel & Tourism Competitiveness Index: An empirical analysis of its reliability José G. Dias ∗,1 Instituto Universitário de Lisboa (ISCTE-IUL), Business Research Unit (UNIDE-IUL), Lisboa, Portugal

a r t i c l e

i n f o

Article history: Received 2 February 2016 Received in revised form 5 October 2016 Accepted 9 October 2016 Keywords: Environmental sustainability Reliability Indices Travel & tourism industry Exploratory and confirmatory factor analyses

a b s t r a c t Indices provide a straightforward summary of the status of an object or concept. Examples of concepts are diverse and go from city quality of life, country level of freedom, human development to environment sustainability. This paper introduces a methodology to assess the reliability of the environmental sustainability index implemented by the Travel & Tourism Competitiveness Index that is published by the World Economic Forum using exploratory and confirmatory factor analyses. Results show that the original index is not reliable as most of the variables are weakly correlated. A simplified version of the original index is obtained by exploratory factor analysis and tested by confirmatory factor analysis. Measures of reliability show that the new index called TTESI – Travel & Tourism Environmental Sustainability Index – is reliable. Results also show that combining data from different sources (e.g., survey data and physical measurements) proves problematic. A z-score value for each country was computed and countries were ranked based on the TTESI. Additionally, the new index is more in line with the HDI – Human Development Index – and can therefore be integrated more easily in an overall index of sustainable development. © 2016 Elsevier Ltd. All rights reserved.

1. Introduction Environmental sustainability (ES) is unquestionably an important concept in the policy-making debate nowadays and one that is under public scrutiny. Its importance is due to the fact that it tends to define trade-offs with the social and economic components of sustainable development (Goodland, 1995). For instance, stakeholders are taking increasing interest in the environmental performance of firms before making investment decisions. Recently, Volkswagen’s market value plunged 50% following the announcement of the automaker’s violations of the Clean Air Act (CNN, 2015). In the aftermath of the scandal, Volkswagen was removed from the Dow Jones Sustainability World Index (S&P Dow Jones Indices and ROBECOSAM, 2015). While the concept of sustainability has been broadly used, little research deals with its assessment and measurement. Klemeˇs (2015) notes only 0.1% of the 96,290 publications on Scopus that include the word “sustainability” also contain “measurement”. Many one-dimensional indicators of sustainable development

∗ Correspondence to: Department of Quantitative Methods for Management and Economics, Av. Forc¸as Armadas, Edifício ISCTE, 1649-026 Lisboa, Portugal. E-mail address: [email protected] 1 I would like to thank both reviewers for their insightful comments, which allowed me to make significant improvements to the paper. http://dx.doi.org/10.1016/j.ecolind.2016.10.008 1470-160X/© 2016 Elsevier Ltd. All rights reserved.

covering the three dimensions of economic, environmental, and social conditions are available in the literature. In this line of thought, it has been argued that the Human Development Index (HDI) is an incomplete measure of development as its use of income, life expectancy, and educational data takes only two development components into account, namely social and economic dimensions. There have been a number of suggestions in the literature on how to make HDI a greener indicator (e.g., Dahme et al., 1998; Morse, 2003; Togtokh, 2011; Iˇsljamovic´ et al., 2015). Togtokh (2011) introduced the Human Sustainable Development Index (HSDI) that adds a fourth variable into its computation: per capita carbon emissions. Other more specific examples on environmental sustainability indices are: the Living Planet Index (LPI) that measures biodiversity (WWF, 1998), the Ecological Footprint (EF) index developed by Wackernagel and Rees (1996), and the Environmental Sustainability Index (ESI) of Samuel-Johnson and Esty (2000) and Esty et al. (2005). For instance, the Ecological Footprint tracks past and current human pressure on the biosphere’s capacity to provide a life-supporting ecosystem and can be used to track the spatial impact of the production and consumption of products and services. Recently, Asici and Acarb (2016) show that as countries grow richer they tend to export the ecological cost of their consumption to poorer economies. However, its conceptual and operational definitions are still topics of active research (e.g. Mancini et al., 2016). Böhringer and Jochem (2007) provide a

590

J.G. Dias / Ecological Indicators 73 (2017) 589–596

detailed survey of these and other less well-known environmental sustainability indices, covering the steps of conceptualization, normalization, weighting, and aggregation. In a recent paper, Babcicky (2013) shows that the ESI’s unidimensional structure2 is problematic and the weight of components is far from perfect. Similarly, Siche et al. (2008) analyze the ESI, the EF, and two emergy performance indices; they conclude that they are weakly correlated and present distinct perspectives on environmental sustainability. Böhringer and Jochem (2007) also conclude that the use of 11 indices to measure environmental sustainability tends to be misleading and inconsistent as a result of the aggregation and normalization used in their computation, and it is therefore useless in terms of policy advice. Bravo (2014) compares the HSDI and alternative indicators of environmental sustainability (e.g., Ecological Footprint) and concludes that HSDI makes only a small advance in the definition and measurement of sustainable development as the correlations between HSDI and these indicators tend to be weak. Given the proliferation of distinct measures of environmental sustainability and the weak correlation between them, Hizsnyik and Toth (2010) recognize that it might be difficult to define a single measurement of sustainable development. The travel & tourism industry is an important source of environmental stress (e.g., airlines, hoteliers, car rental companies) and so it needs access to reliable indicators. This study empirically assesses the psychometric properties of the Environmental Sustainability Index under the pillar T&T Policy and Enabling Conditions reported by The Travel & Tourism Competitiveness Reports 2015 (Crotti and Misrahi, 2015). This research introduces a methodology that is commonly applied in the development of measurement scales in social sciences to compute the reliability of unobserved constructs. In order to assess the reliability of the environmental sustainability indices, I assume that the indicators are a manifestation of the underlying level of environmental sustainability which is measured by these indicators (Churchill, 1979; Peter, 1979). Not only have few studies employed principal component analysis (e.g., Bolcárová and Koloˇsta (2015) in the context of sustainable development), but most of these did not apply confirmatory factor analysis to test the reliability of the scales.3 The main hypothesis underlying this research is that the combination of distinct sustainability indicators – physical vs. attitudinal data – may create problems for the reliability of aggregate indicators. I take a confirmatory factorial approach (CFA) to assess the psychometric properties of a multi-item scale of measurement (Jöreskog, 1971; Jöreskog and Sörbom, 1982; Gerbing and Anderson, 1988; Nunnally and Bernstein, 1994; Kline, 2016). Additionally, I recommend further changes to the index as a consequence of these results. In summary, this research conducts an analysis of the reliability of the ESI derived from the data in The Travel & Tourism Competitiveness Report 2015 and explores further improvements to the index. The paper is structured as follows. Section 2 introduces the data used in this study, which comes from The Travel & Tourism Competitiveness Report 2015. Section 3 describes the methodology for assessing the reliability of Environmental Sustainability indices. Section 4 presents the results, addressing the reliability of the original Environmental Sustainability Index and proposing a modified index that I call the Travel & Tourism Environmental Sustainability

2 Contrary to multidimensional structures, unidimensional structures assume that all observed variables converge and are clustered into a single composite figure. This is explained by the fact that the indicators are correlated and measure a unique concept or construct, e.g., environmental sustainability. 3 For instance, Böhringer and Jochem (2007) provide a summary in Table 3 in which two of 11 sustainability indices apply principal component analysis (PCA) and none applies confirmatory factor analysis.

Index (TTESI). Section 5 concludes the paper with further discussion of potential extensions and applications of this method.

2. Data Data comes from The Travel & Tourism Competitiveness Report 2015, a more detailed description of which can be found elsewhere (Crotti and Misrahi, 2015). The 10 indicators used in the Environmental Sustainability pillar are: (1) Stringency of environmental regulations (2013–2014 weighted average of the responses from the Executive Opinion Survey of the World Economic Forum to the question How would you assess the stringency of your country’s environmental regulations? using the scale of measurement: 1 = Very lax; 7 = Among the world’s most stringent); (2) Enforcement of environmental regulations (2013–2014 weighted average of the responses from the Executive Opinion Survey of the World Economic Forum to the question How would you assess the enforcement of environmental regulations in your country?, using the scale of measurement: 1 = Very lax; 7 = Among the world’s most rigorous); (3) Sustainability of travel and tourism industry development (2013–2014 weighted average of the responses from the Executive Opinion Survey of the World Economic Forum to the question How would you assess the effectiveness of your government’s efforts to ensure that the Travel & Tourism sector is being developed in a sustainable way?, using the scale of measurement: 1 = Very ineffective – development of the sector does not take into account issues related to environmental protection and sustainable development; 7 = Very effective – issues related to environmental protection and sustainable development are at the core of the government’s strategy); (4) Particulate matter (2.5) concentration (Populationweighted exposure to PM[2.5 (micro-grams per cubic meter)], 2012 Yale University and Columbia University, Environmental Performance Index (EPI) 2012 edition based on NASA MODIS and MISR data); (5) Environmental treaty ratification (Total number of ratified environmental treaties, 2014 The International Union for Conservation of Nature (IUCN), Environmental Law Center ELIS Treaty Database). This indicator measures the total number of international treaties from a set of 27 in which a state is a participant); (6) Baseline water stress (Normalized (0–5) ratio of total annual water withdrawals (municipal, industrial and agricultural) to total available annual renewable supply, 2010 World Resources Institute, Aqueduct Country and River Basin Rankings); (7) Threatened species (Threatened species as a percentage of total species (mammals, birds and amphibians); 2014 The International Union for Conservation of Nature (IUCN), Red List of Threatened Species); (8) Forest cover change (Forest cover change between 2000 and 2012, 2012 Yale Center for Environmental Law & Policy (YCELP) and the Center for International Earth Science Information Network (CIESIN) at Columbia University, Environmental Performance Index 2014); (9) Wastewater treatment (Percentage of wastewater that receives treatment weighted by connection to wastewater treatment rate; 2012 Yale Center for Environmental Law & Policy (YCELP) and the Center for International Earth Science Information Network (CIESIN) at Columbia University); (10) Costal shelf fishing pressure (Trawling catch per exclusive economic zone (EEZ) (tonnes per square kilometer) 2006 Yale Center for Environmental Law & Policy (YCELP) and the Center for International Earth Science Information Network (CIESIN) at Columbia University). The TTCI data set covers 141 countries. There is data for all countries for indicators 1, 2, 3, 4, and 7. For the remaining indicators, missing data is generated by two mechanisms: (a) data is not available due to measurement difficulties; (b) the indicator lacks meaning given specific contexts. For instance, the four missing observations for the Environmental treaty ratification indicators

X1

J.G. Dias / Ecological Indicators 73 (2017) 589–596

0.646

−0.083

0.596

0.040

−0.094

0.138

0.743

0.314

0.663

−0.066

0.566

0.004

−0.103

0.116

0.744

0.304

−0.162

0.277

0.129

0.190

0.100

0.376

0.206

0.051

0.163

−0.026

0.049

0.002

0.093

−0.018

−0.238

0.168

0.523

0.184

0.183

0.147

0.196

0.056

−0.063

−0.193

−0.129

0.154

0.094

X9

X8

X7

X6

X5

X4

X3

X2

0.979

591

X10

0.353

0

5

X1

10 0

5

X2

10 0

5

X3

10 0

5010

X4

20

30 0

X5

5 0

X6

20

X7

40 −50

0

X8

50 0

50

X9

100 −20

0

20

X10

Fig. 1. Indicators used in the Environmental Sustainability Index (ESI).

are: Hong Kong, Lao, Puerto Rico, and Taiwan. Given the jurisdictional nature of three of these regions, the data is only truly missing for Lao. For the Baseline water stress indicator, five missing values are due to the lack of measurement observed: Cape Verde, Hong Kong SAR, Mauritius, Puerto Rico, and Seychelles. The data missing for 28 countries on Forest cover change may result from a mixture of these two mechanisms (e.g., Namibia, Jordan, Kuwait). Data on the Wastewater treatment indicator is missing for two countries: Barbados and Seychelles. Finally, data on Costal shelf fishing pressure is missing for 32 landlocked countries e.g. Austria and Zimbabwe. Fig. 1 depicts these 10 indicators (bivariate correlations, histogram, and scatter plot). The first insight is that these indicators do not seem to be sufficiently correlated to be summarized into a single index. Apart from that, the departure of the distribution from normality is very strong for some indicators. To improve normality and shrink the impact of outliers, indicators 4 (Particulate matter (2.5) concentration) and 7 (Threatened species) were log-transformed. For indicators 9 (Wastewater treatment) and 10 (Costal shelf fishing pressure), the log-transformation did not improve the convergence of their empirical distribution towards normality. For indicators

4 (Particulate matter (2.5) concentration), 6 (Baseline water stress), 7 (Threatened species), and 10 (Costal shelf fishing pressure) higher values mean lower sustainability. Therefore, these indicators were reversed before performing the analyses. 3. Methods This study followed established procedures for measuring constructs in social and exact sciences by analyzing the unidimensionality of the index (set of indicators) as a reliable measure of environmental sustainability. First, an exploratory factor analysis (EFA) was performed retaining one factor (unidimensional) using maximum likelihood estimation. In order to retain an indicator, factor loading higher than 0.4 and also the communality higher than 0.5 for each indicator were examined. Some authors recommend that the factor loading of each indicator should be at least equal to or higher than 0.70 to ensure construct reliability (e.g., Anderson and Gerbing, 1988). Results from the EFA should provide a clear structure of the unidimensional definition of the index. Confirmatory factor analysis (CFA) was performed on the indicator selection

592

J.G. Dias / Ecological Indicators 73 (2017) 589–596

Table 1 Model estimates (TTESI-10 and TTESI). Model 1 – TTESI-10

Stringency of environmental regulations Enforcement of environmental regulations Sustainability of travel and tourism industry development Particulate matter (2.5) concentration (LN, REV) Environmental treaty ratification Baseline water stress (REV) Threatened species (LN, REV) Forest cover change Wastewater treatment Costal shelf fishing pressure (REV)

Model 2 – TTESI

Loading

Communalities

Loading

Communalities

0.992 0.985 0.702 −0.059 0.505 0.048 0.300 0.187 0.720 0.004

0.985 0.969 0.493 0.003 0.255 0.002 0.090 0.035 0.519 0.000

0.979 0.989 0.720 – – – – – 0.639 –

0.958 0.978 0.519 – – – – – 0.409 –

Note: Expressions for the computation of loadings and communalities are given in Appendix A.

from EFA using MPlus 6.0 to further investigate the reliability of the scale identified by EFA. The full information maximum likelihood (FIML) procedure was applied to handle missing data (Enders, 2001). Appendix A provides a technical description of exploratory and confirmatory factor analyses. The following three measurements were used to assess the reliability or internal consistency of the index (Hair et al., 2010): Cronbach’s alpha, composite reliability (CR), and average variance extraction (AVE). The reliability of the underlying dimension was measured by Cronbach’s alpha with a cut-off point of 0.7 (Nunnally, 1978; Hair et al., 2010; Kline, 2016). The CR value should be above the rule of thumb threshold of 0.70 (Fornell and Larcker, 1981). Finally, the AVE value should be above the recommended 0.50 threshold (Bagozzi and Yi, 1988). The definition of these three measures of reliability is set out in Appendix B. Country scores based on the index are obtained from the CFA solution. These scores are standardized (null mean and standard deviation of one) and are called z-scores. Country ranking is based on the z-score. Appendix C provides the mathematical expression and details on the computation of the index and ranking of countries based on the index. 4. Results The ten indicators of the original index (TTESI-10) were used in an exploratory factor analysis (EFA). After running a unifactorial model in SPSS 22.0, the Kaiser-Meyer-Olkin (KMO) index and Bartlett’s test of sphericity were used to ensure that the correlation between data indicators was sufficient to perform an EFA (Kaiser, 1974). A KMO index of 0.668 and a significant Bartlett’s test of sphericity (p < 0.001) suggest there was inherent sufficient correlation between indicators to perform an EFA. Table 1 shows the results for the original model (TTESI-10) given by the ten indicators in the TTCI. The estimation of the model shows that some indicators do not converge into a unidimensional structure (one factor). In particular Particulate matter (2.5) concentration has a negative loading (−0.059). The next step was to explore the unidimensionality of the TTESI-10, but eliminating the indicators that do not agree with the common standards of reliability. First, I eliminated indicators with negative loadings (Particulate matter (2.5) concentration). Indicators Baseline water stress, Threatened species, Forest cover change, and Costal shelf fishing pressure were also excluded as they do not co-vary with the remaining indicators (very low communality). Countries like Singapore perform well in most of the indicators but are at odds in some others such as Costal shelf fishing pressure. Intermediate analyses show that the indicator Environmental treaty ratification presents low communality of 0.255 and it was therefore excluded. Thus, the final selection of indicators is: Stringency of environmental regulations, Enforcement of environmental

Table 2 Reliability measures for TTESI. Cronbach’s alpha (standardized indicators) Composite reliability (CR) Average variance extracted (AVE)

0.888 0.907 0.716

Note: Expressions for the computation of these reliability measures are provided in Appendix B.

regulations, Sustainability of travel and tourism industry development, and Wastewater treatment. Confirmatory factor analysis confirms the structure of four indicators found by exploratory factor analysis. Table 1 (Model 2 – TTESI) gives the loadings and communalities of the final model. I observe a very strong correlation between the TTESI and the first two indicators (Stringency of environmental regulations and Enforcement of environmental regulations), which explains 95.8% and 97.8% (communality), respectively. More than half of the variance of the Sustainability of travel and tourism industry development indicator is also explained by TTESI. Finally, the Wastewater treatment indicator is also kept, even though it is not as well represented by TTESI as the first three indicators. Table 2 shows reliability estimates for Cronbach’s alpha, composite reliability (CR), and average variance extracted (AVE) for TTESI. Indicators are computed using standardized loadings. Cronbach’s alpha is 0.888 and indicates good reliability. The CR value of 0.907 is well above the threshold value of 0.70. The value of AVE (Fornell and Larcker, 1981) is 0.716, thus suggesting that the amount of variance explained by the indicators is, on average, higher than the unexplained error, i.e., the AVE value is above the recommended 0.50 threshold (Bagozzi and Yi, 1988). Thus, it can be concluded that these 4 indicators define a valid summated scale. The final step in the analysis is to look at the individual data for each country. Table 3 provides the score of each country for both indices: ESI (a score from minimum value of 1 to the maximum value of 7) and TTESI (a z-score). Additionally, it reports the ranking of countries based on ESI and TTESI and the difference between the two. Overall, I conclude that these indices are in agreement. The Pearson correlation between the ESI and TTESI of 0.678 (p < 0.001) indicates a high association between these scores. In terms of rankings, the ordering of both rankings also shows strong agreement (Spearman’s rho correlation = 0.651; p < 0.001). While the difference between the two rankings is small for many countries, for others we can observe a large gap in both directions and in some cases the difference is more than 60 positions. Countries with a big improvement in their position with TTESI are: Malaysia, United States, Tajikistan, Saudi Arabia, Indonesia, India, and China. On the other hand, the position of Mali, Bulgaria, Burkina Faso, Suriname, and Egypt deteriorates severely when measured by TTESI. For example, for TTESI we observe that the first place is occupied

J.G. Dias / Ecological Indicators 73 (2017) 589–596

593

Table 3 Country ranking based on the TTESI. Country

Albania Algeria Angola Argentina Armenia Australia Austria Azerbaijan Bahrain Bangladesh Barbados Belgium Bhutan Bolivia Botswana Brazil Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Chad Chile China Colombia Costa Rica Côte d’Ivoire Croatia Cyprus Czech Republic Denmark Dominican Republic Egypt El Salvador Estonia Ethiopia Finland France Gabon Gambia Georgia Germany Ghana Greece Guatemala Guinea Guyana Haiti Honduras Hong Kong SAR Hungary Iceland India Indonesia Iran Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Korea. Rep. Kuwait Kyrgyz Republic Lao PDR Latvia Lebanon

ES indices

ES ranking

Country

ESI

TTESI

ESI

TTESI

Diff.

3.60 3.51 3.41 3.32 3.79 4.64 5.13 3.83 3.73 3.01 4.77 4.28 4.07 3.83 4.42 3.89 4.62 4.47 3.78 3.43 3.93 4.63 4.32 4.34 4.38 2.93 3.88 4.11 4.09 4.38 3.92 4.90 4.92 3.23 3.99 3.70 4.73 4.00 5.25 4.67 4.30 4.49 4.31 4.90 3.99 4.19 3.62 3.81 4.18 2.88 4.12 3.96 5.16 4.92 2.89 3.11 3.47 5.31 3.76 4.34 3.33 4.30 3.92 3.84 4.42 3.86 2.95 3.38 3.55 4.59 3.29

−1.33 −1.31 −1.43 −0.95 −0.97 1.44 1.96 −0.10 0.14 −1.02 0.21 1.61 1.41 −0.34 0.27 0.57 −0.66 −0.68 −1.20 −0.88 −0.53 1.06 −0.29 −0.23 0.45 −0.25 −0.58 0.71 −0.84 0.10 0.50 0.73 2.19 −0.64 −1.91 −0.70 1.25 −0.38 2.21 0.91 −0.42 0.35 −0.68 2.04 −0.30 −0.45 −0.53 −1.65 −0.39 −1.75 −0.08 0.27 −0.10 1.26 −0.18 −0.08 −0.55 1.08 −0.04 −0.33 −0.52 2.01 −0.01 −0.14 0.29 0.17 −1.18 −1.17 −0.06 0.71 −2.02

109 113 120 128 97 25 8 94 104 135 18 57 73 93 37 86 27 35 98 118 82 26 49 48 43 137 87 68 71 42 83 13 12 131 77 105 20 76 3 23 54 32 52 14 78 61 108 96 62 140 67 79 7 11 139 134 115 2 101 47 127 53 84 91 39 90 136 124 112 30 129

133 132 134 120 122 17 7 66 54 124 51 13 18 83 48 37 106 108 130 115 96 28 78 75 39 76 101 33 113 57 38 32 2 105 139 111 23 85 1 29 90 43 109 5 79 92 97 137 86 138 64 49 67 22 72 65 99 27 62 80 95 6 61 69 47 53 129 128 63 34 140

24 19 14 −8 25 −8 −1 −28 −50 −11 33 −44 −55 −10 11 −49 79 73 32 −3 14 2 29 27 −4 −61 14 −35 42 15 −45 19 −10 −26 62 6 3 9 −2 6 36 11 57 −9 1 31 −11 41 24 −2 −3 −30 60 11 −67 −69 −16 25 −39 33 −32 −47 −23 −22 8 −37 −7 4 −49 4 11

Lesotho Lithuania Luxembourg Macedonia. FYR Madagascar Malawi Malaysia Mali Malta Mauritania Mauritius Mexico Moldova Mongolia Montenegro Morocco Mozambique Myanmar Namibia Nepal Netherlands New Zealand Nicaragua Nigeria Norway Oman Pakistan Panama Paraguay Peru Philippines Poland Portugal Puerto Rico Qatar Romania Russian Federation Rwanda Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Slovak Republic Slovenia South Africa Spain Sri Lanka Suriname Swaziland Sweden Switzerland Taiwan Tajikistan Tanzania Thailand Trinidad and Tobago Tunisia Turkey Uganda United Arab Emirates United Kingdom United States Uruguay Venezuela Vietnam Yemen Zambia Zimbabwe

Note: ESI scores come from Crotti and Misrahi (2015) and the TTESI z-score expression is provided in Appendix C.

ES indices

ES ranking

ESI

TTESI

ESI

TTESI

Diff.

3.87 4.42 5.23 3.65 3.46 4.01 3.42 4.55 4.17 4.15 4.67 3.37 4.22 3.24 4.30 4.09 3.93 3.57 4.41 3.14 4.81 4.94 3.90 3.86 5.22 4.07 2.82 4.25 3.83 3.75 3.41 4.62 4.42 4.84 4.32 4.35 3.70 4.68 3.41 4.36 4.08 5.17 3.77 4.31 4.49 4.74 4.29 4.61 3.74 4.36 4.48 5.03 5.63 4.09 3.40 3.78 3.46 3.48 4.22 3.83 3.96 4.39 4.79 3.56 4.72 3.38 3.16 2.92 4.16 4.18

0.20 0.64 1.86 −0.15 −0.99 −0.67 1.12 −0.88 −0.13 −0.40 0.03 −0.34 −0.91 −1.51 −0.22 −0.41 −0.94 −1.05 0.59 −0.57 1.70 1.77 −0.60 −0.95 1.76 1.45 −1.09 −0.51 −1.46 −0.53 −0.14 0.11 0.91 1.19 1.58 −0.27 −0.61 2.07 0.44 −0.40 −0.86 1.38 −0.68 1.60 0.14 0.81 0.07 0.34 0.30 −0.89 0.04 1.75 2.13 0.44 0.44 −0.33 −0.46 −1.13 −0.44 −0.21 −0.61 1.41 1.31 1.22 0.27 −1.28 −0.78 −2.14 0.35 −0.33

88 38 4 107 117 75 119 31 64 66 24 126 60 130 55 70 81 110 40 133 16 10 85 89 5 74 141 58 92 102 122 28 36 15 50 46 106 22 121 45 72 6 100 51 33 19 56 29 103 44 34 9 1 69 123 99 116 114 59 95 80 41 17 111 21 125 132 138 65 63

52 35 8 71 123 107 26 116 68 87 60 84 118 136 74 89 119 125 36 100 12 9 102 121 10 16 126 94 135 98 70 56 30 25 15 77 103 4 40 88 114 20 110 14 55 31 58 45 46 117 59 11 3 41 42 81 93 127 91 73 104 19 21 24 50 131 112 141 44 82

−36 −3 4 −36 6 32 −93 85 4 21 36 −42 58 6 19 19 38 15 −4 −33 −4 −1 17 32 5 −58 −15 36 43 −4 −52 28 −6 10 −35 31 −3 −18 −81 43 42 14 10 −37 22 12 2 16 −57 73 25 2 2 −28 −81 −18 −23 13 32 −22 24 −22 4 −87 29 6 −20 3 −21 19

594

J.G. Dias / Ecological Indicators 73 (2017) 589–596

Table 4 The ten top and bottom countries based on the TTESI. TTESI ranking

Stringency of environmental regulations (Indicator X1)

Enforcement of environmental regulations (Indicator X2)

Sustainability of travel and tourism industry development (Indicator X3)

Wastewater treatment (Indicator X9)

1 2 3 4 5 6 7 8 9 10

6.23 6.34 6.33 5.96 6.14 6.08 6.20 5.94 5.81 5.90

6.26 6.24 6.17 6.12 6.08 6.05 5.99 5.90 5.80 5.79

5.31 4.38 5.67 5.92 4.86 4.90 5.68 5.12 5.87 4.73

84.25 93.45 96.95 0.00 95.18 71.26 79.05 95.00 77.90 77.05

2.79 2.68 2.57 3.29 2.73 2.64 2.32 2.24 2.00 1.83

2.58 2.57 2.47 2.41 2.37 2.23 2.13 1.95 1.84 1.72

3.17 3.99 2.17 3.57 3.00 3.06 4.05 3.63 3.14 2.21

34.64 3.36 0.00 0.16 19.80 0.75 0.00 49.50 15.08 0.49

Top 10 countries Finland Denmark Switzerland Rwanda Germany Japan Austria Luxembourg New Zealand Norway

Bottom 10 countries Algeria Albania Angola Paraguay Mongolia Guinea Haiti Egypt Lebanon Yemen

132 133 134 135 136 137 138 139 140 141

by Finland, going up 3 positions; in contrast, Switzerland achieves a lower relative score and moves from first to third position. As a complement to the internal consistency of the ESI that was improved by the proposal of the TTESI, I further investigate its external validity by analyzing the relationship of both indices with the HDI.4 Data on HDI from 2014 comes from the UNDP. Puerto Rico and Taiwan are not included in these analyses as they are two territorial jurisdictions. The Pearson correlation between the HDI and these two indices increases from 0.399 (p < 0.001) to 0.567 (p < 0.001), respectively for ESI and TTESI. Additionally, comparing the ranking of the countries based on the HDI with both indices, I conclude that the TTESI is more correlated with the HDI than the ESI (Spearman’s correlation: 0.442 and 0.613 for ESI and TTESI, respectively). Consequently, TTESI proposes a measure of environmental sustainability that is better linked to the other two components of sustainable development – social and economic dimensions – measured by the HDI. Thus, TTESI can be better integrated with other indicators as a measure of this subdimension of development (Goodland, 1995). Table 4 shows the detailed indicators used to compute the scores for top and bottom positions in TTESI. As noted previously, all data come from the official report entitled The Travel & Tourism Competitiveness Report 2015. For instance, data for Rwanda is on page 287 (Crotti and Misrahi, 2015). At first glance, we see that this country is in fourth position, which is unexpected given traditional rankings. If we carefully inspect the rating in the first three indicators, we conclude that they are quite high and explain the fourth overall position. Nevertheless, Rwanda is penalized by indicator 4 as it treats 0% of wastewater. On the other hand, Egypt treats half of its wastewater, but occupies position 139 due to respondents’ the very low opinion on indicators 1 and 2. Therefore, despite the reduction of indicators to ensure reliability for some countries, the raw data may suffer from measurement error and this affects both ESI and

4 I would like to thank an anonymous reviewer for raising the issue on the relationship between these two indices and the HDI; this allowed me to assess the external validation of the TTESI index.

TTESI. I note that Rwanda is in the 22nd position for ESI and this may also be inflated.

5. Discussion and conclusion The environmental sustainability concept has been around for some time. Different frameworks have been suggested for its definition and measurement. I selected the indicator of environmental sustainability made available by The Travel & Tourism Competitiveness Reports. Using a comprehensive and rigorous methodology, this study showed that the original index has consistency problems. To the best of my knowledge, this is the first study that systematically analyzes the reliability of this index and develops a modified index of environmental sustainability for each country: the Travel & Tourism Environmental Sustainability Index (TTESI). Through the use of the TTESI, a more consistent and reliable measure of the environmental sustainability is reported. The paper also shows that combining physical and survey data raises reliability problems for indices and these two types of data source should be analyzed separately. This dichotomy between physical/environmental data and attitudinal/opinion data is due to the fact that the opinion of respondents in many countries is better than the evidence provided by environmental data. Thus, both ESI and TTESI combine indicators of a distinct nature, and both may suffer from measurement problems. Official statistics in developing countries are prone to measurement error (e.g., heaping effects). On the other hand, because these indices are heavily dependent on the opinion of executives about each country, it is important to investigate data reliability and its cross-country comparability. The unmixing nature of these indicators may indicate the need for further research on the multidimensional nature of this construct.5

5 It can be observed from Fig. 1 that the association between the environmental variables was too weak to provide indicators for a specific dimension. Therefore, the EFA was unable to extract two dimensions from these 10 indicators. Thus, the development of a bidimensional or multidimensional index of environmental sustainability entails exploring further indicators.

J.G. Dias / Ecological Indicators 73 (2017) 589–596

This research reveals that for some countries, the opinion data being collected may not be reliable and may need further adjustments in new versions of TTESI-10 and TTESI. Further improvements are required to the measuring of the invariance of these indicators across different cultural backgrounds as different cultures may assess constructs differently (Mullen, 1995). Therefore, it is important to understand the cross-cultural comparability of this opinion data, which is also used in the original index. One difficulty lies in deciding whether the differences are real or result from the use of the measurement instrument (e.g., questionnaire). Because cross-national research applies the same scale of measurement, it is necessary to ensure that the scale of measurement (indicators) is invariant. We observe that results from the survey for some countries (e.g., Rwanda) are unexpectedly high. Without evidence of measurement equivalence, the differences might be due to the way the scale is used by respondents rather than to the underlying dimension being measured (Steenkamp and Baumgartner, 1998; Davidov et al., 2014). Thus, further field work and research is necessary to verify whether the construct is invariant across different cultural settings. This result has implications for future updates of the ESI and TTESI, and further qualitative studies (e.g., use of stakeholders and experts in the field) or quantitative studies can be applied to improve the validity of opinion surveys thus ensuring that different respondents are using similar definitions of environmental sustainability. The TTESI resulted from the indicators available in The Travel & Tourism Competitiveness Reports. While the initial 10 indicators were regarded as indicators of environmental sustainability, only 4 were retained in this unidimensional definition of the construct. In the case of a unidimensional index, TTESI is preferable for the same available indicators. Further conceptual discussion and data collection may provide directions for a multidimensional definition of Environmental Sustainability as other indicators may be relevant in measuring environmental sustainability. An additional advantage of the TTESI is that it is more in line with the HDI, which is commonly used as a measure of economic and social development. TTESI could also work better in the definition of a composite index in conjunction with the HDI, as an alternative to the HSDI – Human Sustainable Development Index (Togtokh, 2011). This research can be complemented with qualitative research, aimed at identifying other indicators to measure this construct. Additional research is required to further validate the conceptual framework and theoretical justification of the concept as most of the original variables are not strongly associated (Fig. 1), which may suggest a multidimensional nature of the construct that cannot be reduced into a single index. As Dahl (2012) recognizes, ranking countries by these indices can stimulate decision-makers to try and improve their positions in the rankings. Nevertheless, it is important that the rankings are based on reliable indices and data to fulfill their mission of a global improvement in environmental sustainability. While this research endeavor was to analyze reliability and to introduce a modified index of environmental sustainability, future research may apply this methodology to develop reliable measurement scales of other constructs that are internationally comparable. Examples abound from the Globalization Index (Dreher, 2006) to the Global Competitiveness Index published by the World Economic Forum. Moreover, research on the relationship between environmental sustainability and other concepts such as globalization or economic freedom could be furthered. The methodology introduced in this study provides a sound approach for testing such relationships, i.e., before defining structural relationships between constructs, we have to ensure reliability in the measurement of the constructs (Bollen, 1989).

595

Appendix A. Overview of exploratory and confirmatory factor analyses Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) are statistical methods used to understand the variance and covariance structure shared by a set of variables that is believed to be explained by a group of factors or latent constructs. Despite this similarity, EFA and CFA are conceptually and statistically distinct. The main goal of EFA is to identify factors that explain the relationship between the observed variables. It is an exploratory technique as the researchers do not assume any a priori hypotheses about either the number of factors underlying the structure or the variables which each factor will comprise. Thus, EFA does not specify a structure on the linear relationships between the observed variables and the factors. It is focused on the dimension of the model, i.e., the number of latent factors needed to explain the observed structure. On the other hand, the CFA evaluates a priori hypotheses, i.e., it allows for constraints on certain parameters in the model. EFA and CFA do not have to be mutually exclusive analyses; after an EFA that explores the dimension of the model, a CFA model is often estimated to compute measures of reliability. Let yij be the indicator j measured in unit i, n is the sample size, and J is the number of observed indicators. The factor model specifies the variation and co-variation of a set of indicators yj (j = 1, . . ., J) as a function of factors fk (k = 1, . . ., q) and residual variables (ij ). For unit i, the general model is given by yij = j + j1 fi1 + · · · + jk fik + · · · + jk fiq + ij and in matrix form yi =  + fi + i where  is the vector of intercepts,  is the matrix of factor loadings, fi is the vector of common factors, and i is the vector of specific factors. Var(fi ) =  is the matrix of factor variances/covariances and Var(i ) =  is the matrix of residual variances/covariances. Therefore, the population covariance matrix of observed variables is Var(yi ) =  =  + , where  is the transposed matrix of loadings. Often  is assumed to be the identity matrix, i.e., the variance of the latent factors is one and the factors are uncorrelated. In this case, Var(yi ) =  =  + . We notice that the EFA model (without constraints) is not identified. Let us define L as an orthogonal matrix i.e., L−1 = L . Therefore,  ∗ ∗ for * = L, the solution is exactly the same as  =   . Thus, the EFA solution can be rotated in an attempt to find the best interpretation. For yi multivariate normal distributed, the log-likelihood function is (; y) = −

n n −1 log || − Tr(S ) 2 2

where  is the variance-covariance matrix implied by the proposed factor model, S is the observed variance-covariance matrix, and Tr(.) is the trace of a matrix. The estimates of the free parameters in  and  are obtained by maximizing the log-likelihood function in order to estimate  and  iteratively (for centered data intercepts are not estimated): 1. For fixed , maximize analytically over ; 2. For fixed , maximize analytically over . To deal with the missing data, the algorithm is adapted (see Enders (2001)).

596

J.G. Dias / Ecological Indicators 73 (2017) 589–596

The communality of the variable or indicator j gives the proportion of variability of the indicator that is explained or retained by the set of factors. It is computed by h2j =

q 

2jk .

k=1

Appendix B. Measures of reliability To examine the reliability of the scale, three measurements are usually applied to assess the quality of the index or factor: the standardized Cronbach’s alpha, the composite reliabilities (CRs), and Average Variance Extracted (AVE). The first measure is computed directly from raw data, whereas the last two are based on the results of the confirmatory factor analysis (CFA). Cronbach’s alpha. The standardized Cronbach’s alpha is defined as ˛=

J r¯ 1 + (J − 1)¯r

where r¯ is the average of the J(J − 1)/2 non-redundant correlation coefficients (i.e., average of an upper or lower triangular correlation matrix). It assumes non-negative values and varies between 0 and 1 for average correlation of 0 and 1, respectively. Composite reliability (CR). Composite reliability (CR) is based on the loadings of the standardized factorial solution and is given by (



CRk =

(

 ) j jk



 ) j jk

2

+



j

2

(1 − 2jk )

.

Average variance extracted (AVE). Average variance extracted (AVE) is based on the loadings of the standardized factorial solution and it is given by AVE k =



2 j jk

 2  j jk 

+

(1 − 2jk ) j

=

1 2 jk . J j

Appendix C. Computation of z-scores and country ranking The estimated factorial scores ˆfi are based on the conditional distribution of the factor given the observed variables (regression method) and estimated parameters. The best predictor is the conditional mean (the z-score) and is given by 



ˆ ( ˆ ˆ + ) ˆ ˆfi = 

−1

¯ (yi − y).

For each country i this estimate provides the z-score that on average is zero and with a standard deviation of one. This corresponds to the value of the TTESI index. The computation of the index using this expression simultaneously and automatically normalizes and aggregates the indicators, which avoids the arbitrariness of traditional methods that may be misleading and inconsistent (Böhringer and Jochem, 2007). Country ranking is based on the ordering of countries by this z-score: maximum z-score means position 1, and so on. References Anderson, J., Gerbing, D., 1988. Structural equation modeling in practice: a review and recommended two-step approach. Psychol. Bull. 103 (3), 423–441. Asici, A.A., Acarb, S., 2016. Does income growth relocate ecological footprint? Ecol. Indic. 61, 707–714. Babcicky, P., 2013. Rethinking the foundations of sustainability measurement: the limitations of the Environmental Sustainability Index (ESI). Soc. Indic. Res. 113 (1), 133–157.

Bagozzi, R.P., Yi, Y., 1988. On the evaluation of structural equation models. J. Acad. Mark. Sci. 16 (1), 74–94. Böhringer, C., Jochem, P.E.P., 2007. Measuring the immeasurable – a survey of sustainability indices. Ecol. Econ. 63 (1), 1–8. Bolcárová, P., Koloˇsta, S., 2015. Assessment of sustainable development in the EU 27 using aggregated SD index. Ecol. Indic. 48, 699–705. Bollen, K.A., 1989. Structural Models with Latent Variable. Wiley, New York. Bravo, G., 2014. The Human Sustainable Development Index: new calculations and a first critical analysis. Ecol. Indic. 37, 145–150. Churchill, G., 1979. A paradigm for developing better measures of marketing constructs. J. Mark. Res. 16 (1), 64–73. CNN, 2015. Volkswagen has plunged 50%. Will it ever recover? http://money.cnn. com/2015/09/24/investing/volkswagen-vw-emissions-scandal-stock/ (accessed 30.11.15). Crotti, R., Misrahi, T., 2015. The Travel & Tourism Competitiveness Report 2015. Growth through shocks. World Economic Forum, Geneva. Dahl, A.L., 2012. Achievements and gaps in indicators for sustainability. Ecol. Indic. 17, 14–19. Dahme, K., Hinterberger, F., Schütz, H., Seifert, E.K., 1998. Sustainable Human Development Index: A Suggestion for “Greening” the UN’s Indicator. Wuppertal Institute for Climate, Environment and Energy, Wuppertal, Mimeo. Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., Billiet, J., 2014. Measurement equivalence in cross-national research. Annu. Rev. Sociol. 40, 55–75. Dreher, A., 2006. Does globalization affect growth? Evidence from a new index of globalization. Appl. Econ. 38 (10), 1091–1110. Enders, C.K., 2001. A primer on maximum likelihood algorithms available for use with missing data. Struct. Equ. Model. 8 (1), 128–141. Esty, D.C., Levy, M., Srebotnjak, T., de Sherbinin, A., 2005. 2005 Environmental Sustainability Index: Benchmarking National Environmental Stewardship. Yale Center for Environmental Law & Policy, New Haven. Fornell, C., Larcker, D.F., 1981. Evaluating structural equation models with unobserved variables and measurement errors. J. Mark. Res. 18 (2), 39–50. Gerbing, D.W., Anderson, J.C., 1988. An updated paradigm for scale development incorporating unidimensionality and its assessment. J. Mark. Res. 25 (2), 186–192. Goodland, R., 1995. The concept of environmental sustainability. Annu. Rev. Ecol. Syst. 26, 1–24. Hair, J., Black, W.C., Babin, B.J., Anderson, R.E., 2010. Multivariate Data Analysis, 7th ed. Pearson Education International, Upper Saddle River, New Jersey. Hizsnyik, E., Toth, F.L., 2010. Literature Findings and Recommendations for Linking SD and Mainstream Macroeconomic Indicators, Internal Summary Report, Relate Existing Measures of Sustainability to Economic Performance, IN STREAM. International Institute for Applied Systems Analysis, Laxenburg. ´ S., Jeremic, ´ V., Petrovic, ´ N., Radojiˇcic, ´ Z., 2015. Colouring the Iˇsljamovic, socio-economic development into green: I-distance framework for countries’ welfare evaluation. Qual. Quan. 49 (2), 617–629. Jöreskog, K.G., 1971. Statistical analysis of sets of congeneric tests. Psychometrika 36 (2), 109–133. Jöreskog, K.G., Sörbom, D., 1982. Recent developments in structural equation modelling. J. Mark. Res. 19 (4), 404–416. Kaiser, H.F., 1974. An index of factorial simplicity. Psychometrika 39 (1), 31–36. Klemeˇs, J.J., 2015. Assessing and measuring environmental impact and sustainability. Clean Technol. Environ. Policy 17 (3), 577–578. Kline, R.B., 2016. Principles and Practice of Structural Equation Modeling, 4th ed. Guilford, New York, NY. Mancini, M.S., Galli, A., Niccolucci, V., Lin, D., Bastianoni, S., Wackernagel, M., Marchettini, N., 2016. Ecological footprint: refining the carbon footprint calculation. Ecol. Indic. 61, 390–403. Morse, S., 2003. Greening the United Nations’ human development index? Sustain. Dev. 11, 183–198. Mullen, M.R., 1995. Diagnosing measurement equivalence in cross-national research. J. Int. Bus. Stud. 26 (3), 573–596. Nunnally, J.C., 1978. Psychometric Theory, 2nd ed. McGraw Hill, New York, NY. Nunnally, J.C., Bernstein, L., 1994. Psychometric Theory. McGraw Hill, New York. Peter, J.P., 1979. Reliability: a review of psychometric basics and recent marketing practices. J. Mark. Res. 16 (1), 6–17. Samuel-Johnson, K., Esty, D.C., 2000. Pilot Environmental Sustainability Index Report. World Economic Forum (Annual Meeting). Davos, Switzerland. Siche, J.R., Agostinho, F., Ortega, E., Romeiro, A., 2008. Sustainability of nations by indices: comparative study between environmental sustainability index, ecological footprint and the emergy performance indices. Ecol. Econ. 66 (4), 628–637. S&P Dow Jones Indices and ROBECOSAM, 2015. Volkswagen AG to be removed from the Dow Jones Sustainability Indices, New York and Zurich, September 29. http://www.sustainability-indices.com/images/150929-statement-vwexclusion vdef.pdf (accessed 08.06.16). Steenkamp, J.-B., Baumgartner, H., 1998. Assessing measurement invariance in cross-national consumer research. J. Consum. Res. 25 (1), 78–90. Togtokh, C., 2011. Time to stop celebrating the polluters. Nature 479, 269. Wackernagel, M., Rees, W., 1996. Our Ecological Footprint: Reducing human impact on the Earth. New Society Publishers, Philadelphia, PA. WWF, 1998. Living Planet Report 1998. World Wildlife Fund (WWF), Gland.