- Email: [email protected]
Measuring coastal cities' resilience toward coastal hazards: Instrument development and validation
Journal Pre-proof Measuring coastal cities' resilience toward coastal hazards: Instrument development and validation
Rina Suryani Oktari, Louise K. Comfort, Syamsidik, Putra Dwitama PII:
S2590-0617(19)30057-2
DOI:
https://doi.org/10.1016/j.pdisas.2019.100057
Reference:
PDISAS 100057
To appear in: Received date:
16 June 2019
Revised date:
2 November 2019
Accepted date:
5 November 2019
Please cite this article as: R.S. Oktari, L.K. Comfort, Syamsidik, et al., Measuring coastal cities' resilience toward coastal hazards: Instrument development and validation, (2019), https://doi.org/10.1016/j.pdisas.2019.100057
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
© 2019 Published by Elsevier.
Journal Pre-proof
Measuring Coastal Cities’ Resilience toward Coastal Hazards: Instrument Development and Validation Rina Suryani Oktari a*, Louise K Comfort b, Syamsidik c, Putra Dwitama d and Disaster Mitigation Research Center (TDMRC), Graduate School of Mathematics and Applied Science, and Faculty of Medicine, Universitas Syiah Kuala, Jl. Tgk. Syech Abdul Rauf, Banda Aceh, 23111 Indonesia, email; [email protected], b Center for Disaster Management (CDM) and Graduate School of Public and International Affairs (GSPIA), University of Pittsburgh, 3601 Wesley W. Posvar Hall, Pittsburgh, Pennsylvania 15260, email: [email protected] c Tsunami and Disaster Mitigation Research Center (TDMRC) and Civil Engineering Department, Faculty of Engineering, Universitas Syiah Kuala, Jl. Tgk. Syech Abdul Rauf No. 7, Banda Aceh, 23111 Indonesia, email; [email protected] d National Action Plans-Climate Resilience Secretariat (RAN API), Wisma Bakrie 2, 6th floor, Jl. H.R. Rasuna Said Kavling B-2, Setiabudi, Jakarta Selatan, 12920 Indonesia, email; [email protected]
-p
ro of
a* Tsunami
re
ABSTRACT
lP
This study aims to design and validate a comprehensive assessment tool that measures the coastal cities’ resilience toward coastal hazards. Instrument development process adhered to
na
the research and development methodology that involves: i) literature review, ii) design of
ur
the assessment tool, and iii) instrument validity test using Content Validity Ratio (CVR) and
Jo
Content Validity Index (CVI). Fourteen experts were involved in the validity test. This study identified five aspects of the resilience: institutional/ governance, environmental, social economic, built environment, and infrastructure. From the expert judgments, 23 items were judged as having excellent content validity (ICVI higher than 0.79). There were two items with unacceptable ICVI that need for revision. The items that need revision have been modified according to the recommendations of the expert panel. The overall SCVI of the instrument was equal to 0.9 (SCVI /Ave) and 0.92 (SCVI /UA). In addition, the calculation of the Kappa statistic showed that the assessment tool have excellent inter-rater reliability at the item level (K>0.74). This study revealed that this instrument had gained an appropriate level of validity to measure the coastal cities’ resilience. 1
Journal Pre-proof
ro of
KEYWORDS: Coastal resilience, assessment tool, content validity.
1. Introduction
-p
Global climate change, population growth, and human-induced vulnerability have caused
re
coastal communities around the world to experience significant rates of change [1], [2]. The effects of this change highlight the coastal area has a high degree of exposure to climate
lP
change-induced rise in sea level coupled with coastal hazards, including storm surges,
na
shoreline erosion, coastal flooding, and tsunami. These hazards pose significant threats to the physical, social, and economic aspect of the coastal communities [3], [4], [5].
ur
Presently, about 40 million people are exposed to a disaster with a frequency of once in
Jo
every 100-year period. This number will be tripled in the next 50 years [6]. Notwithstanding this condition, coastal cities in developing countries have to solve two types of problems, i.e., un-timely effective responds to disasters and degradation of coastal ecosystem that resulted in poor protection to the coastal communities [7]. Therefore, effective and long-term disaster and climate change risk management are required immediately. Developing effective risk management and mitigation to reduce the impacts of climate change will be essential to secure the sustainable development of coastal zones. Thus, there is a critical need to assess coastal resilience by providing the tools to better inform both scientific debates and policymakers in improving the mitigation and adaptation strategies of
2
Journal Pre-proof the coastal cities toward climate change-induced rise in sea level coupled with coastal hazards [3]. In the 21st century, Asia and Africa are projected to be a center for population growth, economic development, and urbanization in the coastal area, as well as small islands, are particularly vulnerable to sea-level rise and climate change [4], [8]. With more than 17,000 islands, Indonesia is highly vulnerable to the impacts of global climate change because of its geographical location. The average temperature is projected to increase by 0.5–3.92°C by
ro of
2100 compared to the current period (1981–2010). Climate change-induced rise in sea level is expected to reach 35–40 cm by 2050 relative to the value of 2000. Models predict that
-p
taking into account the melting ice factor at the North and South Pole, sea levels will rise
re
about 5 cm in the year 2100. Almost 65% of the population that live in the coastal areas of
sea level coastal flooding [9].
lP
Indonesia is vulnerable to the impacts of climate change, especially that caused by a rise in
na
Considering the high vulnerability to climate change impact, Indonesia should enhance its capacity to mitigate and to adapt to climate change, especially in the most vulnerable
ur
communities [10]. As part of climate change adaptation in the coastal area, to ensure the
Jo
protection of the natural environment that provides the basis for carrying out the various human activities in the coastal area, an integrated approach to coastal management is essential [11]. An increase in the frequency and severity of climate-related disaster is likely because of the impacts of climate change, making it essential for many countries and also a global need to increase resilience, especially in the coastal area [7]. The word “resilience” has been accepted as a synthesis to combine Climate Change Adaption (CCA) and Disaster Risk Reduction (DRR) [12], [13]. Disaster risk reduction should include measures to adapt to the impacts of climate change. However, there is lack of integration between the concepts of DRR and CCA [14], [15], [16]. Coastal hazards, such as
3
Journal Pre-proof tsunami, coastal flooding, and coastal erosion, are rather faster on-set disasters than the impacts of climate change-induced rise in sea level. This resulted in a lack of understanding and a lack of preparedness of coastal cities to integrate CCA and DRR into one comprehensive long-term development plan [17]. Combining the two concepts into coastal cities’ resilience is a challenging work for most of the cases. Stakeholders who work on CCA and DRR often have different development agendas and gather in different forums [16]. Inter-connection between the two issues is infrequent. This makes it difficult to assess the
ro of
coastal cities’ resilience toward the impacts of climate change coupled with certain types of coastal hazards. Another reason is also because of the absence of an assessment tool that
-p
enables the evaluation of the coastal cities’ resilience toward the impacts of climate change
re
coupled with coastal hazards.
The aforementioned condition motivated this study. The objective of this study is to
lP
develop a comprehensive assessment tool on the coastal cities’ resilience toward impacts of
ur
flooding, and tsunami).
na
climate change-induced rise in sea level coupled with coastal hazards (erosion, coastal
Jo
2. Concept of Community Resilience applied to Coastal Areas The definitions of resilience that is relevant to communities reflect the adaptive capacity of a community or a system to manage the disturbance of an adverse event or crisis situation [18], [19]. In addition, resilience provides the capability of the community to recover from the adversity and to mitigate future impacts [20], [21]. There are three characteristics of the resilience of socio-ecological systems: i) the magnitude of the adverse events that the community can absorb shocks while maintaining function, ii) the degree to which the community is capable of self-organization, and iii) the degree to which the community can build capacity to learn and adapt [22]. Despite the fact
4
Journal Pre-proof that human systems are naturally resilient, building resilience into a human-environment system is an effective strategy to cope with change caused by future shocks or unknowable risk [23]. Coastal areas face the challenge of increasing hazards driven by environmental change, unavoidable climate change impact and by human activities. To respond and to cope with the uncertainty surrounding the rate and extent of the rise in sea level and other climate-related hazards arising as a result of the global environmental change, the delivery of adaptive
ro of
responses to coastal management will be required [24], [25].
In fostering coastal resilience, it is important to balance environmental and development
-p
issues while promoting safe and livable communities. From the social perspective, the
re
resilience of the coastal communities can be enhanced through social networks, community engagement and participation, insurance, and access to other financial resources. In the built
lP
environment aspect, improvement in construction practices, building design and practices are
na
some measures that enhance resilience. Some other factors that enhance resilience include the
3. Method
Jo
etc.) [26].
ur
availability of public infrastructure and lifelines (such as roads, bridges, electricity, water,
The development process of the assessment tool adhered to the research and development methodology that involves three steps. First, preliminary studies were conducted to collect information via literature review on various resilience assessment tools. In the second step, the design of the assessment tool was prepared to determine the purpose of the instrument, which the users are, as well as the identification of the parameters, variables, and indicators of the assessment tool. The third step was the validity test of the assessment tool using Content Validity Ratio (CVR) and Content Validity Index (CVI) [27], [28], [29]
Step 1: Preliminary Studies Development stage
Literature review Literature
4 assessment tools reviewed
parameters,
5
ro of
Journal Pre-proof
Figure 1. The flowchart of research methodology
3.1 Experts in the Validity study
-p
Content validity will show that the contents reflect the complete range of attributes studied
re
and is usually done by seven or more experts [28]. Participants in this study consisted of 14 local and national experts, including from academia (3 people), government (7 people) and
lP
practitioners (4 people). Table 1 below shows the details of participants.
Category
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Government Academic Government Academic Practitioner Practitioner Government Government Government Academic Practitioner Government Practitioner Government
Level of Institution
Gender
Education
Local National Local National Local National National Local Local National National Local Local Local
Female Male Male Male Female Female Female Female Male Male Male Male Male Male
Bachelor Doctorate Master Doctorate Bachelor Master Master Master Master Doctorate Master Master Master Bachelor
Jo
ur
Expert
na
Table 1. The distribution of experts in the study
The experts involved were stakeholders representing policymakers, practitioners in the field, and academics who have focused on the areas of disaster and climate change. By conducting 6
Journal Pre-proof FGD that brought together experts from a variety of different backgrounds, communication between science-policy-practice was achieved, thereby strengthening the validity of the parameters and indicators in the assessment tools.
3.2 Data Validation The content validity study combined qualitative and quantitative method by conducting expert judgments through Focus Group Discussion (FGD) which was held in September 2017
ro of
in Banda Aceh, Indonesia. The FGD was aimed at assessing whether the contents of the parameters and variables of the assessment tool are appropriate and relevant for the purpose
-p
of the tool development. Each expert was supplied the list of evidence-based items
re
(parameters and variables) chosen by the researcher. The experts were requested to use the
lP
scoring method shown in Table 2 below to rate each item in the assessment tool.
Table 2. The scale used for rating items (DeVon et al. 2007)
ur
not relevant item needs some revision relevant but need minor revision very relevant
Essentiality not necessary useful but not essential essential
Jo
1 2 3 4
Relevancy
na
Score
To estimate the content validity of the assessment tool we use Lawshe’s CVR and CVI [30], [31]. The CVR was calculated using the level of agreement for each item’s inclusion through each expert’s “essential” rating. The CVR was calculated using Equation (1) as follows. 𝐶𝑉𝑅 =
(𝑁𝑒 −𝑁/2) (𝑁/2)
,
(1)
where Ne = the number of experts indicating “essential” ratings per item and N = total number of experts.
7
Journal Pre-proof According to Table 3, a CVR greater than 0.51 for 14 experts is considered acceptable. It means that the item is maintained in the instrument. If the CVR value is smaller than 0.51, we should remove the item from the instrument [27]. Table 3. Minimum values of CVR for the different number of experts. (One-Tailed Test, P-value=0.05) (Lawshe 1975) Minimum CVR value
5 6 7 8 9 10 11 12 13 14 15 20 25 30 35 40
0.99 0.99 0.99 0.75 0.78 0.62 0.59 0.56 0.54 0.51 0.49 0.42 0.37 0.33 0.31 0.29
na
lP
re
-p
ro of
Number of experts
The CVI is the mean of the CVR values of items and represents the content validity of the
ur
entire instrument. There are two types of CVI: i) ICVI (item level) and ii) SCVI (scale-level).
Jo
The ICVI is defined as the proportion of agreement on the relevancy of each item. The following equation (2) was used to calculate the ICVI value. 𝐼𝐶𝑉𝐼 = 𝑛𝑎 /𝑁,
(2)
where ICVI is Item-Content Validity Index, and na is the number of experts giving rate 3 (relevant) and 4 (very relevant).
The SCVI is defined as the proportion of items in an instrument judged content valid. There are two approaches to calculate the SCVI. The first approach is the SCVI/UA (Universal Agreement) that calculates the proportion of items of the instrument that achieved rate 3
8
Journal Pre-proof (relevant) and 4 (very relevant) by all the experts. The second approach is the SCVI/Ave (Average) that calculates the ICVI for each item in the instrument and calculates the average ICVI across the items. The instrument will gain an excellent content validity if the SCVI /UA value ≥ 0.8 and the SCVI /Ave value ≥ 0.9 [31], [32]. As an important supplement for CVI, the multirater kappa statistic will be also calculated in the content validity study, as it provides information about the degree of agreement that adjusts for chance agreement. Chance agreement is very important while studying agreement
ro of
indices among judges, especially when there are four-scale scoring within two relevant and not relevant classes [33].
-p
To calculate the Kappa Statistic, we should calculate the probability of chance agreement
𝑁!
] 5𝑁
𝐴!(𝑁−𝐴)!
(3)
lP
𝑃𝑐 = [
re
using the following formula (3).
where, Pc is probability of chance agreement and A is the number of experts who agree that
ur
na
the item is relevant.
Jo
The formula used to calculate the Kappa Statistic is as follow. 𝐾=
(𝐼𝐶𝑉𝐼 −𝑃𝐶 ) (1−𝑃𝐶 )
.
(4)
Here, K is the Kappa value.
The Kappa values will be classified according to four criteria: i) excellent, if the kappa value is larger than 0.74, ii) good, if kappa value is between 0.60 and 0.74, and iii) fair, if the kappa value is between 0.40 and 0.59, and iv) poor, if the kappa value is smaller than 0.40 [33], [34].
4. Results and Discussion 9
Journal Pre-proof During the process of the development of the assessment tool, we found that most of the parameters we proposed were valid and relevant to assess the coastal cities’ resilience. In this section, we present the discussions of the results. Section 4.1 discusses the studies that have been done to review the four of existing resilience assessment tools. Section 4.2 explains the development of the assessment tool, including the identification of instrument objectives and items generation of the assessment tool (parameters, variables, and indicators). Section 4.3 presents the results of the content validity study containing the calculation of CVR, CVI, and
ro of
Kappa Statistic (K).
-p
4.1 Review of Existing Resilience Assessment Tools
Preliminary studies were conducted to collect information (via literature review, field
re
observations, and interviews) on various resilience assessment tools.
lP
All these tools have been developed mainly by international organizations, and there is
na
still a lack of tools developed by local authorities that reflect the local needs and conditions. Table 4 below describes information on the selected tools.
Name of Assessment Tools
Jo
ur
Table 4. Basic information on the selected resilience assessment tools (TRF 2014; UNISDR 2015; Shaw et al. 2010; Rubinoff and Courtney 2007) City Resilience Framework (CRF)
UNISDR City Resilience Scorecard V.2.2
Year Format Indicators
2014 Toolkit Infrastructure and environment, leadership and strategy, health and wellbeing, economy and society (12 Goals, 52 indicators and 156 variables) Literature review, stakeholder
2015 Scorecard “Ten Essentials” which cover governance and financial issues (Essentials 1–3); many dimensions of planning and disaster preparation (Essentials 4–8); the disaster response itself and post-event recovery (Essentials 9–10).
Development Method
Literature review
Climate and Disaster Resilience Index (CDRI) 2010 Toolkit Social, physical, economic, institutional, natural
Expert Opinions
NOAA Coastal Community Resilience Guide 2007 Scorecard Governance, society & economy, coastal resource management, land use & structural design, risk knowledge, warning & evacuation, emergency response, disaster recovery. Workshops and discussions with practitioners,
10
Journal Pre-proof input, field testing
Developer
Type of Risk Target Audience Equal Weighting
The Rockefeller Foundation
BM and AECOM
Academia
Multiple
Natural
Multiple
Local Authorities
Local authorities, insurance companies, private industry Yes
Community leaders/local authorities No
Yes
specialists from government agencies and organizations U.S. Indian Ocean Tsunami Warning System Program Tsunami and coastal hazard Local authorities, community leaders, coastal managers Yes
ro of
Most of the tools were designed to address multiple hazards. However, none of them attempt to combine the assessment on impacts of climate change-induced rise in sea level
-p
coupled with the impacts on coastal hazards. The target audiences of the tools are mainly for
re
the local authorities, community leaders, community members, insurance companies, private industry, and public (both individual and household). Three major formats are used in the
lP
assessment tools, including toolkit, scorecard, and index. The analysis shows different
na
resilience aspects addressed by each of the assessment tools. The tools have treated the
ur
resilience aspects as being equally important [35], [36].
Jo
4.2 Design of the Assessment Tool Originality and Practical Implications Despite the existence of several resilience assessment tools, this assessment tool is the first validated to be the practical tool for use to assess coastal cities’ resilience toward impacts of climate change-induced rise in sea level and coastal hazards. There are four objectives of the assessment tool on the coastal cities’ resilience toward the impacts of climate change-induced rise in sea level coupled with coastal hazards. First, is to have a sustainability assessment for continuous improvement on coastal cities’ resilience toward coastal hazards. Second, is to acquire/ identify the current situation of cities’
11
Journal Pre-proof resilience to improve the effectiveness of the existing strategies to increase cities’ resilience, make priorities and action plans (planning) as well as in performance evaluation (monitoring and evaluation) of the coastal cities’ resilience toward coastal hazards. Third, is to provide input to the policy or program in achieving sustainable development through identification of hazards that may hinder development and increase resilience toward the hazards. Fourth, is to provide feedbacks on long-term development policies of coastal cities’ to enhance resilience toward the impacts of climate change-induced rise in sea level and coastal hazards.
ro of
In practical terms, this assessment tool is intended for use by different stakeholder groups in sustaining long-term, consistent efforts in building coastal cities’ resilience. Among the
-p
stakeholders are: i) national and local government agencies; ii) practitioners working with
re
coastal community, iii) International aid agencies, donors, private sectors, and iv)
lP
Academicians.
Parameters, variables, and indicators of the instruments
na
A previous study has proposed many parameters/ indicators to measure the resilience. The
ur
Rockefeller Foundation (TRF) suggested that resilience includes the following parameters, namely: i) infrastructure and environment, ii) leadership and strategy, iii) health and
Jo
wellbeing, and iv) economy and society with a total of 12 goals, 52 indicators, and 156 variables [37]. UNISDR (2015) has developed a City Resilience Scorecard V.2.2 that proposed “Ten Essentials” which cover governance and financial issues (Essentials 1–3); many dimensions of planning and disaster preparation (Essentials 4–8); the disaster response itself and post-event recovery (Essentials 9–10) [38]. In addition, Shaw et al. (2010) proposed the following resilience indicators: social, physical, economic, institutional, and natural [39]. While Rubinoff and Courtney (2007) included some indicators in the Coastal Community Resilience Guide; they were governance, society & economy, coastal resource management,
12
Journal Pre-proof land use & structural design, risk knowledge, warning & evacuation, emergency response, and disaster recovery [40]. Following a thorough review of the parameters used by the assessment tools, we identified eight parameters namely: i) institutional/ governance, ii) social and economic, iii) coastal resource management, iv) land use management and infrastructure, v) risk knowledge, vi) warning and evacuation, vii) emergency response, and viii) disaster recovery [40], [41], [42], [43]. The variables and indicators of each resilience parameter have been selected from
ro of
indicators proposed by previous studies. We have conducted FGD to discuss this initial design of the assessment tool to obtain expert judgments.
-p
Based on the comments and inputs of experts, we extracted at least five resilience
re
parameters, including: i) institutional/ governance, ii) social and economic, iii) coastal resource management, iv) land use management and infrastructure, and v) adaptation and
lP
mitigation strategies.
na
Figure 2 shows a linear hierarchy of the parameters and variables of the coastal cities’
Jo
ur
resilience toward coastal hazard.
13
Journal Pre-proof
COASTAL CITIES' RESILIENCE
1. Institutional/ governance
2. Social and Economic
3. Coastal Resource Management
4. Land Use Management and Infrastructure
5. Adaptation and Mitigation Strategies
2.1 Social capital and skills
3.1 Implementation and monitoring of coastal resources
4.1 Land use policy and building standards
5.1 Risk knowledge
1.2 Basic services
2.2 Livelihood
3.2 Protection of habitats, ecosystems and sensitive natural features
4.2 Structural design
5.2 Early warning and evacuation
1.3 Participatory technical cooperation
2.3 Social and cultural networks
1.4 Technical and financial support
2.4 Economic stability
3.3 Community involvement in planning, implementation and monitoring
4.3 Mainstreaming risk reduction to location and structural design
5.3 Emergency response
ro of
1.1 Policies, plans and programs
5.4 Resource mobilisation
4.4 Education, research, and training
-p
3.4 Investment in management and conservation
re
Figure 2. Linear Hierarchy of the Coastal Cities’ Resilience toward Coastal Hazard
lP
Table 5 presents the item generation for each parameter and variable of the assessment
primary and secondary data.
na
tool. The data and information required to address each indicator in the instrument consist of
ur
Primary data is collected through focus group discussion (FGD), direct interviews and questionnaire surveys. Secondary data is obtained through literature review and an
Jo
institutional visit to gather data already produced or collected by the government department. Before collecting primary data, it is essential to analyse relevant secondary data exist.
Table 5. Indicators of Assessment Tool for Each Parameter/ Variable
Parameter 1: Institutional/Governance
No 1.1
1.2
1.3
Parameter/ Variable Policies, plans, and programs Indicators: i) The existence of policy/ regulations to protect coastal areas, ii) The existence of planning documents for coastal protection, iii) The existence of policies to control the movement and settlement, and iv) The existence of integration of climate change and disaster risk management into development policy. Basic services Indicators: i) Percentage of household that have access to electricity, ii) Percentage of household with access to clean water, iii) Percentage of household that have access to sanitation facilities and solid waste, and iv) Number of public transport. Participatory technical cooperation Indicators: i) The existence of forums/ working groups to undertake programs to enhance the resilience of coastal areas, ii) The existence of cooperation with various stakeholders including NGOs, private parties, academics, etc., iii) The coastal resource management program has included risk reduction issues, and iv) The socio-
References [40], [41], [42], [43]
[42], [43], [45]
[40], [41], [46], [47]
14
Journal Pre-proof
2.2
2.3
2.4
[40], [41], [42], [43]
[42], [43], [47], [48]
[42], [43], [47]
[42], [43], [45], [48]
[42], [43], [45], [47]
[17], [40], [41]
lP
3.4
na
3.3
ur
3.2
Jo
Parameter 3: Coastal Resource Management
3.1
re
-p
Parameter 2: Social and Economic
2.1
economic development program has included risk reduction issues. Technical and financial support Indicators: i) The existence of routine budget allocations as well as assistance to support activities that can reduce the risk of damage caused by coastal hazards, ii) Community leaders have the resources and tools to build community resilience in dayto-day activities, iii) Village budgets have included priorities for managing, upgrading, and mitigating critical facilities and infrastructure, and iv) Government has identified alternatives for increasing the village budget. Social capital and skills Indicators: i) The existence of economic development efforts to reduce the vulnerability of the community, ii) The existence of a program to provide skills to the community for alternative livelihoods, iii) The availability of social safety networks to assist vulnerable sectors of society, and iv) The availability of social safety networks to assist vulnerable sectors of society. Livelihood Indicators: i) Percentage of the community working in the agricultural sector, ii) Percentage of the community working in the fishery sector, iii) Percentage of the community working in the trade sector, and iv) The existence of established strategies to address the economic recovery caused by the disaster. Social and cultural networks Indicators: i) The existence of social and cultural networks involving community, cultural, private, and other non-governmental groups that support activities to increase community resilience, ii) The existence of social networks or community groups that can help during and post disaster, iii) The existence of a problem-solving mechanism to create a peaceful and orderly society, and iv) The mechanisms used to increase community participation in development planning. Economic stability Indicators: i) The existence of technical resources that provide assistance to the community in the diversification of environmentally friendly livelihoods (such as from universities, government programs, donor projects, etc.), ii) Availability of grant aid, technical assistance or loans to develop business alternatives, iii) The development/ financing of micro enterprises has been undertaken to provide sustainable livelihood alternatives, and iv) Insurance services are available in the event of a loss of business production in the event of a disaster. Implementation and monitoring of coastal resources Indicators: i) The existence of mechanism for assessing coastal resources and hazards on a regular basis, ii) The assessment results were used to identify risks in the community and become inputs for planning the management of coastal resources, iii) The existence of feedback mechanism for updating coastal resource management plans, and iv) The existence of community-based forums involved in solving problems in coastal areas. Protection of habitats, ecosystems, and sensitive natural features Indicators: i) Forest cover/ water catchment area, ii) Mangrove forest cover, iii) Coral Reefs cover, and iv) The existence of regulations governing the utilization of natural resources based on priority conservation and risk reduction. Community involvement in planning, implementation and monitoring Indicators: i) The existence of procedure for reviewing plans based on coastal issues and community feedback, ii) Communities are actively involved in the planning of coastal protection programs, iii) Communities are actively involved in the implementation of coastal protection programs, and iv) Communities are actively involved in monitoring coastal protection programs. Investment in management and conservation Indicators: i) The government has undertaken natural resource management, such as forests, rivers, beaches etc., to reduce risks, ii) The existence of rehabilitation and conservation activities of coastal ecosystems (Mangrove/coral reefs, etc.), iii) The existence of coastal ecosystem management group (coral reef/mangrove, etc.), and iv) The existence of monitoring activities of coastal ecosystem condition.
ro of
1.4
[17], [49]
[40], [41], [42]
[40], [41], [49]
Table 5. (Continued) Indicators of Assessment Tool for Each Parameter/ Variable No Parameter 4: Land Use Management and Infrastructure
4.1
4.2
4.3
Parameters/ Variables Land use policy and building standards Indicators: i) Percentage of concrete roads, ii) Level of pollution in coastal areas, iii) Condition of residential building (permanent/ semi-permanent), and iv) Percentage of houses across the coastline. Structural design Indicators: i) Percentage of built-up land, ii) Percentage of public facilities located outside of risk areas (schools, houses of worship, government offices, health facilities), iii) Siting and design for housing, hospitals and other critical infrastructure are based on land use plans and coastal hazard risk assessments, and iv) Coastal engineering structures have been designed to reduce vulnerability to coastal hazards and minimize impacts on coastal habitats. Mainstreaming risk reduction to location and structural design Indicators: i) Builders and architects have an understanding and are able to apply building
References [40], [41]
[40], [42], [43]
[26], [42], [43]
15
Journal Pre-proof
5.2
5.3
[42], [43], [50]
[42], [43], [50]
[42], [43], [50]
[42], [43], [50]
-p
5.4
[26], [42], [43]
re
Parameter 5: Adaptation & Mitigation Strategies
5.1
ro of
4.4
standards that integrate risk reduction, ii) Designer/ structural expert has an understanding and ability to design and build a secure infrastructure, iii) Building standards in hazard areas have been adopted by adjusting location, design and building infrastructure, and iv) The existence of socialization program on building design and practices that integrates risk reduction has reached the community. Education, research, and training Indicators: i) The existence of training/ counseling and dissemination of information to the community about land use and building standards, ii) The existence of training for builders, architects and contractors, iii) The existence of a hazard mitigation certification program for architects and contractors, and iv) Universities/ training institutions have included curricula on land use policies, building standards, and hazard mitigation. Risk knowledge Indicators: i) The existence of mapping and analysis of threats, vulnerabilities, and capacity to identify risk, ii) Public access to information from the analysis of threats, vulnerabilities, capacities and risks, iii) The existence of physical development (mitigation) to reduce disaster risk in coastal areas, and iv) Risk knowledge of the community. Early warning and evacuation Indicators: i) Establishment of an early warning system to provide time for self-rescue and assets for the community, ii) The existence of disaster information facilities, iii) The existence of evacuation maps and routes, evacuation sites and shelter, and iv) Community access to early warning information and evacuation strategies. Emergency response Indicators: i) Availability of disaster evacuation routes, ii) Availability of evacuation shelter, iii) Availability of systems and mechanisms of distribution of resources/ assistance to the community after the disaster, and iv) Community Emergency Response Plan. Resource mobilization Indicators: i) The existence of protection of the community's main productive assets from the impact of the disaster, ii) The existence of capacity building of government personnel to carry out/ train and provide equipment and equipment, facilities and pre-facilities, logistics, and personnel, iii) The existence of contingency plans in the face of disaster, and iv) Resources mobilization capacity of community.
lP
4.3 Content Validity Test of the Assessment Tool Both the qualitative and quantitative approaches were used to assess the content validity test
na
of the assessment tool. The qualitative content validity of the assessment tool was assessed by
ur
14 experts. The experts were asked to assess and to comment on the wording, item allocation, and item scaling. The assessment tool was revised based on the feedbacks and comments
Jo
received. In the quantitative step, the assessment tool was assessed through the calculation of the CVR and CVI [44].
Accordingly, 14 experts were asked to rate the essentiality (for CVR) and the relevancy (for CVI) of the items in the assessment tool. Table 6 presents the calculation of CVR for the items of the assessment tool. Table 6. Calculation of CVR Items 1 1.1 1.2 1.3
Ne 12 12 11 13
CVR 0.71 0.71 0.57 0.86
Interpretation Remained Remained Remained Remained
16
1.4 2 2.1 2.2 2.3 2.4 3 3.1 3.2 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
13 13 12 11 13 13 13 13 13 13 13 13 12 12 12 13 13 13 13 13 13
0.86 0.86 0.71 0.57 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.71 0.71 0.71 0.86 0.86 0.86 0.86 0.86 0.86
ro of
Journal Pre-proof Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained
re
-p
Ne=Number of experts evaluated the item essential, **CVR or Content Validity Ratio = (Ne-N/2)/(N/2) with the number of panelists 14 (N=14), the items with the CVR bigger than 0.51 remained at the instrument and the rest eliminated.
lP
According to Table 6, all the experts in the panel judged that the measurement items in the assessment tool are essential. The CVR values higher than 0.51 were considered as accepted
na
items and are to remain in the assessment tool. All the remaining items (n=25) were
ur
satisfactory with regard to CVR value.
Table 7 provides the calculation of both ICVI and SCVI using 14 experts’ judgment on the
Jo
relevancy items of the assessment tool.
Table 7. Calculation of ICVI Items
Not relevant (rating 1–2)
Relevant (rating 3–4)
ICVI
Interpretation
1 1.1 1.2 1.3 1.4 2 2.1 2.2 2.3 2.4 3
2 2 3 1 1 1 2 3 1 1 1
12 12 11 13 13 13 12 11 13 13 13
0.86 0.86 0.79 0.93 0.93 0.93 0.86 0.79 0.93 0.93 0.93
Appropriate Appropriate Need for Revision Appropriate Appropriate Appropriate Appropriate Need for Revision Appropriate Appropriate Appropriate
17
Journal Pre-proof 3.1 3.2 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
1 1 1 1 1 2 2 2 1 1 1 1 1 1
13 13 13 13 13 12 12 12 13 13 13 13 13 13
0.93 0.93 0.93 0.93 0.93 0.86 0.86 0.86 0.93 0.93 0.93 0.93 0.93 0.93
Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate
ro of
Number of experts=14 (N=14), Number of items considered relevant by all the panelists (I CVI > 0.79)= 23, Number of items=25, SCVI/Ave*** or Average of ICVI =0.9, SCVI/UA**=23/25=0.92. NOTE: *Item-Content Validity Items, **Scale-Content Validity Item/Universal Agreement,***Scale-Content Validity Item/Average.
From the expert judgments of the Item-Content Validity Items (ICVI) suggested that 23
-p
items of the instruments were judged as having excellent content validity (ICVI values were
re
higher than 0.79). There were two items with unacceptable ICVI values that needed revision. The items were modified based on the recommendations of the expert panel in the second
lP
round of judgment.
na
The overall scale-level content validity index (SCVI) of the instrument was equal to 0.9 for Scale-Content Validity Item/Average (SCVI/Ave) and 0.92 for Scale-Content Validity
ur
Item/Universal agreement. We can conclude that the assessment tool have gained an
Jo
excellent content validity.
Table 8 presents the calculation of the probability of chance agreement (PC) and Kappa statistic (K) at the second round of judgment. Table 8. Calculation of PC and K *
Items
Relevant (rating 3–4)
ICVI
1 1.1 1.2 1.3 1.4 2 2.1 2.2 2.3 2.4 3 3.1 3.2
12 12 12 13 13 13 12 12 13 13 13 13 13
0.86 0.86 0.86 0.93 0.93 0.93 0.86 0.86 0.93 0.93 0.93 0.93 0.93
**
PC
0.022 0.022 0.022 0 0 0 0.022 0.022 0 0 0 0 0
***
K
0.85 0.85 0.85 0.93 0.93 0.93 0.85 0.85 0.93 0.93 0.93 0.93 0.93
Interpretation Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent
18
Journal Pre-proof 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
13 13 13 12 12 12 13 13 13 13 13 13
0.93 0.93 0.93 0.86 0.86 0.86 0.93 0.93 0.93 0.93 0.93 0.93
0 0 0 0.022 0.022 0.022 0 0 0 0 0 0
0.93 0.93 0.93 0.85 0.85 0.85 0.93 0.93 0.93 0.93 0.93 0.93
Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent
*I-CVIs: item-level content validity index, **PC (Probability of a chance agreement) was computed using the formula: P C = [N! /A! (N - A)!] *.5N, where N = number of experts and A = number of panelists who agree that the item is relevant. Number of experts = 14 (N = 14), ***K(Kappa statistic) was computed using the formula: K = (ICVI - PC)/(1 - PC). Interpretation criteria for Kappa, Fair = K of 0.40 to 0.59; Good = K of 0.60 to 0.74; and Excellent=K>0.74
ro of
According to Table 8, the results have shown that the assessment tool has excellent inter-
-p
rater reliability at the item level (K>0.74).
5. Conclusions
re
The paper presents a process to develop and validate the assessment tools on the coastal
lP
cities’ resilience toward the impacts of climate change-induced rise in sea level coupled with coastal hazards, by using a two-step method. Both called for a comprehensive literature
na
review, items creation, and expert judgment about the items’ and the entire instrument’s
ur
validity. First was the developmental stage which assessed four of existing assessment tools
Jo
and identified the parameters, variables, and indicators of the new assessment tools. Second was the judgment stage which calculated the content validity of the assessment tools, using CVR and CVI that involved 14 local and national experts. From a set of 25 items, the content validity process identified five parameters, including institutional/ governance, environmental, social economic, built environment and infrastructure. From the expert judgments, 23 items were judged as having excellent content validity (ICVI higher than 0.79). There were two items with unacceptable ICVI that need for revision. The items that need revision have been modified according to the recommendations of the expert panel. The overall SCVI of the instrument was equal to 0.9 (SCVI /Ave) and 0.92
19
Journal Pre-proof (SCVI /UA). In addition, the calculation of the Kappa statistic showed that the assessment tool have excellent inter-rater reliability at the item level (K>0.74). This study revealed that this instrument had gained an appropriate level of content validity. The results of this study support the content validity of the assessment tool to measure the coastal cities’ resilience. This assessment tool is the first validated to be the practical tool for use to assess coastal cities’ resilience toward impacts of climate changeinduced rise in sea level and coastal hazards. This assessment tool is intended for use by
ro of
government, non-government, practitioner and academicians in sustaining long-term,
-p
consistent efforts in building coastal cities’ resilience.
[4]
[5]
[6]
[7] [8]
[9]
lP
na
[3]
ur
[2]
McMichael, A. J. 2014. Earth as humans’ habitat: global climate change and the health of populations, International journal of health policy and management, 2(1): 9. doi:10.15171/ijhpm.2014.03. Levy, B. S., and J. A. Patz. 2015. Climate change, human rights, and social justice. Annals of global health, 81 (3):310-322. doi:10.1016/j.aogh.2015.08.008. UNESCO. 2012. Coastal Management Approaches for Sea-level Related Hazards: Case Studies and Good Practices. (Intergovernmental Oceanographic Commission (IOC) Manuals and Guides, 61) 46 pp. (IOC/2012/MG/61Rev). Neumann, B., A. T. Vafeidis, J. Zimmermann, and R. J. Nicholls. 2015. Future coastal population growth and exposure to sea-level rise and coastal flooding-a global assessment. PloS one, 10 (3): e0118571. doi:10.1371/journal.pone.0118571. Bevacqua, A., D. Yu and Y. Zhang. 2018. Coastal vulnerability: Evolving concepts in understanding vulnerable people and places. Environmental Science & Policy, 82:1929. doi:10.1016/j.envsci.2018.01.006. Nicholls R. J., S. Hanson, C. Herweijer, N. Patmore, S. Hallegatte, J. Corfee-Morlot, J. Chateau, and R. Muir-Wood. 2007. Ranking port cities with high exposure and vulnerability to climate extremes—exposure estimates. Environmental Working Paper No 1, Organisation for Economic Co-operation and Development (OECD), Paris. Barbier, E. B. 2014. A global strategy for protecting vulnerable coastal populations. Science, 345 (6202):1250-1251. doi:10.1126/science.1254629. Mimura, N. 2013. Sea-level rise caused by climate change and its implications for society. Proceedings of the Japan Academy. Series B, Physical and Biological Sciences, 89(7): 281–301. doi:10.2183/pjab.89.281. Third National Communication (TNC). Under the United Nations Framework Convention on Climate Change. 2017. Ministry of Environment and Forestry, Republic of Indonesia.
Jo
[1]
re
References
20
Journal Pre-proof
Jo
ur
na
lP
re
-p
ro of
[10] Case, M., F. Ardiansyah and E. Spector. 2007. Climate change in Indonesia: implications for humans and nature. Climate change in Indonesia: implications for humans and nature. [11] USIOTWSP. 2007. How resilient is your coastal community? A guide for evaluating coastal community resilience to tsunamis and other hazards. In: U.S. Indian Ocean Tsunami Warning System Program Supported by the United States Agency for International Development and Partners, Bangkok, Thailand. [12] Howes, M. 2015. Disaster risk management and climate change adaptation: A new approach. In Applied studies in climate adaptation, ed. J. P. Palutikof, S. L. Boulter, J. Bernett, and D. Rissik, 407–415. Oxford: Wiley. [13] O'Brien, G., and P. Read. 2005. Future UK emergency management: New wine, old skin? Disaster Prevention and Management, 14(3): 353–361. [14] Rivera, C. 2014. Integrating Climate Change Adaptation into Disaster Risk Reduction in Urban Contexts: Perception and Practice. Ploss Curr. Disasters. doi:10.1371%2Fcurrents.dis.7bfa59d37f7f59abc238462d53fbb41f. [15] Few, R., H. Osbahr, L. Bouwer, V. David, and F. Sperling. 2006. Linking Climate Change Adaptation and Disaster Risk Management for Sustainable Poverty Reduction, VARG Synthesis Report. [16] Mitchell, T., and M. Van Aalst. 2008. Convergence of Disaster Risk Reduction and Climate Change Adaptation. Review for DFID, UK. [17] Torabi, E., A. Dedekorkut-Howes, and M. Howes. 2018. Adapting or maladapting: Building resilience to climate-related disaster in coastal cities. Cities, 72: 295-309. doi:10.1016/j.cities.2017.09.008. [18] UNISDR. 2009. Terminology on Disaster Risk Reduction. United Nations International Strategy for Disaster Risk Reduction, Geneva. [19] Alexander, D. E. 2013. Resilience and disaster risk reduction: an etymological journey. Natural hazards and earth system sciences, 13 (11):2707-2716. doi:10.5194/nhess-13-2707-2013. [20] Rose, A. 2007. Economic resilience to natural and man-made disasters: Multidisciplinary origins and contextual dimensions. Environmental Hazards, 7(4):383-398. doi:10.1016/j.envhaz.2007.10.001. [21] Plodinec, M. J. 2009. Definitions of resilience: An analysis. Oak Ridge: Community and Regional Resilience Institute (CARRI). [22] Folke, C., S. Carpenter, T. Elmqvist, L. Gunderson, C. S. Holling, and B. Walker. 2002. Resilience and sustainable development: building adaptive capacity in a world of transformations. AMBIO: A journal of the human environment, 31(5): 437-440. doi:10.1579/0044-7447-31.5.437. [23] Tompkins, E., and W. N. Adger. 2004. Does adaptive management of natural resources enhance resilience to climate change?. Ecology and society, 9(2). [24] Dolan, A. H., and I. J. Walker. 2006. Understanding vulnerability of coastal communities to climate change related risks. Journal of Coastal Research, 3 (39): 1316-1323. [25] Rahman, M. A., and S. Rahman. 2015. Natural and traditional defense mechanisms to reduce climate risks in coastal zones of Bangladesh. Weather and Climate Extremes, 7, 84-95. doi:10.1016/j.wace.2014.12.004. [26] Cutter, S. L., and H. Director. 2008. A framework for measuring coastal hazard resilience in New Jersey communities. White Paper for the Urban Coast Institute. [27] Lawshe, C. 1975. A qualitative approach to content validity. Personnel Psychology, 25: 563–575. doi:10.1111/j.1744-6570.1975.tb01393.x.
21
Journal Pre-proof
Jo
ur
na
lP
re
-p
ro of
[28] DeVon, H. A., M. E. Block, P. Moyle-Wright, D. M. Ernst, S. J. Hayden, and D. J. Lazzara. 2007. A psychometric Toolbox for testing Validity and Reliability. Journal of Nursing scholarship, 39 (2): 155-164. doi:10.1111/j.1547-5069.2007.00161.x. [29] Pellikka, J. 2008. Innovation support services and commercialisation process of innovation in small technology firms. International Journal of Innovation and Regional Development, 1(3): 319-334. [30] Bolarinwa, O. A. 2015. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgraduate Medical Journal, 22 (4):195. doi:10.4103/1117-1936.173959. [31] Gilbert, G. E., & Prion, S. 2016. Making sense of methods and measurement: Lawshe's Content Validity Index. Clinical Simulation in Nursing, 12 (12): 530-531. doi:10.1016/j.ecns.2016.08.002. [32] Rodrigues, I. B., J. D. Adachi, K. A. Beattie, and J. C. MacDermid. 2017. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC musculoskeletal disorders, 18(1): 540. doi:10.1186/s12891-017-1914-5. [33] Wynd, C. A., B. Schmidt, and M. A. Schaefer. 2003. Two quantitative approaches for estimating content validity. Western Journal of Nursing Research, 25(5): 508-518. doi:10.1177/0193945903252998. [34] Cicchetti, D. V., and S. A. Sparrow. 1981. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. American journal of mental deficiency. [35] Schipper, E. L. F., and L. Langston. 2015. A comparative overview of resilience measurement frameworks. Analysing indicators and approaches. Overseas Development Institute. [36] Sharifi, A. 2016. A critical review of selected tools for assessing community resilience. Ecological Indicators, 69: 629-647. doi:10.1016/j.ecolind.2016.05.023. [37] TRF. 2014. City Resilience Index: City Resilience Framework. The Rockefeller Foundation and ARUP. [38] UNISDR. 2015. Disaster Resilience Scorecard for Cities. United Nations Office for Disaster Risk Reduction. [39] Shaw, R., Y. Takeuchi, J. Joerin, and G. Fernandez. 2010. Climate and Disaster Resilience Initiative Capacity-Building Program. Kyoto University, Kyoto, Japan. [40] Rubinoff, P., and C. Courtney. 2007. How resilient is your coastal community? A guide for evaluating coastal community resilience to tsunamis and other coastal hazards. Basins and Coasts News, Integrated Management for Coastal and Freshwater Systems. United States Agency for International Development, 2(1): 24-28. Chicago. [41] KKP .2012. Panduan Penilaian Kondisi Ketangguhan Desa Pesisir. Direktorat Pesisir dan Lautan. Kementerian Kelautan dan Perikanan. [42] UNISDR. 2017. Disaster Resilience Scorecard for Cities. United Nations. [43] BNPB. 2012. Perka BNPB No. 1 Tahun 2012 tentang Pedoman Umum Desa/ Kelurahan Tangguh Bencana. [44] Colton, D., and R. W. Covert. 2007. Designing and constructing instruments for social research and evaluation. New Jersey: Wiley. [45] DasGupta, R., and R. Shaw. 2015. An indicator based approach to assess coastal communities’ resilience against climate related disasters in Indian Sundarbans. Journal of Child and Family Studies, 24 (3):85-101. doi:10.1007/s11852-014-0369-1. [46] Tompkins, E. L., Few, R., & Brown, K. (2008). Scenario-based stakeholder engagement: incorporating stakeholders preferences into coastal planning for climate
22
Journal Pre-proof
ro of -p re lP
[50]
na
[49]
ur
[48]
Jo
[47]
change. Journal of environmental management, 88(4): 1580-1592. doi:10.1016/j.jenvman.2007.07.025. Miamidian, E., M. Arnold, M. Jacquand, and K. Burritt. 2005. Surviving Disasters and Supporting Recovery: A Guidebook for Microfinance Institutions. Huynh, L. T. M., and L. C. Stringer. 2018. Multi-scale assessment of social vulnerability to climate change: An empirical study in coastal Vietnam. Climate Risk Management. doi:10.1016/j.crm.2018.02.003. Wells, S., and C. Ravilious. 2006. In the front line: shoreline protection and other ecosystem services from mangroves and coral reefs (No. 24). UNEP/Earthprint. Joerin, J., R. Shaw, Y. Takeuchi, and R. Krishnamurthy. 2012. Assessing community resilience to climate-related disasters in Chennai, India. International Journal of Disaster Risk Reduction, 1: 44-54. doi:10.1016/j.ijdrr.2012.05.006.
23
Journal Pre-proof Declaration of interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Jo
ur
na
lP
re
-p
ro of
☐The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:
24
Figure 1
Figure 2
Rina Suryani Oktari, Louise K. Comfort, Syamsidik, Putra Dwitama PII:
S2590-0617(19)30057-2
DOI:
https://doi.org/10.1016/j.pdisas.2019.100057
Reference:
PDISAS 100057
To appear in: Received date:
16 June 2019
Revised date:
2 November 2019
Accepted date:
5 November 2019
Please cite this article as: R.S. Oktari, L.K. Comfort, Syamsidik, et al., Measuring coastal cities' resilience toward coastal hazards: Instrument development and validation, (2019), https://doi.org/10.1016/j.pdisas.2019.100057
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
© 2019 Published by Elsevier.
Journal Pre-proof
Measuring Coastal Cities’ Resilience toward Coastal Hazards: Instrument Development and Validation Rina Suryani Oktari a*, Louise K Comfort b, Syamsidik c, Putra Dwitama d and Disaster Mitigation Research Center (TDMRC), Graduate School of Mathematics and Applied Science, and Faculty of Medicine, Universitas Syiah Kuala, Jl. Tgk. Syech Abdul Rauf, Banda Aceh, 23111 Indonesia, email; [email protected], b Center for Disaster Management (CDM) and Graduate School of Public and International Affairs (GSPIA), University of Pittsburgh, 3601 Wesley W. Posvar Hall, Pittsburgh, Pennsylvania 15260, email: [email protected] c Tsunami and Disaster Mitigation Research Center (TDMRC) and Civil Engineering Department, Faculty of Engineering, Universitas Syiah Kuala, Jl. Tgk. Syech Abdul Rauf No. 7, Banda Aceh, 23111 Indonesia, email; [email protected] d National Action Plans-Climate Resilience Secretariat (RAN API), Wisma Bakrie 2, 6th floor, Jl. H.R. Rasuna Said Kavling B-2, Setiabudi, Jakarta Selatan, 12920 Indonesia, email; [email protected]
-p
ro of
a* Tsunami
re
ABSTRACT
lP
This study aims to design and validate a comprehensive assessment tool that measures the coastal cities’ resilience toward coastal hazards. Instrument development process adhered to
na
the research and development methodology that involves: i) literature review, ii) design of
ur
the assessment tool, and iii) instrument validity test using Content Validity Ratio (CVR) and
Jo
Content Validity Index (CVI). Fourteen experts were involved in the validity test. This study identified five aspects of the resilience: institutional/ governance, environmental, social economic, built environment, and infrastructure. From the expert judgments, 23 items were judged as having excellent content validity (ICVI higher than 0.79). There were two items with unacceptable ICVI that need for revision. The items that need revision have been modified according to the recommendations of the expert panel. The overall SCVI of the instrument was equal to 0.9 (SCVI /Ave) and 0.92 (SCVI /UA). In addition, the calculation of the Kappa statistic showed that the assessment tool have excellent inter-rater reliability at the item level (K>0.74). This study revealed that this instrument had gained an appropriate level of validity to measure the coastal cities’ resilience. 1
Journal Pre-proof
ro of
KEYWORDS: Coastal resilience, assessment tool, content validity.
1. Introduction
-p
Global climate change, population growth, and human-induced vulnerability have caused
re
coastal communities around the world to experience significant rates of change [1], [2]. The effects of this change highlight the coastal area has a high degree of exposure to climate
lP
change-induced rise in sea level coupled with coastal hazards, including storm surges,
na
shoreline erosion, coastal flooding, and tsunami. These hazards pose significant threats to the physical, social, and economic aspect of the coastal communities [3], [4], [5].
ur
Presently, about 40 million people are exposed to a disaster with a frequency of once in
Jo
every 100-year period. This number will be tripled in the next 50 years [6]. Notwithstanding this condition, coastal cities in developing countries have to solve two types of problems, i.e., un-timely effective responds to disasters and degradation of coastal ecosystem that resulted in poor protection to the coastal communities [7]. Therefore, effective and long-term disaster and climate change risk management are required immediately. Developing effective risk management and mitigation to reduce the impacts of climate change will be essential to secure the sustainable development of coastal zones. Thus, there is a critical need to assess coastal resilience by providing the tools to better inform both scientific debates and policymakers in improving the mitigation and adaptation strategies of
2
Journal Pre-proof the coastal cities toward climate change-induced rise in sea level coupled with coastal hazards [3]. In the 21st century, Asia and Africa are projected to be a center for population growth, economic development, and urbanization in the coastal area, as well as small islands, are particularly vulnerable to sea-level rise and climate change [4], [8]. With more than 17,000 islands, Indonesia is highly vulnerable to the impacts of global climate change because of its geographical location. The average temperature is projected to increase by 0.5–3.92°C by
ro of
2100 compared to the current period (1981–2010). Climate change-induced rise in sea level is expected to reach 35–40 cm by 2050 relative to the value of 2000. Models predict that
-p
taking into account the melting ice factor at the North and South Pole, sea levels will rise
re
about 5 cm in the year 2100. Almost 65% of the population that live in the coastal areas of
sea level coastal flooding [9].
lP
Indonesia is vulnerable to the impacts of climate change, especially that caused by a rise in
na
Considering the high vulnerability to climate change impact, Indonesia should enhance its capacity to mitigate and to adapt to climate change, especially in the most vulnerable
ur
communities [10]. As part of climate change adaptation in the coastal area, to ensure the
Jo
protection of the natural environment that provides the basis for carrying out the various human activities in the coastal area, an integrated approach to coastal management is essential [11]. An increase in the frequency and severity of climate-related disaster is likely because of the impacts of climate change, making it essential for many countries and also a global need to increase resilience, especially in the coastal area [7]. The word “resilience” has been accepted as a synthesis to combine Climate Change Adaption (CCA) and Disaster Risk Reduction (DRR) [12], [13]. Disaster risk reduction should include measures to adapt to the impacts of climate change. However, there is lack of integration between the concepts of DRR and CCA [14], [15], [16]. Coastal hazards, such as
3
Journal Pre-proof tsunami, coastal flooding, and coastal erosion, are rather faster on-set disasters than the impacts of climate change-induced rise in sea level. This resulted in a lack of understanding and a lack of preparedness of coastal cities to integrate CCA and DRR into one comprehensive long-term development plan [17]. Combining the two concepts into coastal cities’ resilience is a challenging work for most of the cases. Stakeholders who work on CCA and DRR often have different development agendas and gather in different forums [16]. Inter-connection between the two issues is infrequent. This makes it difficult to assess the
ro of
coastal cities’ resilience toward the impacts of climate change coupled with certain types of coastal hazards. Another reason is also because of the absence of an assessment tool that
-p
enables the evaluation of the coastal cities’ resilience toward the impacts of climate change
re
coupled with coastal hazards.
The aforementioned condition motivated this study. The objective of this study is to
lP
develop a comprehensive assessment tool on the coastal cities’ resilience toward impacts of
ur
flooding, and tsunami).
na
climate change-induced rise in sea level coupled with coastal hazards (erosion, coastal
Jo
2. Concept of Community Resilience applied to Coastal Areas The definitions of resilience that is relevant to communities reflect the adaptive capacity of a community or a system to manage the disturbance of an adverse event or crisis situation [18], [19]. In addition, resilience provides the capability of the community to recover from the adversity and to mitigate future impacts [20], [21]. There are three characteristics of the resilience of socio-ecological systems: i) the magnitude of the adverse events that the community can absorb shocks while maintaining function, ii) the degree to which the community is capable of self-organization, and iii) the degree to which the community can build capacity to learn and adapt [22]. Despite the fact
4
Journal Pre-proof that human systems are naturally resilient, building resilience into a human-environment system is an effective strategy to cope with change caused by future shocks or unknowable risk [23]. Coastal areas face the challenge of increasing hazards driven by environmental change, unavoidable climate change impact and by human activities. To respond and to cope with the uncertainty surrounding the rate and extent of the rise in sea level and other climate-related hazards arising as a result of the global environmental change, the delivery of adaptive
ro of
responses to coastal management will be required [24], [25].
In fostering coastal resilience, it is important to balance environmental and development
-p
issues while promoting safe and livable communities. From the social perspective, the
re
resilience of the coastal communities can be enhanced through social networks, community engagement and participation, insurance, and access to other financial resources. In the built
lP
environment aspect, improvement in construction practices, building design and practices are
na
some measures that enhance resilience. Some other factors that enhance resilience include the
3. Method
Jo
etc.) [26].
ur
availability of public infrastructure and lifelines (such as roads, bridges, electricity, water,
The development process of the assessment tool adhered to the research and development methodology that involves three steps. First, preliminary studies were conducted to collect information via literature review on various resilience assessment tools. In the second step, the design of the assessment tool was prepared to determine the purpose of the instrument, which the users are, as well as the identification of the parameters, variables, and indicators of the assessment tool. The third step was the validity test of the assessment tool using Content Validity Ratio (CVR) and Content Validity Index (CVI) [27], [28], [29]
Step 1: Preliminary Studies Development stage
Literature review Literature
4 assessment tools reviewed
parameters,
5
ro of
Journal Pre-proof
Figure 1. The flowchart of research methodology
3.1 Experts in the Validity study
-p
Content validity will show that the contents reflect the complete range of attributes studied
re
and is usually done by seven or more experts [28]. Participants in this study consisted of 14 local and national experts, including from academia (3 people), government (7 people) and
lP
practitioners (4 people). Table 1 below shows the details of participants.
Category
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Government Academic Government Academic Practitioner Practitioner Government Government Government Academic Practitioner Government Practitioner Government
Level of Institution
Gender
Education
Local National Local National Local National National Local Local National National Local Local Local
Female Male Male Male Female Female Female Female Male Male Male Male Male Male
Bachelor Doctorate Master Doctorate Bachelor Master Master Master Master Doctorate Master Master Master Bachelor
Jo
ur
Expert
na
Table 1. The distribution of experts in the study
The experts involved were stakeholders representing policymakers, practitioners in the field, and academics who have focused on the areas of disaster and climate change. By conducting 6
Journal Pre-proof FGD that brought together experts from a variety of different backgrounds, communication between science-policy-practice was achieved, thereby strengthening the validity of the parameters and indicators in the assessment tools.
3.2 Data Validation The content validity study combined qualitative and quantitative method by conducting expert judgments through Focus Group Discussion (FGD) which was held in September 2017
ro of
in Banda Aceh, Indonesia. The FGD was aimed at assessing whether the contents of the parameters and variables of the assessment tool are appropriate and relevant for the purpose
-p
of the tool development. Each expert was supplied the list of evidence-based items
re
(parameters and variables) chosen by the researcher. The experts were requested to use the
lP
scoring method shown in Table 2 below to rate each item in the assessment tool.
Table 2. The scale used for rating items (DeVon et al. 2007)
ur
not relevant item needs some revision relevant but need minor revision very relevant
Essentiality not necessary useful but not essential essential
Jo
1 2 3 4
Relevancy
na
Score
To estimate the content validity of the assessment tool we use Lawshe’s CVR and CVI [30], [31]. The CVR was calculated using the level of agreement for each item’s inclusion through each expert’s “essential” rating. The CVR was calculated using Equation (1) as follows. 𝐶𝑉𝑅 =
(𝑁𝑒 −𝑁/2) (𝑁/2)
,
(1)
where Ne = the number of experts indicating “essential” ratings per item and N = total number of experts.
7
Journal Pre-proof According to Table 3, a CVR greater than 0.51 for 14 experts is considered acceptable. It means that the item is maintained in the instrument. If the CVR value is smaller than 0.51, we should remove the item from the instrument [27]. Table 3. Minimum values of CVR for the different number of experts. (One-Tailed Test, P-value=0.05) (Lawshe 1975) Minimum CVR value
5 6 7 8 9 10 11 12 13 14 15 20 25 30 35 40
0.99 0.99 0.99 0.75 0.78 0.62 0.59 0.56 0.54 0.51 0.49 0.42 0.37 0.33 0.31 0.29
na
lP
re
-p
ro of
Number of experts
The CVI is the mean of the CVR values of items and represents the content validity of the
ur
entire instrument. There are two types of CVI: i) ICVI (item level) and ii) SCVI (scale-level).
Jo
The ICVI is defined as the proportion of agreement on the relevancy of each item. The following equation (2) was used to calculate the ICVI value. 𝐼𝐶𝑉𝐼 = 𝑛𝑎 /𝑁,
(2)
where ICVI is Item-Content Validity Index, and na is the number of experts giving rate 3 (relevant) and 4 (very relevant).
The SCVI is defined as the proportion of items in an instrument judged content valid. There are two approaches to calculate the SCVI. The first approach is the SCVI/UA (Universal Agreement) that calculates the proportion of items of the instrument that achieved rate 3
8
Journal Pre-proof (relevant) and 4 (very relevant) by all the experts. The second approach is the SCVI/Ave (Average) that calculates the ICVI for each item in the instrument and calculates the average ICVI across the items. The instrument will gain an excellent content validity if the SCVI /UA value ≥ 0.8 and the SCVI /Ave value ≥ 0.9 [31], [32]. As an important supplement for CVI, the multirater kappa statistic will be also calculated in the content validity study, as it provides information about the degree of agreement that adjusts for chance agreement. Chance agreement is very important while studying agreement
ro of
indices among judges, especially when there are four-scale scoring within two relevant and not relevant classes [33].
-p
To calculate the Kappa Statistic, we should calculate the probability of chance agreement
𝑁!
] 5𝑁
𝐴!(𝑁−𝐴)!
(3)
lP
𝑃𝑐 = [
re
using the following formula (3).
where, Pc is probability of chance agreement and A is the number of experts who agree that
ur
na
the item is relevant.
Jo
The formula used to calculate the Kappa Statistic is as follow. 𝐾=
(𝐼𝐶𝑉𝐼 −𝑃𝐶 ) (1−𝑃𝐶 )
.
(4)
Here, K is the Kappa value.
The Kappa values will be classified according to four criteria: i) excellent, if the kappa value is larger than 0.74, ii) good, if kappa value is between 0.60 and 0.74, and iii) fair, if the kappa value is between 0.40 and 0.59, and iv) poor, if the kappa value is smaller than 0.40 [33], [34].
4. Results and Discussion 9
Journal Pre-proof During the process of the development of the assessment tool, we found that most of the parameters we proposed were valid and relevant to assess the coastal cities’ resilience. In this section, we present the discussions of the results. Section 4.1 discusses the studies that have been done to review the four of existing resilience assessment tools. Section 4.2 explains the development of the assessment tool, including the identification of instrument objectives and items generation of the assessment tool (parameters, variables, and indicators). Section 4.3 presents the results of the content validity study containing the calculation of CVR, CVI, and
ro of
Kappa Statistic (K).
-p
4.1 Review of Existing Resilience Assessment Tools
Preliminary studies were conducted to collect information (via literature review, field
re
observations, and interviews) on various resilience assessment tools.
lP
All these tools have been developed mainly by international organizations, and there is
na
still a lack of tools developed by local authorities that reflect the local needs and conditions. Table 4 below describes information on the selected tools.
Name of Assessment Tools
Jo
ur
Table 4. Basic information on the selected resilience assessment tools (TRF 2014; UNISDR 2015; Shaw et al. 2010; Rubinoff and Courtney 2007) City Resilience Framework (CRF)
UNISDR City Resilience Scorecard V.2.2
Year Format Indicators
2014 Toolkit Infrastructure and environment, leadership and strategy, health and wellbeing, economy and society (12 Goals, 52 indicators and 156 variables) Literature review, stakeholder
2015 Scorecard “Ten Essentials” which cover governance and financial issues (Essentials 1–3); many dimensions of planning and disaster preparation (Essentials 4–8); the disaster response itself and post-event recovery (Essentials 9–10).
Development Method
Literature review
Climate and Disaster Resilience Index (CDRI) 2010 Toolkit Social, physical, economic, institutional, natural
Expert Opinions
NOAA Coastal Community Resilience Guide 2007 Scorecard Governance, society & economy, coastal resource management, land use & structural design, risk knowledge, warning & evacuation, emergency response, disaster recovery. Workshops and discussions with practitioners,
10
Journal Pre-proof input, field testing
Developer
Type of Risk Target Audience Equal Weighting
The Rockefeller Foundation
BM and AECOM
Academia
Multiple
Natural
Multiple
Local Authorities
Local authorities, insurance companies, private industry Yes
Community leaders/local authorities No
Yes
specialists from government agencies and organizations U.S. Indian Ocean Tsunami Warning System Program Tsunami and coastal hazard Local authorities, community leaders, coastal managers Yes
ro of
Most of the tools were designed to address multiple hazards. However, none of them attempt to combine the assessment on impacts of climate change-induced rise in sea level
-p
coupled with the impacts on coastal hazards. The target audiences of the tools are mainly for
re
the local authorities, community leaders, community members, insurance companies, private industry, and public (both individual and household). Three major formats are used in the
lP
assessment tools, including toolkit, scorecard, and index. The analysis shows different
na
resilience aspects addressed by each of the assessment tools. The tools have treated the
ur
resilience aspects as being equally important [35], [36].
Jo
4.2 Design of the Assessment Tool Originality and Practical Implications Despite the existence of several resilience assessment tools, this assessment tool is the first validated to be the practical tool for use to assess coastal cities’ resilience toward impacts of climate change-induced rise in sea level and coastal hazards. There are four objectives of the assessment tool on the coastal cities’ resilience toward the impacts of climate change-induced rise in sea level coupled with coastal hazards. First, is to have a sustainability assessment for continuous improvement on coastal cities’ resilience toward coastal hazards. Second, is to acquire/ identify the current situation of cities’
11
Journal Pre-proof resilience to improve the effectiveness of the existing strategies to increase cities’ resilience, make priorities and action plans (planning) as well as in performance evaluation (monitoring and evaluation) of the coastal cities’ resilience toward coastal hazards. Third, is to provide input to the policy or program in achieving sustainable development through identification of hazards that may hinder development and increase resilience toward the hazards. Fourth, is to provide feedbacks on long-term development policies of coastal cities’ to enhance resilience toward the impacts of climate change-induced rise in sea level and coastal hazards.
ro of
In practical terms, this assessment tool is intended for use by different stakeholder groups in sustaining long-term, consistent efforts in building coastal cities’ resilience. Among the
-p
stakeholders are: i) national and local government agencies; ii) practitioners working with
re
coastal community, iii) International aid agencies, donors, private sectors, and iv)
lP
Academicians.
Parameters, variables, and indicators of the instruments
na
A previous study has proposed many parameters/ indicators to measure the resilience. The
ur
Rockefeller Foundation (TRF) suggested that resilience includes the following parameters, namely: i) infrastructure and environment, ii) leadership and strategy, iii) health and
Jo
wellbeing, and iv) economy and society with a total of 12 goals, 52 indicators, and 156 variables [37]. UNISDR (2015) has developed a City Resilience Scorecard V.2.2 that proposed “Ten Essentials” which cover governance and financial issues (Essentials 1–3); many dimensions of planning and disaster preparation (Essentials 4–8); the disaster response itself and post-event recovery (Essentials 9–10) [38]. In addition, Shaw et al. (2010) proposed the following resilience indicators: social, physical, economic, institutional, and natural [39]. While Rubinoff and Courtney (2007) included some indicators in the Coastal Community Resilience Guide; they were governance, society & economy, coastal resource management,
12
Journal Pre-proof land use & structural design, risk knowledge, warning & evacuation, emergency response, and disaster recovery [40]. Following a thorough review of the parameters used by the assessment tools, we identified eight parameters namely: i) institutional/ governance, ii) social and economic, iii) coastal resource management, iv) land use management and infrastructure, v) risk knowledge, vi) warning and evacuation, vii) emergency response, and viii) disaster recovery [40], [41], [42], [43]. The variables and indicators of each resilience parameter have been selected from
ro of
indicators proposed by previous studies. We have conducted FGD to discuss this initial design of the assessment tool to obtain expert judgments.
-p
Based on the comments and inputs of experts, we extracted at least five resilience
re
parameters, including: i) institutional/ governance, ii) social and economic, iii) coastal resource management, iv) land use management and infrastructure, and v) adaptation and
lP
mitigation strategies.
na
Figure 2 shows a linear hierarchy of the parameters and variables of the coastal cities’
Jo
ur
resilience toward coastal hazard.
13
Journal Pre-proof
COASTAL CITIES' RESILIENCE
1. Institutional/ governance
2. Social and Economic
3. Coastal Resource Management
4. Land Use Management and Infrastructure
5. Adaptation and Mitigation Strategies
2.1 Social capital and skills
3.1 Implementation and monitoring of coastal resources
4.1 Land use policy and building standards
5.1 Risk knowledge
1.2 Basic services
2.2 Livelihood
3.2 Protection of habitats, ecosystems and sensitive natural features
4.2 Structural design
5.2 Early warning and evacuation
1.3 Participatory technical cooperation
2.3 Social and cultural networks
1.4 Technical and financial support
2.4 Economic stability
3.3 Community involvement in planning, implementation and monitoring
4.3 Mainstreaming risk reduction to location and structural design
5.3 Emergency response
ro of
1.1 Policies, plans and programs
5.4 Resource mobilisation
4.4 Education, research, and training
-p
3.4 Investment in management and conservation
re
Figure 2. Linear Hierarchy of the Coastal Cities’ Resilience toward Coastal Hazard
lP
Table 5 presents the item generation for each parameter and variable of the assessment
primary and secondary data.
na
tool. The data and information required to address each indicator in the instrument consist of
ur
Primary data is collected through focus group discussion (FGD), direct interviews and questionnaire surveys. Secondary data is obtained through literature review and an
Jo
institutional visit to gather data already produced or collected by the government department. Before collecting primary data, it is essential to analyse relevant secondary data exist.
Table 5. Indicators of Assessment Tool for Each Parameter/ Variable
Parameter 1: Institutional/Governance
No 1.1
1.2
1.3
Parameter/ Variable Policies, plans, and programs Indicators: i) The existence of policy/ regulations to protect coastal areas, ii) The existence of planning documents for coastal protection, iii) The existence of policies to control the movement and settlement, and iv) The existence of integration of climate change and disaster risk management into development policy. Basic services Indicators: i) Percentage of household that have access to electricity, ii) Percentage of household with access to clean water, iii) Percentage of household that have access to sanitation facilities and solid waste, and iv) Number of public transport. Participatory technical cooperation Indicators: i) The existence of forums/ working groups to undertake programs to enhance the resilience of coastal areas, ii) The existence of cooperation with various stakeholders including NGOs, private parties, academics, etc., iii) The coastal resource management program has included risk reduction issues, and iv) The socio-
References [40], [41], [42], [43]
[42], [43], [45]
[40], [41], [46], [47]
14
Journal Pre-proof
2.2
2.3
2.4
[40], [41], [42], [43]
[42], [43], [47], [48]
[42], [43], [47]
[42], [43], [45], [48]
[42], [43], [45], [47]
[17], [40], [41]
lP
3.4
na
3.3
ur
3.2
Jo
Parameter 3: Coastal Resource Management
3.1
re
-p
Parameter 2: Social and Economic
2.1
economic development program has included risk reduction issues. Technical and financial support Indicators: i) The existence of routine budget allocations as well as assistance to support activities that can reduce the risk of damage caused by coastal hazards, ii) Community leaders have the resources and tools to build community resilience in dayto-day activities, iii) Village budgets have included priorities for managing, upgrading, and mitigating critical facilities and infrastructure, and iv) Government has identified alternatives for increasing the village budget. Social capital and skills Indicators: i) The existence of economic development efforts to reduce the vulnerability of the community, ii) The existence of a program to provide skills to the community for alternative livelihoods, iii) The availability of social safety networks to assist vulnerable sectors of society, and iv) The availability of social safety networks to assist vulnerable sectors of society. Livelihood Indicators: i) Percentage of the community working in the agricultural sector, ii) Percentage of the community working in the fishery sector, iii) Percentage of the community working in the trade sector, and iv) The existence of established strategies to address the economic recovery caused by the disaster. Social and cultural networks Indicators: i) The existence of social and cultural networks involving community, cultural, private, and other non-governmental groups that support activities to increase community resilience, ii) The existence of social networks or community groups that can help during and post disaster, iii) The existence of a problem-solving mechanism to create a peaceful and orderly society, and iv) The mechanisms used to increase community participation in development planning. Economic stability Indicators: i) The existence of technical resources that provide assistance to the community in the diversification of environmentally friendly livelihoods (such as from universities, government programs, donor projects, etc.), ii) Availability of grant aid, technical assistance or loans to develop business alternatives, iii) The development/ financing of micro enterprises has been undertaken to provide sustainable livelihood alternatives, and iv) Insurance services are available in the event of a loss of business production in the event of a disaster. Implementation and monitoring of coastal resources Indicators: i) The existence of mechanism for assessing coastal resources and hazards on a regular basis, ii) The assessment results were used to identify risks in the community and become inputs for planning the management of coastal resources, iii) The existence of feedback mechanism for updating coastal resource management plans, and iv) The existence of community-based forums involved in solving problems in coastal areas. Protection of habitats, ecosystems, and sensitive natural features Indicators: i) Forest cover/ water catchment area, ii) Mangrove forest cover, iii) Coral Reefs cover, and iv) The existence of regulations governing the utilization of natural resources based on priority conservation and risk reduction. Community involvement in planning, implementation and monitoring Indicators: i) The existence of procedure for reviewing plans based on coastal issues and community feedback, ii) Communities are actively involved in the planning of coastal protection programs, iii) Communities are actively involved in the implementation of coastal protection programs, and iv) Communities are actively involved in monitoring coastal protection programs. Investment in management and conservation Indicators: i) The government has undertaken natural resource management, such as forests, rivers, beaches etc., to reduce risks, ii) The existence of rehabilitation and conservation activities of coastal ecosystems (Mangrove/coral reefs, etc.), iii) The existence of coastal ecosystem management group (coral reef/mangrove, etc.), and iv) The existence of monitoring activities of coastal ecosystem condition.
ro of
1.4
[17], [49]
[40], [41], [42]
[40], [41], [49]
Table 5. (Continued) Indicators of Assessment Tool for Each Parameter/ Variable No Parameter 4: Land Use Management and Infrastructure
4.1
4.2
4.3
Parameters/ Variables Land use policy and building standards Indicators: i) Percentage of concrete roads, ii) Level of pollution in coastal areas, iii) Condition of residential building (permanent/ semi-permanent), and iv) Percentage of houses across the coastline. Structural design Indicators: i) Percentage of built-up land, ii) Percentage of public facilities located outside of risk areas (schools, houses of worship, government offices, health facilities), iii) Siting and design for housing, hospitals and other critical infrastructure are based on land use plans and coastal hazard risk assessments, and iv) Coastal engineering structures have been designed to reduce vulnerability to coastal hazards and minimize impacts on coastal habitats. Mainstreaming risk reduction to location and structural design Indicators: i) Builders and architects have an understanding and are able to apply building
References [40], [41]
[40], [42], [43]
[26], [42], [43]
15
Journal Pre-proof
5.2
5.3
[42], [43], [50]
[42], [43], [50]
[42], [43], [50]
[42], [43], [50]
-p
5.4
[26], [42], [43]
re
Parameter 5: Adaptation & Mitigation Strategies
5.1
ro of
4.4
standards that integrate risk reduction, ii) Designer/ structural expert has an understanding and ability to design and build a secure infrastructure, iii) Building standards in hazard areas have been adopted by adjusting location, design and building infrastructure, and iv) The existence of socialization program on building design and practices that integrates risk reduction has reached the community. Education, research, and training Indicators: i) The existence of training/ counseling and dissemination of information to the community about land use and building standards, ii) The existence of training for builders, architects and contractors, iii) The existence of a hazard mitigation certification program for architects and contractors, and iv) Universities/ training institutions have included curricula on land use policies, building standards, and hazard mitigation. Risk knowledge Indicators: i) The existence of mapping and analysis of threats, vulnerabilities, and capacity to identify risk, ii) Public access to information from the analysis of threats, vulnerabilities, capacities and risks, iii) The existence of physical development (mitigation) to reduce disaster risk in coastal areas, and iv) Risk knowledge of the community. Early warning and evacuation Indicators: i) Establishment of an early warning system to provide time for self-rescue and assets for the community, ii) The existence of disaster information facilities, iii) The existence of evacuation maps and routes, evacuation sites and shelter, and iv) Community access to early warning information and evacuation strategies. Emergency response Indicators: i) Availability of disaster evacuation routes, ii) Availability of evacuation shelter, iii) Availability of systems and mechanisms of distribution of resources/ assistance to the community after the disaster, and iv) Community Emergency Response Plan. Resource mobilization Indicators: i) The existence of protection of the community's main productive assets from the impact of the disaster, ii) The existence of capacity building of government personnel to carry out/ train and provide equipment and equipment, facilities and pre-facilities, logistics, and personnel, iii) The existence of contingency plans in the face of disaster, and iv) Resources mobilization capacity of community.
lP
4.3 Content Validity Test of the Assessment Tool Both the qualitative and quantitative approaches were used to assess the content validity test
na
of the assessment tool. The qualitative content validity of the assessment tool was assessed by
ur
14 experts. The experts were asked to assess and to comment on the wording, item allocation, and item scaling. The assessment tool was revised based on the feedbacks and comments
Jo
received. In the quantitative step, the assessment tool was assessed through the calculation of the CVR and CVI [44].
Accordingly, 14 experts were asked to rate the essentiality (for CVR) and the relevancy (for CVI) of the items in the assessment tool. Table 6 presents the calculation of CVR for the items of the assessment tool. Table 6. Calculation of CVR Items 1 1.1 1.2 1.3
Ne 12 12 11 13
CVR 0.71 0.71 0.57 0.86
Interpretation Remained Remained Remained Remained
16
1.4 2 2.1 2.2 2.3 2.4 3 3.1 3.2 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
13 13 12 11 13 13 13 13 13 13 13 13 12 12 12 13 13 13 13 13 13
0.86 0.86 0.71 0.57 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.71 0.71 0.71 0.86 0.86 0.86 0.86 0.86 0.86
ro of
Journal Pre-proof Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained Remained
re
-p
Ne=Number of experts evaluated the item essential, **CVR or Content Validity Ratio = (Ne-N/2)/(N/2) with the number of panelists 14 (N=14), the items with the CVR bigger than 0.51 remained at the instrument and the rest eliminated.
lP
According to Table 6, all the experts in the panel judged that the measurement items in the assessment tool are essential. The CVR values higher than 0.51 were considered as accepted
na
items and are to remain in the assessment tool. All the remaining items (n=25) were
ur
satisfactory with regard to CVR value.
Table 7 provides the calculation of both ICVI and SCVI using 14 experts’ judgment on the
Jo
relevancy items of the assessment tool.
Table 7. Calculation of ICVI Items
Not relevant (rating 1–2)
Relevant (rating 3–4)
ICVI
Interpretation
1 1.1 1.2 1.3 1.4 2 2.1 2.2 2.3 2.4 3
2 2 3 1 1 1 2 3 1 1 1
12 12 11 13 13 13 12 11 13 13 13
0.86 0.86 0.79 0.93 0.93 0.93 0.86 0.79 0.93 0.93 0.93
Appropriate Appropriate Need for Revision Appropriate Appropriate Appropriate Appropriate Need for Revision Appropriate Appropriate Appropriate
17
Journal Pre-proof 3.1 3.2 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
1 1 1 1 1 2 2 2 1 1 1 1 1 1
13 13 13 13 13 12 12 12 13 13 13 13 13 13
0.93 0.93 0.93 0.93 0.93 0.86 0.86 0.86 0.93 0.93 0.93 0.93 0.93 0.93
Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate Appropriate
ro of
Number of experts=14 (N=14), Number of items considered relevant by all the panelists (I CVI > 0.79)= 23, Number of items=25, SCVI/Ave*** or Average of ICVI =0.9, SCVI/UA**=23/25=0.92. NOTE: *Item-Content Validity Items, **Scale-Content Validity Item/Universal Agreement,***Scale-Content Validity Item/Average.
From the expert judgments of the Item-Content Validity Items (ICVI) suggested that 23
-p
items of the instruments were judged as having excellent content validity (ICVI values were
re
higher than 0.79). There were two items with unacceptable ICVI values that needed revision. The items were modified based on the recommendations of the expert panel in the second
lP
round of judgment.
na
The overall scale-level content validity index (SCVI) of the instrument was equal to 0.9 for Scale-Content Validity Item/Average (SCVI/Ave) and 0.92 for Scale-Content Validity
ur
Item/Universal agreement. We can conclude that the assessment tool have gained an
Jo
excellent content validity.
Table 8 presents the calculation of the probability of chance agreement (PC) and Kappa statistic (K) at the second round of judgment. Table 8. Calculation of PC and K *
Items
Relevant (rating 3–4)
ICVI
1 1.1 1.2 1.3 1.4 2 2.1 2.2 2.3 2.4 3 3.1 3.2
12 12 12 13 13 13 12 12 13 13 13 13 13
0.86 0.86 0.86 0.93 0.93 0.93 0.86 0.86 0.93 0.93 0.93 0.93 0.93
**
PC
0.022 0.022 0.022 0 0 0 0.022 0.022 0 0 0 0 0
***
K
0.85 0.85 0.85 0.93 0.93 0.93 0.85 0.85 0.93 0.93 0.93 0.93 0.93
Interpretation Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent
18
Journal Pre-proof 3.3 3.4 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4
13 13 13 12 12 12 13 13 13 13 13 13
0.93 0.93 0.93 0.86 0.86 0.86 0.93 0.93 0.93 0.93 0.93 0.93
0 0 0 0.022 0.022 0.022 0 0 0 0 0 0
0.93 0.93 0.93 0.85 0.85 0.85 0.93 0.93 0.93 0.93 0.93 0.93
Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent Excellent
*I-CVIs: item-level content validity index, **PC (Probability of a chance agreement) was computed using the formula: P C = [N! /A! (N - A)!] *.5N, where N = number of experts and A = number of panelists who agree that the item is relevant. Number of experts = 14 (N = 14), ***K(Kappa statistic) was computed using the formula: K = (ICVI - PC)/(1 - PC). Interpretation criteria for Kappa, Fair = K of 0.40 to 0.59; Good = K of 0.60 to 0.74; and Excellent=K>0.74
ro of
According to Table 8, the results have shown that the assessment tool has excellent inter-
-p
rater reliability at the item level (K>0.74).
5. Conclusions
re
The paper presents a process to develop and validate the assessment tools on the coastal
lP
cities’ resilience toward the impacts of climate change-induced rise in sea level coupled with coastal hazards, by using a two-step method. Both called for a comprehensive literature
na
review, items creation, and expert judgment about the items’ and the entire instrument’s
ur
validity. First was the developmental stage which assessed four of existing assessment tools
Jo
and identified the parameters, variables, and indicators of the new assessment tools. Second was the judgment stage which calculated the content validity of the assessment tools, using CVR and CVI that involved 14 local and national experts. From a set of 25 items, the content validity process identified five parameters, including institutional/ governance, environmental, social economic, built environment and infrastructure. From the expert judgments, 23 items were judged as having excellent content validity (ICVI higher than 0.79). There were two items with unacceptable ICVI that need for revision. The items that need revision have been modified according to the recommendations of the expert panel. The overall SCVI of the instrument was equal to 0.9 (SCVI /Ave) and 0.92
19
Journal Pre-proof (SCVI /UA). In addition, the calculation of the Kappa statistic showed that the assessment tool have excellent inter-rater reliability at the item level (K>0.74). This study revealed that this instrument had gained an appropriate level of content validity. The results of this study support the content validity of the assessment tool to measure the coastal cities’ resilience. This assessment tool is the first validated to be the practical tool for use to assess coastal cities’ resilience toward impacts of climate changeinduced rise in sea level and coastal hazards. This assessment tool is intended for use by
ro of
government, non-government, practitioner and academicians in sustaining long-term,
-p
consistent efforts in building coastal cities’ resilience.
[4]
[5]
[6]
[7] [8]
[9]
lP
na
[3]
ur
[2]
McMichael, A. J. 2014. Earth as humans’ habitat: global climate change and the health of populations, International journal of health policy and management, 2(1): 9. doi:10.15171/ijhpm.2014.03. Levy, B. S., and J. A. Patz. 2015. Climate change, human rights, and social justice. Annals of global health, 81 (3):310-322. doi:10.1016/j.aogh.2015.08.008. UNESCO. 2012. Coastal Management Approaches for Sea-level Related Hazards: Case Studies and Good Practices. (Intergovernmental Oceanographic Commission (IOC) Manuals and Guides, 61) 46 pp. (IOC/2012/MG/61Rev). Neumann, B., A. T. Vafeidis, J. Zimmermann, and R. J. Nicholls. 2015. Future coastal population growth and exposure to sea-level rise and coastal flooding-a global assessment. PloS one, 10 (3): e0118571. doi:10.1371/journal.pone.0118571. Bevacqua, A., D. Yu and Y. Zhang. 2018. Coastal vulnerability: Evolving concepts in understanding vulnerable people and places. Environmental Science & Policy, 82:1929. doi:10.1016/j.envsci.2018.01.006. Nicholls R. J., S. Hanson, C. Herweijer, N. Patmore, S. Hallegatte, J. Corfee-Morlot, J. Chateau, and R. Muir-Wood. 2007. Ranking port cities with high exposure and vulnerability to climate extremes—exposure estimates. Environmental Working Paper No 1, Organisation for Economic Co-operation and Development (OECD), Paris. Barbier, E. B. 2014. A global strategy for protecting vulnerable coastal populations. Science, 345 (6202):1250-1251. doi:10.1126/science.1254629. Mimura, N. 2013. Sea-level rise caused by climate change and its implications for society. Proceedings of the Japan Academy. Series B, Physical and Biological Sciences, 89(7): 281–301. doi:10.2183/pjab.89.281. Third National Communication (TNC). Under the United Nations Framework Convention on Climate Change. 2017. Ministry of Environment and Forestry, Republic of Indonesia.
Jo
[1]
re
References
20
Journal Pre-proof
Jo
ur
na
lP
re
-p
ro of
[10] Case, M., F. Ardiansyah and E. Spector. 2007. Climate change in Indonesia: implications for humans and nature. Climate change in Indonesia: implications for humans and nature. [11] USIOTWSP. 2007. How resilient is your coastal community? A guide for evaluating coastal community resilience to tsunamis and other hazards. In: U.S. Indian Ocean Tsunami Warning System Program Supported by the United States Agency for International Development and Partners, Bangkok, Thailand. [12] Howes, M. 2015. Disaster risk management and climate change adaptation: A new approach. In Applied studies in climate adaptation, ed. J. P. Palutikof, S. L. Boulter, J. Bernett, and D. Rissik, 407–415. Oxford: Wiley. [13] O'Brien, G., and P. Read. 2005. Future UK emergency management: New wine, old skin? Disaster Prevention and Management, 14(3): 353–361. [14] Rivera, C. 2014. Integrating Climate Change Adaptation into Disaster Risk Reduction in Urban Contexts: Perception and Practice. Ploss Curr. Disasters. doi:10.1371%2Fcurrents.dis.7bfa59d37f7f59abc238462d53fbb41f. [15] Few, R., H. Osbahr, L. Bouwer, V. David, and F. Sperling. 2006. Linking Climate Change Adaptation and Disaster Risk Management for Sustainable Poverty Reduction, VARG Synthesis Report. [16] Mitchell, T., and M. Van Aalst. 2008. Convergence of Disaster Risk Reduction and Climate Change Adaptation. Review for DFID, UK. [17] Torabi, E., A. Dedekorkut-Howes, and M. Howes. 2018. Adapting or maladapting: Building resilience to climate-related disaster in coastal cities. Cities, 72: 295-309. doi:10.1016/j.cities.2017.09.008. [18] UNISDR. 2009. Terminology on Disaster Risk Reduction. United Nations International Strategy for Disaster Risk Reduction, Geneva. [19] Alexander, D. E. 2013. Resilience and disaster risk reduction: an etymological journey. Natural hazards and earth system sciences, 13 (11):2707-2716. doi:10.5194/nhess-13-2707-2013. [20] Rose, A. 2007. Economic resilience to natural and man-made disasters: Multidisciplinary origins and contextual dimensions. Environmental Hazards, 7(4):383-398. doi:10.1016/j.envhaz.2007.10.001. [21] Plodinec, M. J. 2009. Definitions of resilience: An analysis. Oak Ridge: Community and Regional Resilience Institute (CARRI). [22] Folke, C., S. Carpenter, T. Elmqvist, L. Gunderson, C. S. Holling, and B. Walker. 2002. Resilience and sustainable development: building adaptive capacity in a world of transformations. AMBIO: A journal of the human environment, 31(5): 437-440. doi:10.1579/0044-7447-31.5.437. [23] Tompkins, E., and W. N. Adger. 2004. Does adaptive management of natural resources enhance resilience to climate change?. Ecology and society, 9(2). [24] Dolan, A. H., and I. J. Walker. 2006. Understanding vulnerability of coastal communities to climate change related risks. Journal of Coastal Research, 3 (39): 1316-1323. [25] Rahman, M. A., and S. Rahman. 2015. Natural and traditional defense mechanisms to reduce climate risks in coastal zones of Bangladesh. Weather and Climate Extremes, 7, 84-95. doi:10.1016/j.wace.2014.12.004. [26] Cutter, S. L., and H. Director. 2008. A framework for measuring coastal hazard resilience in New Jersey communities. White Paper for the Urban Coast Institute. [27] Lawshe, C. 1975. A qualitative approach to content validity. Personnel Psychology, 25: 563–575. doi:10.1111/j.1744-6570.1975.tb01393.x.
21
Journal Pre-proof
Jo
ur
na
lP
re
-p
ro of
[28] DeVon, H. A., M. E. Block, P. Moyle-Wright, D. M. Ernst, S. J. Hayden, and D. J. Lazzara. 2007. A psychometric Toolbox for testing Validity and Reliability. Journal of Nursing scholarship, 39 (2): 155-164. doi:10.1111/j.1547-5069.2007.00161.x. [29] Pellikka, J. 2008. Innovation support services and commercialisation process of innovation in small technology firms. International Journal of Innovation and Regional Development, 1(3): 319-334. [30] Bolarinwa, O. A. 2015. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgraduate Medical Journal, 22 (4):195. doi:10.4103/1117-1936.173959. [31] Gilbert, G. E., & Prion, S. 2016. Making sense of methods and measurement: Lawshe's Content Validity Index. Clinical Simulation in Nursing, 12 (12): 530-531. doi:10.1016/j.ecns.2016.08.002. [32] Rodrigues, I. B., J. D. Adachi, K. A. Beattie, and J. C. MacDermid. 2017. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC musculoskeletal disorders, 18(1): 540. doi:10.1186/s12891-017-1914-5. [33] Wynd, C. A., B. Schmidt, and M. A. Schaefer. 2003. Two quantitative approaches for estimating content validity. Western Journal of Nursing Research, 25(5): 508-518. doi:10.1177/0193945903252998. [34] Cicchetti, D. V., and S. A. Sparrow. 1981. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. American journal of mental deficiency. [35] Schipper, E. L. F., and L. Langston. 2015. A comparative overview of resilience measurement frameworks. Analysing indicators and approaches. Overseas Development Institute. [36] Sharifi, A. 2016. A critical review of selected tools for assessing community resilience. Ecological Indicators, 69: 629-647. doi:10.1016/j.ecolind.2016.05.023. [37] TRF. 2014. City Resilience Index: City Resilience Framework. The Rockefeller Foundation and ARUP. [38] UNISDR. 2015. Disaster Resilience Scorecard for Cities. United Nations Office for Disaster Risk Reduction. [39] Shaw, R., Y. Takeuchi, J. Joerin, and G. Fernandez. 2010. Climate and Disaster Resilience Initiative Capacity-Building Program. Kyoto University, Kyoto, Japan. [40] Rubinoff, P., and C. Courtney. 2007. How resilient is your coastal community? A guide for evaluating coastal community resilience to tsunamis and other coastal hazards. Basins and Coasts News, Integrated Management for Coastal and Freshwater Systems. United States Agency for International Development, 2(1): 24-28. Chicago. [41] KKP .2012. Panduan Penilaian Kondisi Ketangguhan Desa Pesisir. Direktorat Pesisir dan Lautan. Kementerian Kelautan dan Perikanan. [42] UNISDR. 2017. Disaster Resilience Scorecard for Cities. United Nations. [43] BNPB. 2012. Perka BNPB No. 1 Tahun 2012 tentang Pedoman Umum Desa/ Kelurahan Tangguh Bencana. [44] Colton, D., and R. W. Covert. 2007. Designing and constructing instruments for social research and evaluation. New Jersey: Wiley. [45] DasGupta, R., and R. Shaw. 2015. An indicator based approach to assess coastal communities’ resilience against climate related disasters in Indian Sundarbans. Journal of Child and Family Studies, 24 (3):85-101. doi:10.1007/s11852-014-0369-1. [46] Tompkins, E. L., Few, R., & Brown, K. (2008). Scenario-based stakeholder engagement: incorporating stakeholders preferences into coastal planning for climate
22
Journal Pre-proof
ro of -p re lP
[50]
na
[49]
ur
[48]
Jo
[47]
change. Journal of environmental management, 88(4): 1580-1592. doi:10.1016/j.jenvman.2007.07.025. Miamidian, E., M. Arnold, M. Jacquand, and K. Burritt. 2005. Surviving Disasters and Supporting Recovery: A Guidebook for Microfinance Institutions. Huynh, L. T. M., and L. C. Stringer. 2018. Multi-scale assessment of social vulnerability to climate change: An empirical study in coastal Vietnam. Climate Risk Management. doi:10.1016/j.crm.2018.02.003. Wells, S., and C. Ravilious. 2006. In the front line: shoreline protection and other ecosystem services from mangroves and coral reefs (No. 24). UNEP/Earthprint. Joerin, J., R. Shaw, Y. Takeuchi, and R. Krishnamurthy. 2012. Assessing community resilience to climate-related disasters in Chennai, India. International Journal of Disaster Risk Reduction, 1: 44-54. doi:10.1016/j.ijdrr.2012.05.006.
23
Journal Pre-proof Declaration of interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Jo
ur
na
lP
re
-p
ro of
☐The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:
24
Figure 1
Figure 2