A comparative assessment of flood susceptibility modeling using Multi-Criteria Decision-Making Analysis and Machine Learning Methods

Journal of Hydrology 573 (2019) 311–323

Contents lists available at ScienceDirect

Journal of Hydrology journal homepage: www.elsevier.com/locate/jhydrol

Research papers

A comparative assessment of flood susceptibility modeling using MultiCriteria Decision-Making Analysis and Machine Learning Methods

T

Khabat Khosravia, Himan Shahabib, Binh Thai Phamc, , Jan Adamowskid, Ataollah Shirzadie, ⁎ ⁎ ⁎ Biswajeet Pradhanf,g, Jie Douh, , Hai-Bang Lyi, Gyula Grófj, Huu Loc Hok, , Haoyuan Hongl, , Kamran Chapie, Indra Prakashm ⁎

a

Department of Watershed Management Engineering, Sari Agricultural Science and Natural Resources University, Iran Department of Geomorphology, Faculty of Natural Resources, University of Kurdistan, Sanandaj, Iran c Institute of Research and Development, Duy Tan University, Da Nang 550000, Viet Nam d Department of Bioresource Engineering, McGill University, Ste Anne de Bellevue, Canada e Department of Rangeland and Watershed Management, Faculty of Natural Resources, University of Kurdistan, Sanandaj, Iran f The Centre for Advanced Modeling and Geospatial Information Systems (CAMGIS), Faculty of Engineering and IT, University of Technology Sydney, Sydney, NSW 2007, Australia g Department of Energy and Mineral Resources Engineering, Choongmu-gwan, Sejong University, 209 NeungdongroGwangjin-gu, Seoul 05006, Republic of Korea h Public Works Research Institute (PWRI), Japan i University of Transport Technology, Hanoi 100000, Viet Nam j Department of Energy Engineering, Budapest University of Technology and Economics, Budapest, Hungary k NTT Hi-Tech Institute, Nguyen Tat Thanh University, Ho Chi Minh City, Viet Nam l Key Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing 210023, China m Department of Science and Technology, Bhaskarcharya Institute for Space Applications and Geo-Informatics (BISAG), Government of Gujarat, Gandhinagar, India b

ARTICLE INFO

ABSTRACT

This manuscript was handled by Emmanouil Anagnostou, Editor-in-Chief, with the assistance of Efthymios Nikolopoulos, Associate Editor

Floods around the world are having devastating effects on human life and property. In this paper, three Multi-Criteria Decision-Making (MCDM) analysis techniques (VIKOR, TOPSIS and SAW), along with two machine learning methods (NBT and NB), were tested for their ability to model flood susceptibility in one of China’s most flood-prone areas, the Ningdu Catchment. Twelve flood conditioning factors were used as input parameters: Normalized Difference Vegetation Index (NDVI), lithology, land use, distance from river, curvature, altitude, Stream Transport Index (STI), Topographic Wetness Index (TWI), Stream Power Index (SPI), soil type, slope and rainfall. The predictive capacity of the models was evaluated and validated using the Area Under the Receiver Operating Characteristic curve (AUC). While all models showed a strong flood prediction capability (AUC > 0.95), the NBT model performed best (AUC = 0.98), suggesting that, among the models studied, the NBT model is a promising tool for the assessment of flood-prone areas and can allow for proper planning and management of flood hazards.

Keywords: Flood susceptibility Machine Learning Multi-Criteria Decision-Making GIS China

1. Introduction Flooding is responsible for significant losses to human life resulting in significant socio-economic impacts, for example in coastal and inland regions affected by heavy monsoon rains. (Chapi et al., 2015; Komi et al., 2017; Minh et al., 2018; Wang et al., 2014). Flash floods are even more dangerous given their ability to flood areas without warning. Rapid erosion and landslides resulting from floods can cause sudden morphological changes in the area (Van Tu et al., 2016). Floods also have an effect on the geo-environment through the transportion of

chemicals and other hazardous industrial wastes, that can contaminate surface and subsurface waters as well as agricultural lands. Over the past two decades, the worldwide frequency of floods has increased by over 40% (Hirabayashi et al., 2013). In fact, between 1995 and 2015, roughly 109 million people were affected by floods; with damages reaching up to US $75 billion per year (Alfieri et al., 2017). Floods can have both natural and anthropogenic causes (Chang and Chen, 2016) with climate change being considered as one of the main causes for changes in flood patterns, intensity and magnitude (Dawod et al., 2011; Hens et al., 2018; Thai et al., 2017; Van Thanh et al.,

Corresponding authors. E-mail addresses: [email protected] (B.T. Pham), [email protected] (J. Dou), [email protected] (H.L. Ho), [email protected], [email protected] (H. Hong). ⁎

https://doi.org/10.1016/j.jhydrol.2019.03.073 Received 23 November 2018; Received in revised form 21 February 2019; Accepted 21 March 2019 Available online 22 March 2019 0022-1694/ © 2019 Elsevier B.V. All rights reserved.

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

2017). Many Asian countries are heavily affected by floods; China, for example, was the site of the deadliest natural disaster of the 20th century, the 1931 Central China flood. This flood led to the deaths of an estimated 3.7 million people (Ma et al., 2010). The estimated annual damage caused by flooding rivers, in China, in 2010, was US $51 billion (Stoynov et al., 2017). In recent years, China has observed changing rainfall patterns leading to heavier rainfall between the period of April to June. This increased rainfall has led to flooding in many parts of the Ningdu County (Report of the Ningdu County government, http:// www.ningdu.gov.cn). Changing rainfall patterns, in general, have been tied to an increase in anthropogenic activities which have been deemed responsible for the increased frequency of flooding (Zhang et al., 2018). Given the impact on human life, floods can have an extreme effect on a region's socioeconomic health. Accordingly, appropriate action is required to mitigate flood hazards. Many different approaches have been used to predict the many different aspects of flood phenomena. One of the first approaches used, was the statistical correlation of flood characteristics to the physical properties of watersheds (Wheater et al., 1993). More recently, conceptual and physically-based models have been used for flood prediction in many parts of the world (Solomatine and Ostfeld, 2008). However, generation of flood maps remains a major challenge (Khosravi et al., 2016a; Khosravi et al., 2016b). Given the inherent complexity of watersheds, floods cannot be modelled using simple, nonlinear models. In recent years, however, data-driven models including bivariate statistical models, Multi-Criteria DecisionMaking (MCDM) and Artificial Intelligence (AI), Machine Learning (ML), Computational Intelligence (CI), Soft Computing (SC), Data Mining (DM), Knowledge Discovery in Databases (KDD), and Intelligent Data Analysis (IDA) models have shown promise in many branches of prediction sciences including Flood Susceptibility Mapping (FMS) (Solomatine et al., 2009). These models and their hybrids have already been applied to many natural disasters in a wide range of studies (Ahmadlou et al., 2018; Chapi et al., 2017; Khosravi et al., 2018; Phuong et al., 2017; Shafizadeh-Moghadam et al., 2018). Having reviewed 128 papers employing MCDM methods, de Brito and Evers (2016) found that the Analytic Hierarchy Process (AHP) was the most widespread MCDM method, but concluded that other MCDM methods should be tested. Recently, de Brito et al. (2018) applied the AHP and Analytic Network Process (ANP) to flood vulnerability assessment and mapping in Brazil. The authors noted that due to dependences among criteria, the ANP had greater predictive power than the AHP. These new approaches have demonstrated many advantages including: (i) that compared to physically-based models, they do not require a great deal of detailed information regarding watershed characteristics, and (ii) that these models generally achieved high prediction accuracy (Abedini et al., 2018; Dou et al., 2015; Pham et al., 2018c). The developement of new individual and hybrid models, however, is still necessary to increase the accuracy of modelled parameters. According to the literature, several bivariate, MCDM and ML models have been applied to flood susceptibility mapping; however, of all the papers consulted, no universal guidelines were found to exist regarding the assessement of the predictive power of these models. Moreover, research has shown that all models have their own advantages and disadvantages. Consequently, a range of models must first be applied to a given area and only after an assessment of the models’ predictive power can one be selected for use (Khosravi et al., 2018). The present study sought to introduce and compare the predictive power of several novel models based on MCDM techniques and ML techniques for the spatial prediction of floods. The novel MCDM techniques include: VIKOR (Vlse Kriterijuska Optamizacija I Komoromisno Resenje), TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) and SAW (Simple Additive Weighting). These are well-known MCDM models though they have yet to be applied to flood susceptibility mapping. The predictive power of the MCDM models was assesed and then compared to two popular ML models: Naïve Bayes Trees (NBT) and Naïve Bayes (NB). Validation and comparison of the models was done using the Receiver Operating Characteristic (ROC) curve method and several statistical indices.

2. Study area Located in southeast Jiangxi Province (long. 115°), the Ningdu County, one of China’s most flood-prone regions, was selected as the study area. Covering an area of 4053.16 km2, Ningdu County has 790,829 people, of which 136,187 (17.22%) live in rural areas. The region is hilly, with varying slopes up to 66°. Elevation in the area varies from 29 m to 1410 m A.M.S.L. There are 638 rivers traversing Ningdu County. Geologically, the area is composed of sedimentary (conglomerate and limestone), igneous (basalt and granite) and metamorphic (Phyllite and schist) rock. Soils in the area mainly belong to the Acrisol soil group; a clay rich soil group associated with humid, tropical climates. Land use patterns in the area include: bare lands, farm lands, forests, grasslands, residential areas and water bodies. The prevailing climate in the area is humid and is a part of the middle subtropical monsoon zone. (Report of the Jiangxi Meteorology Bureau http://www.weather.org.cn/). The annual average temperature of the region varies between 14 °C and 19 °C, with an average annual sunshine duration of 1938.8 h and an average annual rainfall of 1600 mm. About 40 to 70% of the annual rainfall occurs between April and June. 3. Data 3.1. Flood inventory map When predicting future flood occurrences, a region’s flood hazards can be analyzed using prior flood records drawn from a flood inventory map (Khosravi et al., 2018; Pham et al., 2018a; Pham et al., 2018d). A flood inventory map of Ningdu County was prepared using available data from 166 flood events supplied by the Chinese Academy of Science (Fig. 1). Using GPS, the locations of flood events were validated from records of flood level marker posts and through extensive field surveys. The flood data was randomly divided into two groups using a 7:3 ratio. To construct the model, 70% of the flood and non-flood locations were used, whereas the remaining 30% of flood and non-flood locations were used to generate the testing dataset in order to validate the models. 3.2. Flood conditioning factors A flood’s magnitude depends on rainfall intensity and duration. Meteorological factors and catchment characteristics including topography, vegetation, and soil characteristics also have an affect on flood susceptibility. In the present study, 12 flood conditioning factors, in conjunction with the catchment’s geo-environmental characteristics, were initially considered for flood susceptibility analysis. These include: Normalized Difference Vegetation Index (NDVI), lithology, land use, distance from river, curvature, altitude, Stream Transport Index (STI), Topographic Wetness Index (TWI), Stream Power Index (SPI), soil type, slope and rainfall (Fig. 2). The Aster Digital Elevation Model (DEM) was used to create maps of the following topographic factors for the watershed in question: elevation, slope, aspect, curvature, TWI, SPI, STI, and river distance buffers. Ground slope is an important factor in regional flood risk assessment as a steeper slope results in increased runoff (Tehrany et al., 2015b). Slopes for the study area were classified into five categories based on a quintile classification scheme: 0 – 5.98, 5.99 – 10.14, 10.15 – 14.82, 14.83 – 21.05, and 21.06 – 66.28° (Fig. 2a). The curvature of an area reflects the shape of the ground surface and can affect flooding in a given area. Curvature maps were, therefore, generated using the DEM and classified into three classes: concave, flat, and convex (Fig. 2b). Convex surfaces are more prone to runoff and are predominantly responsible for down-slope flooding (Il'Inskii and Yakimov, 1987). At higher elevations, the flood risk is relatively low except for as a result of glacial meltwaters. Flat areas at low elevation, however, are very prone to flooding (Meybeck et al., 2001). An elevation map was therefore 312

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 1. The study area showing the distribution of training and testing flood locations.

generated and classified into five categories: 29–227, 228–278, 279–347, 348–477, and 478–1410 m (Fig. 2c). A distance river buffer map was also created with the following five classes: 0–199, 200–477, 478–775, 776–1172, and 1173–5045 m to assess the effect of its extent of flooding (Fig. 2d). The NVDI is considered to be one of the most important factors affecting floods (Khosravi et al., 2016a), and is calculated using Eq. (1) below:

NDVI =

RNIR RR RNIR + RR

values were classified into five classes (1665–1945, 1946–2023, 2024–2122, 2123–2162, and 2163–2309 mm) by analyzing 10 rainfall stations’ data using an inverse distance weighted method (Fig. 2i). TWI is a geomorphometric parameter used to quantify local relief and quantitatively evaluate runoff in flood studies (Pourghasemi et al., 2012). It can be calculated using Eq. (2) below:

TWI = ln (1)

(2)

tan

where, α is the total upslope catchment area draining downward from a point with a slope angle of . The TWI map was prepared and classified into five categories: 2.7 ≤ TWI ≤ 5.7, 5.8 ≤ TWI ≤ 7.7, 7.8 ≤ TWI ≤ 14.9, 15 ≤ TWI ≤ 22.8, and 22.9 ≤ TWI ≤ 28.1 (Fig. 2j). SPI is a measure of erosive power and the intensity of surface run off. It can be calculated using Eq. (3) below:

where, RNIR is reflectance in the infrared portion of the electromagnetic spectrum, and RR is reflectance in the red portion. An NDVI map was constructed using Operational Land Imager (OLI) sensor images from the Landsat 8 satellite. NDVI values were split into five categories: – – 0.12 NDVI 0.43 ≤ NDVI≤–0.02, 0.01 ≤ NDVI ≤ 0.11, 0.23, 0.24 NDVI 0.34 , and 0.35 NDVI 0.60 (Fig. 2e). A lithographic map with nine categories was obtained from the Chinese Academy of Science. The categories are as follows: A (conglomerate, Limestone), B (granite), C (basalt), D (Quartzite), E (Phyllite and slate), F (Diorite), G (moyite), H (Siliceous rock), and I (Schist) (Fig. 2f). A land use map was then generated using aerial images, and split into six categories: bare, farm, grass and forest lands, residential areas, and water bodies (Fig. 2g). Soil is one of the most important factors when attempting to manage infiltration and run off. Data from the National Soil Survey of China (http://www.issas.ac.cn) was used to prepare soil maps of the area; these maps featured four soil groups: Ach (Haplic acrisols), ACu (Humic acrisols), ALh (Haplic alisols), and ATc (Cumulic anthrosols) (Fig. 2h). For any given area, rainfall is the single most important factor related to flood occurrence. As such, a rainfall map was prepared and rainfall

(3)

SPI = A stan

where, A s represent the area of the basin (Jaafari et al., 2014). The SPI map was prepared and classified into six categories: 0 ≤ SPI ≤ 0.01, 0.01 ≤ SPI ≤ 99.65, 99.66 ≤ SPI ≤ 199.30, 199.31 ≤ SPI ≤ 498.25, and 498.26 ≤ SPI ≤ 25410.81 (Fig. 2k). The STI index is used to assess landscape erosion and accounts for flow convergence and divergence. This, in turn, can have an effect on the flood occurance. The STI is calculated using Eq. (4) below (Moore et al., 1993):

STI =

As 22.13

0.6

sin 0.0896

1.3

(4)

The STI map was generated and classified into five categories: 313

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 2. Maps of flood conditioning factors: (a) Slope map, (b) Curvature map, (c) Altitude map, (d) Distance from the river map, (e) NDVI map, (f) Lithology map, (g) Land use map, (h) Soil type map, (i) Rainfall map, (j) TWI map, (k) SPI map, and (l) STI map.

314

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 2. (continued)

0 ≤ STI ≤ 0.01, 0.01 ≤ STI ≤ 3.69, 3.7 ≤ STI ≤ 11.07, 11.08 ≤ STI ≤ 22.14, and 22.15 ≤ STI ≤ 941.13 (Fig. 2l). When generating the maps, elevation, slope, aspect, curvature, TWI, SPI, STI, rainfall, NDVI, and distance to rivers were classified using a

quintile classification scheme (Khosravi et al., 2018; Tehrany et al., 2015b). Other factors like soil, land cover, and lithology were classified using a standard classification scheme (de MOEL and Aerts, 2011; Tehrany et al., 2014; Termeh et al., 2018). 315

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 2. (continued)

4. Methods

(MCDM) or Multiple-Criteria Decision Analysis (MCDA), evaluates multiple conflicting criteria in decision-making. The VIKOR, TOPSIS and SAW methods were applied for the development of the models. The second technique looks at ML methods specifically using the Naive

Two main techniques were used to develop flood susceptibility models. The first technique, Multiple-Criteria Decision-Making 316

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Bayes Tree (NBT) and Naive Bayes (NB) classifiers.

univariate splits are made of each node as a normal DT classifier, and (iii) a given instance is classified using the following classification rule (Eq. (5)) (Kohavi, 1996):

4.1. Multiple-Criteria Decision-Making (MCDM) techniques Each conditioning factor was first converted to stretch raster format with a 30 m × 30 m pixel size. Distance from the river and other similar distances were calculated using the Euclidean distance tool in ArcGIS 10.2. This format consists of several columns and rows for each factor resulting in a single matrix. The required steps for each model were then performed, resulting in data that was used for flood susceptibility mapping. Finally, all factors were normalized between 0 and 1 using the proper fuzzy membership function in ArcGIS 10.2 to reduce any biases stemming from expert judgment.

c * = arg max cj

C

P (cj |s1, s1, ...,sn) =

P (cj ) m j=1

[P (cj )

n i=1

P (si |cj ) n i=1

P (si |cj )

n

= µ arg max P (cj ) cj

P (si |cj )

C i=1

(5)

where S1, S2, …, Sn are the predictive attributes, cj is a class attribute (flood and non-flood) of class set C, µ is the mean of observations, and m is the total number of classes. 4.2.2. Naive Bayes (NB) The NB is a Bayes-based classifier in which no dependency exists between conditioning factors and the variables used to maximize posterior probability (Pham et al., 2017; Soni et al., 2011). The NB is constructed by: 1) collecting relevant examples, 2) estimating the prior probability for each class label (flood and non-flood locations), 3) calculating each class label and computing covariance matrices for each class, and 4) determining their inverse and determinant to ultimately construct the discriminant function for each class. In this study, r = (r1, r2, , rn ), is the vector of flood conditioning factors, whereas f = (f1, f2) corresponds to the vector of classifier variables (flood, non-flood). The conditional probability, for all class labels of the study area, is then obtained for the NB Classifier. The prediction is based on the largest posterior probability and can be formulated using Eq. (6) (McCallum and Nigam, 1998):

4.1.1. Vise kriterijumska optimizacijaik ompromisno Resenje (VIKOR) The VIKOR method, developed by Duckstein and Opricovic (1980) is one of the most popular MCDM methods for the optimization of complex systems. It is used to identify the compromise ranking list and solution and is also used to weight stability intervals. This method introduces a multicriteria ranking index that is nearest to the ideal (Opricovic and Tzeng, 2004). The alternatives are then evaluated according to the established criteria (Chitsaz and Banihabib, 2015; Opricovic and Tzeng, 2004). 4.1.2. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) The TOPSIS method, presented by Ching and Yoon (1981), is based on the Euclidean distance between decision making alternatives (Ameri et al., 2018; Chitsaz and Banihabib, 2015). It was developed to solve decision-making problems consisting of conflicting and non-commensurable criteria. In this method, the ranking of alternatives is based on 1) the shortest distance from the ideal solution and 2) the farthest distance from the negative ideal solution (Jozaghi et al., 2018; Opricovic and Tzeng, 2004).

fNB = arg max P (fi ) fi

{flood , non

n r P ( fi ) i=1 i

flood}

(6)

where, P (fi ) is the prior probability and is calculated as the proportion r of observations with output class fi in the training dataset, and P ( fi ) is i the conditional probability, given by Eq. (7):

4.1.3. Simple Additive Weighting (SAW) The SAW technique, introduced by Fishburn (1967), is based on weighted averages (Jain and Raj, 2013). An evaluation score is first calculated for each alternative by multiplying the scaled value, given to the alternative of a given attribute, with the weighted importance assigned by decision makers. Products for all criteria are then summed (Pourkhabbaz et al., 2014). The advantage of this method is that it is a proportional linear transformation of the raw data. This means that the relative order of magnitude of the standardized scores remains equal (Jain and Raj, 2013).

r P( i) = fi

1 2

e(

(ai µ)2 /2 2

(7)

where, µ is the mean deviation and λ is the standard deviation of observations (ri). 4.3. Model evaluation criteria 4.3.1. Statistical measures criteria For both the training and validation datasets, the effectiveness of the models was evaluated using statistical criteria. Results from the training dataset indicate degree of fit, whereas results from the validation dataset provide information about the predictive ability of the proposed models (Pham and Prakash, 2018; Pham et al., 2018b). Different model prediction results were evaluated: True Positive (TP), representing the number of pixels correctly classified as positive (flood) predictions, and True Negative (TN), representing the number of pixels correctly classified as negative (non-flood) predictions. Furthermore, False Positive (FP) and False Negative (FN), representing the number of pixels incorrectly classified as positive (floods) or negative (non-floods), respectively, were also evaluated (Althuwaynee et al., 2014; Onan, 2015). The statistical measures of Accuracy, MAE, Kappa index (K), and RMSE served to evaluate model performance; they can be determined using Eqs. (8–11) below (Armstrong and Collopy, 1992; Dawson et al., 2006):

4.2. Machine Learning (ML) algorithms In order to obtain the highest prediction performance using ML methods, an optimum value of each operator for the proposed models is determined using trial and error method. By using proper conditioning factors, the NBT and NB models were built and validated using training and validation datasets. Furthermore, these models were constructed and run using a batch size of 100 with two decimal places. A 10-fold cross-validation method was used to prevent over-fitting. 4.2.1. Naive Bayes tree (NBT) NBT classifiers, generally used for solving classification problems, can improve upon the performance of other individual classifiers such as Naïve Bayes (NB) and Decision Tree (DT) (Kohavi, 1996). One of the main assumptions of the NBT is that the class conditional independence factor (s) does not show a correlation between the values of the conditioning factor (Shirzadi et al., 2017). For each pixel in the leaves, the computation of posterior probability proceeds as follows (Chen et al., 2017; Kohavi, 1996; Pham et al., 2016): (i) after creating the tree and following its growth pre-pruning occurs, (ii) to segment the data

Accuracy =

MAE =

317

1 n

TP + TN TP+ TN + FP+FN

(8)

i=n

|X ei i= 1

X oi|

(9)

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Kappaindex (K ) =

RMSE =

1 n

PC

Pexp

1

Pexp

Table 1 Multicollinearity diagnosis test (Tolerance and VIF) and IGR test (Average Merit) for conditioning factors used in flood modeling in the Ningdu Catchment, China.

(10)

i=n

(X ei

X oi) 2

i= 1

(11)

where, Pc is the number of pixels to be classified correctly as flood or non-flood pixels; Pexp are the expected agreements; X oi and X ei are the ith observed and model predicted values, respectively, and n is the number of data points. 4.3.2. Receiver Operating Characteristic (ROC) curve The ROC curve, widely used in geoscience, is a standard technique to determine the general performance of models, and is one of the most important and widely used methods in spatial modeling (Chen et al., 2018b). This curve can be constructed by plotting the values of two statistical indexes, “sensitivity” and “100-specificity”, on the ordinate and abscissa, respectively (Shahabi and Hashim, 2015). The prediction of an event’s occurrence or non-occurrence can be quantitatively evaluated using the Area Under the ROC curve (AUC) (Chen et al., 2018a), which highlights the performance of a suggested flood model. It ranges between 0.5 (inaccurate) and 1 (highly accurate) (Shirzadi et al., 2017).

No

Conditioning Factor

Tolerance

VIF

Average Merit (AM)

1 2 3 4 5 6 7 8 9 10 11 12

NDVI Lithology Land-use Distance from river Curvature Altitude TWI STI SPI Soil type Ground slope Rainfall

0.177 0.880 0.756 0.408 0.632 0.184 0.275 0.310 0.527 0.928 0.196 0.905

5.65 1.13 1.32 2.45 1.58 5.44 3.64 3.22 1.89 1.07 5.10 1.10

0.69 0.48 0.32 0.81 0.00 0.99 0.18 0.23 0.056 0.59 0.49 0.058

5.2. Training and validating the models Validation criteria were applied to investigate the accuracy and prediction effectiveness of each built model on both training and testing parts. Result show that the NB model outperforms the NBT in terms of ACC (96.8%), Kappa (0.95), lower RMSE (0.115) and MAE (0.12) in the training phase (Table 2). Both NB and NBT models were then evaluated in the testing phase with the obtained results being similar to those of the training phase. Prediction capability of the NB model is higher than the NBT model (Table 2) and a comparison of the results of the Kappa index show almost perfect agreement between estimation and actual (observations) in both the training and testing phases. Applied to all models, the ROC curve method showed that the NBT model (AUC = 0.984) outperforms the NB (AUC = 0.979), SAW (AUC = 0.97), TOPSIS (AUC = 0.968), and VIKOR (AUC = 0.965) models in the validation phase (Fig. 3a). The prediction rate (Fig. 3b) showed that despite all models providing similarly suitable results, the NB model outperformed the rest in terms of prediction effectiveness (AUC = 0.98). The NB model was followed by the NBT (AUC = 0.97), SAW and VIKOR (AUC = 0.96) and TOPSIS (AUC = 0.95) I, respectively.

4.4. Feature selection methods The current study applies two main methods: (i) Variance Inflation Factors (VIF) and tolerances (Dormann et al., 2013) and (ii) Information Gain Ratio (IGR) test (Kannan et al., 2009) to identify the most effective conditioning factors for flood occurrence and to remove null factors from the model in order to improve the prediction power and performance of the models studied (Chapi et al., 2017). In the Multicollinearity diagnosis test VIF and tolerance criteria, Pearson’s correlation coefficients, variance decomposition proportions and the conditional index are used (Khosravi et al., 2018). Multicollinearity evaluates the possible correlation between the 12 conditioning factors used. Multicollinearity between factors suggests that correlated factors can be predicted by other factors and must thusly be removed from the modeling process. VIF and tolerance values of > 10 and < 0.1, respectively, indicate a multicollinearity problem (Khosravi et al., 2018). The IGR is considered one of the best methods for feature selection (Chen et al., 2017; Khosravi et al., 2018), and is used to determine the relative importance of each conditioning factor on event occurance. The Average Merit (AM) was calculated as a quantitative criterion for identifying the relative importance of factors. The greater the AM for each conditioning factor, the greater its effect on the occurrence.

5.3. Generation of flood susceptibility map Flood susceptibility maps were generated separately using the TOPSIS, VIKOR and SAW methods. Using the VIKOR method (Fig. 4a, b), the positive and negative impacts varied between 0.09 and 3.27 and 0.046 and 0.63, respectively. In comparison, for the TOPSIS method, positive impact ranged from 0.06 to 1.26 while negative impacts ranged from 0.33 to 1.33 (Fig. 4c and Fig. 4d). The flood susceptibility maps for VIKOR, TOPSIS and SAW are shown in Fig. 5.

5. Results

Table 2 Evaluation of the performance of the NB and NBT models in modeling floods in the Ningdu Catchment, China (training and validation phases).

5.1. Selection of the conditioning factors The VIF and tolerance values of all factors are less than 10 and exceed 0.1, respectively (Table 1). This indicates that no multicollinearity problems exist among the conditioning factors. Application of the IGR test, to evaluate the importance of each factor in flood modeling (Table 1), showed that for the Ningdu Catchment, altitude (AM = 0.99) had the greatest effect on flood occurrence. This is followed by distance from the river (AM = 0.81), NDVI (AM = 0.69), soil type (AM = 0.59), ground slope (AM = 0.49), lithology (AM = 0.48), land-use (AM = 0.32), STI (AM = 0.23), TWI (AM = 0.18), rainfall (AM = 0.058), SPI (AM = 0.056) and curvature (AM = 0), respectively. These results show that among the 12 factors, curvature had no effect on flood occurrence. It was therefore removed from model developement.

Model Accuracy Statistic

Dataset Training

TP TN FP FN ACC (%) Kappa RMSE MAE

318

Validation

NB

NBT

NB

NBT

108 105 3 6 96.8 0.95 0.115 0.12

107 103 4 8 96.0 0.93 0.136 0.13

45 45 3 3 0.91 0.91 0.15 0.14

43 44 5 4 0.90 0.89 0.16 0.17

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

susceptibility mapping is both the first and the key step in flood modeling and risk assessment. By mapping flood susceptibility, flood prone areas can be identified, and the appropriate structural and non-structural measures can be applied to reduce losses related to flooding. The present study applied and compared various MCDM methods (SAW, VIKOR and TOPSIS) and ML methods (NB and NBT) for the mapping of flood susceptibility in the Ningdu Catchment of China. In general, flood susceptibility mapping isolates vulnerable areas based on the consideration of a variety of flood-affecting factors (Hong et al., 2018). However, floods are affected by many other factors related to geological, hydrological, topographical and morphological conditions (Tehrany et al., 2015a). Moreover, only some of these factors contribute to flood susceptibility models; thus, the selection of suitable flood-affecting factors is a important step in flood susceptibility modeling. The effective feature selection method (IGR) employed in the current study allowed for the selection of input factors for flood susceptibility modeling. It was determined that the most effective conditioning factor for flood susceptibility mapping was altitude, while the least effective conditioning factors were rainfall, SPI and curvature. These results are in line with many other recent studies (Khosravi et al., 2018; Khosravi et al., 2016b; Tehrany et al., 2015a). While rainfall is the main source of flooding, it does not necessarily have a linear effect on flooding. Results showed that at higher elevations, an increase in rainfall can actually reduce the occurance of flooding. Validation of model performance was achieved through the use of several statistical model performance criteria. The validation phase results showed that ML methods (NB and NBT) outperform MCDM methods (SAW, VIKOR and TOPSIS) for flood susceptibility assessment. ML methods used two strategies, post-pruning and pre-pruning, to prevent over-fitting and noise problems; hence greater accuracy was achieved (Kotsiantis et al., 2007). In contrast, MCDM methods (SAW, VIKOR and TOPSIS) have several disadvantages: (i) they are extremely data intensive in terms of recording a decision-maker’s preferences, (ii) the required amount of data may not be available for all decisionmaking problems, (iii) due to the weighting of variables, they require strong assumptions from decision makers making them relatively subjective (Velasquez and Hester, 2013). Among the ML methods, the NBT outperformed the NB for flood susceptibility assessment. This can be explained due to the NBT being a hybrid model. As a hybrid model, the NBT has several advantages over the NB model, including its ability to enhance the classification accuracy by using the naïve Bayes algorithm and the decision tree algorithm to construct the classification trees. Overall, it is a simple, effective, and interpretable model that allows for quick learning from the training dataset (Kohavi, 1996). In addition, the NB faces a disadvantage of assuming independence between input variables, which is not always the case with flood prediction. The findings of the present study may be of use to flood hazard managers or researchers when selecting the most suitable methods for flood susceptibility modeling. However, the limitation of the present approach is the non-availability of temporal changes and real-time flood data. Another main limitation of the methods used and of flood susceptibility mapping in general, is that these maps can only show where floods are likely to occur but provide no information relating to flood depth or velocity. In the future, it is recommended that studies such as this one be combined with hydraulic modeling, such as the new version of the Hydrologic Engineering Center's (CEIWR-HEC) River Analysis System (HEC-RAS 5 and newer versions) that can provide 2D maps, for both depth and velocity. In addition, future studies must evaluate whether overlapping criteria based on a DEM be included as part of the model.

Fig. 3. Model evaluation using (a) success and (b) prediction rate.

Flood susceptibility maps were also generated using NBT and NB methods. The NB and NBT models were constructed in the training phase using optimal parameters to calculate the flood susceptibility indices (FSI) while considering the entirety of the pixels within the study area. First, the entire area was converted to pixels of 30 m × 30 m. Then, during the learning phase of the model, the FSI were assigned to each pixel based on the training dataset. Flood susceptibility maps were constructed by classifying susceptibility into five classes: very low, low, moderate, high and very high. This classification was based on the quantile method in ARC GIS 10.2 (Tehrany et al., 2015b) (Fig. 5). 6. Discussion

7. Conclusions

Complex natural disasters such as flash floods can never be fully prevented. It is, therefore, extremely important to improve flood prediction and mitigation methods in order to minimize the loss of human life and the socio-economic impacts related to flooding. Flood

In this study, the MCDM techniques, VIKOR, TOPSIS and SAW, and the ML methods, NBT and NB, were applied for flood susceptibility mapping of the Ningdu catchment, China. Models were constructed 319

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 4. The criteria having a positive and negative impact for: (a) positive, VIKOR, (b) negative, VIKOR, (c) positive, TOPSIS, and (d) negative, TOPSIS.

using a spatial database which included twelve geo-environmental and topographic flood conditioning factors as well as data from 166 past flooding events. The importance of these factors was evaluated following multi-collinearity diagnostic tests, a consideration of tolerance

and VIF methods, and an IGR test. Test results showed that no multicollinearity problem existed among the conditioning factors selected. The IGR test, however, indicated that land surface curvature had no effect on flood occurrence in the area; curvature was thusly removed 320

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Fig. 5. Food susceptibility maps using: (a) VIKOR, (b) TOPSIS, (c) SAW, (d) NBT, and (e) NB models.

and the final analysis was performed using the remaining eleven conditioning factors (NDVI, lithology, land use, distance from river, curvature, altitude, STI, TWI, SPI, soil type, slope and rainfall). The training and validation phase accuracy for the studied models

was evaluated using several statistical measures: ACC, Kappa, RMSE, MAE, and AUC. Validation results showed that all models studied (NBT, NB, SAW, TOPSIS and VIKOR models) performed well, but that the NBT model performed best. The NBT model can therefore be explored for the 321

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al.

Environ. 626, 1121–1135. Chen, W., et al., 2017. A novel hybrid artificial intelligence approach based on the rotation forest ensemble and naïve Bayes tree classifiers for a landslide susceptibility assessment in Langao County, China. Geomatics, Natural Hazards and Risk 8 (2), 1955–1977. Chen, W., Zhang, S., Li, R., Shahabi, H., 2018b. Performance evaluation of the GIS-based data mining techniques of best-first decision tree, random forest, and naïve Bayes tree for landslide susceptibility modeling. Sci. Total Environ. 644, 1006–1018. Ching, L.H., Yoon, P., 1981. Multiple Attribute Decision Making. Springer-Verlag, Berlin. Chitsaz, N., Banihabib, M.E., 2015. Comparison of different multi criteria decisionmaking models in prioritizing flood management alternatives. Water Resour. Manage. 29 (8), 2503–2525. Dawod, G.M., Mirza, M.N., Al-Ghamdi, K.A., 2011. GIS-based spatial mapping of flash flood hazard in Makkah City, Saudi Arabia. Journal of Geographic Information System 3 (3), 217. Dawson, C.W., Abrahart, R.J., Shamseldin, A.Y., Wilby, R.L., 2006. Flood estimation at ungagged sites using artificial neural networks. J. Hydrol. 319, 391–409. de Brito, M.M., Evers, M., 2016. Multi-criteria decision-making for flood risk management: a survey of the current state of the art. Nat. Hazards Earth Syst. Sci. 16 (4), 1019–1033. de Brito, M.M., Evers, M., Almoradie, A.D.S., 2018. Participatory flood vulnerability assessment: a multi-criteria approach. Hydrol. Earth Syst. Sci. 22 (1). de Moel, H., Aerts, J., 2011. Effect of uncertainty in land use, damage models and inundation depth on flood damage estimates. Nat. Hazards 58 (1), 407–425. Dormann, C.F., et al., 2013. Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 36 (1), 27–46. Dou, J., et al., 2015. An integrated artificial neural network model for the landslide susceptibility assessment of Osado Island, Japan. Natural Hazards 78 (3), 1749–1776. Duckstein, L., Opricovic, S., 1980. Multiobjective optimization in river basin development. Water Resour. Res. 16 (1), 14–20. Fishburn, P., 1967. Additive utilities with incomplete product set: applications to priorities and sharings. Google Scholar. Hens, L., et al., 2018. Sea-level rise and resilience in Vietnam and the Asia-Pacific: a synthesis. Vietnam Journal Of Earth Sciences 40 (2), 126–152. Hirabayashi, Y., et al., 2013. Global flood risk under climate change. Nat. Clim. Change 3 (9), 816. Hong, H., et al., 2018. Application of fuzzy weight of evidence and data mining techniques in construction of flood susceptibility map of Poyang County, China. Sci. Total Environ. 625, 575–588. Il'Inskii, N., Yakimov, N., 1987. Determination of the shape of the downstream slope of an earth-fill dam from the seepage strength conditions at the face. Fluid Dyn. 22 (3), 414–419. Jaafari, A., Najafi, A., Pourghasemi, H., Rezaeian, J., Sattarian, A., 2014. GIS-based frequency ratio and index of entropy models for landslide susceptibility assessment in the Caspian forest, northern Iran. Int. J. Environ. Sci. Technol. 11 (4), 909–926. Jain, V., Raj, T., 2013. Evaluation of flexibility in FMS using SAW and WPM. Decision Science Letters 2 (4), 223–230. Jozaghi, A., et al., 2018. A Comparative Study of the AHP and TOPSIS Techniques for Dam Site Selection Using GIS: A Case Study of Sistan and Baluchestan Province, Iran. Geosciences 8 (12), 494. Kannan, G., Pokharel, S., Kumar, P.S., 2009. A hybrid approach using ISM and fuzzy TOPSIS for the selection of reverse logistics provider. Resour. Conserv. Recycl. 54 (1), 28–36. Khosravi, K., Nohani, E., Maroufinia, E., Pourghasemi, H.R., 2016a. A GIS-based flood susceptibility assessment and its mapping in Iran: a comparison between frequency ratio and weights-of-evidence bivariate statistical models with multi-criteria decision-making technique. Nat. Hazards 83 (2), 947–987. Khosravi, K., et al., 2018. A comparative assessment of decision trees algorithms for flash flood susceptibility modeling at Haraz watershed, northern Iran. Sci. Total Environ. 627, 744–755. Khosravi, K., Pourghasemi, H.R., Chapi, K., Bahri, M., 2016b. Flash flood susceptibility analysis and its mapping using different bivariate models in Iran: a comparison between Shannon’s entropy, statistical index, and weighting factor models. Environ. Monit. Assess. 188 (12), 656. Kohavi, R., 1996. Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid. KDD. Citeseer 202–207. Komi, K., Neal, J., Trigg, M.A., Diekkrüger, B., 2017. Modelling of flood hazard extent in data sparse areas: a case study of the Oti River basin, West Africa. J. Hydrol.: Reg. Stud. 10, 122–132. Kotsiantis, S.B., Zaharakis, I., Pintelas, P., 2007. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering 160, 3–24. Ma, J., Tan, X., Zhang, N., 2010. Flood management and flood warning system in China. Irrigation and Drainage: The journal of the International Commission on Irrigation and Drainage 59 (1), 17–22. McCallum, A., Nigam, K., 1998. A comparison of event models for Naive Bayes text classification, AAAI-98 workshop on learning for text categorization, pp. 752–763. Meybeck, M., Green, P., Vörösmarty, C., 2001.. A new typology for mountains and other relief classes: an application to global continental water resources and population distribution. Mt. Res. Dev. 21 (1), 34–45. Minh, P.T., Tuyet, B.T., Thao, T.T.T., 2018. Application of ensemble Kalman filter in WRF model to forecast rainfall on monsoon onset period in South Vietnam. Vietnam Journal Of Earth Sciences 40 (4), 367–394. Moore, I.D., Gessler, P.E., Nielsen, G.A., Peterson, G., 1993. Soil attribute prediction using terrain analysis. Soil Sci. Soc. Am. J. 57, 443–452. Onan, A., 2015. A fuzzy-rough nearest neighbor classifier combined with consistency-

Fig. 5. (continued)

construction of flood susceptibility maps. Flood susceptibility maps generated from this study may be useful for land use planning of areas likely to be affected by floods. The results of this study can also be used to improve and inform flood risk management of all flood prone areas, not only Ningdu, China. Acknowledgement This work was supported by International Partnership Program of Chinese Academy of Sciences (115242KYSB20170022). References Abedini, M., et al., 2018. A novel hybrid approach of bayesian logistic regression and its ensembles for landslide susceptibility assessment. Geocarto International (just-accepted) 1–44. Ahmadlou, M., et al., 2018. Flood susceptibility assessment using integration of adaptive network-based fuzzy inference system (ANFIS) and biogeography-based optimization (BBO) and BAT algorithms (BA). Geocarto International 1–21. Alfieri, L., et al., 2017. Global projections of river flood risk in a warmer world. Earth's Future 5 (2), 171–182. Althuwaynee, O.F., Pradhan, B., Park, H.-J., Lee, J.H., 2014. A novel ensemble decision tree-based CHi-squared Automatic Interaction Detection (CHAID) and multivariate logistic regression models in landslide susceptibility mapping. Landslides 11 (6), 1063–1078. Ameri, A.A., Pourghasemi, H.R., Cerda, A., 2018. Erodibility prioritization of sub-watersheds using morphometric parameters analysis and its mapping: A comparison among TOPSIS, VIKOR, SAW, and CF multi-criteria decision making models. Sci. Total Environ. 613, 1385–1400. Armstrong, J.S., Collopy, F., 1992. Error measures for generalizing about forecasting methods: Empirical comparisons. Int. J. Forecast. 8, 69–80. Chang, H.-S., Chen, T.-L., 2016. Spatial heterogeneity of local flood vulnerability indicators within flood-prone areas in Taiwan. Environmental Earth Sciences 75 (23), 1484. Chapi, K., et al., 2015. Spatial-temporal dynamics of runoff generation areas in a small agricultural watershed in southern Ontario. J. Water Resour. Prot. 7 (01), 14. Chapi, K., et al., 2017. A novel hybrid artificial intelligence approach for flood susceptibility assessment. Environ. Modell. Software 95, 229–245. Chen, W., et al., 2018a. Landslide susceptibility modelling using GIS-based machine learning techniques for Chongren County, Jiangxi Province, China. Sci. Total

322

Journal of Hydrology 573 (2019) 311–323

K. Khosravi, et al. based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Syst. Appl. 42 (20), 6844–6852. Opricovic, S., Tzeng, G.-H., 2004. Compromise solution by MCDM methods: a comparative analysis of VIKOR and TOPSIS. Eur. J. Oper. Res. 156 (2), 445–455. Pham, B.T., Bui, D., Prakash, I., Dholakia, M., 2016. Evaluation of predictive ability of support vector machines and naive Bayes trees methods for spatial prediction of landslides in Uttarakhand state (India) using GIS. J Geomatics 10, 71–79. Pham, B.T., Bui, D.T., Pourghasemi, H.R., Indra, P., Dholakia, M., 2017. Landslide susceptibility assesssment in the Uttarakhand area (India) using GIS: a comparison study of prediction capability of naïve bayes, multilayer perceptron neural networks, and functional trees methods. Theor. Appl. Climatol. 128 (1–2), 255–273. Pham, B.T., Jaafari, A., Prakash, I., Bui, D.T., 2018a. A novel hybrid intelligent model of support vector machines and the MultiBoost ensemble for landslide susceptibility modeling. Bull. Eng. Geol. Environ. 1–22. Pham, B.T., Prakash, I., 2018. Machine Learning Methods of Kernel Logistic Regression and Classification and Regression Trees for Landslide Susceptibility Assessment at Part of Himalayan Area, India. Indian Journal of Science and Technology 11 (12). Pham, B.T., Prakash, I., Jaafari, A., Bui, D.T., 2018b. Spatial Prediction of RainfallInduced Landslides Using Aggregating One-Dependence Estimators Classifier. J. Indian Soc. Remote Sens. 1–14. Pham, B.T., et al., 2018c. A Comparison of Support Vector Machines and Bayesian Algorithms for Landslide Susceptibility Modeling. Geocarto International (just-accepted) 1–36. Pham, B.T., Tien Bui, D., Prakash, I., 2018d. Bagging based Support Vector Machines for spatial prediction of landslides. Environmental Earth Sciences 77 (4), 146. https:// doi.org/10.1007/s12665-018-7268-y. Phuong, N.T.B., Duy, N.B., Nghiem, N.C., 2017. Remote Sensing for Monitoring Surface Water Quality in the Vietnamese Mekong Delta: The Application for Estimating Chemical Oxygen Demand in River Reaches in Binh Dai, Ben Tre. Vietnam Journal Of Earth Sciences 39 (3), 256–268. Pourghasemi, H., Pradhan, B., Gokceoglu, C., Moezzi, K.D., 2012. Landslide susceptibility mapping using a spatial multi criteria evaluation model at Haraz Watershed, Iran, Terrigenous mass movements. Springer 23–49. Pourkhabbaz, H., Javanmardi, S., Faraji Sabokbar, H., 2014. Suitability analysis for determining potential agricultural land use by the multi-criteria decision making models SAW and VIKOR-AHP (Case study: Takestan-Qazvin Plain). Journal of Agricultural Science and Technology 16 (5), 1005–1016. Shafizadeh-Moghadam, H., Valavi, R., Shahabi, H., Chapi, K., Shirzadi, A., 2018. Novel forecasting approaches using combination of machine learning and statistical models for flood susceptibility mapping. J. Environ. Manage. 217, 1–11. Shahabi, H., Hashim, M., 2015. Landslide susceptibility mapping using GIS-based statistical models and Remote sensing data in tropical environment. Sci. Rep. 5, 9899. Shirzadi, A., et al., 2017. Shallow landslide susceptibility assessment using a novel hybrid

intelligence approach. Environmental Earth Sciences 76 (2), 60. Solomatine, D., See, L.M., Abrahart, R., 2009. Data-driven modelling: concepts, approaches and experiences, Practical hydroinformatics. Springer 17–30. Solomatine, D.P., Ostfeld, A., 2008. Data-driven modelling: some past experiences and new approaches. J. Hydroinf. 10 (1), 3–22. Soni, J., Ansari, U., Sharma, D., Soni, S., 2011. Predictive data mining for medical diagnosis: an overview of heart disease prediction. International Journal of Computer Applications 17 (8), 43–48. Stoynov, P., Zlateva, P., Velev, D., Zong, X., 2017. Modelling of Major Flood Arrivals on Chinese Rivers by Switch-time Processes. IOP Conference Series: Earth and Environmental Science. IOP Publishing, 012006. Tehrany, M.S., Pradhan, B., Jebur, M.N., 2014. Flood susceptibility mapping using a novel ensemble weights-of-evidence and support vector machine models in GIS. J. Hydrol. 512, 332–343. Tehrany, M.S., Pradhan, B., Jebur, M.N., 2015a. Flood susceptibility analysis and its verification using a novel ensemble support vector machine and frequency ratio method. Stoch. Env. Res. Risk Assess. 29 (4), 1149–1165. Tehrany, M.S., Pradhan, B., Mansor, S., Ahmad, N., 2015b. Flood susceptibility assessment using GIS-based support vector machine model with different kernel types. Catena 125, 91–101. Termeh, S.V.R., Kornejady, A., Pourghasemi, H.R., Keesstra, S., 2018. Flood susceptibility mapping using novel ensembles of adaptive neuro fuzzy inference system and metaheuristic algorithms. Sci. Total Environ. 615, 438–451. Thai, T.H., Thao, N.P., Dieu, B.T., 2017. Assessment and simulation of impacts of climate change on erosion and water flow by using the soil and water assessment tool and GIS: case study in upper Cau River basin in Vietnam. Vietnam Journal Of Earth Sciences 39 (4), 376–392. Van Thanh, N., Le, D.T., Thinh, N.A., Lan, T.D., Hens, L., 2017. Shifting challenges for coastal green cities. Vietnam Journal of Earth Sciences 39 (2), 109–129. Van Tu, T., Duc, D.M., Tung, N.M., 2016. Preliminary assessments of debris flow hazard in relation to geological environment changes in mountainous regions, North Vietnam. Vietnam Journal Of Earth Sciences 38 (3), 277–286. Velasquez, M., Hester, P.T., 2013. An analysis of multi-criteria decision making methods. International Journal of Operations Research 10 (2), 56–66. Wang, S.-Y.S., et al., 2014. Changes in the autumn precipitation and tropical cyclone activity over Central Vietnam and its East Sea. Vietnam Journal of Earth Sciences 36 (4), 489–496. Wheater, H., Jakeman, A., Beven, K., 1993. Progress and directions in rainfall-runoff modelling. Zhang, Q., Gu, X., Singh, V.P., Shi, P., Sun, P., 2018. More frequent flooding? Changes in flood frequency in the Pearl River basin, China, since 1951 and over the past 1000 years. Hydrol. Earth Syst. Sci. 22 (5).

323