Object-specific optimization of hierarchical multiscale segmentations for high-spatial resolution remote sensing images

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing journal homepage: www.elsevier.com/locate/isprsjprs

Object-specific optimization of hierarchical multiscale segmentations for high-spatial resolution remote sensing images

T

Xueliang Zhang, Pengfeng Xiao , Xuezhi Feng ⁎

Department of Geographic Information Science, School of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu 210023, China Collaborative Innovation Center of South China Sea Studies, Nanjing, Jiangsu 210023, China Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, Jiangsu 210023, China

ARTICLE INFO

ABSTRACT

Keywords: Image segmentation Segmentation scale Hierarchical multiscale segmentation Region merging Geographic object-based image analysis

Accurate segmentation of high-spatial resolution remote sensing images remains a challenging problem for geographic object-based image analysis. An object-specific optimization method for hierarchical multiscale segmentations is proposed in this study by fusing multiple segmentations into an optimized segmentation with specific consideration of each object. Based on a segment tree model representing hierarchical multiscale segmentations, the framework of object-specific optimization is achieved by identifying and fusing the meaningful nodes in each path originating from a leaf node. Within the optimization framework, an optimization measure for identifying meaningful node is designed according to the maximum change of homogeneity in a path. The proposed optimization method is experimentally validated to hold the advantage of improving segmentation accuracy by the manner of object-specific optimization as well as the potential of automatically producing optimized segmentation for successive object-based analysis.

1. Introduction High-spatial resolution remote sensing (HR) images provide detailed geometric information for geographic objects, in which geographic objects are presented with great heterogeneity and thus brings challenges for automatically extracting geographic information from HR images. To deal with the challenge caused by increased intra-object heterogeneity, the geographic object-based image analysis (GEOBIA) method provide an effective solution (Blaschke et al., 2014; Chen et al., 2018a,b), in which accurate image segmentation serves as a key prerequisite by partitioning an image into disjoint regions as image objects (Dey et al., 2010; Hossain and Chen, 2019). From geometric view, the benefit of image segmentation for GEOBIA lies in providing the extent and boundary information of geographic objects without involving semantic meaning. Another challenge caused by increased spatial resolution is the increased variety among different geographic objects. Specifically, geographic objects even belonging to the same land cover class present various shapes and sizes in HR images. Usually, image segmentation algorithms are controlled by a global threshold parameter for image features involved in segmentation procedure, through which a single segmentation is not able to separate various geographic objects into distinct regions. In this case, the multiscale segmentation strategy ⁎

remains popular to deal with the inter-object heterogeneity, where coarse-scale segmentations are suited for separating relatively large objects and fine-scale segmentations for small objects (Benz et al., 2004). Region merging method is widely used for producing multiscale segmentations by setting different thresholds, also named as scale parameters (Benz et al., 2004; Hay et al., 2005; Chen et al., 2012). Specially, hierarchical multiscale segmentation requires that segments at different scales are nested (Beaulieu and Goldberg, 1989; Zhang et al., 2013), which is beneficial for combining multiple segmentation scales for object-based analysis (Johnson and Xie, 2013; Wang et al., 2017). A remaining challenge of utilizing multiscale segmentations is to automatically select the optimal segmentation scale for successive analysis (Ma et al., 2017), for which the unsupervised segmentation evaluation method is widely adopted (Zhang et al., 2008; Chen et al., 2018a,b). Originally, a lot of efforts have been made to select a single optimal segmentation scale. On one hand, the global measure combining intrasegment homogeneity and inter-segment heterogeneity was widely used for unsupervised evaluation (Espindola et al., 2006; Corcoran et al., 2010; Johnson and Xie, 2011; Ming et al., 2015; Wang et al., 2019), through which the optimal segmentation scale is defined as the scale with maximized intra-segment homogeneity and inter-segment

Corresponding author at: Department of Geographic Information Science, School of Geography and Ocean Science, Nanjing University, China. E-mail addresses: [email protected] (X. Zhang), [email protected] (P. Xiao), [email protected] (X. Feng).

https://doi.org/10.1016/j.isprsjprs.2019.11.009 Received 15 July 2019; Received in revised form 8 October 2019; Accepted 12 November 2019 0924-2716/ © 2019 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

heterogeneity. On the other hand, the change of homogeneity in terms of all the segments was also explored for unsupervised evaluation, through which the optimal segmentation scale is identified as the scale exactly before abrupt decrement of homogeneity (Drǎguţ et al., 2010). However, the selected optimal segmentation scale still contains segments that are either too coarse or too fine because a single segmentation produced by a global scale parameter is difficult to separate various geographic objects (Johnson and Xie, 2011). To overcome the limitation of selecting a single optimal segmentation scale, the method of selecting multiple meaningful segmentation scales was developed with the assumption that the meaningful segmentation scale should be different for various geographic objects (Drǎguţ et al., 2010; 2014; Yang et al., 2014), in which a few meaningful segmentation scales covering those from coarse to fine were identified for successive analysis. However, since the evaluation measure is still calculated based on all the segments in a segmentation, it cannot identify the exactly optimal segmentation scale for different geographic objects. Furthermore, it needs additive efforts to select or fuse the optimal segmentation scales for successive analysis. Instead of automatically selecting single optimal segmentation scale (s) by a global evaluation measure, the idea of determining locally adaptive scale parameters catches great attractions in recent years, aiming at selecting optimal segmentation scale parameters for different regions or objects (Zhang and Du, 2016). There are mainly two types of automatic methods for determining local scale parameters. One is to locally tune a global optimal scale parameter according to the heterogeneity of local structure (Yang et al., 2017; Zhou et al., 2017; Dekavalla and Argialas, 2018; Xiao et al., 2018; Su, 2019), which could be called as local-structure specific optimization strategy. The other is to first partition the image into different regions or landscapes, and then to determine the optimal segmentation scale for each region by a global evaluation measure (Kavzoglu et al., 2017; Georganos et al., 2018), which could be called as region-specific or landscape-specific optimization strategy. The above methods achieved to set different local scale parameters to produce a better segmentation result. As a whole, the segmentation performance of these methods still depends on the effectiveness the adopted global evaluation measure. In addition, the theoretical foundation for the scale parameter localization strategy deserves being further explored to ensure that the local scale parameters are optimal for each geographic object. The aim of this study is to determine the optimal segmentation scale for each geographic object by exploring the change of homogeneity for each segment between adjacent segmentation scales. To achieve this idea, a technical basis is to build the inclusion relations between segments at adjacent scales by a segment tree model (Guigues et al., 2006; Salembier and Garrido, 2000; Lu et al., 2007; Li et al., 2016). Akçay and Aksoy (2008) proposed an optimization method to identify objectspecific optimal segmentation scales, but the effectiveness was only validated for a limited number of multiscale segmentations produced by morphological operations. Felzenszwalb and Huttenlocher (2004) and Navon et al. (2005) also achieved object-specific optimization method for natural images, but the effectiveness of which remains unknown for HR images that have higher complexity than natural images. Hu et al. (2017) developed a method to simplify the segment tree, but not to identify the optimal segmentation scale for each object. An object-specific optimization method for hierarchical multiscale segmentations based on region merging method is proposed in this study. It is accomplished by identifying and fusing the meaningful nodes for each path in a segment tree representing the hierarchical multiscale segmentations. The main contributions include: (1) a general object-specific optimization framework is developed to produce optimized segmentation for each geographic object; (2) a new optimization measure of selecting optimal segmentation scale for each geographic object is developed and the effectiveness of which is experimentally demonstrated by comparing with existed object-specific optimization measures; and (3) the proposed object-specific optimization method is

Fig. 1. Flow diagram of the proposed object-specific optimization framework for hierarchical multiscale segmentations.

experimentally validated to hold the advantage of improving segmentation accuracy and the potential of automatically selecting optimal segmentation scales for each object. The rest of this paper is organized as follows. The details of the proposed object-specific optimization method are described in Section 2. The experimental data, results, and discussions are presented in Section 3. The conclusions are finally drawn in Section 4. 2. Methodology 2.1. Overview The flow diagram of the proposed object-specific optimization framework for hierarchical multiscale segmentation is presented in Fig. 1. At first, the hierarchical multiscale segmentation is performed based on region merging method for a HR image. During the segmentation procedure, the binary partition tree (BPT) model is built to represent all the produced regions and the inclusion relation between segments at adjacent scales. An optimization measure is designed and then applied to identify optimal nodes in each path of BPT originating from a leaf node, where the optimal nodes appear at different segmentation scales for different geographic objects. After that, the identified optimal nodes are fused and output as optimized segmentation with full coverage of the image extent. The three main steps of building BPT, identifying optimal nodes, and fusing optimal nodes are described in detail as below. 2.2. Building binary partition tree during hierarchical multiscale segmentation The BPT model is built during region merging procedure by recording the whole merging sequence starting from the initial segmentation, as shown in Fig. 2. The leaf nodes represent the regions in the initial segmentation and the root node represents the whole image. A merging is recorded by creating a parent node representing the new region from the merging and linking it to its two child nodes representing the pair of regions that are merged. Therefore, the nodes of BPT represent regions produced during region merging and the links represent the inclusion relationship between regions at different segmentation scales. A path (or named as ancestry path) refers to a set of nodes and links originating from a leaf node to the root node, e.g. nodes 4, 8, 9 and links (4, 8), (8, 9) in Fig. 2(b), expressing the coarsening procedure of an initial segment by iterative merging. Hierarchical multiscale segmentation is achieved by a local-oriented region merging method (Zhang et al., 2013). Specifically, multiscale segmentations are produced by applying a set of equally increased scale parameters {S1, S2,…, Smax|S1 < S2 < … < Smax} from an initial segmentation that is apparently over-segmented. The segmentation at 309

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

Fig. 2. Constructing binary partition tree during region merging procedure: (a) the region merging procedure of producing each new region by merging two adjacent regions, and (b) the binary partition tree model built by recording the region merging procedure.

scale Sk+1 is produced by performing region merging iterations on that at scale Sk, which results in nested multiscale segmentations. It is noted that no matter how to control the region merging procedure to produce multiscale segmentations, the region merging procedure is accomplished by merging two adjacent regions iteratively. Hence, the BPT model can be built for any region merging method to record the hierarchical multiscale segmentations. The reasons of adopting the multiscale segmentation method (Zhang et al., 2013) include: (1) it can produce hierarchical multiscale segmentations that can be recorded by BPT; (2) it is convenient to output single-scale segmentations for comparison with the proposed object-specific optimization method; and (3) it is convenient to control the minimal and the maximal segmentation scales for the proposed optimization procedure, which will be illustrated in detail as below.

larger i indicates a greater homogeneity decrease caused by the merging for segment i, and the node with the greatest i in a path is identified as the optimal node. To illustrate the effectiveness of the measure of homogeneity change on identifying optimal nodes in each path, three examples showing the change of i and i in a path from the leaf node to the root node are presented in Fig. 3. Generally, i tends to increase as the segment is getting coarser by iteratively merging, indicating the trend of decreased homogeneity from the leaf to the root node. It is noted that when the segment is being very coarse after merging different geographic objects, i tends to change randomly. In this case, we consider to exclude the nodes that are too coarse for optimization. The greatest i in a path is able to identify the optimal scale for the paths in Fig. 3(a) and (b). However, if a geographic object is of high inner-heterogeneity, as shown in Fig. 3(c), i could be very large at the initial merging iterations before reaching the optimal scale. In this case, we consider to exclude the nodes that are apparently over-segmented for optimization. According to the above analysis, the optimal node in a path is identified as the node with the greatest i in the constrained scale range of [Smin, Smax], where Smin and Smax are set by users and adaptive to different images. The optimization measure is described as:

2.3. Identifying optimal nodes in each path of BPT During the region merging procedure, a segment is getting expanded by iteratively merging its adjacent segments. Accordingly, the homogeneity of a segment tends to decrease along with its expansion. At the initial merging iterations, the merged two segments are belonging to the same geographic object, the homogeneity presents a changing trend of gradual decrease. After that, if the two segments belonging to different geographic objects are merged, the homogeneity would decrease abruptly because the two segments are different with each other. The optimal node in each path is thus identified based on the assumption that the homogeneity of an optimal segment is going to abruptly decrease in the next merging with a segment belonging to different object. Specifically, the node with the greatest homogeneity decrease is identified as optimal in each path originating from a leaf node. The spectral standard deviation (σi) of a segment i is used to indicate its homogeneity, where a larger σi indicates a lower homogeneity, and vice versa. It is calculated as the mean standard deviation of each spectral band for a segment. i

B

=

j =1

ij/ B ,

Nopt = Argmax Ni ( i),Ni

=

where

p i

i, p i

(3)

where Ni represents a node in a path and Nopt is the identified optimal node in the path. The apparently over-segmented scales smaller than Smin are not involved in optimization procedure to avoid the negative influence caused by geographic objects with high inner-heterogeneity, and the apparently under-segmented scales larger than Smax are not involved to avoid the negative influence caused by the random homogeneity changes of very coarse segments. Even though two parameters Smin and Smax need to be set for the optimization measure, it should not be difficult for users to judge apparently over- and under-segmentation. In addition, how to set the two parameters will be discussed by analyzing the influence of different Smin and Smax on optimization results in the followed section for experiments.

(1)

where ij is the standard deviation of segment i for spectral band j, and B is the number of spectral bands in the image. The change of homogeneity i from node i to its parent node is thus defined as the change of the spectral standard deviation. i

{Segments produced in scale range[Smin,Smax ]}.

2.4. Fusing multiscale segmentations to achieve object-specific optimization Given the optimization measure as described above, how to identify and fuse the optimized nodes in each path to produce the object-specific optimization result is described in this subsection. Identifying the optimal node of a path is achieved by a bottom-up tracking strategy from the leaf node to the root node as shown in

(2)

is the spectral standard deviation of the parent node of node i. A 310

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

merging several initial segments. For a node Ni in the upper layer, one of its child nodes Nic may be identified as optimal in path j originating from leaf node j, while Ni is identified as optimal in another path k. As a result, there are two nodes Ni and Nic identified as optimal for path j. To deal with the uncertainty caused by more than one identified optimal node in a path, we suggest to output the optimized segmentation by a two-way fusion of the optimal nodes in a top-down manner, namely coarse fusion and fine fusion, as shown in Fig. 4(b). Since all the identified optimal nodes could be meaningful to represent the geographic objects, we choose not to abandon optimal nodes when there is more than one optimal node in a path. Specifically, the coarse fusion outputs the coarsest optimal nodes in each path while the fine fusion outputs the other optimal nodes. As to accomplish the two-way fusion, the coarse fusion is first performed followed by the fine fusion, producing two optimized segmentations. The coarse fusion is performed by tracking the nodes in each path by a top-down order. Specifically, during the tracking procedure for a path starting from the root node, when the first node identified as optimal is met, the optimal node is output as a segment in the coarse fusion result and the tracking procedure for this path stops and turns to another path. When all the paths have been tracked in the same way, the coarse fusion result is produced with full coverage of the image. Based on the coarse fusion result, the fine fusion continues to perform starting from the coarsest optimal node for each path by the top-down order. If a path has only one optimal node, the corresponding segment in the fine fusion result is same as that in the coarse fusion result. If a path has more than one optimal node, the child nodes identified as optimal are output as single segments in the fine fusion result, which actually further divides the corresponding segment in the coarse fusion result. As for the computational complexity of the proposed object-specific optimization procedure, it mainly comes from the bottom-up tracking of each path to identify optimal nodes and the top-down tracking of each path to output the optimal nodes. Hence, the computational complexity could be described as 2 × L × P, where L is the mean length of all the paths that is equal to the number of nodes in a path and P is the number of paths that equals to the number of initial segmentations. Since L is far smaller than P, the computational complexity could be viewed as linear to the number of paths in a BPT. 2.5. Existed object-specific optimization measures for comparison To validate the effectiveness of the proposed object-specific optimization framework and the optimization measure, three existed optimization measures proposed by Akçay and Aksoy (2008), Navon et al. (2005), and Felzenszwalb and Huttenlocher (2004), named as AA, NMA, and FH, respectively, are applied in the proposed framework. It can help to prove the compatibility of the proposed optimization framework by successfully exploring different object-specific optimization measures. Moreover, since the different measures are applied in the same optimization framework, the comparison results could clearly reflect the effectiveness of the proposed optimization measure. The core ideas of applying these measures in the proposed optimization framework are briefly introduced as below. The technical details of each measure are referred to the original publications. The optimization measure AA was originally developed for optimizing multiscale segmentations produced by morphological operations (Akçay and Aksoy, 2008). The change of homogeneity between node i and its parent is calculated as i × Ai and used as the optimization measure, where Ai is the number of pixels in node i. As noted in Section 2.4, a node in upper layers could belong to several paths. For AA, the optimal node is identified as the one with greater homogeneity change by comparing with its child nodes in all the paths it belongs to, rather than by comparing with all the other nodes in a path as the proposed method. In this case, AA produces a single optimized segmentation, but the identified optimal node may not achieve the greatest

Fig. 3. Examples showing the spectral standard deviation changes from the leaf to the root node in a path of the BPT. The red cross indicates the location of the geographic object. The labeled segmentation scale refers to that the corresponding merging happens. The scale marked as red indicates the optimal node in a path and those marked as blue indicate examples of over- and under-segmentation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 4(a). For the nodes in a path that are produced in the scale range [Smin, Smax], the i of each node is calculated in the bottom-up order according to Eq. (2). The optimal node of this path is then identified as that with the greatest i by Eq. (3). The optimal node is successively identified for each path in the same way. It has two properties for the identified optimal nodes: (1) each path has at least one node identified as optimal; and (2) a path may have more than one identified optimal node. The first property assures a full coverage for the optimized segmentation by outputting the optimal nodes of all the paths. The second property leads to additional uncertainty of identifying an exactly optimal segmentation scale for an object. Why a path could have more than one identified optimal node by the proposed method? It is noted that a leaf node exactly corresponds to a path, and a node in the upper layer of the tree belongs to several paths because the corresponding region is produced after 311

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

Fig. 4. Object-specific optimization procedure based on a binary partition tree.

homogeneity change in a path. The optimization measure NMA was originally developed for optimizing multiscale segmentations for natural images (Navon et al., 2005). The optimization measure is same as the homogeneity change i in the proposed method. In the original version of NMA, there is a threshold parameter for homogeneity change to exclude too fine scales for optimization. Since the scale range is constrained to exclude apparently over-segmented scales in the proposed framework, the threshold strategy is not involved when applying NMA in this study. Similar to the proposed method, the optimal node is also identified in each path for NMA. However, the optimal node in a path is identified as the first local maxima that satisfies i > ip and i > ic , where p and c represent the parent and the child of node i respectively. The difference between NMA and the proposed method comes from identifying the optimal node as the first local maxima or the maxima in each path. NMA also identifies more than one optimal node for a path, and the two-way fusion can be applied to produce two optimization results. Since the fine fusion result of NMA is apparently over-segmented according to visual judgement, only the coarse fusion result of NMA is presented for quantitative comparison in the followed section. The optimization measure FH was originally developed for optimizing segmentation during graph-based region merging procedure for natural images (Felzenszwalb and Huttenlocher, 2004). The spectral difference between a segment and its most similar neighbor is used as the optimization measure FH. The optimized node is identified as the node with spectral difference greater than its internal difference, where the internal difference of a node refers to the maximal spectral difference among all its child nodes in BPT. In this case, FH produces a single optimized segmentation, but the identified optimal node may also not achieve the greatest homogeneity change in a path. Originally, FH has a threshold function about region size to exclude apparently over-segmentations. Since the proposed optimization framework applies a scale range constraint to exclude over-segmented scales, the threshold function is not involved when applying FH in this study.

3.1. Datasets and segmentation accuracy assessment method The QuickBird images in Hangzhou City and Nanjing City, China, are used for experiments, which are composed of blue, green, red, and near-infrared bands. Three subsets from the image in Hangzhou are used for quantitative evaluation of the segmentation result and presented in Fig. 5, named as T1, T2, and T3, respectively. The spatial resolution is 0.6 m after pan-sharpening. The size of T1, T2, and T3 is 658 × 504, 538 × 546, and 996 × 550 pixels, respectively. A subset from the image in Nanjing, named as T4, is used for qualitative evaluation of the segmentation result with a size of 1478 × 974 pixels. The spatial resolution of T4 is 2.4 m without pan-sharpening. The optimized segmentations are quantitatively evaluated by the region-based accuracy measures of Precision, Recall, and F-measure (Zhang et al., 2015). A high Precision value combined with a low Recall value indicate the over-segmentation error, while a high Recall value combined with a low Precision value indicate the under-segmentation error. F-measure achieves a trade-off balance for Precision and Recall, indicating the overall segmentation accuracy, and a greater F-measure value indicates a higher segmentation accuracy. The accuracy measures are calculated by comparing the segmentation result with the reference. The references for T1, T2, and T3 were manually delineated by specialists in remote sensing and reviewed by others to catch any obvious errors, with 165, 107, and 292 reference segments, respectively. 3.2. Influence of the segmentation scale range on optimized segmentation accuracy The segmentation scale range [Smin, Smax] is the only parameter for the proposed object-specific optimization method, the influence of which on optimized segmentation result is evaluated by treating Smin and Smax separately, aiming at giving a guidance for how to effectively set the parameters. Specifically, Smin is set as 50 and Smax is gradually increased for exploring the influence of Smax. By contrary, Smax is set as 200 and Smin is gradually decreased for exploring the influence of Smin. It is noted that the segmentation at scale 50 and 200 is apparently overand under-segmented for the test images T1–T3 through visual analysis. Accordingly, the accuracies of the optimized segmentations by setting different scale ranges are calculated and presented in Fig. 6 for test images T1–T3. In addition, the number of regions in each optimized segmentation is also presented in Fig. 6 for further illustrating the influence of scale range. Given Smin, the influence of Smax on optimized segmentation accuracies is reflected by the left part of Fig. 6. The number of regions in the optimized segmentations keeps decreased as Smax increases for both coarse and fine fusion results, indicating that more segments with large size are identified as optimal because coarser segmentation scales are involved for optimization, which results in the decreased Precision and the increased Recall. It also shows that the overall segmentation

3. Experimental results and discussions The optimized segmentations are presented and evaluated in this section to validate the effectiveness of the proposed object-specific optimization method. Specifically, the influence of the constrained scale range on the optimization result is first analyzed. The potential of automatically selecting object-specific optimal segmentation scales and improving segmentation accuracy by the proposed method are then demonstrated. Furthermore, the effectiveness of the proposed method is experimentally validated by comparing with existed optimization measures of AA, NMA, and FH. Finally, the segmentation time of the proposed object-specific optimization method is evaluated.

312

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

Fig. 5. Test QuickBird images of T1 (a), T2 (b), and T3 (c) shown with color combination of near-infrared, red, and green bands. The boundaries of reference segments are shown with yellow lines. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

accuracy indicated by F-measure tends to quickly increase at first and achieves relatively steady state when Smax is getting very large. This shows that involving coarse segmentation scales could benefit for the object-specific optimized segmentation, and thus Smax should not be set a small value. When Smax is large enough, it cannot further improve the optimized segmentation accuracy by involving coarser scales, but does not lowering the F-measure values, which means that Smax should be set a large enough value with apparent under-segmentation. As for the difference between coarse fusion and fine fusion given Smin as 50, the accuracies of coarse fusion results are higher than those of fine fusion results. This is because the number of regions in fine fusion result is apparently larger than that in the coarse fusion result with the same Smax, which leads to the higher Precision and the lower Recall and thus the lower trade-off accuracy F-measure for the fine fusion result. Given Smax, the influence of Smin on optimized segmentation accuracies is shown in the right part of Fig. 6. The number of regions in optimized segmentations keeps increasing with the decreased Smin for both coarse and fine fusions because more segmentations with finer scales are involved for optimization. However, the overall segmentation accuracies do not change apparently for different Smin values by comparing with that for different Smax values, which shows that the sensitivity of Smin is lower than that of Smax for the proposed optimization method. Nevertheless, it is suggested not to set Smin as a large segmentation scale parameter, because a large Smin could result in higher Recall value than the Precision value for the optimized segmentation, indicating under-segmentation errors.

As for the difference between coarse fusion and fine fusion given Smax as 200, the values of F-measure between them are close to each other when Smin is relatively large. The F-measure of the fine fusion result tends to decrease slowly when Smin is small, while that of the coarse fusion result is almost unchanged. It is thus suggested not to set Smin very small for the proposed optimization method, especially for fine fusion, to achieve high segmentation accuracy. 3.3. Potential of automatic selection of optimal segmentation A basic application of the proposed optimization method is to automatically select an optimal segmentation from hierarchical multiscale segmentations. Since the scale range has a great influence on the optimized segmentation, four different scale ranges of [50, 125], [125, 200], [50, 200], and [90, 160] are applied and compared to validate this application potential. The scale range [50, 125] represents the case that is Smax too small, [90, 160] represents the case that Smax is not large enough, and [125, 200] and [50, 200] are compared to further illustrate the influence of Smin. The accuracies of optimized segmentations by applying different scale ranges are presented in Fig. 7 and Table 1, where Fig. 7 only shows the F-measure and Table 1 shows values for all the adopted accuracy measures. In addition, the F-measure for each segmentation scale between 50 and 200 with an equal increment of 5 are also presented in Fig. 7 for comparing with the optimized segmentations. It shows that the accuracies of optimized segmentations with scale range of [50, 125] and [90, 160] are apparently lower than those of the other 313

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

Given minimal segmentation scale 50

Given maximal segmentation scale 200

(a) T1

(b) T1

(c) T2

(d) T2

(e) T3

(f) T3

Fig. 6. Influence of the maximal/minimal segmentation scale on optimized segmentation accuracy with the given minimal/maximal segmentation scale.

two scale ranges because of the relatively smaller Smax. The accuracies of the optimized segmentations with scale range of [50, 200] and [125, 200] could be close to or even a bit higher than the greatest accuracy indicated by F-measure. In this case with large enough Smax, Smin needs to be cautiously set to be not too small for the fine fusion, otherwise it could result in relatively low overall segmentation accuracy by identifying over-segmented segment as optimal, as shown in Fig. 7(a) and (b) for the fine fusion result with scale range of [50, 200]. The above results demonstrate the effectiveness of the proposed method to select the optimal segmentation from hierarchical multiscale segmentations with a proper scale range constraint. When using global unsupervised evaluation measure to select a single optimal segmentation scale, the upper bound of accuracy for the selected optimal segmentation would be the best segmentation indicated by the supervised evaluation measure. The proposed method could produce optimized segmentation with accuracy close to or even higher than the best single segmentation, showing the advantage of automatically selecting optimal segmentation. To further illustrate the effectiveness of the proposed method in terms of automatically selecting optimal segmentation, taking test image T2 as an example, the optimized segmentations with Smax 200

are presented in Fig. 8 and compared with the single segmentation that achieves the highest F-measure. Both the coarse and fine fusion results with scale range of [125, 200] in Fig. 8(c) and (d) present similar segmentation pattern with the best single segmentation in Fig. 8(e). In addition, as marked by the green rectangles, the under-segmentation errors in the best single segmentation are eliminated by the proposed object-specific optimization method because multiple segmentation scales are involved for optimization, which results in a bit increase of the segmentation accuracy as shown in Fig. 7(b). The negative effects of too small Smin are illustrated in Fig. 8(a) and (b), as highlighted by the yellow rectangles, where the over-segmented regions are marked as optimal. A possible risk of coarse fusion about missing details is marked as the blue rectangles in Fig. 8, where the small water body is merged into its surroundings by the coarse fusion. Hence, the application potentials of the coarse fusion and the fine fusion should be different. The coarse fusion result can well segment the objects with relatively large size to achieve high Recall accuracy as shown in Table 1, but may have the risk of missing details of small objects and thus bring under-segmentation errors for these objects. The fine fusion result can reduce the risk of under-segmentation and thus 314

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

(a) T1

(b) T2

(c) T3

Fig. 7. Accuracies of the optimized segmentations with different scale range constraints compared with those the single segmentation. The different colors of the dashed lines for coarse fusion and the triangles for fine fusion represent the different scale ranges for optimization.

achieve high Precision accuracy as shown in Table 1, but it needs to cautiously set the parameter Smin to avoid over-segmentation.

the above problem. To validate the potential of improving accuracy for a single segmentation, the accuracies of the optimized segmentations with different scale ranges are compared with a single segmentation at scale 125 for T1–T3, which is neither too coarse nor too fine for each test image. Specifically, two strategies in terms of setting the scale range for optimization are applied and the results are shown in Fig. 9. The left part of Fig. 9 shows the optimized segmentation accuracies by equally expanded scale ranges, referring to the set of scale ranges {[120, 130], [115, 135], …, [50, 200]}. In this manner, the accuracies of the coarse fusion results can achieve higher overall segmentation accuracies, where the improvement of accuracy tends to increase when

3.4. Potential of improving segmentation accuracy Another application of the proposed object-specific optimization method is to improve the accuracy of a single segmentation produced by a global threshold parameter. As discussed in the Introduction, it always has over- or under-segmentation errors in a single segmentation because of the inter-object heterogeneity in HR images. Since the proposed optimization method aims at identifying specific optimal segmentation scale for each object, it could have the potential to overcome Table 1 Segmentation accuracies for fusion results with different scale ranges. Test image

Scale range for fusion

Coarse fusion

Fine fusion

Number of segments

Precision

Recall

F-measure

Number of segments

Precision

Recall

F-measure

T1

50–125 125–200 50–200 90–160

358 122 204 182

0.84 0.70 0.74 0.76

0.56 0.74 0.70 0.65

0.67 0.72 0.72 0.70

614 167 474 267

0.88 0.76 0.84 0.82

0.47 0.68 0.57 0.58

0.61 0.72 0.68 0.68

T2

50–125 125–200 50–200 90–160

190 63 111 98

0.88 0.81 0.82 0.82

0.72 0.81 0.78 0.78

0.79 0.81 0.80 0.80

284 82 214 125

0.91 0.85 0.90 0.87

0.66 0.77 0.70 0.73

0.76 0.81 0.78 0.79

T3

50–125 125–200 50–200 90–160

449 181 262 265

0.83 0.72 0.74 0.77

0.60 0.72 0.70 0.66

0.70 0.72 0.72 0.71

767 222 575 361

0.87 0.75 0.83 0.81

0.54 0.69 0.63 0.61

0.67 0.72 0.72 0.70

315

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

(a) coarse [50, 200]

(b) fine [50, 200]

(c) coarse [125, 200]

(d) fine [125, 200]

(e) best single 145

Fig. 8. Visual comparison of optimized segmentations (a)–(d) with the single segmentation with the greatest overall accuracy (e) for test image T2, where coarse and fine in (a)–(d) represent coarse fusion and fine fusion respectively. The detailed information of accuracies and number of segments for the optimized segmentations are presented in Table 1.

Fusion with similar region number

Fusion with equally expanded scale range (a) T1

(b) T1

(c) T2

(d) T2

(e) T3

(f) T3

Fig. 9. Comparison of optimized segmentation accuracies with that of the single segmentation.

316

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

(a) coarse [50, 200]

(b) coarse [50, 200]

(c) fine [105, 200]

(d) fine [105, 200]

(e) single 125

(f) single 125

Fig. 10. Visual comparison of the optimized segmentations from both coarse and fine fusions with the single segmentation for test image T1 and T3, where the optimized segmentations have similar number of segments with that of the single segmentation for each image.

the scale range is expanded. However, the accuracies of the fine fusion results do not present constant improvement compared with that of the single segmentation. This is because the number of regions in the fine fusion results are apparently larger than that in the single segmentation and thus resulting in lower overall segmentation accuracy. It is noted that the accuracy improvement by coarse fusion is not apparent for T2, which could relate to the difference of the region number in the optimized segmentations. Therefore, the strategy of equally expanded scale ranges needs to be cautiously applied with consideration of the number of regions in the optimized segmentations. Considering the limitation of the above strategy, the optimized segmentations with similar region number to the single segmentation are produced for comparison. The accuracies are presented in the right part of Fig. 9. In this manner, the accuracy difference caused by different region number is removed. It shows that the accuracies of the optimized segmentations by setting different scale ranges are improved comparing with that of the single segmentation in most cases. Specifically, the different scale ranges are represented by different Smax, where the Smin is tuned for each Smax to produce optimized segmentations with similar region number. The differences between the coarse and the fine fusion results are not apparent in terms of the overall segmentation accuracy, both of which tend to increase as the scale range expands.

This shows the benefit of involving large scale range for improving segmentation accuracy by the proposed optimization method. The optimized segmentations of T1 and T3 by both coarse and fine fusions using the strategy of producing similar region number are presented in Fig. 10 for visual comparison with the single segmentation, where the examples of removed under-segmentation and over-segmentation errors in the optimized segmentations are marked as green and yellow rectangles. It is noted that even though the accuracy of coarse fusion result is improved in comparison with the single scale segmentation, it may have the risk of bring under-segmentation errors in the optimized segmentation, as shown in Fig. 10(a). Hence, it is suggested to use the fine fusion strategy to avoid the risk of undersegmentation errors. To further illustrate the effectiveness of the fine fusion strategy, the optimized segmentation with scale range of [50, 200] is presented in Fig. 11 in comparison with the single segmentation at scale 60 that has similar region number. Generally, the single segmentation at scale 60 appears to be over-segmented. It clearly shows that the optimized segmentation can remove many over-segmentation errors for relatively homogeneous objects, such as the vegetation and water objects, while the geometric details for heterogeneous object could be well preserved, such as the building objects.

317

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

(a) Single segmentation at scale 60 with 5463 segments

(b) Optimized segmentation by fine fusion constrained by scale range [50, 200] with 5321 segments Fig. 11. Visual comparison of the optimized segmentation by fine fusion with the single segmentation for test image T4.

3.5. Comparison with other object-specific optimization measures

values for fine fusion results are not the smallest even though the number of regions is the most, which results in relatively high overall segmentation accuracies for fine fusion. Coarse fusion results can achieve the highest F-measure among the five measures, followed by fine fusion and NMA results, and FH measure has the lowest F-measure. The above results could demonstrate the effectiveness of the proposed optimization measures through quantitative evaluation. Taking test image T3 as an example, the optimized segmentations by different optimization measures with scale range [50, 200] are presented in Fig. 12 for visual comparison. The fine fusion result in Fig. 12(a) still looks clean even though it has much more regions than other optimized segmentations, which is because the fine fusion strategy can segment homogeneous objects into single regions and preserve the details for heterogeneous objects at the same time. The coarse fusion result is similar to the NMA result because the node with

The three existed object-specific optimization measures AA, FH, and NMA are applied in the proposed optimization framework as described in Section 2.5 for comparison. Given the same scale range [50, 200], the proposed optimization method and the three measures for comparison are implemented based on the same conditions for T1–T3. Accordingly, the differences among them can reflect the effectiveness of the optimization measures. The accuracies and the number of regions for the optimized segmentations produced by different optimization measures are presented as Table 2. Generally, the number of regions tends to decrease from fine fusion, AA, FH, NMA, to coarse fusion. Accordingly, the Precision values tend to decrease as the number of regions decreases, but the Recall values do not show a constant increasing trend. Specifically, the Recall 318

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

Table 2 Segmentation accuracies by different object-specific optimization methods with scale range [50, 200]. Image

Method

Number of segments

Precision

Recall

F-measure

T1

Fine fusion AA FH NMA Coarse fusion

473 374 328 222 204

0.84 0.80 0.79 0.74 0.74

0.57 0.58 0.55 0.66 0.70

0.68 0.67 0.65 0.70 0.72

T2

Fine fusion AA FH NMA Coarse fusion

214 219 156 122 111

0.90 0.84 0.83 0.86 0.82

0.70 0.68 0.72 0.75 0.78

0.78 0.75 0.77 0.80 0.80

Fine fusion AA FH NMA Coarse fusion

575 487 442 313 262

0.83 0.81 0.80 0.77 0.74

0.63 0.58 0.56 0.64 0.70

0.72 0.67 0.66 0.70 0.72

T3

Table 3 Comparison of segmentation time between the proposed optimization method and normal multiscale segmentation method. The segmentation time is calculated as a mean of 10 runs. Image

Normal multiscale segmentation method (s)

Proposed method (s)

T1 T2 T3 T4

1.8 1.4 2.8 12.8

1.9 1.5 2.9 18.3

coarse fusion results. It is noted that the other locally adaptive scale optimization methods are not adopted for comparison with the proposed objectspecific optimization method, such as the local-structure specific optimization strategy and the region-specific optimization strategy described in Introduction Section. In our opinion, all these strategies provide an effective solution to overcome the limitation caused by a global segmentation scale parameter. As for which strategy is the best solution, it deserves a systematic comparison study to answer the question in the future, including the theoretical foundation, the computational complexity, the robustness and the segmentation accuracy.

maximal i in a path belongs to the nodes with local maximal i . The difference is that the first local maxima may not be the maximal node in a path and thus resulting in several over-segmentations in the NMA result compared with the coarse fusion result. The optimization procedures of AA and FH are similar with each other by considering all the paths connected to a node, rather than identifying an optimal node for each path by the proposed method. In this manner, the optimal node identified by AA and FH may not achieve the maxima in a path and thus resulting in several over-segmentations compared with the fine and

3.6. Segmentation time The proposed object-specific optimization method is implemented using C# language. The codes are available upon the authors request. From the view of implementation, the segment tree model is built during the multiscale segmentation procedure, and the additional

(a) Fine fusion

(b) AA

(c) FH

(d) NMA

(e) Coarse fusion

Fig. 12. Visual comparison of optimized segmentations produced by different optimization measures with scale range [50, 200] for test image T3. 319

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al.

computation time compared with normal multiscale segmentation mainly comes from the identification and fusion of optimal nodes in each path, as described in Section 2.4. To clearly show the additional computation time for the proposed optimization method, it is calculated and compared with the segmentation time of the adopted multiscale segmentation method based on a PC with Core i5 CPU and 16G RAM, as shown in Table 3. For test images T1–T3, since the image size is relatively small, the additional segmentation time is almost neglected. For the relatively large test image T4, the additional time is about half of the time of multiscale segmentation method.

Benz, U.C., Hofmann, P., Willhauck, G., Lingenfelder, I., Heynen, M., 2004. Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information. ISPRS J. Photogram. Rem. Sens. 58 (3–4), 239–258. Blaschke, T., Hay, G.J., Kelly, M., Lang, S., Hofmann, P., Addink, E., Feitosa, R.Q., van der Meer, F., van der Werff, H., van Coillie, F., Tiede, D., 2014. Geographic object-based image analysis-towards a new paradigm. ISPRS J. Photogram. Rem. Sens. 87, 180–191. Chen, G., Weng, Q., Hay, G.J., He, Y., 2018a. Geographic object-based image analysis (GEOBIA): emerging trends and future opportunities. GISci. Rem. Sens. 55 (2), 159–182. Chen, J., Li, J., Pan, D., Zhu, Q., Mao, Z., 2012. Edge-guided multiscale segmentation of satellite multispectral imagery. IEEE Trans. Geosci. Rem. Sens. 50 (11), 4513–4520. Chen, Y., Ming, D., Zhao, L., Lv, B., Zhou, K., Qing, Y., 2018b. Review on high spatial resolution remote sensing image segmentation evaluation. Photogramm. Eng. Remote Sens. 84 (10), 629–646. Corcoran, P., Winstanley, A., Mooney, P., 2010. Segmentation performance evaluation for object-based remotely sensed image analysis. Int. J. Remote Sens. 31 (3), 617–645. Dekavalla, M., Argialas, D., 2018. A region merging segmentation with local scale parameters: applications to spectral and elevation data. Rem. Sens. 10 (12), 2024. Dey, V., Zhang, Y., Zhong, M., 2010. A review on image segmentation techniques with remote sensing perspective. In: Wagner, W., Székely, B. (Eds.), ISPRS TC VII Symposium – 100 Years ISPRS, XXXVIII (7A): 31–42. IAPRS, Vienna, Austria, pp. 2010. Drǎguţ, L., Tiede, D., Levick, S.R., 2010. ESP: a tool to estimate scale parameter for multiresolution image segmentation of remotely sensed data. Int. J. Geogr. Inform. Sci. 24 (6), 859–871. Drǎguţ, L., Csillik, O., Eisank, C., Tiede, D., 2014. Automated parameterisation for multiscale image segmentation on multiple layers. ISPRS J. Photogram. Rem. Sens. 88, 119–127. Espindola, G.M., Camara, G., Reis, I.A., Bins, L.S., Monteiro, A.M., 2006. Parameter selection for region-growing image segmentation algorithms using spatial autocorrelation. Int. J. Remote Sens. 27 (14), 3035–3040. Felzenszwalb, P.F., Huttenlocher, D.P., 2004. Efficient graph-based image segmentation. Int. J. Comput. Vision 59 (2), 167–181. Georganos, S., Grippa, T., Lennert, M., Vanhuysse, S., Johnson, B.A., Wolff, E., 2018. Scale matters: spatially partitioned unsupervised segmentation parameter optimization for large and heterogeneous satellite images. Remote Sens. 10 (9), 1440. Guigues, L., Cocquerez, J.P., Le Men, H., 2006. Scale-sets image analysis. Int. J. Comput. Vision 68 (3), 289–317. Hay, G.J., Castilla, G., Wulder, M.A., Ruiz, J.R., 2005. An automated object-based approach for the multiscale image segmentation of forest scenes. Int. J. Appl. Earth Observat. Geoinform. 7 (4), 339–359. Hossain, M.D., Chen, D., 2019. Segmentation for Object-Based Image Analysis (OBIA): a review of algorithms and challenges from remote sensing perspective. ISPRS J. Photogramm. Rem. Sens. 150, 115–134. Hu, Z., Li, Q., Zhang, Q., Zou, Q., Wu, Z., 2017. Unsupervised simplification of image hierarchies via evolution analysis in scale-sets framework. IEEE Trans. Image Process. 26 (5), 2394–2407. Johnson, B., Xie, Z., 2011. Unsupervised image segmentation evaluation and refinement using a multi-scale approach. ISPRS J. Photogram. Rem. Sens. 66 (4), 473–483. Johnson, B., Xie, Z., 2013. Classifying a high resolution image of an urban area using super-object information. ISPRS J. Photogram. Rem. Sens. 83, 40–49. Kavzoglu, T., Erdemir, M.Y., Tonbul, H., 2017. Classification of semiurban landscapes from very high-resolution satellite images using a regionalized multiscale segmentation approach. J. Appl. Remote Sens. 11 (3), 035016. Li, M., Stein, A., Bijker, W., Zhan, Q., 2016. Region-based urban road extraction from VHR satellite images using Binary Partition Tree. Int. J. Appl. Earth Observat. Geoinformat. 44, 217–225. Lu, H., Woods, J.C., Ghanbari, M., 2007. Binary partition tree for semantic object extraction and image segmentation. IEEE Trans. Circuits Syst. Video Technol. 17 (3), 378–383. Ma, L., Li, M., Ma, X., Cheng, L., Du, P., Liu, Y., 2017. A review of supervised object-based land-cover image classification. ISPRS J. Photogram. Rem. Sens. 130, 277–293. Ming, D., Li, J., Wang, J., Zhang, M., 2015. Scale parameter selection by spatial statistics for GeOBIA: using mean-shift based multi-scale segmentation as an example. ISPRS J. Photogram. Rem. Sens. 106, 28–41. Navon, E., Miller, O., Averbuch, A., 2005. Color image segmentation based on adaptive local thresholds. Image Vis. Comput. 23 (1), 69–85. Salembier, P., Garrido, L., 2000. Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans. Image Process. 9 (4), 561–576. Su, T., 2019. Scale-variable region-merging for high resolution remote sensing image segmentation. ISPRS J. Photogram. Rem. Sens. 147, 319–334. Wang, M., Cui, Q., Wang, J., Ming, D., Lv, G., 2017. Raft cultivation area extraction from high resolution remote sensing imagery by fusing multi-scale region-line primitive. ISPRS J. Photogram. Rem. Sens. 123, 104–113. Wang, Y., Qi, Q., Liu, Y., Jiang, L., Wang, J., 2019. Unsupervised segmentation parameter selection using the local spatial statistics for remote sensing image segmentation. Int. J. Appl. Earth Observat. Geoinform. 81, 98–109. Xiao, P., Zhang, X., Zhang, H., Hu, R., Feng, X., 2018. Multiscale optimized segmentation of urban green cover in high resolution remote sensing image. Rem. Sens. 10 (11), 1813. Yang, J., Li, P., He, Y., 2014. A multi-band approach to unsupervised scale parameter selection for multi-scale image segmentation. ISPRS J. Photogram. Rem. Sens. 94, 13–24. Yang, J., He, Y., Caspersen, J., 2017. Region merging using local spectral angle

4. Conclusions An object-specific optimization method for hierarchical multiscale segmentations of HR image was proposed in this study to identify the optimal segmentation scale for each object separately. Specifically, a general object-specific optimization framework is developed based on a BPT model representing the hierarchical multiscale segmentations, in which a scale range is constrained to remove the negative influence caused by apparently over- and under-segmentations. An optimization measure is designed to identify an optimal node in each path according to the homogeneity decrease across segmentation scales. Accordingly, the coarse and fine fusion strategies are developed to produce the final optimized segmentation. The influence of the scale range for both coarse and fine fusion results is thoroughly discussed, showing that a relatively large Smax is preferred for both of them and Smin needs to be cautiously set for fine fusion to not be too small. The proposed optimization method, including both coarse and fine fusion, is experimentally demonstrated to have the potential of automatically select the optimal segmentation and to improve accuracy for a single segmentation. The effectiveness is further validated by comparing with three existed object-specific measures, showing that the proposed method can achieve higher accuracies and fewer over-segmentation errors. As for the difference between coarse fusion and fine fusion strategies in the proposed framework, coarse fusion may have the risk of producing undersegmentations, and fine fusion may produce over-segmentations that could be reduced by setting Smin properly. Nevertheless, one of the benefits to produce coarse and fine fusion results is that the users can select each of them according to the requirement for specific applications. The future work will be focused on further improving the objectspecific optimization measure within the proposed framework from two aspects: (1) designing optimization measure to release the constraint of scale range; (2) designing class-specific optimization measure for optimizing segmentations for each land cover class. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements This work was supported by the National Natural Science Foundation of China (Grant No. 41601366, 41871235, 41871326), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160623), and the Fundamental Research Funds for the Central Universities (Grant No. 020914380068). References Akçay, H.G., Aksoy, S., 2008. Automatic detection of geospatial objects using multiple hierarchical segmentations. IEEE Trans. Geosci. Rem. Sens. 46 (7), 2097–2111. Beaulieu, J.M., Goldberg, M., 1989. Hierarchy in picture segmentation: a stepwise optimization approach. IEEE Trans. Pattern Anal. Mach. Intell. 11 (2), 150–163.

320

ISPRS Journal of Photogrammetry and Remote Sensing 159 (2020) 308–321

X. Zhang, et al. thresholds: a more accurate method for hybrid segmentation of remote sensing images. Remote Sens. Environ. 190, 137–148. Zhang, H., Fritts, J.E., Goldman, S.A., 2008. Image segmentation evaluation: a survey of unsupervised methods. Comput. Vis. Image Underst. 110 (2), 260–280. Zhang, X., Du, S., 2016. Learning selfhood scales for urban land cover mapping with veryhigh-resolution satellite images. Remote Sens. Environ. 178 (C), 172–190. Zhang, X., Xiao, P., Song, X., She, J., 2013. Boundary-constrained multi-scale segmentation method for remote sensing images. ISPRS J. Photogram. Rem. Sens. 78, 15–25.

Zhang, X., Feng, X., Xiao, P., He, G., Zhu, L., 2015. Segmentation quality evaluation using region-based precision and recall measures for remote sensing images. ISPRS J. Photogram. Rem. Sens. 102, 73–84. Zhou, Y., Li, J., Feng, L., Zhang, X., Hu, X., 2017. Adaptive scale selection for multiscale segmentation of satellite images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10 (8), 3641–3651.

321