Planar surface detection for sparse and heterogeneous mobile laser scanning point clouds

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing journal homepage: www.elsevier.com/locate/isprsjprs

Planar surface detection for sparse and heterogeneous mobile laser scanning point clouds

T

Hoang Long Nguyen , David Belton, Petra Helmholz ⁎

Department of Spatial Sciences, Curtin University, Perth, WA, Australia

ARTICLE INFO

ABSTRACT

Keywords: Mobile laser scanning Sparse point clouds Planar surface Detection Segmentation Planarity value

Plane detection and segmentation is one of the most crucial tasks in point cloud processing. The output from this process can be used as input for further processing steps, such as modelling, registration and calibration. However, the sparseness and heterogeneity of Mobile Laser Scanning (MLS) point clouds may lead to problems for existing planar surfaces detection and segmentation methods. This paper proposes a new method that can be applicable to detect and segment planar features in sparse and heterogeneous MLS point clouds. This method utilises the scan profile patterns and the planarity values between different neighbouring scan profiles to detect and segment planar surfaces from MLS point clouds. The proposed method is compared to the three most stateof-the-art segmentation methods (e.g. RANSAC, a robust segmentation method based on robust statistics and diagnostic principal component analysis – RDCPA as well as the plane detection method based on line arrangement). Three datasets are used for the validation of the results. The results show that our proposed method outperforms the existing methods in detecting and segmenting planar surfaces in sparse and heterogeneous MLS point clouds. In some instances, the state-of-the-art methods produce incorrect segmentation results for façade details which have a similar orientation, such as for windows and doors within a façade. While RDCPA produces up to 50% of outliers depending on the neighbourhood threshold, another method could not detect such features at all. When dealing with small features such as a target, some algorithms (including RANSAC) were unable to perform segmentation. However, the propose algorithm was demonstrated to detect all planes in the test data sets correctly. The paper shows that these mis-segmentations in other algorithms may lead to significant errors in the registration process of between 1.047 and 1.614 degrees in the angular parameters, whereas the propose method had only resulted in 0.462 degree angular bias. Furthermore, it is not sensitive to the required method parameters as well as the point density of the point clouds.

1. Introduction Mobile laser scanning (MLS) has become a mainstream technology for 3D spatial data capturing. Since, the captured point clouds from the MLS are purely point data and only provide information or knowledge about the scanned area in an implicit manner, there is a need to extract explicit geometric features such as planes (e.g. building facets, roofs), cylinders (e.g. lamp pole), octagonal lamp poles, catenaries (e.g. power lines and transmission lines) from the captured MLS point clouds. Among those features, planar features are the most dominant features since they exist at most survey sites and especially in urban areas. Extraction of planar features is the initial step for many MLS point cloud processing steps such as modelling, registration and calibration (Chan et al., 2013; Previtali et al., 2014; Skaloud and Lichti, 2006). The plane based matching techniques, specifically the least squares

⁎

plane fitting adjustment (LSPFA) is widely used for these tasks (Rabbani, 2006; Skaloud and Lichti, 2006). The inputs for this technique are points belonging to planes and their plane parameters. Therefore, the points representing the planes in point clouds have to be determined. The quality of the extracted points representing these planes directly impacts the final quality of the registration and calibration processes. Hence, plane detection and segmentation is an essential first step for automating point cloud processing. Plane detection and segmentation is the process of grouping spatially neighbouring points that belong to the same planar surfaces into groups or regions. Numerous methods have been proposed to detect and segment planar surfaces in point cloud datasets. Most of them utilise the normal vectors of points that are usually estimated using either the Random sample consensus (RANSAC) (Vosselman and Maas, 2010), Principal Components Analysis (PCA) (Vosselman and Maas,

Corresponding author. E-mail addresses: [email protected] (H.L. Nguyen), [email protected] (D. Belton), [email protected] (P. Helmholz).

https://doi.org/10.1016/j.isprsjprs.2019.03.006 Received 27 April 2018; Received in revised form 12 March 2019; Accepted 12 March 2019 0924-2716/ © 2019 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

2010) or Robust and Diagnostic Principal Components Analysis (RDPCA) (Nurunnabi et al., 2014). Those methods do not take into account the point density of the point clouds. However, this is essential for MLS point clouds captured at high speed which leads to sparse and heterogeneous point density (more details provided in Section 2.2). Therefore, these properties of the MLS point cloud data may lead to special challenges to the recent normal vector estimation and planar surface detection approaches. For instance, Lari and Habib (2014) proposed a method for detecting planar as well as linear/cylindrical features which considered local point densities in the segmentation process. Nevertheless, this method has problems with features in low densities point clouds due to the PCA classification errors. The aim of this paper is to develop an algorithm to accurately detect and segment planar surfaces in sparse and heterogeneous MLS point clouds, especially for the registration process. Unlike TLS point cloud data or point clouds obtained using range images, the captured MLS point cloud data can be considered as partially organized because points are captured in sequences. This property of MLS point cloud data has not been utilised in plane detection and segmentation process. Our approach utilises this property of MLS point clouds. The proposed method is evaluated and compared with other state of the art method using three different datasets. This paper is organised as follows; Section 2 presents the background of this research, and Section 3 reviews related literature. Next, the new segmentation method is proposed in Section 4 and evaluation results are given along with the discussion in Section 5. The paper closes with the conclusion and future work.

height difference between the scanner and the target and (d) the orientation of the target and the orientation of the scanner. In summary, there are two MLS system dependant characteristics which affect the point density of the captured point clouds: (1) the scanner rate and (2) the scanner pulse rate. While the scanner rate affects the scan profile spacing the scanner pulse rate affects the point spacing. The other point cloud density influencing factors depend on the distance and orientation of the object to the scanner and will not be further discussed in this paper. 2.2. Sparse and heterogeneous MLS point clouds Currently, there are several MLS systems available. Puente et al. (2012) reviewed seven of them, more specifically the following systems: ROAD-SCANNER, IP-S2, TRIMBLE MX8, STREETMAPPER, VMX250 – RIEGL, MDL Dynascan, LYNX Mobile Mapper. Due to the different configurations of the MLS systems, the point clouds collected by these MLS systems will have different densities. For instance, the MDL Dynascan has the scanner rate 10 times slower than the RIEGL VQ-450 (see Table 1) leading to the average profile spacing being 10 times larger. In addition, the pulse rate of the MDL Dynascan system is over 15 times slower than the one of the RIGL VQ 450, leading to a much larger distanced between scan profiles. Therefore, profile spacing and point spacing of a point cloud captured from the MDL Dynascan S250 (Renishaw, 2015) are normally quite large. Specifically, using Eq. (1) the standard profile spacing of point clouds collected by the MDL Dynascan S250 and the RIEGL VQ-450 can be calculated and is presented in Table 2. The standard profile spacing of point clouds collected by the RIEGL VQ-450 for the same speeds are ten times smaller (see Table 2a). Furthermore, Table 2b also shows that using the MDL Dynascan S250, the profile spacing is much larger than its point spacing. These point spacing values are manually measured from the real captured point cloud data. In addition, surfaces in the captured scene can have different orientations, heights and distances to the scanner. As a result, point clouds representing different surfaces may have different point densities (i.e. different scan profiles and point spacing distances). To conclude, the captured point cloud datasets from a MLS system with a low scanner rate and low scanner pulse rate will have a low and heterogeneous point densities, and their profile spacing is usually much larger than its point spacing. In this paper, a MLS point cloud is considered as a sparse point cloud if it has all of the above properties.

2. Background 2.1. Types and point densities of a MLS point clouds MLSs can be classified into two types: MLS that has laser head(s) placed perpendicular (type 1) and MLS that has laser scanner(s) placed at an oblique angle (type 2) to the trajectory of the carrier vehicle. Point clouds captured by these two different MLS classes have different scanline patterns. Traditionally, the point density of a point cloud is considered as the number of points per m2 or number of points per m3. MLSs use line scanner(s) and utilise the movement of the carrier vehicle to obtain 3D point clouds. Each full circle scan is termed a ‘scanline’. Points of the same scanline on the same surface form a scanline segment or a ‘scan profile’. Cabo et al. (2015) defined a group of neighbouring scan profiles of the same scanline as a polyline. As mentioned in Cahalane et al. (2014), point cloud density of a MLS line scanners can be defined by the distance between two adjacent scan profiles called ‘profile spacing’ and the distance between two adjacent points of a scan profile called ‘point spacing’ (Fig. 1). The profile spacing is influenced by (a) the speed of the vehicle, (b) the orientation of the scanner, (c) the scanner rate and (d) the orientation of the target surface. In this paper, based on Cahalane et al. (2010) we define the standard profile spacing Dsl as the profile spacing of a vertical surface with orientation perpendicular to the laser beam which can be calculated by using Eq. (1).

Dsl =

V S

2.3. Local saliency features estimation problem with sparse point clouds Local saliency features such as surface normal and curvature are widely used in the processes of plane segmentation and registration of point cloud data (Nurunnabi et al., 2012; Rabbani et al., 2006). In general, local saliency features of a point are estimated based on the point and its neighbours. These neighbours are normally determined by using K-nearest neighbourhood (KNN) or fixed distance neighbour (FDN) algorithm. These two algorithms determine the local neighbouring points based on their distances to the query points. Consequently, since the point spacing is much smaller than the scan profile spacing, neighbouring points will tend to be selected from the same scanline if these two algorithms are used for sparse MLS point clouds. In the worst case it may occur that all of the neighbour points are selected from the same scanline. Because the selection of neighbouring points only belong to one scanline, there is no definition of a normal vector for a line, which will lead to inaccurate local saliency feature estimation. Moreover, a small surface (e.g. 1 m square or less) may be represented by less than 20 points. Using KNN or FND may lead to the problem that the number of neighbour points belonging to another closely adjacent surfaces outnumbers the number of points on the query surface. In other words, the number of outliers will outnumber the number of inliers. Consequently, the true normal and curvature of points on this

(1)

where

Dsl = standard profile spacing V = vehicle speed (m/s) S = scan rate (Hz) According to Cahalane et al. (2014), unlike the profile spacing, the spacing of points within the same scan profile is not influenced by the vehicle speed. Instead, it depends on (a) the scanner pulse rate (pulse repetition rate), (b) the distance from the scanner to the surface, (c) the 142

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 1. Scan profiles patterns of type 1 MLS in a simple environment.

region/surface growing based (Nurunnabi et al., 2012; Rabbani et al., 2006; Vosselman and Maas, 2010) and robust model fitting (Fischler and Bolles, 1981; Hough, 1962; Vosselman and Maas, 2010).

Table 1 MDL Dynascan S250’s and RIEGL VQ 450’s specifications.

Range Scanner rate Scanner pulse rate Accuracy

MDL Dyanscan S250

RIEGL VQ-450

up to 250 m up to 20 Hz 36,000 pps 1 cm

up to 800 m up to 200 Hz 550,000 pps 8 mm

3.1. Edge based segmentation Edge based segmentation approaches start by detecting edge and boundary points, followed by grouping the points that are inside the boundary into a region (Huang and Menq, 2001). The borders of each region are defined based on the curvatures or the smoothness of points (Kang et al., 2016). Those values are normally calculated using PCA on the local neighbourhood points of the query point. Due to the challenges of point spacing and scanline spacing in sparse MLS point clouds (Section 2.3), finding the boundary is not always possible. Edge based segmentation is not a common approach in point cloud processing.

Table 2 (a) Comparisons of the profile spacing distances between MDL Dyanscan S250 system and RIGL VQ 450 system at different speeds. For the profile spacing constant speeds of 20, 30, 40 km/h are estimated using Eq. (1); (b) comparisons of the point spacing distances of MDL Dyanscan S250 system collected at different distances between the scanner to the object. (a)

(b)

Profile spacing [m]

Point spacing [m]

3.2. Robust model fitting

Speed

MDL Dyanscan S250

RIEGL VQ450

Distance to the object

MDL Dyanscan S250

20 km/h 30 km/h 40 km/h

0.27 0.41 0.55

0.027 0.041 0.055

7 27 40

0.03 0.09 0.15

Robust model fitting approaches detect parameterized features and estimate the mathematical parameters of the interested features from the point clouds using voting techniques. Random sample consensus (RANSAC) (Fischler and Bolles, 1981) and Hough Transform (Hough, 1962) are the two most popular methods in the model fitting group. Deschaud and Goulette (2010) claimed that Hough Transform compared to RANSAC is too time consuming. Furthermore, Hough Transform is very sensitive to the segmentation parameters (Tarsha-Kurdi et al., 2007). RANSAC is possibly the most popular approach in detecting, fitting and segmenting planes in point cloud data. As the original objective of RANSAC does not aim to detect and segment planar features in point cloud data, numerous variants of RANSAC have been proposed to achieve this goal. Bauer et al. (2003) introduced a RANSAC based method to detect building façades in dense point clouds. The 3D point cloud data in this publication were not captured by laser scanning but obtained by using dense image matching. Boulaassal et al. (2007) applied their RANSAC variant to detect planar features in Terrestrial Laser Scanning (TLS) point clouds and archived outputs with 90% accuracy with their case study datasets. Deschaud and Goulette (2010) concluded

surface cannot be accurately estimated by using the existing methods (e.g. RANSAC, PCA and RDPCA). Finally, in a sparse point clouds, different surfaces may have different point densities, hence a fixed number of neighbour points (with KNN) or a fixed distance (FDN) seems to be not suitable to apply for the whole point cloud. Consequently, those issues may lead to special challenges to the process of plane detection in the sparse MLS point clouds. 3. Related literature Different approaches for segmentation can be categorized into four groups: edge based (Huang and Menq, 2001), scanline based (Cabo et al., 2015; Jiang and Bunke, 1994; Sithole and Vosselman, 2003), 143

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

3.4. Scanline based

that large planar features in noisy point clouds can be efficiently detected by using RANSAC, but RANSAC is very slow in detecting small planar features. Fujiwara et al. (2013) claimed that the distance threshold parameter for RANSAC needs to be adjusted relative to the point density in order to have accurate segmentation outputs. As the density in sparse point clouds can vary, this adjustment is a challenge. Li et al. (2017) proposed a variant of RANSAC which utilises normal distribution transform cells. Normal distribution transform cells is used to represent a point-cloud space. It considers points within a cell instead of neighbouring points. While this method is suitable for indoor MLS point clouds with its stable and controlled scanning environments, sparse and heterogeneous MLS point clouds with its different scanning environments require different cell sizes carefully determined. Previtali et al. (2014) proposed another variant of RANSAC that utilizes the normal vectors of points as well as the spatial proximity between points in detecting planar features. Because, the influences of the distance threshold parameter may be reduced, this approach will be chosen as the representative of RANSAC algorithm for the comparison in this research.

Scanline base segmentation approaches break each scanline into smaller scan profiles and then merge scan profiles of the same surfaces based on similarity criteria. Jiang and Bunke (1994) proposed a fast method for segmenting range images into planar patches. In range image data, each row is considered as a scanline. Firstly, each scanline is split into different scan profiles. Then, groups of three scan profiles are selected as seed regions for the region growing process if they meet three conditions (Hoover et al., 1996). These are: (1) the length of all of three scan profiles must be bigger than a pre-defined threshold, (2) these scan profiles must overlap with each other for a pre-defined percentage, and (3) the distances of every pair of points on any two segments have to be smaller than a specific distance. Sithole and Vosselman (2003) proposed a method for segmenting Airborne Laser Scanning (ALS) point cloud data that uses 2D profiles sampled in different directions. These profiles are generated by connecting points that satisfy pre-defined conditions (e.g. the slope between two points has to be smaller than a pre-defined threshold). This approach cannot be applied to segment MLS point clouds due to the difference in scanning geometry between ALS and MLS. Cabo et al. (2015) proposed a method to detect planar surfaces in MLS point cloud data by arranging scan profiles. In order to create straight scan profiles, firstly, point clouds are broken into different scanlines based on the scanline information captured by the scanner. Then, each scanline is split into different polylines based on the distances between consecutive points. If the distance between any two consecutive points is bigger than a fixed and pre-defined threshold Dg, they will be assigned to different polylines. Then, the 3D version of the Douglas Peucker algorithm (Douglas and Peucker, 2011; Ebisch, 2002) is used to split polylines into different straight scan profiles by setting a maximum distance Ddp to the original line. According to Cabo et al. (2015), the Dg threshold has to be manually specified. In theory, this parameter has to be bigger than the longest point spacing distance Dlps of a surface in order to properly detect this straight segment of the surface. In addition, this parameter should be smaller than the smallest distance Dsg between any two surfaces that are on the same scanline in order to ensure that the surfaces can be separated. However, according to Cahalane et al. (2014) different surfaces in the same point cloud may have different point spacings. Hence, problems may arise in determining this parameter. For instance, the point spacing of a surface (L3) can be equal or longer than the distance between two surfaces on the same scanline direction (L1 and L2) as shown in Fig. 2. In this case, in order to preserve L3 Dg needs to be set bigger than d – point spacing of L3. However, this will lead to L1 and L2 being considered as the same segment (Fig. 2). Consequently, a fixed Dg parameter is not suitable. Even in a point cloud that has the maximum point spacing less than the distance between two closest scan profiles, finding a desired parameter for a specific point cloud is not easy, especially with MLS system with low scanner pulse rate. Moreover, when forming scan profiles from polylines this method does not consider noise or outliers that may be present in the point

3.3. Region growing and robust diagnostic principal component analysis Region/surface-based methods start with a seed point and the estimation of the seed points’ local saliency features (e.g. normal vectors). Then adjacent points with similar characteristics (i.e. angle between the normal vectors of any two adjacent points has to be smaller than a threshold) are added to the growing region. Region based approaches are more robust to noise and usually give better performances than edge based method (Nurunnabi et al., 2012; Rabbani et al., 2006). Rabbani et al. (2006) proposed a segmentation method for TLS point clouds using smoothness constraints. It segments point clouds by labelling connected points that have similar normal vectors. Seed points for region growing are selected based on their residuals. The normal vectors of points are obtained using PCA on points and their neighbours. Neighbouring points are determined by using KNN or FDN methods. El-Sayed et al. (2018) proposed a plane detection algorithm that firstly divides the captured point cloud into different cubes based on octree-balanced density down-sampling approach. Then, the seed planes are selected based on the planarity values of the random points, which is estimated using FDN to determine the local neighbourhoods and PCA to calculate the planarity values. Finally, any points in the original dataset will be assigned to the seed planes if their distance to the seed planes are smaller than a threshold. Nurunnabi et al. (2012) proposed a region growing segmentation algorithm that considers the local saliency features of a point as well as the Euclidian distances between neighbouring points, and the orthogonal distances between a point and its neighbouring point’s plane. The first step of this approach is the determination of neighbourhoods for each point in the point cloud using the KNN algorithm. Then the normal vectors and curvatures of each point are estimated. As PCA is very sensitive to outliers, Nurunnabi et al. (2015) proposed the Robust and Diagnostic PCA (RDPCA) method to detect outliers and generate robust normal-curvature estimations. RDPCA uses Median and Mean Absolute Deviation (MAD) instead of mean and standard deviation as well as a robust normal and curvature estimation for outlier detection. RDPCA has been proven to outperform segmentation based on PCA and be more robust and accurate than RANSAC (Nurunnabi et al., 2015). For this approach RDPCA was utilised for the local saliency feature estimation. Finally, region growing is used with the condition that neighbouring points belonging to the same region must have similar normal directions and have orthogonal distances and Euclidian distances smaller than predefined thresholds. The final segmentation results in this research had a very high accuracy. Hence, we will consider this approach as one of the benchmark methods in this paper.

Fig. 2. Example of scan profiles with different point spacings; d is distance between two consecutive points. 144

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

than a predefined threshold. Fig. 5 shows examples of the split of scanlines into scan profiles using direction vectors. In order to segment scan profiles from the scanlines, two sub-steps are required: (a) direction vectors estimation and (b) forming scan profiles. 4.1.1. Direction vector estimation The direction vector of a point is considered to be identically with the direction vector of the scan profile that this point belongs to. Similar to the process of the estimation of the normal vector of a point, the first step is to select the local neighbouring points of the query points. The selected points need to belong to the same scan profiles with the query point. Then, PCA is used to calculate the eigenvectors of the point group. The eigenvector corresponding to the biggest eigenvalue of the covariance matrix formed by the query point and its neighbours is considered to be the direction vector of the point group. An example is presented in Fig. 6, where the vector E1 (i.e. the first eigenvector) denotes the direction vector of the point group.

Fig. 3. Split a polyline into different scan profiles using Cabo method: (a) input polyline, (b) outcome after the first iteration, and (c) final outcome.

cloud data. Depending on the geometry of the polylines, outliers may cause fail segmentation as shown in Fig. 3. In this example, there are three scan profiles in the polyline (1–4, 5–9 and 10–13). Due to the presence of an outlier (the 7th point), the final outcome is inaccurate. Furthermore, depending on the Ddp value, a point can belong to two different scan profiles (e.g. the 4th and 7th points), or they can form ‘fake’ scan profiles. Theoretically, a point only belongs to a single scan profile unless it is the point exactly on the boudary edge of two adjacent surfaces. Nevertheless, in reality, this event very rarely occurs. This may lead to bias angles between the normal vectors of the segmented planar features and the normal vectors of the ‘true’ planar features. Finally, the longest scan profile is assigned as the seed line for line arrangements. Then, the scan profile that meets the three conditions will be labelled as the next seed line. The conditions are: (1) belong to the consecutive scanline; (2) parallel with the seed line; (3) the distance D between begin or the end points of the seed line and this scan profiles is smaller than a threshold Dbe. This D is the profile spacing (distance between two adjacent profiles). Similar to RDPCA method by Nurunnabi et al. (2015) the method by Cabo et al. (2015) will be used for the comparison with the novel method introduced in the next section.

4.1.1.1. Local neighbourhood selection. In order to obtain the “true” direction vector of the query point, all of the selected points need to belong to the same scan profile. Nevertheless, current local selection algorithms such as KNN and FDN cannot assure that only neighbouring points situated in the same scan profile are selected, i.e. depending on the predefined parameters of the used algorithm (e.g. k parameter in KNN or fd parameter in FDN), the neighbouring points may be selected from various scanlines. This may lead to difficulties in removing outliers for the direction vectors estimation process. In fact, it is not feasible to select the “true” neighbouring points for all the points in just one step. Hence, at this step we aim to select the local neighbouring points belonging to the same scanline, and not necessary to the same scan profile as the query point. The scanline can be identified using the property of the MLS point clouds due to the points being captured in sequence (and many MLS systems providing this information). For this selection, we can use either KNN or FND within the scanline. Nevertheless, due to the property of the MLS point cloud data in some cases (Fig. 7), most of the selected neighbouring points of the query point (i.e. the yellow point inside the red circle) are not the “true” neighbours. The “true” local neighbouring points of the query points are the yellow points. Hence, determining the local neighbourhoods based on the distance criterion (i.e. using KNN or FDN algorithm) does not work here. This circumstance mainly occurs with the points near the boundary of two consecutive scan profiles that have different point spacing distances. To overcome this issue a new staged approached is proposed. Firstly, the initial local neighbouring points are selected by setting the number of neighbouring point’s parameter (nnp). The nnp number of points captured before and after the query point are selected as the neighbouring points of the query point. Hereby, the nnp parameter acts similar to the k parameter in the KNN algorithm but is considering the point indexes instead of distances criteria. Hence, the issues related to point density can be overcome. In example in Fig. 7, the nnp parameter is set to 5, leading to 5 yellow points and 5 green points be selected. If there is no noise in the captured point cloud as well as the value of the nnp parameter is smaller than the number of points in the scan profile, then the number of inliers in the selected local neighbouring points is always bigger than the number of outliers. In other words, the percentage of outliers is always smaller than 50%. Even if the value of the nnp parameter is bigger than the number of points in the scan profile (which rarely occurs), all of the inliers will still be selected. As a result, the number selected inlier points identified as the “true” neighbouring points is always maximised. They can then be used for further processing steps. Another advantage of this approach over KNN and FDN is even if the information about scanlines is not provided, this approach can be still applied since points in MLS point clouds are normally captured in

4. Methodology Unlike TLS point clouds or point clouds obtained using range images data, the captured MLS point clouds data can be considered as partially organized because points are captured in sequences as discussed before. Nevertheless, this information has often not been used in MLS point clouds processing, particularly in plane detection and segmentation process. Moreover, inspiring from the fact that points on the same scan profile belong to the same planar surface, we proposed a new planar surfaces detection and segmentation method namely Planes detection and Segmentation based on the Planarity values of Scan profile groups (PSPS). This method has three main steps: (1) forming scan profiles; (2) grouping scan profiles and (3) planes are detected and segmented based on the planarity values of different neighbouring scan profiles. The workflows of the method are shown in Fig. 4. 4.1. Segmenting scan profiles from the scanlines By definition, points of the same scan profile belong to the same planar surface. Hence the captured point cloud should be segmented into different scan profiles firstly. The 3D version of Douglas Peucker that was proposed by Cabo et al. (2015) is sensitive to parameters and not robust against noise. Additionally, the existing local saliency features such as normal vector and curvature seem not to be suitable for this task. Therefore, a new local saliency feature, namely the direction vector, is proposed. The idea is based on the assumption that neighbouring points have a high likelihood to belong to the same scan profile if they have similar direction vectors. In this paper, two direction vectors are considered to be similar if the angle between them is smaller 145

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 4. Plane detection and segmentation based on the planarity value of the scan profiles method workflows.

sequence. Finally, by using this approach, no data structure such as K-D tree or Octree is required. 4.1.1.2. Modified RANSAC algorithm. Although the proposed neighbourhood selection algorithm outperforms KNN and FND algorithm, it still cannot assure only inliers are selected. Therefore, a post-processing step needs to be performed to remove outliers from the selected points. Thus, we introduce a modified RANSAC algorithm; the workflow of this algorithm is illustrated in Fig. 8. Unlike the original RANSAC, this proposed algorithm does not select samples completely at random. Based on the assumption that the two consecutive points (e.g. captured before and after) of the query points have a high probability to be “true” inliers, for the first two iterations the selected samples are the query point and one of these two points. Specifically, for the first iteration the samples are the query point and the point captured before; for the second iteration the samples are the query point and the point captured after. Similar to the original RANSAC, for each iteration the size of the consensus is the number of points that have distances to the line formed by the samples smaller than a threshold Tr. If all of the selected points are inliers, the number of points will be equal to the number of inliers. In this case, the algorithm will be terminated, and all of the neighbouring points will be used to estimate the direction vector of the query point. If it is not the case, more iterations will be performed. At this stage, samples will be selected randomly from either neighbouring points captured before or after the query point. The reason behinds this technique is based on the assumption that points on the same side of the query point have higher likelihood to belong to the same scan profiles. The size of the consensus set for each iteration is determined, and the process is iterated a number of times. Finally, the subset that has the biggest number of inliers and contains the query point will be assigned as the “clean” subset. The reason behind these conditions is expanded in Fig. 9. There may exists the case that for the query points (red point), the number of “inlier” points (black points) in the selected local neighbourhood of points is less than the number of “incorrect inlier” points (purple points). Specifically, Fig. 9 illustrates a query point with its neighbouring points that are selected from three different scan profiles SP1, SP2 and SP3 (with the nnp parameter set to 8). In this case the maximum consensus consists of seven purple points, while the correct consensus only has six points including the query point. Consequently, if this criterion is not taken into account the query point will be considered to have the same direction vector as the purple points which is incorrect. One may argue that this issue can also solved by reducing the value for the nnp parameter (e.g. to 6 or 7). Nevertheless, it may not be a good solution in similar situations where the number of black points (correct consensus points) is just 5 or less. Furthermore, the aim is to propose a method that is not sensitive to the parameters. In other words, the required parameters for the proposed method can be

Fig. 5. Example of splitting a scanline into different scan profiles: (a) input scanline, (b) direction vectors of each point, and (c) different scan profiles.

Fig. 6. Direction vector of a group of points by using PCA.

Fig. 7. Problems in selecting local neighbouring points using KNN or FDN in a particular case with a real captured MLS point cloud.

146

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 10. Example of direction vectors of points with different Tr parameters: (a) direction vector when Tr is equal or bigger than d; (b) direction vector when Tr is smaller than d.

fixed for any sparse and heterogeneous MLS point cloud dataset. By introducing this criterion, the influence of the value of the nnp parameter on the quality of the outputs of the proposed approach is significantly reduced. Practically, it may occur the case that the query point lies very close to the intersection of the two scan profiles, e.g. point 9 in Fig. 10. In this case, the distance from the query point to both of the scan profiles may be smaller than the threshold Tr. Consequently, there will be two subsets that have the same number of inliers as well as containing the query point. Hence, in order to make an accurate decision, the linearity values of these two subsets will be compared. The linearity value of a group of points is considered as the second smallest eigenvalue of the covariance matrix of those points. The linearity value of a perfectly collinear group of points is zero. Therefore, the subset that has the smaller linearity value will be assigned as the ‘clean’ subset. By doing this, the direction vector of point “9” will be estimated as shown in Fig. 10(b) as the linearity values of group of “*” points is smaller than group of (o) points. This condition not only improves the quality of the estimation outputs, but also to compensate the sensitivity of the Tr parameter. Finally, in order to improve the processing time, the modified RANSAC will be terminated at any time when the number of inliers is equal to two third of the number of neighbouring points and the query point is in the recent consensus. This termination condition is based on the assumption that in the local neighbourhood selection step, at least 50% of “true” neighbouring points are always selected. The output of this algorithm will be used to estimate the direction vector of the query point.

Fig. 8. Modified RANSAC algorithm workflows.

4.1.2. Forming scan profiles Finally, a process similar to region growing is performed to determine the different scan profiles as proposed in Nguyen et al. (2016). Once again, the information about the captured order is utilised. The seed point of each scan profile is determined based on the point indices (e.g. point that has the smallest index will be chosen as the next seed point). Then neighbouring points will be assigned to the scan profile if they satisfy two criteria: (1) have similar direction vectors as the seed point and (2) have the orthogonal distances to the line formed by the seed point and the direction vector of the seed point less than a predefined threshold. Normally, a group of neighbouring points that do not belong to a planar surface (e.g. vegetation points), do not form a straight scan profile. Hence, those points will be removed from the outputs of this method.

Fig. 9. Example of direction vector estimation different colours indicate different scan profiles: purple points are points of scan profile SP1; Black points and the red point are points of scan profile SP2; Orange points are points of scan profile SP3 and green point is outlier points; the red point is the query point. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

147

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

4.2. Grouping scan profiles In theory, if different scan profiles belong to the same plane, they must be parallel or intersect. However, if two scan profiles in a MLS point cloud intersect with each other, then they belong to the same scanline and not belong to the same plane. Hence, two neighbouring scan profiles have a high likelihood to belong to the same planar feature if they are parallel. Based on this rule, after different scan profiles are formed, the neighbouring scan profiles (i.e. scan profiles of two consecutive scanline) which are parallel (i.e. have similar direction orientation) are grouped. This can be done by checking the angle between the direction vectors of each neighbouring scan profile with each other. If the calculated angle is smaller than a threshold, then it is assumed that they belong to the same object. Each group may represent one surface or contain scan profiles of more than one surface if these surfaces are adjacent to each other and have similar orientation with the ground. At this stage, the number of surfaces in each group cannot be determined. Therefore, further processing steps are needed.

Fig. 11. An example of two adjacent planar objects that have similar orientations.

calculated from any of two consecutive scan profiles (i.e. two scan profiles belong to two consecutive scanlines) in the group. After that, the planarity value of each group is calculated and compare with its planarity threshold. Any group that has the planarity value smaller than its planarity value threshold will be considered as a single planar object. The remaining groups are considered containing more than one planar feature. In order to accurately segment them from each group, we introduce region growing based on the planarity (RGPL) algorithm. RGPL start by determining the scan profile Li that only has neighbours on the left or on the right and its neighbour Li+1 are added to the seed segment group Sc and be removed from Gi. Then, the planarity threshold for this surface will be calculated by applying PCA on the points on Li and Li+1 multiplied with a factor. Next, the planarity of points on these this Li+1 and Li and the neighbour on the opposite side of Li (Li+2) (Fig. 11) will be estimated. If the calculated planarity value is equal or smaller than the planarity threshold, they will be added to Sc, and removed from Gi. Otherwise, Sc will be saved into the R list, and Sc will be cleared. The remaining scan profiles will be considered as a new group. The process is repeated until all scan profiles in all groups are assigned to different regions. RGPL is illustrated in Algorithm 1 as follows. After RGPL is performed on all of the unsegmented groups, the scan profiles near the boundaries between different planar objects needs to be carefully analysed when determining which surface, they belong to in order to have accurate results. Practically, in the scanning area, the case may exist that there are two planar objects that have similar orientation are adjacent to each other as shown in Fig. 11. In this case, there are two planar objects indicated by different colours (e.g. planar object 1 – green and planar object 2 – blue). If the middle scan profile (i.e. the longest scan profile) lies very close to the boundary between two surfaces, both of the planarity values calculated from the two green scan profile with the middle scan profile and from three blue scan profiles will be very small and may be smaller than the planarity threshold value of each corresponding plane. Hence, depending on the seed scan profile selection order, the middle scan profiles may be assigned to the surface 1 or it may be assigned to surface 2. Hence, in order to overcome this issue and provide accurate results, another step is introduced for the cased that a group contain more than two planar objects. After the algorithm 1 is completed, the last scan profile of each planar object is considered as the “undecided” scan profile (e.g. the middle scan profile in Fig. 11). Then, the planarity value of the planar object 1 with the “undecided” scan profile and the planarity value of the planar object 2 with the “undecided” scan profile are compared. The “undecided” scan profile will be assigned to the planar object that has smaller planarity value.

4.3. Planar detection and segmentation on the planarity of group of scan profiles (PSPS) Each group of scan profiles resulted from the grouping scan profiles step may contain one or more planar objects. Planar objects are detected and segmented based on the planarity value between different scan profiles. The planarity value of a group of points is considered as the smallest eigenvalue of the covariance matrix of those points. In perfect conditions (e.g. the planar object is perfectly smooth, and the accuracy of the scanner is 0 mm) the planarity value of a group of points representing a planar object will be equal to zero. These conditions cannot be met in reality as the planar objects are normally not perfectly smooth, as well as errors always exist in any measurements. In addition to the planarity value, in this study we make the assumption that each plane in the captured MLS point clouds are formed by at least two scan profiles due to the fact that at least two scan profiles are needed to identify a planar object parameters. Based on these, a novel plane detection and segmentation method namely plane detection and segmentation based on the planarity of group of scan profiles (PSPS) is proposed. As discussed in the previous section (i.e. Section 4.2) each group of scan profiles resulted from the grouping scan profiles step may contain one or more than one planar objects Therefore, there is a need to determine whether a group contain one planar object or more than one. In order to archive this objective, a planarity value threshold (Tp) needs to be set. If the calculated value of the planarity value of a group is smaller than this threshold, all of the points in this group are considered to belong to the same planar objects. If it is larger than the threshold, further steps will be performed. This planarity value threshold has the same role as the distance threshold in RANSAC algorithm. Practically, different planes may have different planarity values. It depends on many factors such as the roughness of the plane and the accuracy of the scanner. Consequently, it is not reasonable to set a fix value for the planarity value threshold for the whole MLS point cloud dataset. Hence, we introduce an adaptively planarity value. We assume that the planarity value of any two scan profiles of a plane is approximately the planarity value of this plane. The planarity value threshold of each group is set dynamically based on the planarity of points on two scan profiles in this group (e.g. the planarity values threshold of green plane in Fig. 11 is set based on the planarity of Li and Li+1). The first step of PSPS is to determine the number of scan profiles in each group. If a group has less than three scan profiles, it represents a single surface. Any group that has the number of scan profiles less than three will be assigned to a region. And the other groups will be used as inputs for the next steps. In the next step, the planarity value threshold of each group will be 148

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

accurately on a scan profile. They may lie on the same side or different side of the scan profile as shown in Fig. 12. The distances from the captured points to the correct scan profile vary (e.g. points 1, 9 and 12 have the largest distances to the correct scan profile resulting in the worst accuracy value) and are normally equal to or smaller than the accuracy of the MLS system. If there is always at least two points lying exactly on the scan profile, then Tr can be set to the accuracy of the MLS system. Nevertheless, this assumption is not always assured. Therefore, the best candidate points for the process that can be achieved are two points lying on the same side and having similar distances to the correct scan profile (e.g. point 2 and 3). In this case, the Tr threshold should not be set to equal to the accuracy of the MLS systems as the distance from point 12 to the L3 line (i.e. purple line) is larger than Tr. Hence, the value for Tr is suggested to be set to twice of the accuracy of the scanner used to account for the worst case when all of the neighbouring points lie on one side of the query point (i.e. point 7) as shown in Fig. 12, as well as to have more inliers in the same iteration (e.g. point 4, 5, 6, 7, 10 and 11 in the case of L3 line in Fig. 12). One may argue that by setting the value of Tr to twice of the accuracy of the scanner may lead to outliers being considered as inliers. However, it cannot be assured that a point that has the distance to a scan profile formed by its neighbouring points equal to twice of the accuracy of the scanner is an outlier (e.g. point 7 in Fig. 13).

4.4. Discussions of the required parameters The required parameters of each step have different impacts to the quality of the plane detection and segmentation outputs. While some of them have insignificant influences and can be fixed for a point cloud dataset; some of them can be determined based on the specifications of the MLS system, particularly from the accuracy of the laser scanner(s). The first parameter to be discussed is the parameter for local neighbourhood selection, the nnp parameter. Estimating the direction vector of a point is much simpler and more reliable than estimating its normal vector. Hence, the value of the nnp parameter has almost no impact on the outputs of the final results, and it can be fixed for all of the datasets. We suggest using nnp equal to 7. For the modified RANSAC algorithm, there are two parameters which need to be set, namely the number of iterations T and the distance threshold Tr. In theory, the number of iterations T should be set based on the number of selected neighbouring points. Hence, it can also be fixed for all of the datasets. Meanwhile, we can set the value for Tr equal to twice of the accuracy of the scanner. Due to imperfect measurements of MLS systems (e.g. the accuracy of the MDL Dynascan S250 and RIEGL VQ-450 is 1 cm and 0.8 cm respectively) the captured points of a scan profile normally do not all lie

Fig. 12. Example of different iterations of the modified RANSAC to detect outliers of the direction vectors; different colours indicate the line formed by different sampling points; the black line is the actual line; other lines are formed by different sampling points. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 13. Example of the particular case of the query point and its neighbouring points; the black line is the actual line; purple line is formed by sampling points. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) 149

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Table 3 Standard setting parameters for PSPS method.

Table 4 Setting parameters for the simulated dataset.

Parameters

Values

Comments

Parameter

Value

nnp T Ai Tr

7 10 10 Twice the accuracy of the scanner Twice the accuracy of the scanner

Can be fixed for all point clouds Can be fixed for all point clouds Can be fixed for all point clouds Can be fixed for point clouds taken with the same scanner Can be fixed for point clouds taken with the same scanner

nnp T Ai Tr DLi

7 points 10 iteration 0 degree 0 mm 0 mm

DLi

and is called approach “RANSAC without NV”. The implementation which does take into account the normal vectors is called “RANSAC with NV”. In contrast, the following methods have been applied to all datasets: robust segmentation method based on RDPCA (called RDPCA) and the plane detection based on the line arrangement (called Cabo based on the name of the first author paper introducing the method).

The step for forming scan profiles requires two parameters, namely the angle threshold Ai and the distance threshold DLi. The distance threshold DLi acts similar like the Tr parameter. As a result, it can also be set to equal a value twice that of the accuracy of the scanner. Meanwhile, if the dataset is errorless, the Ai parameter can be set equal to 0. However, due to the roughness of the surfaces and the accuracy of the scanner(s), it should be set to less than 10 degree as suggested by previous research for plane detection and segmentation purpose (Cabo et al., 2015; Previtali et al., 2014). This value can also be applied for the angle threshold in the grouping of scan profiles step. Table 3 summarizes the standard setting parameters for PSPS.

5.1. Evaluation using the simulated dataset The simulated dataset is an errorless simulated point cloud of a house consisting of four planes: a roof – plane 1, a building façade – plane 2, a window – plane 3 and a door – plane 4 as shown in Fig. 14. The point cloud dataset is heterogeneous, has a sparse point density with different point spacing distances (i.e. point spacing values vary from 0.01 to 0.014 m) and different profile spacing distances (i.e. profile spacing values vary from 0.008 to 0.095 m). In this dataset, there are three planar objects that have similar normal vectors (the window, the door and the house façade surface). The orthogonal distances between the door surface to the building façade and between the window surface and the building façade surface are 6 cm and 5 cm respectively. The reason behind this simulated dataset is based on the assumption that the simulated dataset is errorless, and the point density is known, so the ideal values parameters for all of the discussed plane detection and segmentation approaches can be determined. Therefore, the dataset can be used to evaluate the performances of all of the discussed.

5. Results and discussion Three datasets will be used for the evaluation and comparison of our proposed method with other state of the art methods. The first dataset is an errorless simulated point cloud of a simple house. The second dataset consists of point clouds of a target which can be used for MLS registration in some special cases (Nguyen et al., 2017). Three point cloud datasets of the target are captured at different conditions (e.g. different speeds and distances). They are all considered as sparse and heterogeneous MLS point clouds and vary in point cloud densities. They are difficult tasks for plane detection and segmentation algorithms due to their sparseness and the size of the planes of the target compared to the point spacing and density. Hence, it is a highly suitable object for the evaluation purposes. Finally, point clouds of a road corridor near the university campus captured by using the MDL Dynascan S250 are used as the third dataset. The point clouds of the road corridor have the “normal” size of the MLS point cloud. In the first two dataset the planes have similar properties (e.g. point density and roughness) and the issue of the normal vector estimation in sparse and heterogeneous point clouds. Therefore, we include the RANSAC approach by (Previtali et al., 2014) into the evaluation process. Nevertheless, there are two different implementations of this approach. The first implementation does not take into account the normal vector

5.1.1. Evaluation of the PSPS method As previously mentioned, there are five parameters required for segmenting scan profiles from the scanlines, namely number of neighbourhoods nnp, angle threshold Ai and number of iterations T, the threshold for the modified RANSAC line fitting, Tr and DLi. All of them are set equal to the suggested values as shown in Table 4. For this simulated dataset, the angle threshold to check the parallelism between adjacent scan profiles is set to 0. After grouping the scan profiles belong to surface 1, 2 and 3 are grouped in one group as their scan profiles are all parallel to each other (Fig. 15). And scan profiles of surface 4 are put into another group.

Fig. 14. The simulated dataset, different planes are indicated by different colours. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 15. Outputs from breaking scanlines into different scan profiles process. 150

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

For this dataset, the RANSAC without NV provides a much better output than RANSAC with normal vectors as shown in Fig. 18. It detects all of the planes. Nevertheless, some points of the roof are wrongly labelled as the wall as their distances to both roof surface and wall surface are equal and the number of points of the wall is higher than the number of points of the roof. 5.1.2.2. Evaluation of the RDPCA method. Similar to RANSAC with NV, there is also no guideline how to determine the most two important parameters of the RDPCA method, i.e. the number of neighbouring points and the percentage of outliers. Therefore, different combinations of values for these two parameters are applied in order to obtain the best outputs (Table 5). The angle threshold parameter is set to 0 degree. For RDPCA, the k parameter is set to equal 15, 20, 30 and 40 the percentage of outliers is set to equal 15%, 25%, 35% and 50%. As can be seen from the outputs of RDPCA (Fig. 19), similar to RANSAC with normal vectors, RDPCA is also sensitive to the parameters’ values when applying to MLS sparse point clouds. Different parameters give different segmentation outputs. For instance, for the case k = 20 and epsilon = 25%, roof surface is detected as two separate regions as the two scan profiles on the left are far away from the rest (Fig. 19(a)). The segmented outputs of the other three features are even worse. They are over-segmented into at least 3 regions. The roof can be properly segmented if we set k = 40 and epsilon = 25% (Fig. 19(d)). Nevertheless, in this case RDPCA fails in detecting and segmenting other three surfaces.

Fig. 16. Different planes are perfectly extracted; different colours indicate different planes. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

At the final step, no parameter values are required. By using PSPS and the automatic planarity threshold adaption, all of the planes are perfectly detected and segmented. All of the scan profiles are assigned to their correct planar surfaces as shown in Fig. 16. 5.1.2. Comparison with other state of the art plane detection and segmentation approaches 5.1.2.1. Evaluation of the RANSAC method. With regards to the RANSAC with NV, besides the distance parameter, two additional parameters are required which are related to the normal vector calculations, i.e. the number of neighbouring points and the angle threshold. The distance threshold and angle threshold are both set to 0 degree. Nevertheless, there is no guideline for choosing the values for the number of neighbouring points even for the errorless dataset and for when the point density is known. We can only ensure that neighbouring points are selected from at least two scanlines. Different values for this parameter are applied to test the effect and to find the best output for comparison. Particularly, the k parameter is set to equal 15, 30 and 40. As can be seen from Fig. 17, different k values provided different outputs. Specifically, for the case of k = 15, all of the planes are extracted. However, they are over-segmented (i.e. one plane is considered as more than one plane). For instance, the building façade surface is segmented into 5 different regions. Increasing the number of neighbourhoods will increase the possibility of having points from other surface in the neighbouring point group. Consequently, the possibility of points on the same surfaces may have different normal vector directions is higher. Indeed, for the case k = 30, only three planes are detected. In order to provide a good visualization, only detected planes that have more than 5 points are kept for visualization.

5.1.2.3. Evaluation of the Cabo method. In the case of Cabo method, the parameters are obtained based on the characteristic of the simulated dataset. Cabo can perfectly extract the roof plane. However, the other planes (i.e. building façade, door and window) are labelled as a single surface (Fig. 20). The reason for this is that the Cabo method only takes into account the line arrangement condition, while scan profiles of these three surfaces are parallel with each other. 5.1.3. Conclusions for the case of the simulated dataset As the simulated dataset is considered to be collected by a perfect MLS system, some of the parameters values can be ideally determined for RANSAC, Cabo and PSPS. However, with RANSAC considering the normal vectors and RDPCA, different parameters values still have to empirically determine in order to obtain best values. RANSAC performs better than the RANSAC considering the normal vector in this case. Different parameters’ values are applied in order to find the best outputs from RDPCA. The outputs from RDPCA are significantly influenced by the parameters’ values for the simulated dataset. Cabo fails in detecting and segmenting adjacent planes that have similar orientations. Finally, among these five methods, PSPS provides the best and perfect plane detection and segmentation outputs.

Fig. 17. Segmentation outputs by using RANSAC considering the normal vectors; (a) k = 15 and (b) k = 30. 151

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 18. Segmentation output by using RANSAC: (a) overview and (b) zoom in to the roof; different colours indicate different planes. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

park, a controlled relatively open environment. The target was scanned three times at 2 different speeds (e.g. 20 km per hour and 30 km per hour) and 3 different distances (e.g. 7 m, 27 m and 40 m) by using the Dynascan MDL S250 MLS system (Table 6). Due to the specification of the scanning system, the speeds of the vehicle and the distances between the target and the scanning system, the three captured point clouds are sparse. Three captured point clouds have three different point densities and have a noise level specific to the capture MLS system. Specifically, Dataset T1 has an average point spacing of 0.033 m and the average profile spacing is 0.32 m; Dataset T2 has an average point spacing of 0.01 m and the average profile spacing is 0.47 m; and Dataset T3 has an average point spacing of 0.15 m and the average profile spacing is 0.47 m. Moreover, in order to evaluate the robustness of all of the discussed plane detection and segmentation methods again

Table 5 Different setting parameters of RDPCA for the simulated dataset. Number of neighbouring points

Percentage of outlier (%)

15 20 30 40

15 25 35 50

5.2. Evaluation using the target dataset A target (Fig. 21) has been designed in order to fulfil the requirement for a successful plane-based registration process in some particular scanning cases. For this experiment, the target is placed in a car

Fig. 19. Segmentation outputs from RDPCA; different colours indicate different planes; (a) k = 20, e = 25%; (b) k = 40, e = 25%; (c) k = 30, e = 50% and (d) k = 40, e = 50%. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) 152

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

minimum error residual for the noise is if noise point has error residual less than the accuracy of the laser scanner, it is impossible to determine whether this point is noise or not. Specifically, seven noise points were added to plane 1 (TP1) (30%), five noise points were added to plane 2 (TP2) (33%) and eight noise points were added to plane 3 (TP3) (29%). Fig. 22 shows the visualisation of the 4 point cloud datasets. These four sparse and heterogeneous MLS point clouds datasets are special challenges for plane detection and segmentation because the planes in all of the four datasets are relatively small and represented by strongly limited number of points. As a result, they are highly suitable test objects for evaluation purposes. The point clouds of four datasets are visualised in Fig. 22. Table 6 shows the number of scan profiles and number of points of each plane of the target in different datasets. In order to obtain the benchmarks for evaluation and comparison the quality of all of the discussed methods, the planar surfaces of the target in the four testing datasets are manually extracted. The results are shown in Fig. 23. In this research, we also focus on the quality of the plane with respect to the inputs for the registration process. The inputs for the least square plane fitting adjustment model come from both the slave point clouds and the master point clouds (Previtali et al., 2014). The comparisons will be separated three two parts. In the first part, segmentation outputs will be compared with regards to the number of oversegmentation and under segmentation features. In the second part, segmentation outputs will be compared regarding to their role in slave point clouds. Finally, we will evaluate the segmentation outputs regarding to their role in master point clouds.

Fig. 20. Segmentation outputs by using Cabo.

1

5.2.1. Evaluation of the PSPS method For this dataset, the captured point clouds were split into different scan lines based on the laser positions. By using the laser position points as “break points”, the captured point clouds can be easily segmented into different scanlines. As discussed previously the required parameters for the scan profiles forming step can performed by either using standard values or based on the accuracy of the laser scanner of the used MLS system. However, in order to investigate the influences of these five parameters different values are applied. Specifically, the neighbourhood size parameter nnp is set to 5, 7 and 10 points; the angle threshold Ai were set 5, 7 and 10 degree; and the number of iterations T was set to 5, 7 and 10 iterations. The threshold for the modified RANSAC line fitting, Tr and DLi was set to 20 mm (twice the range accuracy of the MDL Dynascan S250 system) and 25 mm. Table 7 illustrates different parameters values for PSPS with the target datasets. Different combinations of parameters’ values are performed. The results show that changing the standard values for the parameters does not change the outputs for the four datasets. All of the scan profiles are successfully detected and segmented. All of the noisy points are removed from Dataset T4. After the scan profiles are formed, unlike in the simulated dataset, the standard value for the angle threshold parameter is set to 10 as discussed in Section 3.4. Nevertheless, similar to the previous step, different values for the angle threshold parameter (e.g. 5, 7 and 10

3

2

Fig. 21. The target consisting of three planes. Table 6 Vehicle speeds and distances from target to scanner of different datasets. Data set

Vehicle speed (km/h)

Distance (metres)

Plane TP1

Plane TP2

Plane TP3

T1 T2 T3 T4

∼20 ∼30 ∼30 ∼30

∼7 ∼27 ∼40 ∼27

4/64 3/16 2/7 3/16

3/71 2/10 2/8 2/10

3/64 3/19 2/8 3/19

noises, Dataset T4 is made up by adding noising points into the dataset 2. Twenty noising points with 100 mm standard deviation and 26 mm minimum error residual are randomly added. The reason behind the

Fig. 22. The visualizations of datasets 2.1–2.4: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4. 153

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 23. Manual segmentation outputs of the four datasets: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4; different symbols indicate different planes.

problems in detecting plane 2 and 3 of the target in Dataset T3. The reason for this failure is that points in the two middle scan profiles form a “fake” planar object that has more points than plane 2 and 3. Problems also arise when applying RANSAC in Dataset T4. Significant impacts of the noises to the segmentation outputs are visible. Different combinations of parameters values were used for RANSAC with NV with the target datasets as shown in Table 8. Due to the number of points of the target in Dataset T3 being too small (i.e. there are only seven, eight and eight points on plane 1, 2 and 3 respectively). RANSAC with normal vectors completely fails to detect the planes in this dataset. With regards to other three datasets, after trying all the values in Table 8, 50 mm for the distance threshold seems to be the most suitable value. While depending on the point density of the point clouds, the neighbourhood size parameter needs to be differently chosen. Specifically, the desire neighbourhood size parameters needs to be set to 30, 10 and 30 for Dataset T1, T2 and T4 respectively in order to get the best outputs. RANSAC with normal vectors again shows worse performances than the RANSAC in the target datasets. It can detect all of the target planes in Dataset T1, but with a lot of missing points on plane 2 and 3, while the segmentation outputs from Dataset T2 and T4 are not inaccurate (see Fig. 26).

Table 7 Different setting parameters for the target dataset. Parameters

Values

nnp T Ai Tr DLi

5, 7 and 10 5, 7 and 10 5, 7 and 10 20 and 25 mm 20 and 25 mm

degree) are applied to investigate its impact. After the grouping step was performed, the outputs from this step were not influenced by the changes of the parameters. For all of the four datasets, the scan profiles belonging to plane 1 are assigned to one group. Meanwhile, plane 2 and plane 3 were labelled to the same group as the scan profiles of these two surfaces are parallel to each other. Finally, the planarity threshold parameter is automatically determined After this step is performed, planes of the target in all of the datasets are properly detected and segmented without over- or under segmentation (Fig. 24(a)–(d)). In the case of Dataset T2, the planarity of the two scan profile belonging to plane 2 (black square) and the planarity of the two segments belonging to plane 3 (circle) as well as the middle segment are all smaller than the planarity threshold. By comparing the planarity value of different neighbouring scan profiles as discussed in Section 4.3, the middle segment is assigned to plane 3.

5.2.2.2. Evaluation of RDPCA. Similar to the experiment with the simulated dataset, different inputs’ values are applied in order to evaluate their effects and to find the best outputs that RDPCA can provide. The output with K = 30 and the percentage of outlier = 25% are shown in Fig. 27. It can be seen from Fig. 27(a) that RDPCA considered there are 7 planes in the point clouds of the target of Dataset T1, while there are 3 and 2 different planar surfaces detected in Dataset T2 and T3 respectively Fig. 27(b). Specifically, all of the planes of the target are detected in Dataset T1. However, many points are wrongly assigned into virtual surfaces, especially points near the boundary of plane 2 and 3. With Dataset T2, RDPCA can only detect plane 1 and 3. Meanwhile it completely failed when applied on Dataset T3 Fig. 27(c). All of three planes are detected by RDPCA in case of Dataset T4. Nevertheless, RDPCA fails in removing the noisy points. Consequently, many of them are labelled as belonging to one of the three planes of the target. The failures of RDPCA in this experiment resulted from the local saliency features estimation issue as discussed in previous section.

5.2.2. Comparison with other state of the art plane detection and segmentation approaches Currently, the quality of the recent plane detection and segmentation approaches is evaluated based on the number of over-segmentation and under-segmentation features. Under segmentation occurs when more than one “real” object are merged. Over-segmentation occurs when a single object is labelled to more than one object. 5.2.2.1. Evaluation of RANSAC. Unlike the simulated dataset, there is no indication of how to set the distance parameter value for the RANSAC without NV. Therefore, the distances threshold parameter was empirically determined. As can be seen from Fig. 25, RANSAC without NV works quite well with the Datasets T1 and T2. However, it has

Fig. 24. Segmentation results of the four datasets by using PSPS: (a) Dataset T1; (b) Dataset T2, (c) Dataset T3, (d) Dataset T4; different symbols indicate different planes. 154

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 25. Segmentation outcomes from RANSAC without normal vectors: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4; different symbols indicate different planes.

(MS). Table 9 shows the number of CS points, WS points and MS points of different plane detection and segmentation techniques in each dataset. As can be seen from Table 9, the quality of the detected planes obtained from RANSAC without NV, RANSAC with NV and RDPCA have negative correlations with the point density of the point clouds. The sparser the point clouds density is, the lower the plane detection and segmentation quality is. Particularly, the percentages of CS points detected by RANSAC are 58.7 percent for Dataset T1, 48.9 percent for Dataset T2 and down to 26.1 percent for Dataset T3. Meanwhile, with RDPCA the percentages of wrongly segmented points increase from 0 percent for Dataset T1 to 34.5 percent for Dataset T3, whereas the outputs obtained by using Cabo and PSPS are not affected by the reduction of the point density of the MLS point clouds. However, Cabo could not separate plane 2 and plane 3 from each other as they have the same orientation with the grounds. Finally, all of the four mentioned state of the art plane detection and segmentation methods are significantly affected by the presence of noises, especially with Cabo method. It could not detect any plane of the target. Meanwhile, PSPS is robust against the presences of the noises.

Table 8 Parameters list for the RANSAC with normal vectors. Neighbourhood size

Distance threshold (mm)

5 10 15 20 30

10 15 20 30 50

5.2.2.3. Evaluation of Cabo method. In this experiment, all of the required parameters are taken from (Cabo et al., 2015). In addition, only scan profiles that have more than 2 points are kept in order to get rid of the fake regions form by them (see Section 2.3.5). As can be seen from the results shown Fig. 28, the 3D version of Douglas Peucker algorithm shows a good performance in segmenting scan profiles from scanlines in the first three datasets as they are free from outliers. However, it completely fails when applying to the noisy dataset. After scan profiles are segmented, Cabo could not detect adjacent planar surfaces that have similar orientation to the ground. This is due to the fact that the scan profiles of these planes are parallel to each other and are neighbours of each other. Consequently, for Datasets T1 to T3, Cabo can only detect plane 1 and labels plane 2 and 3 as a single plane. The details about the segmentation outputs of Cabo method on the four datasets are illustrated in Fig. 28.

5.2.4. Quality of the segmentation output regarding to their role in master point clouds With respect to inputs from the master, the planes’ parameters are also the inputs for the LSPFA. The four parameters of a plane are the three elements of its normal vector and its distance to the origin. The number of equations that are affected by the plane parameters depends on the number of points on its corresponding plane on the slave point cloud. Depending on the number point on the corresponding plane on the slave point cloud, an incorrect plane in the master point cloud or master model can introduce several to thousands of incorrect equations to the least square model. Thus, the accurate of planes’ parameters play a very important role in the accurate of the final registration results. In order to evaluate the quality of the segmented planes from different techniques, we compare the bias angles (in degree) between the normal vectors of the ‘true’ planes (i.e. benchmarking planes) with the normal vectors of the extracted planes from all of the discussed methods. The parameters of the benchmarking planes as well as the detected planes parameters from different methods are derived by applying PCA on the

5.2.3. Quality of the segmentation output regarding to their role in slave point clouds With respect to inputs from the slave, each extracted point contributes an equation to the least square model. As a result, each point assigned to its correct surface introduces a correct equation, while each point assigned to it incorrect surface contributes an incorrect equation to the least square model. Each point that does not assigned to its surface also reduces the number of incorrect equations of the model. Therefore, after getting the segmentation results from the five discussed methods, the quality of the detected planes are assessed based on: (1) the number of points properly labelled to their surfaces – correctly segmented (CS); (2) the number of points wrongly labelled to one of the three surfaces – wrongly segmented (WS); and (3) the number of points which are not labelled to any of the three surfaces – miss segmented

Fig. 26. Segmentation outcomes from RANSAC: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4; different symbols indicate different planes. 155

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 27. Segmentation outcomes from RDPCA: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4; different symbols indicate different planes.

extracted groups of points representing different planes of the target. As discussed in previous section RANSAC, RDPCA and Cabo failed in detecting and segmenting planes in Dataset T4. As such we only compare the segmentation results of Dataset T1, T2 and T3. The differences between the angles of the detected planes and the benchmarks are shown in Table 10. As can be seen from Table 10, RDPCA and RANSAC with NV provide the worst quality inputs from the master point clouds for the least square model with the maximum difference in the benchmark of 12.5 degree and 5.3 degree respectively. Cabo offers quite good results when detecting plane 1, but it has problems with the adjacent planes that have similar orientation to the ground. The gaps between segmentation outputs from RANSAC without NV and the benchmarks are relatively small for the first two datasets. However, it could not distinguish plane 2 and plane 3 in Dataset T3. Meanwhile PSPS provides ideal segmentation outputs for master’s inputs due to is perfectly plane detection and segmentation outputs for all of the cases.

Table 9 Number of points of the final segmentation outcomes from different methods with Datasets T1-T4: CS – correctly segmented points; WS – wrongly segmented points and MS – miss segmented points; x – segmentation fail.

5.2.5. Conclusions for the case of the target dataset After considering all of the quality measurement approaches including traditional approaches and our proposed approaches with the target dataset, although the experiments are conducted with a small dataset our proposed plane detection and segmentation method shows a promising potential to be the most accurate method. However, it still needs to be proven to work efficiently with the normal size MLS dataset.

Methods

CS

WS

MS

Dataset T1

RANSAC with NV RANSAC without NV RDPCA Cabo PSPS

117 194 88 135 199

0 1 0 66 0

82 4 111 64 0

Dataset T2

RANSAC with NV RANSAC without NV RDPCA Cabo PSPS

22 42 31 26 45

5 2 12 22 0

23 3 14 19 0

Dataset T3

RANSAC with NV RANSAC without NV RDPCA Cabo PSPS

6 6 9 15 23

1 0 8 9 0

17 17 6 8 0

Dataset T4

RANSAC with NV RANSAC without NV RDPCA Cabo PSPS

20 37 25 x 45

6 11 31 x 0

25 8 16 x 0

and a front surface of an air condition (i.e. plane 6) near the supermarket, two phone boxes (i.e. plane 8 and plane 9), and two façade of a building near the supermarket (i.e. plane 10 and plane 11). On the other side of the road (Fig. 30(b)), there are 4 planes including 1 façade of a house (i.e. plane 14) with a window (i.e. plane 13), the roof of this house (i.e. plane 15) and the fence (i.e. plane 12). In these fifteen planes, besides the target, there are other planes that may cause challenges for detection and segmentation. For instances, plane 4 and plane 6 are adjacent to each other and have similar normal vectors. Plane 13, plane 14 and plane 15 are similar in circumstance with the simulated dataset but having the practically noise level. Plane 8 and plane 9 have outliers on their top (Fig. 31). In this experiment, ground points were removed in order to reduce the processing time. Ground points can be automatically removed by using ground surface extraction techniques

5.3. Evaluation using the road corridor dataset The third dataset was captured along a road corridor near the university campus using MDL Dynascan S250 system. This dataset has an average profile spacing is 25 cm. This dataset can be considered to have the normal size of the MLS point clouds dataset. The scanning area has planes at both of the left and the right sides of the scanner (see Fig. 29). Many of them are isolated from other, thus they are easy to detect and segment. Therefore, we only choose fifteen planes as shown in Fig. 30(a) and (b) for evaluation and comparison. There are 11 planar features on the left hand side of the road (Fig. 30(a)) including three planes of the target (i.e. plane 1, plane 2 and plane 3), a façade of the supermarket building (i.e. plane 4), 1 small planar object (i.e. plane 5),

Fig. 28. Segmentation outcomes from Cabo method: (a) Dataset T1; (b) Dataset T2; (c) Dataset T3 and (d) Dataset T4; different symbols indicate different planes. 156

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Table 10 Bias angle from different methods (x: undetected).

Dataset Dataset Dataset Dataset Dataset Dataset Dataset Dataset Dataset

T1 T1 T1 T2 T2 T2 T3 T3 T3

– – – – – – – – –

plane plane plane plane plane plane plane plane plane

1 2 3 1 2 3 1 2 3

RANSAC without NV

RANSAC with NV

RDPCA

Cabo

PSPS

0.3 0.2 0.2 0.1 0.2 0.6 0 x x

0.1 1.1 3.6 0.3 5.3 2.1 x x x

1.2 1.5 3.3 12.5 5.6 2.4 x x x

0.4 x x 2.1 x x 0 x x

0 0 0 0 0 0 0 0 0

Fig. 29. Visualization of road corridor dataset.

such as the robust ground surface extraction method proposed by Nurunnabi et al. (2015) or simply based on the height values of the points. For instance, for this dataset, a height threshold was set to 16.5 to remove the ground points. The benchmark used for the comparison was created by manually segmenting the dataset. Similar to the target datasets, there is no guidance to determine the required parameters for the state-of-the-art methods for this dataset. Hence, the different parameter values that were used for the target dataset are used for this dataset in order to find the optimal outputs from the different methods. Meanwhile for our proposed method, the same input parameters are used in order to investigate their sensitivity to the final segmentation outputs. In order to accurately detect and segment planes with different point densities and roughness, it has been proven that with RANSAC without NV requires different distance parameters. This will not be taken into account for comparison with this dataset. Next, the same subtests will be performed which also have been applied to the target dataset, i.e.:

– Test 1: Quality of the correct segmentation labelling (traditional approach). – Test 2: Quality of the segmentation outputs compared regarding to their role in slave point clouds. – Test 3: Quality of the segmentation outputs regarding to their role in master point clouds. 5.3.1. Evaluation of the PSPS method The same parameters’ values are applied in order to investigate the sensitivity of our proposed method to the final segmentation outputs. For this dataset, we only discuss the final plane detection and segmentation results. After all of the segmentation steps are performed, all of the planes are detected as shown in Fig. 32. Specifically, all of the target’s planes on the left-hand side are properly segmented; outliers near plane 8 and plane 9 were also removed; plane 4 and plane 6 are split into two different features. The window, the façade and the roof of the house on the right-hand side (i.e. plane 13, plane 14 and plane 5) are detected and segmented properly similar like the simulated dataset. 157

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 30. Visualization of fifteen planes in the road corridor dataset.

Regarding the non-planar points (e.g. vegetation points) in this dataset, most of them were correctly detected (i.e. they were not in the outputs). However, there were several groups of non-planar points that were detected as planar features. The reason is that those groups of points formed two parallel straight lines by accident. However, most of those mis-detected planes consisted two “incorrect” straight scan profile with a small number of points (e.g. several points). Hence, they can be removed from the outputs if needed by adding a condition that the outputs will only keep planes, which have more than a pre-defined number of points. 5.3.2. Comparison with other state of the art plane detection and segmentation approaches 5.3.2.1. Evaluation of RANSAC. As can be seen from Fig. 33, RANSAC with NV detects all the planar features in the dataset. Nevertheless, there are many missing segmented points in most of the planar objects except plane 12 and plane 7 as they are isolated from other objects. There are also a number of wrongly segmented points on plane 5 and plane 8. The reason for the erroneously segmented points in this case resulted from the issues with the normal vector estimation as discussed in Section 2.3. 5.3.2.2. Evaluation of RDPCA. Unlike RANSAC, RDPCA does not have many mis-segmentation problems. However, it is sensitive to the parameter values in the case where there are adjacent planes that have similar normal vectors. For instance, when the number of neighbourhood points is set to 40 and the percentage of outliers is set to 30 percent, RDPCA assigned plane 6 and plane 8 as a single surface. Nevertheless, by changing the values of those two parameters to 30 points and 25 percent the problem is solve. RDPCA also has issues with the points near the boundary of adjacent objects that have different normal vectors. For instances, with this dataset, points near the middle of plane 2 and plane 3 of the target are not assigned to their corresponding planes (see Fig. 34).

Fig. 31. Closer visualization of plane 8 and plane 9; points inside the yellow ellipse are outliers. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Plane 4 and plane 6 on the left-hand side are assigned into two separate regions. There are minor discrepancies between the detected planes and the benchmarks. Nevertheless, they are insignificant.

Fig. 32. Plane detection and segmentation outputs of the road corridor dataset by using PSPS.

158

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

Fig. 33. Plane detection and segmentation outputs of the road corridor dataset by using RANSAC.

Fig. 34. Plane detection and segmentation outputs of the road corridor dataset by using RANSAC with NV.

5.3.2.3. Evaluation of Cabo method. Similar to the target dataset, Cabo fails in detecting and segmenting planes that are adjacent and have similar orientation with the grounds (i.e. plane 2 and plane 3, plane 4 and plane 6, and plane 13 and plane 14). The polyline gap threshold value and Distance threshold between end-nodes parameter values need to be set carefully in order to prevent Cabo labels plane 10 and 11 to the same feature. Meanwhile the other planes are accurately detected as they do not have similar orientations with their neighbours (see Fig. 35).

mm level, a zero value does not mean the planes are perfectly extracted. As can be seen from Table 11, most of the detected planes from Cabo have very low mean error values (e.g. close to 1 mm). Among other three techniques, RANSAC seems to be the worse with the average mean error value is 9 mm and the maximum value is 61 mm of plane 6. Meanwhile, PSPS still provides the highest quality planes outputs with most of the mean error values are approximate 1 mm. 5.3.4. Quality of the segmentation outputs compared regarding to their role in master point clouds The results of the bias angle values between the detected planes and the benchmarks are shown in Table 12. Similar to previous section, the average bias angle value from Cabo is not calculated. As can be seen from Table 12, RANSAC is still the worse approach to obtain inputs from the master with the average bias angle from the benchmarks is 1.61 degree and the maximum bias angle is bigger than 7 degree (i.e. 7.32 degree – plane 14). RDPCA provides slightly higher quality results with the average bias angle around 1 degree and the maximum bias angle approximate 4 degree. The plane detection and segmentation outputs obtained from PSPS is the best inputs for the master input for MLS registration among all the discussed approaches with the average bias angle is less than 0.5 degree and the maximum value is less than 2 degree.

5.3.3. Quality of the segmentation outputs compared regarding to their role in slave point clouds Unlike the target dataset, for this dataset we evaluate the quality of the plane detection and segmentation results with respect to inputs from slave based on mean errors values of the detected planes instead of number of CS points, WS points and MS points. We assume that there is correlation between the mean error values and the quality of the LSPFA. Due to Cabo fails in detecting several planes, such as plane 2, plane 3, plane 4, plane 6, plane 13 and plane 14, the average mean error value from Cabo is not calculated. Hence, we will not compare Cabo with other techniques in this section. Nevertheless, the mean error values of the detected planes obtained by using Cabo are still calculated. Table 11 shows the mean error values of different planes as well as the average values of each technique. As the mean error values are rounded up to

Fig. 35. Plane detection and segmentation outputs of the road corridor dataset by using Cabo. 159

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al.

that have similar orientation with the ground surface. After implementing the evaluation with the three datasets, PSPS has been shown to outperform the existing state-of-the art methods and gives promising results with all of the datasets. By taking into account the property of the MLS point clouds (e.g. points are captured in sequence), the elements (e.g. points and scan profiles) near the boundary of different primitives can be easily determined. By performing carefully analysis, the quality of the final segmentation outputs can be improved. Although PSPS is only tested to detect planes in sparse and heterogeneous MLS point clouds, there is a potential to apply this method to dense MLS point clouds. Another advantage of PSPS is that it is not sensitive to the parameters’ values. The required parameters can be either automatically determined based on the specifications of the MLS system such as the accuracy of the scanner or they can be fixed. To conclude, with the high accuracy of the outcomes from PSPS, it seems to be very suitable to provide inputs for plane-based registration process of the sparse MLS point clouds as well as other processes.

Table 11 Mean errors of different planes detected and segmented by different approaches with the benchmarks in millimetre. Bold values indicate high values.

Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Average

RANSAC

RDPCA

Cabo

PSPS

1 3 1 1 12 61 18 7 1 0 3 0 2 17 7

2 2 6 0 1 15 2 11 13 0 1 0 0 35 3

1 x x x 0 x 15 0 0 0 0 0 x x 2

1 0 1 1 1 6 1 0 1 0 0 0 3 15 1

9

6

x

2

Acknowledgements This research is supported by the Curtin International Postgraduate Research Scholarship (CIPRS)/Vietnamese Ministry of Education and Training – Vietnamese International Education Development Scholarship.

Table 12 Bias angles of different planes detected and segmented by different approaches with the benchmarks in degree. RANSAC

RDPCA

Cabo

PSPS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0.619 0.797 0.159 0.021 1.624 3.168 3.676 1.077 3.255 0.179 0.204 0.841 0.296 7.326 0.978

0.235 2.352 0.450 0.085 0.380 2.390 3.561 2.843 1.411 0.027 0.232 0.344 0.134 0.397 0.878

0.580 x x x 0.633 x 3.972 1.852 2.598 0.007 0.395 0.072 x x 0.253

0.387 0.218 0.428 0.020 0.282 1.814 0.834 1.746 0.279 0.028 0.243 0.205 0.057 0.038 0.354

Average

1.614

1.047

x

0.462

Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane Plane

References Bauer, J., Karner, K., Schindler, K., Klaus, A., Zach, C., 2003. Segmentation of building models from dense 3D point clouds. Paper presented at the In Workshop of the Austrian Association for Pattern Recognition, Laxenburg, Austria. Cabo, C., García Cortés, S., Ordoñez, C., 2015. Mobile Laser Scanner data for automatic surface detection based on line arrangement. Autom. Constr. 58, 28–37. https://doi. org/10.1016/j.autcon.2015.07.005. Cahalane, C., McElhinney, C.P., Lewis, P., McCarthy, T., 2014. Calculation of targetspecific point distribution for 2D mobile laser scanners. Sensors (Basel, Switzerland) 14 (6), 9471–9488. https://doi.org/10.3390/s140609471. Cahalane, C., McElhinney, C.P., McCarthy, T., 2010. Mobile mapping system performance: an initial investigation into the effect of vehicle speed on laser scan lines. Paper Presented at the Remote Sensing Photogrammetry Society Annual Conference – “From the Sea-Bed to the Cloudtops”, Cork, Ireland. Chan, T.O., Lichti, D.D., Glennie, C.L., 2013. Multi-feature based boresight self-calibration of a terrestrial mobile mapping system. ISPRS J. Photogramm. Rem. Sens. 82, 112–124. https://doi.org/10.1016/j.isprsjprs.2013.04.005. Deschaud, J.-E., Goulette, F., 2010. A fast and accurate plane detection algorithm for large noisy point clouds using filtered normals and voxel growing. Paper Presented at the Proceedings of the 5th International Symposium 3D Data Processing, Visualization and Transmission, Paris, France. Douglas, D.H., Peucker, T.K., 2011. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or its Caricature Classics in Cartography. John Wiley & Sons Ltd., pp. 15–28. Ebisch, K., 2002. A correction to the Douglas-Peucker line generalization algorithm. Comput. Geosci. 28 (8), 995–997. https://doi.org/10.1016/S0098-3004(02)00009-2. El-Sayed, Emad, Abdel-Kader, Rehab F., Nashaat, Heba, Marei, Mahmoud, 2018. Plane detection in 3D point cloud using octree-balanced density down-sampling and iterative adaptive plane extraction. IET Image Proc. Fischler, M.A., Bolles, R.C., 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24 (6), 381–395. https://doi.org/10.1145/358669.358692. Fujiwara, T., Kamegawa, T., Gofuku, A., 2013. Evaluation of plane detection with RANSAC according to density of 3D point clouds. CoRR abs/1312.5033. Boulaassal, H., Landes, T., Grussenmeyer, P., TarshaKurdi, F., 2007. Automatic segmentation of building facades using terrestrial laser data. Paper Presented at the ISPRS Workshop on Laser Scanning. Hoover, A., Jean-Baptiste, G., Jiang, X., Flynn, P.J., Bunke, H., Goldgof, D.B., et al., 1996. An experimental comparison of range image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 18 (7), 673–689. https://doi.org/10.1109/34.506791. Hough, P.V.C., 1962. Method and means for recognizing complex patterns: Google Patents. Huang, J., Menq, C.-H., 2001. Automatic data segmentation for geometric feature extraction from unorganized 3-D coordinate points. IEEE Trans. Robot. Autom. 17 (3), 268–279. https://doi.org/10.1109/70.938384. Jiang, X., Bunke, H., 1994. Fast segmentation of range images into planar regions by scan line grouping. Mach. Vision Appl. 7 (2), 115–122. https://doi.org/10.1007/ bf01215806. Kang, Z., Zhong, R., Wu, A., Shi, Zhenwei, Luo, Zhongfei, 2016. An efficient planar feature fitting method using point cloud simplification and threshold-independent BaySAC. IEEE Geosci. Rem. Sens. Lett. PP (99), 1–5.

6. Conclusion and outlook In this paper we introduced a novel method for detection and segmentation planar features in a MLS point clouds. This method is especially suitable for sparse (low point density and heterogeneous) point clouds as frequently present in mobile mapping laser scan data collected at high speed. This proposed approach utilises the fact that points on the same scan profile belong to the same surface and the planarity between different scan profiles to detect and segment planar surfaces in MLS point clouds. A new saliency feature, namely the direction vector of a point and a novel local neighbouring points selection approaches are presented to split scan profiles from the scanline. The proposed plane detection and segmentation method was evaluated and compared with three state-of-the art segmentation methods: (1) RANSAC, (2) the segmentation method based on RDPCA and (3) the plane extraction method proposed Cabo et al. (2015) using three different test datasets. The new criteria to evaluate the quality of the plane detection outputs with respect to the inputs for LSPFA are introduced. While RANSAC with NV has limitations in detecting small planar features with many under segmented points, the method based on RDPCA fails in segmenting sparse point clouds. Furthermore, the inputs parameters for these two methods need to be adjusted in order to be suitable with the point density of the point clouds. Cabo method can only handle sparse point clouds without noise and fails to detect neighbouring surfaces 160

ISPRS Journal of Photogrammetry and Remote Sensing 151 (2019) 141–161

H.L. Nguyen, et al. Lari, Z., Habib, A., 2014. An adaptive approach for the segmentation and extraction of planar and linear/cylindrical features from laser scanning data. ISPRS J. Photogramm. Remote Sens. 93, 192–212. https://doi.org/10.1016/j.isprsjprs.2013. 12.001. Li, L., Yang, F., Zhu, H., Li, D., Li, Y., Tang, L., 2017. An improved RANSAC for 3D point cloud plane segmentation based on normal distribution transformation cells. Remote Sens. 9, 433. Nguyen, H.L., Belton, D., Helmholz, P., 2017. Comparative study of automatic plane fitting registration for mls sparse point clouds with different plane segmentation methods. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. IV-2/W4, 115–122. https://doi.org/10.5194/isprs-annals-IV-2-W4-115-2017. Nguyen, H.L., Belton, D., Helmholz, P., 2016. Scan profiles based method for segmentation and extraction of planar objects in mobile laser scanning point clouds. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLI-B3, 351–358. https://doi.org/ 10.5194/isprs-archives-XLI-B3-351-2016. Nurunnabi, A., Belton, D., West, G., 2012, 3-5 Dec. 2012. Robust segmentation in laser scanning 3D point cloud data. Paper presented at the Digital Image Computing Techniques and Applications (DICTA), 2012 International Conference on. Nurunnabi, A., Belton, D., West, G., 2014. Robust statistical approaches for local planar surface fitting in 3D laser scanning data. ISPRS J. Photogramm. Remote Sens. 96, 106–122. https://doi.org/10.1016/j.isprsjprs.2014.07.004. Nurunnabi, A., West, G., Belton, D., 2015. Outlier detection and robust normal-curvature estimation in mobile laser scanning 3D point cloud data. Pattern Recogn. 48 (4), 1404–1419. https://doi.org/10.1016/j.patcog.2014.10.014. Previtali, M., Barazzetti, L., Brumana, R., Scaioni, M., 2014. Laser scan registration using

planar features. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XL-5, 501–508. https://doi.org/10.5194/isprsarchives-XL-5-501-2014. Puente, I., González-Jorge, H., Arias, P., Armesto, J., 2012. Land-based mobile laser scanning systems: a review. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XXXVIII-5/W12, 163–168. https://doi.org/10.5194/isprsarchives-XXXVIII-5-W12163-2011. Rabbani, T., 2006. Automatic reconstruction of industrial installations using point clouds and images. PhD (Dissertation). Nederlandse Commissie voor Geodesie. Rabbani, T., van den Heuvel, F.A., Vosselmann, G., 2006. Segmentation of point clouds using smoothness constraint. Paper presented at the IEVM06. Renishaw, 2015. MDL Dynascan S250. < http://www.renishaw.com/en/dynascan-s250– 27332 > (retrieved 21/07, 2015). Sithole, G., Vosselman, G., 2003, 22–23 May 2003. Automatic structure detection in a point-cloud of an urban landscape. Paper presented at the Remote Sensing and Data Fusion over Urban Areas, 2003. 2nd GRSS/ISPRS Joint Workshop on. Skaloud, J., Lichti, D., 2006. Rigorous approach to bore-sight self-calibration in airborne laser scanning. ISPRS J. Photogramm. Remote Sens. 61 (1), 47–59. https://doi.org/ 10.1016/j.isprsjprs.2006.07.003. Tarsha-Kurdi, F., Landes, T., Grussenmeyer, P., 2007, 2007-09. Hough-transform and extended RANSAC algorithms for automatic detection of 3D building roof planes from lidar data. Paper presented at the ISPRS Workshop on Laser Scanning 2007 and SilviLaser 2007, Espoo, Finland. Vosselman, G., Maas, H.G., 2010. Airborne and Terrestrial Laser Scanning. Whittles Publishing/CRC Press, Scotland, UK.

161