Aircraft Skin Rivet Detection Based on 3D Point Cloud via Multiple Structures Fitting

Computer-Aided Design 120 (2020) 102805

Contents lists available at ScienceDirect

Computer-Aided Design journal homepage: www.elsevier.com/locate/cad

Aircraft Skin Rivet Detection Based on 3D Point Cloud via Multiple Structures Fitting✩ Qian Xie a , Dening Lu a , Kunpeng Du b , Jinxuan Xu a , Jiajia Dai a , HongHua Chen a , ∗ Jun Wang a , a b

Nanjing University of Aeronautics and Astronautics, Nanjing, China Xi’an Aircraft Industry (Group) Company Ltd., Xi’an, China

article

info

Article history: Received 10 August 2019 Received in revised form 14 October 2019 Accepted 7 December 2019 Keywords: Raw scanned point cloud Local point density Geometric model fitting Rivet contour

a b s t r a c t Rivet detection is usually the first step for almost all surface and rivet inspection methods in aircraft skins. With 3D laser scanners, one can rapidly obtain the precise 3D information, i.e. point cloud, of the surface and rivets. Subsequently, rivet detection can be converted to a multiple-structure fitting problem from 3D point clouds. However, robust structure fitting from scanned 3D point cloud remains an open problem due to its challenging nature, such as noise and outliers, irregular sampling density and missing scanning. To reduce the fitting variability, this paper presents an automated densityaware multiple-structure fitting algorithm to perform rivet detection based on a 3D point cloud. The key observation is that the local density of points belonging to the rivet contour is relatively higher. We hereby formulate rivet detection as a multiple structure fitting problem with a densitybased significance measure. By considering the local distribution characteristics, we first perform adaptive density enhancement on the basic local density. Subsequently, we detect the potential circle hypotheses and thereby extract rivet contours. By performing the mode-seeking algorithm on hypergraphs, all the circle structures can be obtained simultaneously. Overall, the proposed extraction algorithm is able to efficiently and effectively detect rivets from the raw scanned point clouds. We also demonstrate that the proposed algorithm achieves significant superiority over several state-of-the-art model fitting methods on the real scanned point cloud via experimental results. Moreover, we give the application of our algorithm on rivet flush inspection, showing that our method can assist in the rapid measurement of riveting quality. © 2019 Elsevier Ltd. All rights reserved.

1. Introduction In the field of aircraft manufacturing, the quality of riveting is a vital factor to the whole aerodynamic performance and long fatigue life of aircraft, which makes the inspection of the riveting quality indispensable [1–4]. In the automation inspection of riveting quality, rivet extraction is the first and most important step. In the past, rivets could only be recognized and qualitatively measured by human eyes. Nevertheless, human vision-based inspection lacks quantitative measurement. As a result, there is a high rate of missed detection, low efficiency and low reliability. Meanwhile, the large number of rivets used on the skin surface, as shown in Fig. 1, makes the manual inspection time-consuming ✩ No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.cad. 2019.102805. ∗ Corresponding author. E-mail address: [email protected] (J. Wang). https://doi.org/10.1016/j.cad.2019.102805 0010-4485/© 2019 Elsevier Ltd. All rights reserved.

and laborious. Recently, machine vision-based techniques are increasingly used in rivet extraction. By leveraging the advanced image processing algorithms, rivets can be automatically obtained to a certain extent. However, machine vision-based methods greatly depend on the quality of the required image data. Comparatively, a 3D-based (point cloud) data acquisition method could get more information, which assists in rapid rivet extraction. A 3D scanner is a rapid and non-destructive inspection technique which is being increasingly used in the aircraft inspection field, such as skin crack detection [5,6], gap measurement between skins and surface deformation inspection. The main advantages of the 3D scanner are rapid inspection and the ease of obtaining 3D point data with high precision. It can rapidly obtain a large number of 3D points, precisely reflecting the real shape of the corresponding surfaces, which is more suitable for aircraft skin-related inspections. Thus, we first scan the skin surface to obtain point clouds, by using a 3D laser scanner. Rivets will be presented as circle patterns in the scanned point cloud. An aircraft skin example scanned by 3D laser scanner is shown in Fig. 2.

2

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Fig. 1. Rivet structures on an aircraft skin. (a) Rivets are usually in regular arrangement. (b) The two-dimensional drawing of one kind of the most used rivet in aircraft manufacturing.

Fig. 2. The scanning procedure of the skin with rivets. (a) The 3D scanner used in this paper. (b) An aircraft skin with rivets inserted. (c) The scanning result represented as a point cloud. As can be seen, the scanning point cloud suffers from several problems, including severe noise and outliers, missing contours and density anisotropy.

Accordingly, points belonging to the rivets can be separately extracted using the proposed circle fitting method. However, circle pattern recognition on raw scanned point data is exceedingly challenging, due to the special characteristics of these data. First, the raw scanned data is inevitably contaminated with severe noise and outliers [7]. The regions without rivets on the surface will be scanned at the same time, which count for the majority of the space in the scanned point cloud. Points belonging to these regions are regarded as noise and outliers in the extraction of rivet points. Second, the density difference problem also hinders the recognition task. There are two kinds of density difference. One is the density difference between rivet contours and the non-rivet regions, which is the main principle used to recognize rivet contours. The other is that the density of rivets varies from one to the other. Rivets in some regions have high density values, while other rivets have a relatively low density value. Thus, it is hard to directly filter out the rivet regions via a global density value threshold. In this work, we address these challenges by introducing a density-aware structure seeking model with significance reference on the point density map to perform circle fitting. To precisely extract rivet contours, our circle fitting method takes into account not only the composition of the circle (i.e., the inliers), but also the density difference of the underlying circle points and their neighborhood points. This combination makes our circle fitting method particularly insensitive to the variation of density distribution. In this paper, our goal is to directly extract rivet contour points on the raw scanned point cloud via searching circle patterns. This is formulated as a multiple-structures fitting problem. However, the raw input point cloud suffers from heavy noise, outliers and density nonuniform distribution, i.e. density anisotropy. Thus, it is non-trivial to extract rivet contours via directly applying an existing circle fitting method, such as RANSAC [8], SCAMS [9], PM [10] and AKSWH [11]. Hence, we propose a density-aware multiplestructures fitting algorithm to automatically discriminate high density points from low ones. We found that the significant

Fig. 3. Illustration of the density anisotropy problem in the circle fitting task. (a) Synthetic point data with density anisotropy problem. (b) The corresponding density value histogram of the black line. As shown in (b), one can only detect the left circle if a global threshold is used to measure the significance. However, the right circle is also an underlying circle pattern when compared to its surrounding environment.

pattern not only depends on the local density, but also on the relative significance difference with the surrounding environment. That is, when we measure the significance of an underlying pattern, we should do it in a local instead of a global way. When local density structure is consistent with a circle, we can be confident in rivet head identification, though the circularity hypothesis might be masked if viewed in the context of the global density distribution, as shown in Fig. 3. Overall, our contributions are as follows:

• We propose a fully automatic rivet extraction algorithm from raw scanned 3D point clouds, which is capable of recognizing and localizing rivets rapidly and precisely. • We design a novel point cloud density computation method, which leverages the density information of the local neighborhood. • We devise a density-aware hypergraph-based multiple circle fitting algorithm to find all the circle patterns in the 3D point cloud. The rest of this paper is organized as follows. Section 2 gives a brief review of the related works. Section 3 describes the proposed adaptive point cloud density computation in detail. We then introduce the density-aware multiple structure fitting method in Section 4 and the post-processing scheme in Section 5. Section 6 presents the experimental evaluation results. The conclusion and future work are finally discussed in Section 7. 1.1. Related work We divide our discussion on the related work into the most two related parts, that is, rivet inspection and geometric model fitting. Rivet detection. Rivets are widely applied to assemble skins of the aircraft fuselage, making it of great importance to inspect the riveting quality during aircraft manufacturing and flight [12–14]. As the first step of most automated rivet inspection methods, the rivet detection problem has been extensively studied over decades [15,16], most of which are based on 2D image

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

3

Fig. 4. Overview of our rivet detection framework, which essentially contains two kernel stages, i.e., adaptive point cloud density computation and density-aware multiple structure fitting. We also show two related subsequent applications.

techniques. In general, various vision-based sensors are first utilized to collect image data. For instance, Park et al. [17] and Fan et al. [18] proposed to use the Hough transformation and morphological image processing method to detect circles in MOI (Magneto-optic Imager) images. Also based on magneto-optical images, Li et al. [19] first acquired the feature of rivet contour using the star radial vector method. The center of the rivet can then be calculated by the threshold method. However, as the authors mentioned in the paper, the center obtained by their method is approximate. To detect rivets in poorly illuminated steel structure environments, Paul et al. [20] proposed to perform rivet detection based on RGB–D sensors. Their method combines color image, depth and surface normal information using a probabilistic occupancy mapping technique. In comparison, we use the 3D scanner to obtain 3D point represented data, which contains more shape and geometric information. Additionally, our method can also be employed to detect and localize rivets from the raw scanned point cloud simultaneously, and be further applied to other geometric measurements in the field of rivet inspection. Geometric structure fitting. The purpose of geometric structure fitting is to estimate a set of underlying structures that best fit the data sampled from a high-dimensional space. The sampled data usually contains severe noise and outliers, which makes model fitting a challenging research problem. RANSAC [8] would be the most popular model fitting method, due to its efficiency and simplicity. Although designed for single-structure fitting, RANSAC can also be extended to multiple structures, by simply using a ‘‘fitting and removing’’ strategy. To robustly seek the underlying model under severe noise and outliers, a number of structure fitting methods have been proposed in the past few decades [21–24]. However, these fitting methods are still unsatisfied, when dealing with real-world data under complicated situations. In computer graphics, Abdul et al. [25] combines Robust Principal Component Analysis (RPCA) with an efficient algebraic circle fitting method to fit the circle from unstructured point clouds. Their method can achieve better circle pattern recognition compared to existing competitive methods under missing data and severe outliers. However, like RANSAC, their method is designed to estimate the parameters of one circle instance at a time. It requires a sequential ‘‘fitting and removing’’ procedure to segment multiple-circles. Nevertheless, the ‘‘fitting and removing’’ manner has several intrinsic drawbacks. For instance, the wrong fitting of the current model would disrupt the fitting operation in the last step, by incorrectly removing the inliers of the remaining models. To address this issue, Wang et al. [26] presented a subspace clustering-based multi-structure modeling algorithm. By considering the density anisotropy problem, their method is able to simultaneously detect multiple structures from the LiDAR point

cloud with severe outliers. However, their method is proposed to focus on detecting planes, and cannot be applied to fit circles in our scanned point data. Similarly, this paper introduces the local significance embedded into the hypergraph searching model to simultaneously segment circle patterns from raw scanned point cloud, corrupted by heavy noise and outliers, missing data and density anisotropy. 2. Overview In this section, we provide a brief overview of our method. The proposed approach takes as input the raw scans of aircraft skin with rivets acquired by 3D laser scanners, represented as unorganized 3D point clouds, as shown in Fig. 2. Our goal is to automatically extract 3D points of rivets directly from the raw scanned point clouds with severe noise and outliers. Fig. 4 shows an overview of our proposed modeling algorithm, which essentially consists of two kernel stages. Adaptive local point cloud density computation. Taking as input the raw point cloud of an aircraft skin with rivets, we generate the corresponding density scalar field. Considering the spatial heterogeneity of a scanned point cloud, we propose to compute the adaptive local point cloud density, instead of classical point cloud density. This is inspired by the Adaptive Contrast Enhancement (ACE) in image processing. Specifically, we first build the octree structure on the point cloud to accelerate neighborhood search. On this basis, we obtain the initial density estimation for a given point, by calculating the number of points within the r-neighboring sphere, followed by recomputing the adaptively local point density. Accordingly, the density scalar field of the point cloud is generated by considering the neighborhood of each point. Density-aware multiple structure fitting. In this step, we directly extract rivet contour points on the raw scanned point cloud via searching circle patterns, which is formulated as a multiple structure fitting problem. The raw input point cloud suffers from heavy noise, outliers and density nonuniform distribution. Thus, it is non-trivial to extract rivets via directly applying an existing fitting method. Hence, we propose a density-aware multiple structure fitting to automatically discriminate points with local significance. Based on the hypergraph searching model, we can simultaneously extract rivets, by fitting all the circle patterns from the raw scanned point clouds. 3. Adaptive local point cloud density computation In this section, we present an adaptive local point cloud density estimation approach to compute the local characteristics.

4

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

3.1. Local density of 3D point cloud

4. Density-aware multiple structure fitting

There are two basic local point density estimating methods [27]. The first one is done by calculating the number of points with a given space, while the other one is to compute the distances within a fixed number of points. Both of the two methods are based on the point cloud neighborhood, that is, r − radius neighborhood and k neighborhood, respectively. However, in our cases, it is hard to determine a fixed r value for all the input point cloud scans. Meanwhile, the points in our paper can be seen as belonging to a planar surface in the local area. Thus, we adopt the second method to compute the basic local point density. Specifically, for a given point pi (xi , yi , zi ) in the whole point cloud P, we first find its k neighborhood points with a pre-built KD-tree structure. The k is set to 25 in all the experiments. The k neighborhood of pi is defined as a point set Nk (pi ), whose Euclidean distance is closer to pi than the other points in P. Accordingly, the basic density d(pi ) of point pi can be simply calculated by the number of points within Nr (pi ), which is:

Given the density scores, we can extract the rivet contour points by searching circle patterns. For the whole input point cloud, its density map is obtained through the last stage. Based on the computed local density, the significance of each circle hypothesis is measured locally. We then extract the points belonging to the rivet contours, according to the obtained local significance values. In this section, we formulate the rivet contours detection problem as the circle pattern searching problem, which can be solved by robust fitting algorithms. Given a set of data points in d dimensions that usually contain outliers and noise, the task of robust fitting algorithm is to simultaneously estimate the number and the parameters of model instances in the given data. We then apply the state-of-the-art multi-structure fitting method called MSHF [29] to recognize circles from the given point cloud. MSHF is able to seek models via simultaneously detecting representative patterns without requiring iterative processes. As a consequence, the parameters and the number of model instances can be directly derived from the detected patterns. To achieve this, MSHF first formulates geometric model fitting as a mode seeking problem on the hypergraph, in which each vertex represents a model hypothesis θ (i.e., circles in our paper). Given a set of data points, MSHF first samples a set of minimum number of data points, which is necessary to estimate a model hypothesis (e.g., 2 for line fitting and 3 for circle fitting). A set of model hypotheses are then generated from the minimal subsets, and their inlier noise scales are also estimated accordingly. They applied the kernel density technique to estimate the noise scale, based on which the inliers belonging to each model hypothesis are distinguished from outliers. Having the initial model hypotheses, they then effectively weight each model hypothesis to measure the goodness of the hypothesis. After that, each vertex (i.e., model hypothesis) is connected to its corresponding hyperedges (i.e., the inliers of the model hypothesis). Based on the constructed hypergraph, they then search for representative modes by analyzing the weighting score of vertices and the similarity between the vertices and their neighbors. We refer the reader to [29] for a detailed description of the method. The most important parameter of MSHF is the weighting score w(θ ) which measures the significance degree for each hypothesis mode in the hypergraph construction stage. Essentially, MSHF searches for the representative modes according to the weighting scores assigned with each mode hypothesis. Thus, the scoring techniques directly determine the performance of the whole multi-structure fitting algorithm. However, the original weighting score formulation cannot be directly employed in our case, owing to the heavy outlier in the scanned point cloud. Hence, we propose to improve the weighting score via embedding the local significance pattern, which is shown in Fig. 5. Like [30], given a set of n points P = {p1 , p2 , . . . , pn }, we first sample a number of k-subsets (the minimum number of points which are required to calculate the model parameters) to generate circle hypothesis set Θ from the given n points. Note that the times of sampling should be sufficiently enough such that at least one subset is sampled for each circle. Here we adopt the adaptive sampling strategy in [31] to sample k-subsets in our paper. k is equal to 3 for circle fitting in this paper. Given an arbitrary circle mode hypothesis θ , we then present a novel weighting score measurement method to score each hypothesis for subsequent representative hypothesis searching. First, the residual set r(θ ) from all points to θ can be represented as {r1 , r2 , . . . , rn }. Then, according to the non-parametric kernel

d(pi ) =

k+1

(1)

2 π rmax

where rmax is equivalent to the maximum distance from point pi to one of Nk (pi ), that is, rmax = max(∥Nk (pi ) − pi ∥)

(2)

3.2. Adaptive local density Generally, the local density of points lying in the rivet contours is relatively high, which is caused by the scanning theory and sharp changes of curvature in the rivet contours. However, due to the influence of noise, there would be some non-contour regions with very high density. These noise regions in the whole point cloud would affect the significance of some rivet contour regions. To alleviate this issue, we propose an adaptive local density computation method to suppress the effect of density nonuniform distribution, which is inspired by the adaptive contrast enhancement (ACE) algorithm [28] in image processing. Specifically, given a point pi , we first get its neighborhood Nk (pi ) and compute the average density value by:

Σpj ∈Nk (pi ) d(pj )

(3) n where n is the number of points in Nk (pi ). Similar to ACE in the image processing field, the local variance is then derived as:

mr (pi ) =

√ σr (pi ) =

Σpj ∈Nk (pi ) [d(pj ) − mr (pi )]2 n

(4)

The new adaptive local density of point pi can then be formulated as: C d∗ (pi ) = mr (pi ) + [d(pi ) − mr (pi )] (5) σr (pi ) where C is the scale factor to adjust the enhancement degree. In Eq. (5), the local average density mr (pi ) is first subtracted from the neighboring point density map at point pi . A variable gain σ C(p ) is then applied to the difference to amplify the local r

i

variations. A portion of the average density is added back to restore the subjective quality of the image. The local gain is itself locally adaptive, being inversely proportional to the local variance, so that areas with small local variance receive large gain. In that way, we can enhance the local contrast (i.e., difference in local density) by increasing the local density gain. As to the scale factor C , the noise would be enhanced when C is too large, while a small value would relieve the effect of enhancement. Thus, we empirically set it to be 10 in our experiments.

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

5

Algorithm 1 : Local Significance Score Computing

Fig. 5. Illustration of the proposed significance measurement for a circle candidate. The fitted circle and its neighboring area are divided into three regions, inlier area ℜI (marked as yellow), near inside area ℜni (blue) and near outside area ℜno (green). A good circle candidate should possess a similar histogram pattern as shown, in terms of local point density. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Require:Raw point cloud ¶ = {pi }, i = {1, . . . , N }, a set of sampled k−subsets. Ensure:Local Significance Score Slss . Initialization: P ← Adaptive Density Computation (P ). function local Significance Score Computing(P , k) for each k−subsets do θ ← Get Circle Hypothesis (k); c = Get Model Center (θ ); r = Get Model Radius (θ ); I ← Get Model Inliers (θ ); for each pi ∈ I do dini = Get Density of Inside Area (pi ,P); diI = Get Density of Inlier Area (pi ,P); dino = Get Density of Outside Area (pi ,P); end for Slss = Get Local Significance Score (dni ,dI ,dno ); end for end function

Fig. 6. Insensitivity to the global density variation. The significance score degrades gradually, as the sampling ratio increases.

density estimate techniques [32], the variable bandwidth kernel density estimate at ri can be defined as:

( ) n 1∑ 1 x − ri (θ ) fθ (x) := KNE n h(θ ) h(θ )

(6)

i=1

where h(θ ) and KNE (·) represent the bandwidth and the Epanechnikov kernel, as used in [11]. On this basis, Wang et al. [11] score the circle hypotheses using the density estimate as the origin (O), i.e.,:

w(θ ) :=

fθ (O)

(7)

σ (θ ) where σ (θ ) is the scale estimate. One can refer to [11] for calculating fθ (O) and σ (θ ). θ is the circle hypothesis. From the

definition, the scoring function encourages those circles with large kernel density, while suppressing circles that produce large scale estimates. However, we notice that this weighting scheme is basically a summation over the inlier points, which is quite sensitive to the sampling density of points, as shown in Fig. 3. In fact, the significance of an underlying pattern should mainly link to the density difference with its neighborhood points, and such a pattern should be measured locally, instead of using global criteria. We, therefore, introduce a local density-aware measure into the kernel density metric, to better reveal significant circles in a local fashion. The intuition of our scoring scheme is to encourage circles with a large density difference to their near local neighborhood. To achieve local density-awareness, we approximate the significance of circles by the density change from the inside to the outside of the circle. Specifically, we introduce three areas, inlier area, near inside area and near outside area, as shown in Fig. 5.

We summarize the proposed weight scoring method in Algorithm 1. Given a circle hypothesis θ , we first get the center point, radius and the inlier points. For each inlier point, we can get its corresponding near inside area ℜni and near outside area ℜno with a angle deviation σt , e.g. 10 degrees in our experiments. ℜni and ℜno are two annular sectors with radius of r + 3h(θ ) and r − 3h(θ ), respectively, as illustrated in Fig. 5. For a given inlier point i, average density values are computed for the corresponding three areas using the points within each area, ℜiI , ℜini and ℜino , respectively. Thus, we can obtain three corresponding average density values diI , dini and dino . We then compute the local significance score Slss as

∑N

i=1

Slss =

f 1 − dini /diI · f 1 − dino /diI

(

)

N

(

) (8)

where N is the inlier point number and f (·) is a indicator function as

{ f (x) =

1 0

x>∆ x≤∆

(9)

where ∆ is the local significance threshold. We set it to be 0.4 in our experiments. That is, only the point whose density difference value (1 − di /diI ) is larger than 0.4 would contribute to the local significance measurement. The new score is then defined as:

w ˆ :=

fθ (O)

σ (θ )

· Slss

(10)

From the definition, our score incorporates both the measure of inliers and the local significance of the underlying circle. The bigger the density difference between the circle hypothesis and its surroundings, the higher the score. Fig. 6 plots the significance score of the underlying circle regarding the corresponding sampling ratio. As shown, the proposed scoring metric is strongly insensitive to sampling density. This is because the scoring metric is mainly related to the local density difference, i.e., the local significance. After scoring each circle hypothesis using the proposed significant measure, we then adopt the Mode-Seeking on Hypergraphs Fitting (MSHF) algorithm in [29] to solve this structure fitting problem, due to its significant superiority over the other state-of-the-art model fitting methods, as demonstrated in [29].

6

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Fig. 7. Circle fitting comparisons. There are eight circles in the whole input 2D point data, four circles on each side. Due to the different densities of the two sides, the outlier rates are 95.14% and 98.06%, respectively. As shown, our method achieves the best result and succeeds in detecting all circles correctly.

Fig. 8. Circle fitting comparisons. There is only one circle in the whole input 2D point data. As shown, our method achieves the best result and succeeds in detecting the center circle correctly.

5. Results and discussions In this section, we experiment with our rivet extraction method on a variety of raw scanned point clouds, and we present its effectiveness regarding circle fitting on 2D and 3D points. In order to demonstrate the superiority of our density-aware multiple structure fitting method, we also compare our algorithm with the other related approaches on both synthetic data and raw scanned point clouds. Moreover, we evaluate quantitative metrics to measure the extraction quality of our algorithm. 5.1. Comparison To demonstrate the effectiveness of our algorithm on circle fitting, we perform comprehensive comparisons with several stateof-the-art model fitting methods, including KF [21], T-linkage [33] and MSHF [29]. We try our best to adjust their respective parameters carefully to guarantee that they achieve the best performance when running those methods. Synthetic data. We test the four algorithms on circle fitting using two sets of synthetic data in Figs. 7 and 8. The corresponding quantitative results are given in Fig. 7. In Fig. 7, we arrange the circles in the data to generate a challenging configuration for circle fitting. To approximate the authentic circle fitting situation as closely as possible, a total of G points of gross outliers are randomly sampled within the range of the data. To further simulate the density anisotropy case, the right four circles with noise and outliers are first generated in Fig. 7. Based on this, we then generate a sparser data arrangement by randomly down-sampling the points to form the right four

circles. The two generated parts are finally concatenated together, yielding the challenging configuration in Fig. 7. Since each circle in the left part contains 200 inliers, while each circle in the right part contains 80 inliers, the outlier rates (OR) of two parts can be calculated by: ORi =

G+

∑n

G+

i′ =1,i′ ̸ =i

∑n

j=1

Lj

Li ′

× 100%

(11)

where ORi is the outlier rate of the ith circle in input data, and Li denotes the inliers associated to the ith circle. Therefore, the outlier rates of these two parts in Fig. 7 are 95.14% and 98.06%, respectively. From the results, all four algorithms can successfully recover the left four circles. However, the right four circles are pretty challenging for the other three methods. The reason is that the significance level of the right four circles is less than the noise and outliers in the left side, from the global view. Thus, the fitted circles are mainly lying on the left half part, which tends to contain more inlier points given a circle candidate. By contrast, our algorithm is inclined to search those circle candidates with a higher significance level, compared to the surrounding points in the local area. As a result, our method succeeded in fitting all the right four circles correctly. Fig. 8 gives another case with synthetic data. The data is composed of two regions, the outermost layer with a relatively high density value and the inner layer with a low value. As shown, the visual circle is lying in the inner area. Although the density of the underlying circle points is obviously lower than the points of the outermost layer, its density difference from inside to outside is still the most significant. These traditional methods tend to

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

7

Fig. 9. Comparisons on raw scanned point cloud corrupted with density anisotropy and outliers. All the comparative methods correctly fit the right circle, due to its global significant characteristic, while our method further detects the left circle pattern successfully. Table 1 Quantitative result shows the fitting error comparisons on the synthetic data from Tlinkage, KF, MSHF and our method, including fitting errors (in percentage) and running time (in seconds). The best results are boldfaced. Data

Tlinkage

KF

MSHF

Ours

Fig. 7

Std. Avg. Min Time

2.10 33.62 29.16 362.53

1.63 36.18 25.29 31.26

0.92 34.24 22.41 27.33

0.18 3.23 1.95 29.96

Fig. 8

Std. Avg. Min Time

0.01 4.91 4.30 153.12

0.01 4.96 4.17 12.01

0.01 4.94 4.06 8.13

0.01 0.53 1.88 10.52

Fig. 9

Std. Avg. Min Time

4.03 26.12 19.28 186.54

2.95 34.4 25.7 15.35

1.71 32.18 23.62 12.15

0.12 2.28 1.56 13.56

Fig. 10. Rivet contour extraction pipeline on a typical skin scan, which is corrupted severely with outliers and contains some noise. All the contours corresponding to underlying circle structures are successfully detected.

find circles containing as many inlier points as possible in the global space. Subsequently, the compared three algorithms all select circles in the outermost layer. Nevertheless, the proposed density-aware structure fitting technique is designed for detecting circles with the most significance in the local area. Thus, our method is still capable of fitting the circle successfully, despite the large number of outliers in the outermost layer. Real data. To further demonstrate the efficiency of our method on real data, we compare our algorithm with these methods on the raw scanned point data in Fig. 9. The point cloud consists of two rivet contours with a large fraction of outliers. The main challenge in this data is the density anisotropy. As seen, most of the points locate in the right part, resulting in a relatively low significance of the left rivet contour from the global view. Thus, all the comparative three methods correctly detect the right contour, but fail to fit the left circle. However, from the local view, the left contour is actually prominent compared with its local surroundings. Based on this observation, our method succeeds in detecting the left one via embedding the local significance into the hypergraph searching model. Further, we quantitatively evaluate the performance of the four methods in Table 1. The superiority of our method lies in detecting circle patterns with a relatively low significance from the global view, which cannot be fitted by the competing three methods. Thus, we calculate the fitting error as: fitting error =

♯ mislabeled data points × 100% ♯ data points

(12)

The equation measures the ability of discovering the circle pattern, which is also adopted in [29,33] and [34]. We calculate the average, the standard variances and the minimum results of the fitting errors via repeating experiments 100 times. We also show the average running performance in CPU time in Table 1. From Table 1, it can be seen that our method achieves the lowest fitting error among the four fitting approaches for Fig. 7. This is because only our method succeeds in correctly fitting all eight circles. For Fig. 8, all the three competing methods fail in fitting the center circle. In contrast, the proposed fitting method correctly segments the circle, resulting in the lowest fitting errors. Moreover, the left circle in Fig. 9 is a challenging problem for the competing methods due to a relative low significance, while our method still correctly fits the two circles, as shown from the fitting error in Table 1. The quantitative comparison results further demonstrate the superiority of our density-aware based method over the other competing approaches. 5.2. Raw point cloud data We further evaluate the effectiveness of the proposed method on several 3D real scans in Figs. 10–13. As seen, these raw scans are always with noise and outliers, density anisotropy and other complicated situations, which are directly processed by our method.

8

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Fig. 11. Circle fitting result from raw scanned point cloud with density anisotropy. The upper region of the point cloud contains more points than the lower region, making it challenging to discover the circles in the lower region.

Fig. 14. Comparison results with and without the proposed density computation method. (a) and (c) represent the density map without improvements and the detection result, respectively. (b) and (d) show the adaptive density computed map and the detection result with it. As shown, the local significance score Slss is improved by the proposed adaptive density estimate strategy, which contributes to the correct detection result.

Fig. 12. Rivet contour detection case on the raw scanned point cloud with contour missing regions.

Fig. 13. Circle extraction result from raw scanned skin point cloud with several problems, including severe noise and outliers, contour missing and density anisotropy.

Noise and outliers. Fig. 10 manifests the effectiveness on the raw scanned point cloud of 12 rivets corrupted by a considerable amount of noise and outliers. Scanned points belonging to the surrounding surface lead to noise and outliers, when detecting rivet contours. Nevertheless, our circle detection algorithm still performs accurate detection of the circle structures and recovers all the rivet contours. As a comparison, we also show the detection results of RANSAC in Fig. 10. Density anisotropy. In Fig. 11, we demonstrate that our algorithm is capable of detecting rivet contours under the density anisotropy situation. As shown in the middle density map, the overall point density of the upper half part is relatively higher than that of the lower half part. When looking at the bottom three rivet contours, the number of points lying on them is relatively smaller than the upper three contours. Nevertheless, by incorporating the measurement of the significance difference in our hypothesis weighting metric, our algorithm can discover the significance of these contours in the local view. Subsequently, these relatively insignificant circles, from the global view, can also be fitted well. Contour missing. We evaluate our method on the raw scan with large regions of missing data in Fig. 12. As we can clearly see in the regions pointed out by red arrows, nearly half of the rivet contours are poorly scanned, leading to the missing data problem. However, according to Section 4, we detect all the circle structures as shown in Fig. 12(c).

Fig. 15. Rivet extraction result on the raw scanned point cloud of an entire skin segment, which contains a total of 144 rivets. Note that the radius from the right side is not the same as the left side, demonstrating that our method is robust to radius variation.

Complicated scene. Fig. 13 shows a complicated case with various situations, including noise and outliers, density anisotropy and contours partially missing. Thanks to our density-aware substructure selection technique, we are able to extract these circle structures from the raw input scan. To further assess the performance of our method, we experiment on two large scale scans with over 100 rivets in Figs. 15 and 16. As seen, our method succeeds in detecting all the rivets automatically, by the proposed circle fitting technique. Effect of the proposed adaptive density computation. To relief the adverse impact of rare regions with the extremely high density value, we present a novel point cloud density computation approach in Section 3.2. Fig. 14 illustrates the effectiveness of the proposed adaptive local point cloud density computation. As shown in Fig. 14(b), owing to the extremely high density calculated by Eq. (1), the density difference of the circle on the bottom is not clear enough. Thus, the consequent detection step leaves that circle, as can seen in Fig. 14(d). However, the local significance score Slss is improved by the proposed adaptive density estimate strategy (from 0.3529 to 0.7081). Consequently, the circles are correctly detected through the proposed density computation, which demonstrates the adaptive density strategy can efficiently improve the circle detection performance.

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Fig. 16. Another large scale rivet extraction result, automatically generated by the proposed method. As can seen, outliers occupy a large fraction of the data.

9

Fig. 18. Robustness to noise and outliers. With an increasing amount of noise and outliers, the accuracy of fitting results declines slightly. As the degree of corruption increases to 98.0%/6% or even more, our method tends to be unstable. Overall, our method exhibits relatively high robustness to a high level of noise and outliers.

we perform the normalization to the input data before fitting, and then we are able to measure the accuracy of our method, by calculating the above quantitative metrics. Robustness to noise and outliers. Represented by the quantitative metrics, the accuracy of the fitting results can be evaluated under various degrees of noise and outliers. To this end, given the ground truth, we generate the Gaussian noise of standard deviation σ , and add outliers randomly in the bounding box, as in Section 5.1. Since the input data is composed of the synthetic noise, outliers and the ground truth, we are able to evaluate the fitting results quantitatively. Our method exhibits relatively high robustness to a high level of noise and outliers, as shown in Fig. 18, where the coverage rate is expressed by: Fig. 17. Two limitation cases (red boxes) of our method. (a) Most parts of the contour are not well scanned, so that the partial point set with high density level would be regarded as noise rather than rivet contour. (b) The rivet contour is not presented obviously due to the scanning operation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

5.3. Quantitative evaluation After demonstrating the performance of our method in terms of effectiveness on the synthetic data and imperfect raw scan data, we introduce some quantitative metrics to measure the accuracy of the fitting results. The robustness of our algorithm to noise and outliers can then be investigated quantitatively, based on the metrics above. Quantitative metrics. As illustrated in Section 5.1, we utilize the fitting error to measure the completeness of fitting results against the ground truth. In addition, we also choose the rootmean-square error (RMSE) metric to evaluate the geometric fidelity of our fitting results. The robustness of our method on different levels of noise and outliers is further shown in Fig. 18. Assume that the ground truth Ω consists of n circles, i.e., Ω = {ci }ni=1 . For each circle ci , the point subset of the corresponding fitting result is denoted by Pi . Therefore, the root-mean-square error metric is then defined as:

√∑ RMSE =

n i=1

∑

∀p∈Pi

∑n

∥p − ci ∥2

i=1 |Pi |

(13)

where ∥p − ci ∥2 is the Euclidean distance between p and ci , and |·| is the cardinality of a set. Specially, to unify the evaluation scale,

Cov erage Rate = 1 − fitting error

(14)

As the degree of noise and outliers increases, the accuracy of fitting results declines slightly. Since our method is density-aware based on the local significance, it is capable of fitting the circles precisely, even in case of 96.8% of outliers plus 4% of noise. However, as the degree of corruption increases to 98.0%/6% or even more, our method tends to be unstable, since the local significance may be blurred. 5.4. Limitations Our method is expected to behave well with noise and density anisotropy cases over other state-of-the-art methods. However, there are a few limitations that must be discussed. First, as demonstrated in Fig. 17(a), when the missing rivet contour regions are fairly large, the significance reflected by the density change could be severely decreased and our circle fitting method could fail to detect the corresponding rivet contour. In addition, as shown in Fig. 17(b), our algorithm may also fail when the density of contour points has no significant difference, compared with the surrounding points. 6. Application Based on the proposed density-aware multiple structures fitting algorithm for rivet contour extraction, we develop several practical applications in the rivet quality inspection field, such as rivet head extraction and rivet flush inspection, which achieve favorable results, in terms of accuracy and efficiency.

10

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Table 2 Standard diameter patterns (mm) of rivet heads. Basic size Tolerance

4

5 0 −0.4

6

8 0 −0.5

10

12

16

0

0

0

−0.6

−0.7

−0.8

6.1. Rivet head extraction Right after each rivet contour extraction, we employ a postprocessing to trim the radius and the normal of the initial circle extracted. With the center point, revised radius and normal of the extracted circle, a cylinder with infinite length could be generated accordingly. Finally, all points lying inside the cylinder would be extracted as the final point set of the rivet. Once individual fitting is completed over each rivet contour, the initial fitted 3D circles are obtained. However, the radius and normal could be inaccurate, due to the influence of noise. Thus, after obtaining the initial 3D circle in the RANSAC based fitting step, we trim the radius and the normal for the final rivet points extraction. To this end, we propose to utilize the priori knowledge. According to the rivet manufacturing standards, the rivet head should have several fixed radius patterns, which can be referred to in Table 2. Only the rivets in compliance with the above requirements can be used in practical projects. Therefore, we suppose those rivets should have fixed radius patterns. According to these fixed radius patterns, we can then get the more precise extraction results via trimming the initial fitting. Specifically, for each fitted 3D circle, its radius would be trimmed to the nearest standard value. For instance, when the initial radius of a circle is 6.151, it would be trimmed to 6.000, according to the corresponding basic size 6 and tolerance −0.4 in Table 2. With the initial center point, normal and the corrected radius, a cylinder with infinite length is generated accordingly. We then extract all the points inside the cylinder, as the initial point set of the extracted rivet head. Given the initial point set, we then utilize a RANSAC based 3D plane fitting method to extract a 3D plane, whose normal would be treated as the corrected normal. We then generate a new cylinder with the new normal, the corrected radius and the initial center point. Given the new cylinder, we can obtain the final point set of the rivet head. Two extraction results are shown in Fig. 19. The extracted heads can be used to further analyze the riveting quality, as described in the next sub-section. 6.2. Flush inspection Riveting quality is a vital factor in the aerodynamic profile of aircraft. Rivets should lie flush with the corresponding surrounded aircraft skin surfaces. Good riveting could reduce bulging aerodynamic surfaces, which in turn reduces drag and improves airplane performance. Rivet flush is normally measured by two values, dc and δc , as defined in Fig. 20. As shown, dc is the distance from the rivet head center to the corresponding surround surface, while δc stands for the angle deviation between the normal direction of the rivet head and the surrounding surface. Through these two values, we can easily measure the rivet flush in a quantitative way. Accordingly, dc and δc should both be close to zero, which would yield a good riveting from the flush view. Nevertheless, as one of the important measurement means of riveting quality, flush inspection is difficult for traditional inspection techniques, e.g. visual-based inspections. In fact, there is still no existing effective approach to automatically inspect rivet flush. The inspection is usually done using human eyes and can only be measured qualitatively. Thousands of rivets are normally used in the skin of a aircraft. Traditional inspection methods are

Fig. 19. Rivet head extraction results on two large scale scanned point clouds. Note that the diameter varies among the rivet heads.

Fig. 20. Definition of rivet flush (dc , δc ). From the side view, we can obviously see that the rivet head is slightly inclined to the surrounding surface.

unsatisfactory in terms of both efficiency and accuracy. Thus, there is an urgent demand for automatic rivet flush inspection techniques. However, fully automatic flush inspection can easily be achieved based on the proposed rivet extraction algorithm. The dc can be obtained via computing the distances between extracted rivet points and the plane fitted by the surrounding points near the rivet. Specifically, with the extracted rivet head point set and its center point, we first expand the radius to 2r for generating a virtual bounding 3D cylinder. r is the radius of the extracted rivet. The points in point set Ps within the created cylinder, except for those points belonging to the extracted rivet head, are then regarded as supporting points of the surrounding surface. Since we only expand the radius to 2r, the extracted point set Ps could be seen as lying on a plane. Thus, we fit a plane on these points using a RANSAC based 3D plane fitting method. Distances between the fitted plane and rivet head center points, i.e. dc , can then be calculated directly. As for δc , we need to first compute the normal nc of the extracted rivet head point set. With the head normal nc and the fitted surrounding plane, we can then measure the angle deviation δc between the rivet head and surrounding surface. Quantitative results of rivet flush computed via our method are shown in Table 3. Besides the quantitative indices, we also develop a visual system to help inspectors to visually and rapidly measure the rivet flush. In particular, with the extracted rivet head point sets and the fitted planes of the corresponding surround surfaces, we can compute the distance from the head points to the fitted planes. The distance scalar field is then associated to the color scale to visualize the distances in the system. Fig. 20 shows several calculated visual results on the rivet flush measure. From the figure, inspectors can rapidly locate rivets with bad flush, and then further check the specific quantitative values.

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

Fig. 21. Visual results of rivet flush. Distances are mapped to the color range for a better visualization. Table 3 Quantitative results of rivet flush calculated by our method. #

dc (mm)

δc (degree)

#

dc (mm)

δc (degree)

1 2 3 4 5

0.241 0.190 0.147 0.128 0.155

0.532 0.417 0.668 0.602 0.625

6 7 8 9

0.176 0.169 0.153 0.130

0.353 0.449 0.581 0.368

7. Conclusions We have presented a circle model fitting method to efficiently detect rivets directly over the raw scanned point clouds. By introducing the relative density change detecting strategy, our circle fitting algorithm is capable of dealing with randomly arranged circles within severe noise and outliers, missing contour and density anisotropy scenes. That is, instead of adapting a unified inlier measurement from the global view, we propose to discover the relative significance of circle candidates in the local view. The proposed method proceeds automatically without user interactions, and thus it is quite appropriate to large-scale point cloud based automate rivet quality inspection applications, such as rivet head extraction and rivet flush inspection. Future work. In this paper, we propose a novel circle fitting method which measures the inlier scale, using density change in the local area. In fact, the main idea in this paper can be totally embedded into other types of shape fitting methods, like line and sphere fitting. In terms of the specific task of this paper, rivet extraction from the scanned point cloud, we may leverage the prior information of the organized rivet location arrangement, as shown in Fig. 21. This can definitely accelerate and simplify the rivet extraction task. Therefore, to seek the regularities and geometric constraints of the rivet location arrangement would be a research direction of our future work, as the authors did in [35]. Acknowledgments The authors are grateful to the anonymous reviewers for their valuable comments. This work was supported in part by National Natural Science Foundation of China under Grant 61772267, the Fundamental Research Funds for the Central Universities, China under Grant NE2016004, and the Natural Science Foundation of Jiangsu Province, China under Grant BK20190016. References [1] Lei C, Bi Y, Li J, Ke Y. Effect of riveting parameters on the quality of riveted aircraft structures with slug rivet. Adv Mech Eng 2017;9(11). 1687814017734710. [2] Skorupa M, Machniewicz T, Skorupa A, Korbel A. Fatigue life predictions for riveted lap joints. Int J Fatigue 2017;94:41–57.

11

[3] Le M, Kim J, Kim S, Lee J. B-scan ultrasonic testing of rivets in multilayer structures based on short-time fourier transform analysis. Measurement 2018;128:495–503. [4] Kafie-Martinez J, Keating PB, Chakra-Varthy P, Correia J, de Jesus A. Stress distributions and crack growth in riveted lap joints fastening thick steel plates. Eng Fail Anal 2018;91:370–81. [5] Jovančević I, Pham H-H, Orteu J-J, Gilblas R, Harvent J, Maurice X, Brèthes L. 3D point cloud analysis for detection and characterization of defects on airplane exterior surface. J Nondestruct Eval 2017;36(4):74. [6] Jovančević I. Exterior inspection of an aircraft using a pan-tilt-zoom camera and a 3d scanner moved by a mobile robot: 2d image processing and 3d point cloud analysis (Ph.D. thesis), Ecole des Mines d’Albi-Carmaux; 2016. [7] Wang Y, Feng H-Y. Outlier detection for scanned point clouds using majority voting. Comput Aided Des 2015;62:31–43. [8] Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981;24(6):381–95. [9] Li Z, Cheong L-F, Zhiying Zhou S. Scams: Simultaneous clustering and model selection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 264–71. [10] Tennakoon RB, Bab-Hadiashar A, Cao Z, Hoseinnezhad R, Suter D. Robust model fitting using higher than minimal subset sampling. IEEE Trans Pattern Anal Mach Intell 2016;38(2):350–62. [11] Wang H, Chin T-J, Suter D. Simultaneously fitting and segmenting multiple-structure data with outliers. IEEE Trans Pattern Anal Mach Intell 2012;34(6):1177–92. [12] Skorupa M, Machniewicz T, Skorupa A, Korbel A. Fatigue strength reduction factors at rivet holes for aircraft fuselage lap joints. Int J Fatigue 2015;80:417–25. [13] Skorupa A, Skorupa M, Machniewicz T, Korbel A. Fatigue crack location and fatigue life for riveted lap joints in aircraft fuselage. Int J Fatigue 2014;58:209–17. [14] Yu H, Zheng B, Xu X, Lai X. Residual stress and fatigue behavior of riveted lap joints with various riveting sequences, rivet patterns, and pitches. Proc Inst Mech Eng B 2019;0–12. [15] Liu Z, Forsyth DS, Marincak A, Vesley P. Automated rivet detection in the eol image for aircraft lap joints inspection. NDT Int 2006;39(6):441–8. [16] Jiang T, Cheng X, Cui H, Shi C, Li Y. Dual-camera-based method for identification and location of scattered self-plugging rivets for robot grasping. Measurement 2019;134:688–97. [17] Park U, Fan Y, Udpa L, Ramuhalli P, Shih W, Stockman GC. Automated rivet inspection for aging aircraft with magneto-optic imager. In: 10th international workshop on electromagnetic nondestructive evaluation. Michigan; 2004. [18] Fan Y, Deng Y, Zeng Z, Udpa L, Shih W, Fitzpatrick G. Aging aircraft rivet site inspection using magneto-optic imaging: Automation and real-time image processing. In Proceedings of the 9th joint FAA/DoD/NASA aging aircraft conference, 2006, p. 9. [19] Li B, Wang X, Yang H, Zhou Z. Aircraft rivets defect recognition method based on magneto-optical images. In: 2010 international conference on machine vision and human-machine interface. IEEE; 2010, p. 788–91. [20] Paul G, Liu L, Liu D. A novel approach to steel rivet detection in poorly illuminated steel structural environments. In: 2016 14th international conference on control, automation, robotics and vision (ICARCV). IEEE; 2016, p. 1–7. [21] Chin T-J, Wang H, Suter D. Robust fitting of multiple structures: The statistical learning approach. In: 2009 IEEE 12th international conference on computer vision. IEEE; 2009, p. 413–20. [22] Fredriksson J, Larsson V, Olsson C. Practical robust two-view translation estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2684–90. [23] Litman R, Korman S, Bronstein A, Avidan S. Inverting RANSAC: Global model detection via inlier rate estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5243–51. [24] Yi C, Zhang Y, Wu Q, Xu Y, Remil O, Wei M, Wang J. Urban building reconstruction from raw lidar point data. Comput Aided Des 2017;93:1–14. [25] Nurunnabi A, Sadahiro Y, Laefer DF. Robust statistical approaches for circle fitting in laser scanning three-dimensional point cloud data. Pattern Recognit 2018;81:417–31. [26] Wang J, Xu K. Shape detection from raw lidar data with subspace modeling. IEEE Trans Vis Comput Graph 2017;23(9):2137–50. [27] Lari Z, Habib A. New approaches for estimating the local point density and its impact on lidar data segmentation. Photogramm Eng Remote Sens 2013;79(2):195–207. [28] Chang D-C, Wu W-R. Image contrast enhancement based on a histogram transformation of local standard deviation. IEEE Trans Med Imaging 1998;17(4):518–31. [29] Wang H, Xiao G, Yan Y, Suter D. Searching for representative modes on hypergraphs for robust geometric model fitting. IEEE Trans Pattern Anal Mach Intell 2019;41(3):697–711.

12

Q. Xie, D. Lu, K. Du et al. / Computer-Aided Design 120 (2020) 102805

[30] Wang H, Chin T-J, Suter D. Simultaneously fitting and segmenting multiple-structure data with outliers. IEEE Trans Pattern Anal Mach Intell 2011;34(6):1177–92. [31] Chum O, Matas J. Matching with prosac-progressive sample consensus. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol. 1. IEEE; 2005, p. 220–6. [32] Wand MP, Jones M. Kernel smoothing. Chapman and Hall/CRC; 1994.

[33] Magri L, Fusiello A. T-linkage: A continuous relaxation of j-linkage for multi-model fitting. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 3954–61. [34] Mittal S, Anand S, Meer P. Generalized projection-based m-estimator. IEEE Trans Pattern Anal Mach Intell 2012;34(12):2351–64. [35] Wang J, Wu Q, Remil O, Yi C, Guo Y, Wei M. Modeling indoor scenes with repetitions from 3d raw point data. Comput Aided Des 2018;94:1–15.