- Email: [email protected]
LiDAR data reduction assisted by optical image for 3D building reconstruction
Optik 125 (2014) 6282–6286
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
LiDAR data reduction assisted by optical image for 3D building reconstruction Lin Yang a,b , Yehua Sheng a,b,∗ , Bo Wang a a b
Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210023, China Key Laboratory of Police Geographic Information Technology, Ministry of Public Security, People’s Republic of China
a r t i c l e
i n f o
Article history: Received 12 November 2013 Accepted 16 June 2014 Keywords: LiDAR data Data reduction Building model 3D Reconstruction Image feature
a b s t r a c t The focus of LiDAR data compression and reduction has been raised in recent years due to the dramatically huge amount of points cloud. To improve the disposal efficiency of LiDAR data, a data reduction method assisted by optical image for 3D reconstruction of building facade is proposed in this article. The method involves a series of procedures of “2D feature line extraction – 3D feature line converting – buffer area of LiDAR points”. The main issue here is finding out a LiDAR point dataset around the 3D feature segment converted from 2D feature lines, which benefits to improving the efficiency of post-processing based on LiDAR data. The reduced LiDAR data can be obtained with reliable structures and accurate geometric position. Furthermore, the experiment of the LiDAR data reduction was conducted over the Xingyuan Building in Nanjing Normal University. The reduction method in this paper proved to be suit for the regular objects modeling, on the basis of the reliable and rigorous mathematical models and computer algorithms, and was a fundamental and useful approach to improve the efficiency of 3D reconstruction of building with the higher modeling accuracy. © 2014 Elsevier GmbH. All rights reserved.
1. Introduction Terrestrial LiDAR (light detection and ranging) is a novel sensor involving the technologies of laser ranging and digital CCD and regular surveying, which is widely being used in geomatics-related fields. With recent advances in hardware and software of laser scanning system, 3D laser scanners have become more accurate and the speed of data acquisition has increased dramatically, and the terrestrial LiDAR is being increasingly used to 3D reconstruction of building. However, they generate up to tens of thousands of points per second, and the post processing of data still relatively lags behind, especially on the handling the huge amount of LiDAR points efficiently. Take Leica HDS7000, FARO Focus 3D 120 and Riegl VZ4000 as instances, whose angle resolutions all are less than 0.001◦ , and the point resolution extends to sub-millimeter level in tens of meters range. High accuracy and huge LiDAR point data are obtained simultaneously, causing great obstacles on the follow-up data processing, data transmission and further application.
∗ Corresponding author at: Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210023, China. Tel.: +86 13813880937. E-mail address: [email protected] (Y. Sheng). http://dx.doi.org/10.1016/j.ijleo.2014.08.016 0030-4026/© 2014 Elsevier GmbH. All rights reserved.
For example, due to the uniform angular resolution scanning there are many redundant data among the original data sampling and modeling of building facade, and the redundant data, excepting the sampling points around the outlines and edges of building construction, are nearly no use for feature extracting and modeling. So it is very necessary to adapt or design corresponding methods or algorithms to reduce the LiDAR data [1–3]. For a practical simplification algorithm of point cloud, the data compression ratio, the compressive error, the processing efficiency and the algorithm applicability should be taken into consideration. The main issue here is offering a reasonable algorithm to maximize to reduce the redundant points under the premise of the prescribed reconstruction error tolerance and geometric feature preservation. Aim at the drawback of the quantity and redundancy of LiDAR points, a number of methods have been developed, which are broadly divided into two categories. The first category is resampling, namely resampling the points from raw LiDAR data by uniform interval, which doubtlessly losing some useful information due to the reduction on the same level. The second approach is merging and resampling of LiDAR data in homogenous area. That means merging redundant points and replacing the original quantity of points with small amount of sample ones (or synthetic sample points) based on some algorithms in the homogeneous area (such as certain building’s facade), in order to achieve the purpose of the data reduction. This method has obvious advantages
L. Yang et al. / Optik 125 (2014) 6282–6286
because of the adaptive data resampling, reducing the amount of scanned data while maintaining useful feature information and the required accuracy [4]. Some research related to point data reduction is described. Liu et al. [5] completed the compression of LiDAR data by a series of procedures including resampling by uniform interval, and median filtering and removal based on the elevation distance and compression test by entropy calculation of echo intensity. Zuo et al. [6] proposed a typical noise removal algorithm of LiDAR points based on 3D finite-element analysis. Firstly, point clouds were partitioned into smaller and similar units by finite elements named space hexahedron model. And then, all of the units were classified into noise units or non-noise units with adjacency-based reasoning rules. Finally, the low noise was removed by iterative processing with finer threshold. The form of finite-element analysis, namely the Boolean function model, was selected to describe local solutions in relatively simple circumstance. Some other forms should be used to solve complex problems. Xu et al. [7] also presented a simplification method for LIDAR ground points based on local scatter characteristics of points. Here, scatter means the difference of elevation between each original point and the even elevation. The larger the difference, the more attribution to the terrain undulation was. So, the point with larger difference of elevation was remained while the one with smaller difference of elevation was removed, and then the topographic patterns were maintained in local region. Their method, however, had a drawback due to the reduction error been sensitive to the threshold of scatter. Dobrucali et al. [8] proposed a compression technique based on compressive sensing by sampling the sparse signal using either simple coding or compressive sampling, which provided a reasonably good compromise between reconstruction accuracy and speed and was superior to some well-known lossy and lossless compression techniques in encoding the scan data. The proposed technique involved efficiently sampling the sparse signal obtained from the difference between the current scan and its estimate, which was generated by shifting the previous scan along the horizontal and vertical axes by certain amounts. The method could be effectively used for 3D representation of both indoor and outdoor environments [9], however, could be improved by further coding the encoder output via lossless coding techniques. Luo et al. [4] suggested a method to reduce point data based on quad-tree partition. In their method, the raw data were totally partitioned in the designed minimize size grids based on quad-tree firstly, and then the data in homogenous area were merged and resampled from bottom to top according to certain homogenous decision rule. The algorithm was suitable for the point reduction in a plan with high compression ratio, keeping the edge of good performance, however, for data compression on the surface (such as spherical, hyperboloid), due to the gradient vector normal, it was difficult to offer a reasonable threshold and the corresponding improvement was needed. Lee et al. [2] proposed non-uniform grid based point data reduction methods from the point of view of part shape and z-axis error occurring in laser scanning. These non-uniform grid methods were implemented either by using one-directional or bi-directional non-uniform grids. The proposed methods were applied to sample models having freeform shapes, and the data reduction could be performed effectively while maintaining the quality of the point data. However, these methods were only useful if the registration of point clouds was performed after reducing the data size for each point cloud. When a complete 3D model of a part needed to be processed, these 2D grid methods might not be suitable, since they were projection-based techniques. The reduction method dealing with 3D scanned data needed further research. Major research efforts in the existing data reduction methods lie in manipulating polyhedral models. Various schemes are also used to reduce the amount of point data from the initial point clouds;
6283
however, none of the existing methods has considered the usage of the characteristics of the geometric feature derived from optical image, which is obtained easily from the most of 3D scanners. In my paper, a novel method for reducing LiDAR data cued by geometric feature derived from optical image is proposed, according to the processing” 2D feature line – 3D feature line-buffer area of LiDAR points”. 2. Methodology 2.1. Feature extraction from optical image and handling Because straight line is the main structure boundary for most regular buildings, straight line is chosen as the basic feature element to analyze and detect more complex features in this article. The feature extraction algorithm based on gradient direction information of edge detection proposed by Von Gioi et al. [10] is adopted in this paper. However, due to the shooting angle, light shadow, non-geometric texture extraction et al., there still exist the following problems: (1) the straight line segment is of fracture or overlap, and so on; (2) the boundary lines are not closed or exceed both ends in the corner of windows and walls, which leads to the lack of topology information between line and line, not only being against of the integrity of geographical objects and having an influence on the modeling of building structure. So further processing based on the structure integrity constraints should be executed for the extracted feature more reliable and more photo-realistic to express the building outlines. For most of building facades, the geometric shapes of construction members are fairly regular, such as the boundary structure of door and window generally is rectangular, and the edge of wall is straight line, etc. So building facade features are defined in advance to describe the semantic features, like the classification of ground features in remote sensing image processing. The facade features classification method proposed by Pu et al. [11] is adopted in this paper, which can be divided into six types: walls, edges of walls, doors, windows and other convex (hollow) edge lines, part of the building roofs. According to the topological relationship between the geometric feature lines and the feature attributes, the integrity constraint rules of building structure are described to establish the mathematical model for refining the feature extraction. The post-processing of 2D features from images based on the structural integrity constraint rules mainly includes the feature lines merging and closure in the corner point. Lines merging are based on some feature attributes, such as the directions and relative lengths of the line segments, and the attributes are regarded as weight parameters to determine the line segments merging [12]. Lines closure in the corner point is based on the rule of wall edge lines are vertical approximately at the wall corner by calculating the intersection point of two feature lines and then revising the feature line segments. 2.2. Space mapping from 2D feature line to 3D feature line In order to assist in reducing point clouds by 2D features, space mapping is necessary. The space mapping from 2D features to 3D ones is to calculate the mathematic projection relation of the image pixel coordinates and 3D point. For optical image, the spatial location coordinates of every pixel are determined by the camera imaging geometric model parameters. Take the simplest pinhole camera model for instance, that means a series of coordinate systems transformations during the camera imaging process. The coordinate systems transformations include the world coordinates to camera coordinates, the camera coordinates to image coordinates and image coordinates to the pixel coordinates. After
6284
L. Yang et al. / Optik 125 (2014) 6282–6286
Fig. 1. LiDAR data searching around 3D feature line segment in buffer area.
the three coordinate transformations, the geographical spatial 3D coordinates are projected into the 2D plane, and vice versa. Take the Riegl LMS series scanners as instance, its attached scan data processing software-RiSCAN provides corresponding transformation parameters, as a result, and the transformation model is achieved [13]. The detailed steps are: calculate the 3D coordinates of each endpoint of 2D image feature line segments, and then search rapidly the LiDAR point closest to the 3D endpoint based on k-d tree neighborhood search method. After the data processing, 2D image feature lines can be converted into 3D features. 2.3. Data organization and quick search of LiDAR points The discrete LiDAR dataset acquired from laser scanner includes the values of all range, azimuth, polar, color and intensity in a scanning coordinate system, but is lack of geometric topology information among the points, which leads to the difficulty on searching the neighbor point and calculating the normal vector and curvature of LiDAR data. So in order to deal with the mass point cloud data, the primary job is to establish the index of the discrete LiDAR points and realize rapid search based on neighborhood relations. The spatial index method based on k-d tree is adopted in this paper to re-organize the point data for improving the efficiency of searching [14]. And then the consequent calculation of the normal vector and the curvature of LiDAR point can be improved rapidly based on k-d tree neighborhood search method. 2.4. Buffer areas of LiDAR data cued by 3D feature lines For refining 3D feature lines from LiDAR data efficiently, the primary job is to search the regional LiDAR points around the 3D feature lines. Generally speaking, in a 2D space, the points searching around a line segment is based on surface buffer and judging whether points are within the buffer area. Correspondingly, in a 3D space, the points searching around the line segment is based on a cylindrical buffer with the 3D line segment as the axis of the cylinder. By iteratively calculating and judging whether the distance of each LiDAR point to the axis is less than or equal to the circle radius of the cylinder, the points within the buffer is determined consequently. Although this method is simple, the computation efficiency is on an extremely low level and requires higher computer processing speed. A new search method based on the spheres with the same radius of the cylinder is proposed as shown in Fig. 1. A series of point datasets within the spheres approximate the inside points of the cylinder instead, so the buffer area of LiDAR data cued by 3D feature lines is determined by searching neighborhood points of the center of the sphere. Based on the above k-d tree index method of LiDAR data, the efficiency of searching the LiDAR points in the buffer area could be improved, thus simplifying the subsequent refining of the 3D features based on LiDAR data. The detailed searching steps are shown in Fig. 2. Firstly, determine the sphere searching radius R. Secondly, determine the searching step size D according to the length of 3D feature segment, and determine the main searching direction (in the direction, the coordinates difference between two endpoints
Fig. 2. Processing of generating buffer area of LiDAR data based on 3D feature line segment.
is largest) according to the coordinates differences in the X, Y, Z direction, respectively. Thirdly, calculate the coordinates of each searching point of division (the center of each searching sphere) on the feature line. Then, in order to ensure the spheres fully filled cylinder, calculate and judge whether D is less than or equal to the R/2, if not, re-calculate the searching step size D and the coordinates of each searching point of division, and if yes, the searching of LiDAR points can be executed based on k-d tree neighborhood search method. Finally, judge whether each LiDAR point is recorded, only the new point should be remained, avoiding to be recorded more than once. Repeat the above process until the LiDAR data searching based on each point of division on the feature segment is completed. 3. Experiment and discussion LiDAR-optical data from a real geographical environment was taken as an instance to test the method proposed in this paper. The data cover partial building facade in Nanjing Normal University, China. The original optical image (as shown in Fig. 3) generated from Nikon D100 camera mounted on top of the scanner, whose focal length is 20 mm and the picture format is 3008 × 2000 pixels. The LiDAR data (as shown in Fig. 4) were acquired by Riegl 3D laser imaging system of LMS-Z420. Fig. 5 is the result of 2D feature lines extraction from image and post-processing. It can be seen that most
Fig. 3. Original optical image.
L. Yang et al. / Optik 125 (2014) 6282–6286
Fig. 4. Original LiDAR data.
6285
Fig. 7 is the result of buffer areas of LiDAR data cued by 3D feature lines. It can be seen from Fig. 7(a) the amount of redundant point clouds were removed compared with Fig. 4 and the points around feature lines were retained. Statistical result shows the number of the original LiDAR points is 746,647, and the point number of the buffer areas is 340,979, accounting for 45.67% of the former (where buffering radius is 0.3 m), which needs less time in post processing based on LiDAR data. Fig. 7(b) is the corresponding detailed drawing. The next work, the feature segments can be refined based on the reduced LiDAR data by the calculation of the normal vector and the curvature of LiDAR data, and the classification and plane fitting and feature lines extraction, and eventually completing 3D modeling of the building facade. 4. Conclusions
Fig. 5. 2D feature lines extraction from image and post-processing.
Fig. 6. The matching result of LiDAR data and optical image. (a) Overall view and (b) detailed drawing.
With the rapid development on the 3D scanning system, the processing efficiency of LiDAR data has been increasingly focused due to the huge amount of points cloud. A new LiDAR data reduction method is proposed in this study. By generating the buffer area of LiDAR points cued by 3D segments converted from 2D feature segments extracted from optical image based on rigorous mathematical models, the reduced LiDAR data can be obtained with reliable structures and accurate geometric position. The main contributions of the proposed approach are as follows: (a) a new technique to utilize the complementary image data to assist the LiDAR data reduction, improving the reliability of segmentation and avoiding information loss, and (b) put forward to the conception of buffer area of LiDAR data. The main issue here is generating LiDAR point datasets around the 3D feature segments, which benefits to improving the efficiency of post-processing of LiDAR data. Further studies may be performed in improving and optimizing the extraction algorithm of 2D features from image due to its importance to the generation of buffer area of LiDAR data. In addition, further regular and adaptive reduction algorithms may be applied to the LiDAR data in the buffer area (e.g., resampling by uniform or non-uniform grid interval). Acknowledgements This work was supported by the National Natural Science Foundation of China (grant no. 41101377 and no. 41271383) and the Priority Academic Program Development of Jiangsu Higher Education Institutions, China (PAPD) (grant no. 164320H101). Sincere thanks are given for the comments and contributions of anonymous reviewers and members of the Editorial team. References
Fig. 7. Buffer areas of LiDAR data cued by 3D feature lines. (a) Overall view and (b) detailed drawing.
of obvious feature lines can be extracted, and with a small number of overlap lines or break lines. Fig. 6 is the matching result of LiDAR data and optical image based on the rigorous registration model. Fig. 6(a) is the overall view of the matching of the building facade and Fig. 6(b) is the corresponding detailed drawing.
[1] L.-Y. Zhang, R.-R. Zhou, W.-B. Cai, Research on cloud data simplification, J. Comput. Aided Des. Comput. Graph. 13 (2001) 1019–1023. [2] K.H. Lee, H. Woo, T. Suk, Data reduction methods for reverse engineering, Int. J. Adv. Manuf. Technol. 17 (2001) 735–743. [3] A.C. Lin, H.T. Liu, Automatic generation of NC cutter path from data points, Comput. Aided Des. 30 (1998) 77–90. [4] D.-A. Luo, L.-Q. Liao, Ground-based LIDAR data simplification based on quadtree partition, Comput. Appl. 25 (2005) 420–421, 425. [5] C. Liu, C. Lu, Compress of the three dimension laser ranger scanning data and topography sampling, Remote Sens. Inf. 2 (2005) 6–10. [6] Z. Zuo, Z. Zhang, J. Zhang, Noise removal algorithm of LIDAR point clouds based on 3D finite-element analysis, J. Remote Sens. 16 (2012) 297–309. [7] J. Xu, Y. Wan, S. Zhang, On simplification method for LiDAR ground points cloud, J. Geomat. 33 (February) (2008) 32–34. [8] O. Dobrucali, B. Barshan, Novel compression algorithm based on sparse sampling of 3-D laser range scans, Comput. J. 56 (2013) 852–870. [9] O. Dobrucali, B. Barshan, A compression method for 3-D laser range scans of indoor environments based on compressive sensing, in: Proceedings
6286
L. Yang et al. / Optik 125 (2014) 6282–6286
of 19th European Signal Processing Conference, Barcelona, Spain, August 29–September 2, 2011, pp. 1115–1119. [10] R.G. Von Gioi, J. Jakubowicz, J. Morel, LSD: a fast line segment detector with a false detection control, IEEE Trans. Pattern Anal. Mach. Intell. 32 (2010) 722–732. [11] S. Pu, G. Vosselman, Automatic extraction of building features from terrestrial laser scanning, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 36 (2006) 25–27.
[12] J.M.R.S. Tavares, A.J. Padilha, A new approach for merging edge line segments, in: Proceedings of 7th Portuguese Conference of Pattern Recognition, 1995, pp. 1–6. [13] B. Qiu, Research on Realistic Texture Mapping for 3D Laser Scanning, Nanjing University of Science and Technology Press, Nanjing, 2009, pp. 13–19. [14] J.L. Bentley, Multidimensional binary search trees used for associative searching, Commun. ACM 18 (1975) 509–517.
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
LiDAR data reduction assisted by optical image for 3D building reconstruction Lin Yang a,b , Yehua Sheng a,b,∗ , Bo Wang a a b
Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210023, China Key Laboratory of Police Geographic Information Technology, Ministry of Public Security, People’s Republic of China
a r t i c l e
i n f o
Article history: Received 12 November 2013 Accepted 16 June 2014 Keywords: LiDAR data Data reduction Building model 3D Reconstruction Image feature
a b s t r a c t The focus of LiDAR data compression and reduction has been raised in recent years due to the dramatically huge amount of points cloud. To improve the disposal efficiency of LiDAR data, a data reduction method assisted by optical image for 3D reconstruction of building facade is proposed in this article. The method involves a series of procedures of “2D feature line extraction – 3D feature line converting – buffer area of LiDAR points”. The main issue here is finding out a LiDAR point dataset around the 3D feature segment converted from 2D feature lines, which benefits to improving the efficiency of post-processing based on LiDAR data. The reduced LiDAR data can be obtained with reliable structures and accurate geometric position. Furthermore, the experiment of the LiDAR data reduction was conducted over the Xingyuan Building in Nanjing Normal University. The reduction method in this paper proved to be suit for the regular objects modeling, on the basis of the reliable and rigorous mathematical models and computer algorithms, and was a fundamental and useful approach to improve the efficiency of 3D reconstruction of building with the higher modeling accuracy. © 2014 Elsevier GmbH. All rights reserved.
1. Introduction Terrestrial LiDAR (light detection and ranging) is a novel sensor involving the technologies of laser ranging and digital CCD and regular surveying, which is widely being used in geomatics-related fields. With recent advances in hardware and software of laser scanning system, 3D laser scanners have become more accurate and the speed of data acquisition has increased dramatically, and the terrestrial LiDAR is being increasingly used to 3D reconstruction of building. However, they generate up to tens of thousands of points per second, and the post processing of data still relatively lags behind, especially on the handling the huge amount of LiDAR points efficiently. Take Leica HDS7000, FARO Focus 3D 120 and Riegl VZ4000 as instances, whose angle resolutions all are less than 0.001◦ , and the point resolution extends to sub-millimeter level in tens of meters range. High accuracy and huge LiDAR point data are obtained simultaneously, causing great obstacles on the follow-up data processing, data transmission and further application.
∗ Corresponding author at: Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210023, China. Tel.: +86 13813880937. E-mail address: [email protected] (Y. Sheng). http://dx.doi.org/10.1016/j.ijleo.2014.08.016 0030-4026/© 2014 Elsevier GmbH. All rights reserved.
For example, due to the uniform angular resolution scanning there are many redundant data among the original data sampling and modeling of building facade, and the redundant data, excepting the sampling points around the outlines and edges of building construction, are nearly no use for feature extracting and modeling. So it is very necessary to adapt or design corresponding methods or algorithms to reduce the LiDAR data [1–3]. For a practical simplification algorithm of point cloud, the data compression ratio, the compressive error, the processing efficiency and the algorithm applicability should be taken into consideration. The main issue here is offering a reasonable algorithm to maximize to reduce the redundant points under the premise of the prescribed reconstruction error tolerance and geometric feature preservation. Aim at the drawback of the quantity and redundancy of LiDAR points, a number of methods have been developed, which are broadly divided into two categories. The first category is resampling, namely resampling the points from raw LiDAR data by uniform interval, which doubtlessly losing some useful information due to the reduction on the same level. The second approach is merging and resampling of LiDAR data in homogenous area. That means merging redundant points and replacing the original quantity of points with small amount of sample ones (or synthetic sample points) based on some algorithms in the homogeneous area (such as certain building’s facade), in order to achieve the purpose of the data reduction. This method has obvious advantages
L. Yang et al. / Optik 125 (2014) 6282–6286
because of the adaptive data resampling, reducing the amount of scanned data while maintaining useful feature information and the required accuracy [4]. Some research related to point data reduction is described. Liu et al. [5] completed the compression of LiDAR data by a series of procedures including resampling by uniform interval, and median filtering and removal based on the elevation distance and compression test by entropy calculation of echo intensity. Zuo et al. [6] proposed a typical noise removal algorithm of LiDAR points based on 3D finite-element analysis. Firstly, point clouds were partitioned into smaller and similar units by finite elements named space hexahedron model. And then, all of the units were classified into noise units or non-noise units with adjacency-based reasoning rules. Finally, the low noise was removed by iterative processing with finer threshold. The form of finite-element analysis, namely the Boolean function model, was selected to describe local solutions in relatively simple circumstance. Some other forms should be used to solve complex problems. Xu et al. [7] also presented a simplification method for LIDAR ground points based on local scatter characteristics of points. Here, scatter means the difference of elevation between each original point and the even elevation. The larger the difference, the more attribution to the terrain undulation was. So, the point with larger difference of elevation was remained while the one with smaller difference of elevation was removed, and then the topographic patterns were maintained in local region. Their method, however, had a drawback due to the reduction error been sensitive to the threshold of scatter. Dobrucali et al. [8] proposed a compression technique based on compressive sensing by sampling the sparse signal using either simple coding or compressive sampling, which provided a reasonably good compromise between reconstruction accuracy and speed and was superior to some well-known lossy and lossless compression techniques in encoding the scan data. The proposed technique involved efficiently sampling the sparse signal obtained from the difference between the current scan and its estimate, which was generated by shifting the previous scan along the horizontal and vertical axes by certain amounts. The method could be effectively used for 3D representation of both indoor and outdoor environments [9], however, could be improved by further coding the encoder output via lossless coding techniques. Luo et al. [4] suggested a method to reduce point data based on quad-tree partition. In their method, the raw data were totally partitioned in the designed minimize size grids based on quad-tree firstly, and then the data in homogenous area were merged and resampled from bottom to top according to certain homogenous decision rule. The algorithm was suitable for the point reduction in a plan with high compression ratio, keeping the edge of good performance, however, for data compression on the surface (such as spherical, hyperboloid), due to the gradient vector normal, it was difficult to offer a reasonable threshold and the corresponding improvement was needed. Lee et al. [2] proposed non-uniform grid based point data reduction methods from the point of view of part shape and z-axis error occurring in laser scanning. These non-uniform grid methods were implemented either by using one-directional or bi-directional non-uniform grids. The proposed methods were applied to sample models having freeform shapes, and the data reduction could be performed effectively while maintaining the quality of the point data. However, these methods were only useful if the registration of point clouds was performed after reducing the data size for each point cloud. When a complete 3D model of a part needed to be processed, these 2D grid methods might not be suitable, since they were projection-based techniques. The reduction method dealing with 3D scanned data needed further research. Major research efforts in the existing data reduction methods lie in manipulating polyhedral models. Various schemes are also used to reduce the amount of point data from the initial point clouds;
6283
however, none of the existing methods has considered the usage of the characteristics of the geometric feature derived from optical image, which is obtained easily from the most of 3D scanners. In my paper, a novel method for reducing LiDAR data cued by geometric feature derived from optical image is proposed, according to the processing” 2D feature line – 3D feature line-buffer area of LiDAR points”. 2. Methodology 2.1. Feature extraction from optical image and handling Because straight line is the main structure boundary for most regular buildings, straight line is chosen as the basic feature element to analyze and detect more complex features in this article. The feature extraction algorithm based on gradient direction information of edge detection proposed by Von Gioi et al. [10] is adopted in this paper. However, due to the shooting angle, light shadow, non-geometric texture extraction et al., there still exist the following problems: (1) the straight line segment is of fracture or overlap, and so on; (2) the boundary lines are not closed or exceed both ends in the corner of windows and walls, which leads to the lack of topology information between line and line, not only being against of the integrity of geographical objects and having an influence on the modeling of building structure. So further processing based on the structure integrity constraints should be executed for the extracted feature more reliable and more photo-realistic to express the building outlines. For most of building facades, the geometric shapes of construction members are fairly regular, such as the boundary structure of door and window generally is rectangular, and the edge of wall is straight line, etc. So building facade features are defined in advance to describe the semantic features, like the classification of ground features in remote sensing image processing. The facade features classification method proposed by Pu et al. [11] is adopted in this paper, which can be divided into six types: walls, edges of walls, doors, windows and other convex (hollow) edge lines, part of the building roofs. According to the topological relationship between the geometric feature lines and the feature attributes, the integrity constraint rules of building structure are described to establish the mathematical model for refining the feature extraction. The post-processing of 2D features from images based on the structural integrity constraint rules mainly includes the feature lines merging and closure in the corner point. Lines merging are based on some feature attributes, such as the directions and relative lengths of the line segments, and the attributes are regarded as weight parameters to determine the line segments merging [12]. Lines closure in the corner point is based on the rule of wall edge lines are vertical approximately at the wall corner by calculating the intersection point of two feature lines and then revising the feature line segments. 2.2. Space mapping from 2D feature line to 3D feature line In order to assist in reducing point clouds by 2D features, space mapping is necessary. The space mapping from 2D features to 3D ones is to calculate the mathematic projection relation of the image pixel coordinates and 3D point. For optical image, the spatial location coordinates of every pixel are determined by the camera imaging geometric model parameters. Take the simplest pinhole camera model for instance, that means a series of coordinate systems transformations during the camera imaging process. The coordinate systems transformations include the world coordinates to camera coordinates, the camera coordinates to image coordinates and image coordinates to the pixel coordinates. After
6284
L. Yang et al. / Optik 125 (2014) 6282–6286
Fig. 1. LiDAR data searching around 3D feature line segment in buffer area.
the three coordinate transformations, the geographical spatial 3D coordinates are projected into the 2D plane, and vice versa. Take the Riegl LMS series scanners as instance, its attached scan data processing software-RiSCAN provides corresponding transformation parameters, as a result, and the transformation model is achieved [13]. The detailed steps are: calculate the 3D coordinates of each endpoint of 2D image feature line segments, and then search rapidly the LiDAR point closest to the 3D endpoint based on k-d tree neighborhood search method. After the data processing, 2D image feature lines can be converted into 3D features. 2.3. Data organization and quick search of LiDAR points The discrete LiDAR dataset acquired from laser scanner includes the values of all range, azimuth, polar, color and intensity in a scanning coordinate system, but is lack of geometric topology information among the points, which leads to the difficulty on searching the neighbor point and calculating the normal vector and curvature of LiDAR data. So in order to deal with the mass point cloud data, the primary job is to establish the index of the discrete LiDAR points and realize rapid search based on neighborhood relations. The spatial index method based on k-d tree is adopted in this paper to re-organize the point data for improving the efficiency of searching [14]. And then the consequent calculation of the normal vector and the curvature of LiDAR point can be improved rapidly based on k-d tree neighborhood search method. 2.4. Buffer areas of LiDAR data cued by 3D feature lines For refining 3D feature lines from LiDAR data efficiently, the primary job is to search the regional LiDAR points around the 3D feature lines. Generally speaking, in a 2D space, the points searching around a line segment is based on surface buffer and judging whether points are within the buffer area. Correspondingly, in a 3D space, the points searching around the line segment is based on a cylindrical buffer with the 3D line segment as the axis of the cylinder. By iteratively calculating and judging whether the distance of each LiDAR point to the axis is less than or equal to the circle radius of the cylinder, the points within the buffer is determined consequently. Although this method is simple, the computation efficiency is on an extremely low level and requires higher computer processing speed. A new search method based on the spheres with the same radius of the cylinder is proposed as shown in Fig. 1. A series of point datasets within the spheres approximate the inside points of the cylinder instead, so the buffer area of LiDAR data cued by 3D feature lines is determined by searching neighborhood points of the center of the sphere. Based on the above k-d tree index method of LiDAR data, the efficiency of searching the LiDAR points in the buffer area could be improved, thus simplifying the subsequent refining of the 3D features based on LiDAR data. The detailed searching steps are shown in Fig. 2. Firstly, determine the sphere searching radius R. Secondly, determine the searching step size D according to the length of 3D feature segment, and determine the main searching direction (in the direction, the coordinates difference between two endpoints
Fig. 2. Processing of generating buffer area of LiDAR data based on 3D feature line segment.
is largest) according to the coordinates differences in the X, Y, Z direction, respectively. Thirdly, calculate the coordinates of each searching point of division (the center of each searching sphere) on the feature line. Then, in order to ensure the spheres fully filled cylinder, calculate and judge whether D is less than or equal to the R/2, if not, re-calculate the searching step size D and the coordinates of each searching point of division, and if yes, the searching of LiDAR points can be executed based on k-d tree neighborhood search method. Finally, judge whether each LiDAR point is recorded, only the new point should be remained, avoiding to be recorded more than once. Repeat the above process until the LiDAR data searching based on each point of division on the feature segment is completed. 3. Experiment and discussion LiDAR-optical data from a real geographical environment was taken as an instance to test the method proposed in this paper. The data cover partial building facade in Nanjing Normal University, China. The original optical image (as shown in Fig. 3) generated from Nikon D100 camera mounted on top of the scanner, whose focal length is 20 mm and the picture format is 3008 × 2000 pixels. The LiDAR data (as shown in Fig. 4) were acquired by Riegl 3D laser imaging system of LMS-Z420. Fig. 5 is the result of 2D feature lines extraction from image and post-processing. It can be seen that most
Fig. 3. Original optical image.
L. Yang et al. / Optik 125 (2014) 6282–6286
Fig. 4. Original LiDAR data.
6285
Fig. 7 is the result of buffer areas of LiDAR data cued by 3D feature lines. It can be seen from Fig. 7(a) the amount of redundant point clouds were removed compared with Fig. 4 and the points around feature lines were retained. Statistical result shows the number of the original LiDAR points is 746,647, and the point number of the buffer areas is 340,979, accounting for 45.67% of the former (where buffering radius is 0.3 m), which needs less time in post processing based on LiDAR data. Fig. 7(b) is the corresponding detailed drawing. The next work, the feature segments can be refined based on the reduced LiDAR data by the calculation of the normal vector and the curvature of LiDAR data, and the classification and plane fitting and feature lines extraction, and eventually completing 3D modeling of the building facade. 4. Conclusions
Fig. 5. 2D feature lines extraction from image and post-processing.
Fig. 6. The matching result of LiDAR data and optical image. (a) Overall view and (b) detailed drawing.
With the rapid development on the 3D scanning system, the processing efficiency of LiDAR data has been increasingly focused due to the huge amount of points cloud. A new LiDAR data reduction method is proposed in this study. By generating the buffer area of LiDAR points cued by 3D segments converted from 2D feature segments extracted from optical image based on rigorous mathematical models, the reduced LiDAR data can be obtained with reliable structures and accurate geometric position. The main contributions of the proposed approach are as follows: (a) a new technique to utilize the complementary image data to assist the LiDAR data reduction, improving the reliability of segmentation and avoiding information loss, and (b) put forward to the conception of buffer area of LiDAR data. The main issue here is generating LiDAR point datasets around the 3D feature segments, which benefits to improving the efficiency of post-processing of LiDAR data. Further studies may be performed in improving and optimizing the extraction algorithm of 2D features from image due to its importance to the generation of buffer area of LiDAR data. In addition, further regular and adaptive reduction algorithms may be applied to the LiDAR data in the buffer area (e.g., resampling by uniform or non-uniform grid interval). Acknowledgements This work was supported by the National Natural Science Foundation of China (grant no. 41101377 and no. 41271383) and the Priority Academic Program Development of Jiangsu Higher Education Institutions, China (PAPD) (grant no. 164320H101). Sincere thanks are given for the comments and contributions of anonymous reviewers and members of the Editorial team. References
Fig. 7. Buffer areas of LiDAR data cued by 3D feature lines. (a) Overall view and (b) detailed drawing.
of obvious feature lines can be extracted, and with a small number of overlap lines or break lines. Fig. 6 is the matching result of LiDAR data and optical image based on the rigorous registration model. Fig. 6(a) is the overall view of the matching of the building facade and Fig. 6(b) is the corresponding detailed drawing.
[1] L.-Y. Zhang, R.-R. Zhou, W.-B. Cai, Research on cloud data simplification, J. Comput. Aided Des. Comput. Graph. 13 (2001) 1019–1023. [2] K.H. Lee, H. Woo, T. Suk, Data reduction methods for reverse engineering, Int. J. Adv. Manuf. Technol. 17 (2001) 735–743. [3] A.C. Lin, H.T. Liu, Automatic generation of NC cutter path from data points, Comput. Aided Des. 30 (1998) 77–90. [4] D.-A. Luo, L.-Q. Liao, Ground-based LIDAR data simplification based on quadtree partition, Comput. Appl. 25 (2005) 420–421, 425. [5] C. Liu, C. Lu, Compress of the three dimension laser ranger scanning data and topography sampling, Remote Sens. Inf. 2 (2005) 6–10. [6] Z. Zuo, Z. Zhang, J. Zhang, Noise removal algorithm of LIDAR point clouds based on 3D finite-element analysis, J. Remote Sens. 16 (2012) 297–309. [7] J. Xu, Y. Wan, S. Zhang, On simplification method for LiDAR ground points cloud, J. Geomat. 33 (February) (2008) 32–34. [8] O. Dobrucali, B. Barshan, Novel compression algorithm based on sparse sampling of 3-D laser range scans, Comput. J. 56 (2013) 852–870. [9] O. Dobrucali, B. Barshan, A compression method for 3-D laser range scans of indoor environments based on compressive sensing, in: Proceedings
6286
L. Yang et al. / Optik 125 (2014) 6282–6286
of 19th European Signal Processing Conference, Barcelona, Spain, August 29–September 2, 2011, pp. 1115–1119. [10] R.G. Von Gioi, J. Jakubowicz, J. Morel, LSD: a fast line segment detector with a false detection control, IEEE Trans. Pattern Anal. Mach. Intell. 32 (2010) 722–732. [11] S. Pu, G. Vosselman, Automatic extraction of building features from terrestrial laser scanning, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 36 (2006) 25–27.
[12] J.M.R.S. Tavares, A.J. Padilha, A new approach for merging edge line segments, in: Proceedings of 7th Portuguese Conference of Pattern Recognition, 1995, pp. 1–6. [13] B. Qiu, Research on Realistic Texture Mapping for 3D Laser Scanning, Nanjing University of Science and Technology Press, Nanjing, 2009, pp. 13–19. [14] J.L. Bentley, Multidimensional binary search trees used for associative searching, Commun. ACM 18 (1975) 509–517.