- Email: [email protected]
Planar segmentation of indoor terrestrial laser scanning point clouds via distance function from a point to a plane
Optics and Lasers in Engineering 64 (2015) 23–31
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Planar segmentation of indoor terrestrial laser scanning point clouds via distance function from a point to a plane Cumhur Sahin 1 Department of Geodetic and Photogrammetric Engineering, Gebze Institute of Technology, PK 141, Gebze, 41400 Kocaeli, Turkey
art ic l e i nf o
a b s t r a c t
Article history: Received 7 April 2014 Received in revised form 26 May 2014 Accepted 12 July 2014
Terrestrial laser scanners are frequently used in most of measurement application, particularly in documentation and restoration studies of indoor historical structures, and in acquiring facade reliefs. When compared to a photogrammetric method, terrestrial laser scanners have the ability to give three dimensional point cloud data directly in a fast and detailed way. High data density of point cloud data is a challenging factor in texture-map operations during documentation and restoration of historical artifacts with more indoor spaces. When coordinate information for terrestrial laser scanner point cloud data is documented, it is seen that there is no regular order and classification for the data. The aim of this study is to suggest the mathematical filtering algorithm for segmentation work towards separation of planar surfaces which have different depths and parallel to each other and which can be frequently encountered in the indoor spaces from the data of terrestrial laser scanner. Filtering function for segmentation used, is based on the distance of a point to the plane. This algorithm has been chosen for the advantage of the rapid and easy results for extracting 3D coordinate data in texture mapping process. The MatLAB interface has been developed for using this method and analyzing the results for application which is detected how many different surfaces exist according to the statistical deviation amount. In the application, test data with 21932 points was segmented by separating it into 16 points in total with four different planes and four corner points per plane. Surfaces with four different depths were obtained as the result of the research. Each of them included four points. These segmented surfaces consisting of four points will facilitate integrated data production by integrating vectorial terrestrial laser scanner data into raster camera data, without the need to conventional measurements that accelerate particularly documentation and modeling in the fields of historical indoor areas. & 2014 Elsevier Ltd. All rights reserved.
Keywords: Segmentation Terrestrial laser scanning Computer programming Integration
1. Introduction Terrestrial laser scanners, which are important parts of today's measurement technology, have an area of usage in documentation and restoration studies on historical monuments. When we compare this new method with the previous photogrammetric measurement technique, we can see that the obtained final products have several advantages and disadvantages in terms of architectural usage. When architectural products are examined, geometrical accuracy and image reality of the photographic product are essential factors. Thus, photogrammetric method in architectural studies has not been completely abandoned. It is obvious that when fast geometric measurement speed coming from terrestrial laser scanner is combined with photogrammetry, there will be an effective measurement method. The aim of this study is to produce a filtering function algorithm for the purpose
E-mail address: [email protected] Tel.: þ90 262 6051810; fax: þ 90 (262) 6530675.
1
http://dx.doi.org/10.1016/j.optlaseng.2014.07.007 0143-8166/& 2014 Elsevier Ltd. All rights reserved.
of segmentation of laser data which is the most significant stage for providing integrated data production by integrating vectorial terrestrial laser scanner data into raster camera data, without the need to conventional measurements that accelerate particularly documentation and modeling in the fields of historical indoor areas and also, the aim of the study is to test that algorithm with the sample data. The digital photogrammetry and laser scanning technique are two of the most frequently used techniques in acquisition of three dimensional digital models [1]. The biggest disadvantage of terrestrial laser scanners is their inability to provide sufficient color precision of the scanned objects for a realistic model. This fact is emphasized frequently in the literature. Laser scanning can produce intense three dimensional point-cloud data which are required to establish high-definition geometric models, but the quality of color information is sometimes lower than the requirement [2]. Because of this scanner inability, in the literature, it is suggested that digital photogrammetry technique and laser scanning technique are combined for a successful three dimensional model. There are also distinctions in terms of accuracy, reliability,
24
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
and detail acquisition ability and automation level between the techniques. Thus, mostly it is necessary to combine the data from different technologies in order to model complex architectures. Image based approaches and scanner based methods are generally complimentary to each other [3]. Photo-realistic modeling is a technique which produces three dimensional virtual copies of an object in high quality textures, by integrating geometrical data from surface textures of the object and from terrestrial laser scanner [4]. As a result, in most of the cases, the best solution would be a combination of the above mentioned methods considering their benefits [5–7]. Today, there are several studies on indoor areas in the context of combining terrestrial laser scanners and photogrammetry studies. [8] are presenting an accuracy assessment of 3D point clouds of complex interiors produced with a fully automated open source photogrammetric software developed within the IGN (French Mapping Agency) [8]. Laser scanners allow millions of points to be recorded in a few minutes. Because of their practicality and versatility, these kinds of instruments are today widely used in the field of architectural, archeological and environmental surveying [9]. Today, detailed, complete and exact 3D models with photo-realistic textures are increasingly demanded for numerous applications in architecture and archaeology [10]. The integration of several technologies in heritage documentation is discussed every time a new technology appears. Photogrammetry has always been technology driven. TLS technology is more recent and changed the documentation approach from vector based models to point cloud based approaches ([11]). For a long time, laser scanning was the main solution to produce dense 3D point-clouds allowing high resolution geometric models, while photogrammetry was more suited to produce high resolution 3D textured models of small objects [12]. TLS techniques have been widely adopted for cultural heritage documentation and therefore many papers have tried to investigate the advantages and disadvantages of TLS and manual close range photogrammetry ([2,7,13,14]). The general conclusion is that due to the complexity of the scenes and of the materials used the choice of the method is heavily correlated with the scene that has to be modeled and hence a combination of the two methods could be interesting in various cases [8]. Because of the rapid changes in technology, some new concepts such as indoor space maps and Historic Building Information Modeling (HBIM) are introduced to us. The HBIM process begins with remote collection of survey data using a terrestrial laser scanner combined with digital photo modeling. The final HBIM product is the creation of full 3D models including detail behind the object's surface concerning its methods of construction and material make-up. The resultant HBIM can automatically create cut sections, details and schedules in addition to the orthographic projections and 3D models (wire frame or textured) for both the analysis and conservation of historic objects, structures and environments [15].
Terrestrial laser scanners can be considered as highly automatic motorized total stations. Unlike total stations however, where the operator directly chooses the points to be surveyed, laser scanners randomly acquire a dense set of points. The operator only selects the portion of the object he wishes to acquire and the density of the points he desires in the scan (usually the angular step of the scan in vertical and horizontal planes can be selected by the operator). Once these initial values have been chosen, the acquisition is completely automatic. The result of the laser survey is a very dense points cloud (also called DDSM – Dense Digital Surface Model). For each point of the model the X, Y, and Z coordinates and the reflectivity value are known. As this set of points is acquired in a completely arbitrary way, with the exception of the parameters imposed by the operator, it is necessary to manage this data in a critical and reasonable way. Particular attention must be paid to the quality of the original data [9]. Intense laser point cloud segmentation is one of the most bust academic study areas related to terrestrial laser scanners. The use of terrestrial laser scanners is increasing in the field of cultural heritage recording due to their high data acquisition rate, relatively high accuracy and high spatial data density. The main problem related to this new technique is the treatment of the collected data. [16] describes an automatic approach in laser scanning point clouds for architectural modeling. The aim of the algorithm is to obtain real surfaces of the scanned object and reduce the data volume [16]. Automatic modeling is applicable for buildings with relatively simple structure, such as rectangular solid, but it is not applicable for buildings with intricate structure. The goal is to model Japanese traditional houses and buildings in a district using TLS and photogrammetry for the sake of disaster simulation. In the proposed methodology, users are requested to define the procedure to estimate planes, edges and vertices [17]. Segmentation is the most important step in the feature extraction process. In practice, most segmentation approaches use geometrical information to segment the 3D point cloud [18]. The segmentation process is the essential step in obtaining surfaces, since the extraction of features of the different building elements basically depends on the accuracy of the lsegmentation step ([19]) [18]. As man-made structures are dominated by planar surfaces, many attempts have been made to segment planar surfaces from point clouds [18]. When segmentation is employed as a preprocessing step before the application of filtering algorithms, it is called segmentation-based filtering ([20]). Therefore, the segmentation processes for planar surfaces on man-made objects can be considered as a first step in the creation of 3D model documentation with a best fit to reality directly from 3D point clouds. However, although segmentation is one of the ;main processing steps, it is far from being solved even for planar features ([21]) [18]. In the past decade, many algorithms have been designed to extract planar surfaces from point clouds using segmentation methods. Usually one of three distinct methods is employed for segmenting points: region growing ([22–24]),
Fig. 1. Segmentation method.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
clustering of features ([16,25–28]) or the model fitting method ([29–32]). While region-growing and feature-clustering methods are based on geometrical criteria for grouping homogeneous
25
regions that are present in the point cloud data, the model fitting algorithms are based on fitting geometric primitive shapes [18]. 1. The region-growing method: ([24]) proposed an approach to automatically extract planar surfaces from TLS point clouds following the region-growing segmentation method of ([33]). The region-growing method does suffer from the main and difficult disadvantage of having to define the correct seed surface, because if the definition of the seed surface is wrong (particularly in the case of large noisy data-sets) the error will grow and all processes will fail [18]. 2. The method based on clustering features: ([28]) proposed an unsupervised clustering approach based on fuzzy methods. Both the Fuzzy C-Means (FCM) algorithm and the Possibilistic C-Means (PCM) mode-seeking algorithm are used incombination with a similarity-driven cluster merging method [18]. 3. The model fitting method: The model fitting method is based on fitting geometric primitive shapes, which can be represented mathematically as planar surfaces; then points are conformed by the mathematical representation that would group them as one segment. Two widely known algorithms in line with model fitting methods are RANSAC (introduced by ([34])) and the Hough transform introduced by ([35]) [18]. By the way, the suggested algorithm in this study has been included model fitting segmentation methods.
Fig. 2. Point cloud data structure in 3D coordinate system.
Fig. 3. Distance of a point to a plane.
A new data integration model is suggested for the operating processes in architectural documentation and cultural heritage studies, with the terrestrial laser scanning that uses new data production methods of photogrammetry [36]. When coordinate information for terrestrial laser scanner point cloud data is documented, it is seen that there is no regular order and classification for the data [36]. Points with known three dimensional coordinates, which are used in image rectification, should be picked out from all the TLS data. In order to classify point location data of surfaces with different depths, raw data obtained with scanner is filtered and data is classified with the help of a mathematical algorithm. By the help of filtering function, it is aimed to automatically produce the points to be used in photogrammetric image rectification without any operation on point cloud data forming TLS data which requires the presence of any operator. Sample point cloud data is classified in Matlab environment by the algorithm of filtering function which has been developed for this purpose. In the second part of the study, the algorithm is described. Third part describes sample application; fourth part focuses on the results and discussion. The fifth part is the conclusion part.
Fig. 4. Matrix form of segmentation algorithm in Matlab software.
26
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
2. Algorithm
Fig. 5. Four corners of the surfaces obtained by algorithm.
Algorithm of this study which aims at filtering laser point cloud data of parallel surfaces in indoor areas by the help of filtering function is shown in Fig. 1. The first step of filtering function algorithm is primarily choosing one of the parallel indoor surfaces as the reference plane by an operator. The point cloud data that is shown in Fig. 2 is defined by distinct surfaces. Indoor areas are usually defined by plane surfaces such as walls. Fig. 2 shows the point cloud data and different surface construction in three dimensional coordinate system. Mathematical function that represents plane surface is given in Eq. (1) [37]. Ax þ By þCz þD ¼ 0
Fig. 6. Application site.
ð1Þ
As shown in Eq. (1), surface function has four parameters, namely A, B, C, and D. Calculation of these four parameters is essential to mathematically express the chosen reference plane surface. For the sake of determining parameters of the chosen reference plane surface, point coordinates which are chosen should be read and entered manually by an operator from the reference plane. A plane is mathematically defined in four parameters. Thus, the only solution is to define the parameters of reference plane with four points. For determining the reference plane parameters in the adjustment computation, choosing more than four points which belong to same reference plane by an operator over the plane is made possible. The second value that
Fig. 7. Application data. (Four indoor plane surfaces with different depths).
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
operator will enter manually is the threshold value. Threshold value is the minimum difference of depth that the filter will be operated. Apart from these two stages in algorithm, all the operations are performed automatically by software developed in Matlab. The second step of algorithm is to calculate parameters of the chosen reference plane. After five or more points are chosen manually by an operator in the first step of algorithm, four parameters that represent reference plane are determined in the adjustment computation automatically by interface which is developed in Matlab. So, adjusted reference plane is generated in this step of algorithm. The third step of algorithm is to calculate the distance of all points within laser point cloud to reference plane with parameters determined in the adjustment computation. Fig. 3 shows the distance of a point to a plane. With the given in Eq. (2) [37], distances of all points within laser point cloud to the adjusted reference plane is calculated. This equation is the filtering
27
function chosen for segmentation. h¼
jAx1 þ By1 þCz1 þ Dj pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi A2 þB2 þ C 2
ð2Þ
Where h is the distance of a laser point to the reference plane, A, B, C, and D are the parameters of the adjusted reference plane and x1, y1, z1 are the three dimensional coordinate of each laser point. These distances are added to segmentation matrix S as a column in vector form. Distances of points into reference plane in segmentation filtering are calculated separately for each point. Matlab operation environment is defined in segmentation matrix form. It is proper to call this as “segmentation matrix”. However, calculations are made with column algorithm and final column vector is used for classification of points into surfaces. According to that, first column in segmentation matrix shown in Fig. 4 is X value, the second column is Y value, the third column is Z value of points in terrestrial laser scanning point cloud data. The fourth column is the distance of points into reference plane, the fifth
Fig. 8. Application data in Matlab software.
28
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
column shows statistical differences of the distances, the sixth column shows exponential values of those differences, and the seventh column illustrates which surface the point belongs to. Fourth step of algorithm is segmentation (classification) part. It is used for statistical change of the vector created for this study. It is the detection of how many different surfaces are in the vector, depending on the amount of statistical deviation. Statistical analysis is made based on standard deviation values. The comparison value for standard deviation is the threshold value. So, the amount of statistical deviation is determined with reference to the searched minimum difference of depth. Thus, points with the less minimum difference of depths are considered on the same surface. In statistical analysis (5th Column) the number of different surfaces in total are detected, and all the distances are shifted into a positive value with exponential function (6th Column). So, raw data is obtained for classification. Exponential of the found value is taken since there are some conditions in which the distance of point to plane is negative and that the point remains in the other side of the plane. Thus the points between two surfaces are assigned to the surface with the smaller absolute value. All points within laser scanning data are assigned to some surface based on this value and to the matrix that belongs to the surface (7th Column). The last step of segmentation algorithm is the stage that defines four limit points of surface coming from laser scanning points which is assigned to matrix of the surface. Minimum and maximum x, y plane coordinate values of the points within surface matrix are used for this purpose. The edge points in each segment of segmented original laser point cloud data are detected according to minX–minY, minX–maxY, maxY–minX, maxX–maxY values. Thus, both the limit points of surface and x, y plane coordinates are determined. However, original values are not taken as the height value for the points with defined plane coordinates in the algorithm. If we created a surface with the Z value obtained from laser scanning data of four points which creates the surface, then every surface would not be parallel to each other. So, when assigning height value to four edge points, average of the Z (height) value for all the points which are assigned for that surface as the result of segmentation, is assigned as the height value. Thus, Z value of four edge points that are segmented for each surface is the mean Z value and it is the same value for each of the four points. This is shown in Fig. 5.
As the result of segmentation, each of the surfaces with thousands of points is converted into planes which include only four points and which take the average of heights of all the points within a segment.
3. Application Laser point cloud of a class, scanned with Leica HDS 3000 on GYTE (shown in Fig. 6) in order to test filtering algorithm. Scanning frequency is 5 mm. While obtaining the determined data in txt format from CYCLONE (Leice) software, affine transformation is performed in a way to give Z data depth for point cloud data. Thus, determining the reference surface will be easier for operator. An original data set containing four different surfaces and 21,932 points was selected. There are various differences in depth ranging from 1 cm to 20 cm between the surfaces of doors, borders, walls and columns. In the indoor space shown in Fig. 7, surface 1 is accepted as reference plane in order to test the filtering function from four plane surfaces which are parallel to each other. In Fig. 7, Surface 1 is the backmost surface and surface 4 is the foremost surface. Five points on reference surface are chosen by an operator in Cyclone screen for the calculation of plane equation parameters. Threshold value for test data (minimum difference of depth) is determined as 1 cm. When the software developed in Matlab is started, 21,932 three dimensional laser point clouds are classified into four separate surfaces. This is shown in Fig. 8. Fig. 9 illustrates the graphics of exponential values. In the first segment of the original 21,932 laser scanning data, divided into four different classes, there are 3906 points, in its second segment 6588 points, in its third segment 1951 points, in its four segment 9487 points (with 5 points taken from surface1). Fig. 10 shows the surfaces passing through each of the laser point cloud data, which are divided into four classes. As shown in Fig. 10, distance threshold value between second and third plane surfaces is around 1 cm and the filter satisfyingly segmented these two different surfaces. All points on laser point cloud are assigned into a plane surface and they are recorded into their appropriate surface matrix. X, y plane coordinate values of four edge points of plane surface are
Fig. 9. Segmantation graphic.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
found by the help of points within the matrix of each surface. Then, only one Z value is assigned to four points for this plane surface.
4. Results and discussion Here, the crucial point is that there is no limitation in choosing the surface which is defined as reference plane by operator. In the suggested algorithm, reference plane can be chosen by operator as any surface. Operator may determine the closest or the farthest plane surface or any surface between these surfaces as the reference surface. This is the reason of using exponential values during creating segmentation matrix which is defined as matrix S. Distances of the points to the reference plane might be negative or positive and this does not effect segmentation and classification results since exponential values are used. So, negative or positive distances to the reference plane (showing which side they stay
29
according to the plane and not affecting the result) do not cause any error in segmentation. Thus, when reference plane is determined as the foremost surface (surface 4) and five points are chosen from that surface, there are 3906 points in the first segment, 6542 in the second segment, 1997 in the third segment, 9487 in the fourth segment (with 5 points taken from surface 4). So, first and fourth surfaces with surface points which are out of the algorithm threshold value give 100% accuracy. And, 8542 points on two parallel surfaces with a distance which is similar to the threshold value give approx. 99% accuracy. Table 1 shows segmented amount of points based on the information that both of the surfaces are reference planes. In this situation, one might question whether faulty edge points are determined on the surfaces close to the threshold values during determination of limit values of the surface; but in fact, this question is irrelevant because x, y plane coordinates of the surface are determined from the matrix points that are assigned to the surface.
Fig. 10. The result of segmentation (Four segmentated surfaces which are different depths).
30
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
Table 1 Segmentation according to two different reference planes. Reference surface
Amount of points
Total amount of points
1. Surface 2. Surface 3. Surface 4. Surface Surface 1 (backmost) 3906 Surface 4 (foremost) 3906 Difference in point 0 amounts
6588 6542 46
1951 1997 46
9487 9487 0
21932 21932 92
Images of four surfaces with different depths which are formed of four corner points obtained by segmentation, taken by terrestrial laser scanner cameras or digital cameras, are textured over those surfaces and this provides simplicity and convenience, compared to other multi-point surfaces. Thus, by the help of planes created in this way, point cloud data is much more easily processed by both the hardware and the operator. This segmentation study provides a data structure which is much more appropriate for texture mapping than point cloud data.
5. Conclusion Considering the core software as a result of the research, a general advantage–disadvantage list is shown below for the functional model. 5.1. Advantages of suggested algorithm Any of the surfaces with difference of depth can be chosen as the reference surface by the operator. In determination of the reference surface, there is no limitation such as choosing the nearest or farthest one for operator. The original aspect of the study is that four corner points within original laser data for each surface are determined with the average of Z values of all the points that are segmented for that surface, for the sake of creating a texture map. So, the walls are made surfaces which are really parallel to each other. If a surface which passes through all the points in segmentation in original laser scanning data segmented into four separate surfaces, this surface obtained by the difference of Z (height values) will not be the plane surface. By the help of filtering function, the points to be used in photogrammetric image rectification are produced automatically, without any operation on point cloud data forming TLS data which requires the presence of any operator. 5.2. Disadvantages of suggested algorithm Difference of depth is determined as the threshold value. If, for example, the difference of depth is determined as 5 cm, then differences of depth less than 5 cm will be considered as one single surface by the filtering function, instead of a separate surface. This is a disadvantage. Thus, rectification points that are most appropriate for parallel surfaces are detected for the surface texture model, by simplifying with a geometric object based filtering function. This geometry is formed by surfaces which have different depths and are parallel to each other. These kind of surfaces are frequently seen in indoor spaces for terrestrial laser scanning with a dense point cloud data. Then, in the integration study for obtaining a true scale surface texture model, these points are detected from laser scanner data. Also, image coordinates of the points are detected by reducing with the help of “projection over image” equations.
With the study that will be made after this point, data obtained by camera equipment in indoor spaces and terrestrial laser scanner data will be integrated and texture models that can be defined only with three dimensionally located and scaled raster data will be obtained. Models created with this integration are much smaller than the laser scanning data in terms of file size. Thus, file sizes decrease in modeling. So, depending on the power of hardware, memory and hard disk problems that we encounter during editing three dimensional model files with software will be resolved. Additionally, since converting them into VRML format is made by the help of texturing, these kind of models will be more preferred in virtual tour studies. Terrestrial laser scanning data which gives three dimensional information directly in terms of product but does not have a high radiometric resolution and photographic image which is necessary for surface texture will complement each other. Combination of these two data source will increase speed and reliability of documentation. References [1] Abdelhafiz A. (Ph.D. Thesis). Integrating digital photogrammetry and terretrial laser scanning. TU, Braunsvhweig, Germany: Institute of Geodesy and Photogrammetry; 2009. [2] Kadobayashi R, Kochi N, Otani H, Furukawa R. Comparision and evaluation of laser scanning and photogrammetry and their combined use for digital recording of cultural heritage. In: Proceedings of the XXth ISPRS congress, Istanbul, Turkey; 12–23 July 2004. p. 401–6. [3] Gonzo L, Voltolini F, Girardi S, Rizzi A, Remondino F, El-Hakim SF. Multiple techniques approach to the 3D virtual reconstructions of cultural heitage. In: Proceedings of the 5th eurographics italian chapter conference, Trento: Italy; 14–16 February 2007. [4] Ortiz P, ve Sanchez H. Virtual city models combining cartography and photorealistic texture mapped range data. In: Proceedings of the XXIIth international cartography conferance, Coruna: Spain; 9–16 July 2005. [5] Fuchs A, Alby E, Begriche R, Grussenmeyer P, Perrin JP. Confrontation du Relevé Laser 3D Aux Techniques de Relevé Conventionnelles et Développement D'outils Numériques Pour la Restitution Architecturale. Revue de la Société Française de Photogrammétrie et de Télédétection n1 173/174 (2004-1/ 2), 2004. p. 36–47. [6] Gonzo L, El-Hakim SF, Picard M, Girardi S, Whiting E. Photo-realistic 3-D reconstruction of castles with multiple-sources image-based techniques. In: Proceedings of the international archives of the photogrammetry, remote sensing and spatial information sciences, vol. 35 (B5); 2004. p. 120–5. [7] Grussenmeyer P, Landes T, Voegtle T, Ringle K. Comparision methods of terrestrial laser scanning photogrammetry and tacheometry data For recording of cultural heritage buildings. In: Proceedings of the international archives of the photogrammetry, remote sensing and spatial information science, XXXVII (B5), Beijing, China; 3–11 July, 2008. [8] Georgantas A, Bredif M, Desseilligny MP. An accuracy assessment of automated photogrammetric techniques for 3D modeling of complex interiors. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIXB3, Melbourne, Australia; 25 August–01 September 2012, 2012. [9] Bornaz L, Rinaudo F. Terrestrial aser scanner data processing. In: Turkey V, editor. Proceedings of the XXth ISPRS congress technical commission Orhan Altan ISPRS Archives – vol. XXXV Part B5, Istanbul: Turkey; 12–23 July 2004. p. 514–9. [10] Kersten TP, Stallmann D. Automatic texture mapping of architectural and archaeological 3D models. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIX-B5, , Melbourne, Australia; 25 August–01 September 2012. [11] Grussenmeyer P, Hanke K. Cultural heritage applications. Airborne and terrestrial laser scanning. Whittles Publishing; 2010; 271–90. [12] Grussenmeyer P, Alby E, Landes T, Koehl M, Guillemin S, Hullo JF. et al. Recording approach of heritage sites based on merging point clouds from high resolution photogrammetry and terrestrial laser scanning. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIX-B5, Melbourne, Australia; 25 August–01 September; 2012. [13] Alshawabkeh Y, Haala N. Integration of digital photogrammetry and laser scanning for heritage documentation. In: Proceedings of the XXth international society for photogrammetry and remote sensing congress, Comission V, Istanbul, Turkey, 12–23 July; 2004. p. 424–9. [14] Boehler W, Marbs A. 3D scanning and photogrammetry for heritage recording: a comparison. In: Proceedings of the 12th international conference on geoinformatics, University of Gavle, Sweden; 2004. p. 291–8.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31 [15] Murphy M, McGovern E, Pavia S. Historic building information modelling – adding intelligence to laser and image based surveys of European classical architecture. ISPRS J Photogramm Remote Sens 2013;76:89–102. [16] Lerma JL, Biosca JM. Segmentation and ifltering of laser scanner data for cultural heritage. In: Proceedings of the CIPA XX international symposium. Torino, Italy; 26 September–01 October; 2005. [17] Susaki J. Modeling of building from terrestrial laser scanning considering popology for occlusion. In: Proceedings of the 22nd CIPA symposium. Kyoto, Japan; October, 11–15; 2009. [18] Awwad TM, Zhu Q, Du z, Zhang Y. An improved segmentation approach for planar surfaces from unstructed 3D point clouds. Photogramm Rec 2010;25 (129):5–23. [19] Rodriguez-Gonzalvez P, Gonzalez-Aguilera D, Gomez-Lahoz J. From point cloud to surface: modeling structures in laser scanner point clouds. In: Proceedings of International archives of the photogrammetry, remote sensing and spatial information sciences, Workshop on Laser Scanning and Silvilaser 2007, Espoo, Finland; vol. 36(3/W52), September 12–14; 2007. p. 338–43. [20] Tovari D, Pfeifer N. Segmentation based robust interpolation—a new approach to laser data filtering. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Laser Scanning 2005, Enschede, The Netherlands; vol. 36(3/W19), September 12–14; 2005. p. 79–84. [21] Hoover A, Jean-Baptiste G, Jiang Xiaoyi, Flynn PJ, Bunke H, Goldgof D. et al. Range image segmentation: the user's dilemma. In: Proceedings of IEEE international symposium on computer vision, Florida, USA; 1995. p. 323–8. [22] Hofmann AD, Maas HG, Streilein A. Knowledge-based building detection based on laser scanner data and topographic map information. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Graz, Austria; vol. 34(3/A); 2002. p. 169–74. [23] Dold C, Brenner C. Automatic matching of terrestrial scan data as a basis for the generation of detailed 3D city models. International archives of photogrammetry, remote sensing and spatial information sciences, Istanbul, Turkey; vol. 35(B3); 2004. p. 1091–6. [24] Pu S, Vosselman G. Automatic extraction of building features from terrestrial laser scanning. International archives of the photogrammetry, remote sensing and spatial information sciences, 25-27 September; vol. 36(5); 2006. p. 5. [25] Filin S. Surface clustering from airborne laser scanning data. International Archives of photogrammetry, remote sensing and spatial information sciences. vol. 34(3/A); 2002. p. 119–124.
31
[26] Hofmann AD. Analysis of TIN-structure parameter spaces in airborne laser scanner data for 3-D building model generation. International archives of photogrammetry, remote sensing and spatial information sciences. vol. 35 (B3); 2004. p. 302–307. [27] Filin S, Pfeifer N. Segmentation of airborne laser scanning data using a slope adaptive neighborhood. ISPRS J Photogramm Remote Sens 2006;60(2):71–80. [28] Biosca JM, Lerma JL. Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods. ISPRS J Photogramm Remote Sens 2008;63(1):84–98. [29] Bauer J, Karner K, Schindler K, Klaus A, Zach C. Segmentation of building models from dense 3D point-clouds. In: Proceedings of 27th workshop of the austrian association for pattern recognition, Laxenburg; 2003. p. 253–8. [30] Bretar F, Roux M. Extraction of 3D planar primitives from raw airborne laser data: a normal driven RANSAC approach. In: Proceedings of IAPR conference on machine vision applications, Tsukuba Science City: Japan; 2005. p. 452–5. [31] Boulaassal H, Landes T, Grussenmeyer P, Tarsha-Kurdi F. Automatic segmentation of building facades using terrestrial laser data. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Workshop on Laser Scanning 2007 and SilviLaser 2007, Espoo, Finland; vol. 36(3/W52); 2007. p. 65–70. [32] Boulaassal H, Landes T, Grussenmeyer P. Automatic extraction of planar clusters and their contours on building fac¸ades recorded by terrestrial laser scanner. Int J Archit Comput 2009;7(1):1–20. [33] Vosselman G, Gorte BGH, Sithole G, Rabbani T. Recognising structure in laser scanner point clouds. International archives of photogrammetry, remote sensing and spatial information sciences. vol. 36(8/W2); 2004. p. 33–38. [34] Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981;24(6):381–95. [35] Hough PVC. Method and means for recognizing complex patterns. U.S. Patent 3,069,654. 1962. [36] Ergun B. A novel 3D geometric object filtering function for application in indoor area with terrestrial laser scanning data. Opt Laser Technol 2010;42:799–804. [37] Balci M. General mathematics, vol. 2. Balci Publishing; 2000.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Planar segmentation of indoor terrestrial laser scanning point clouds via distance function from a point to a plane Cumhur Sahin 1 Department of Geodetic and Photogrammetric Engineering, Gebze Institute of Technology, PK 141, Gebze, 41400 Kocaeli, Turkey
art ic l e i nf o
a b s t r a c t
Article history: Received 7 April 2014 Received in revised form 26 May 2014 Accepted 12 July 2014
Terrestrial laser scanners are frequently used in most of measurement application, particularly in documentation and restoration studies of indoor historical structures, and in acquiring facade reliefs. When compared to a photogrammetric method, terrestrial laser scanners have the ability to give three dimensional point cloud data directly in a fast and detailed way. High data density of point cloud data is a challenging factor in texture-map operations during documentation and restoration of historical artifacts with more indoor spaces. When coordinate information for terrestrial laser scanner point cloud data is documented, it is seen that there is no regular order and classification for the data. The aim of this study is to suggest the mathematical filtering algorithm for segmentation work towards separation of planar surfaces which have different depths and parallel to each other and which can be frequently encountered in the indoor spaces from the data of terrestrial laser scanner. Filtering function for segmentation used, is based on the distance of a point to the plane. This algorithm has been chosen for the advantage of the rapid and easy results for extracting 3D coordinate data in texture mapping process. The MatLAB interface has been developed for using this method and analyzing the results for application which is detected how many different surfaces exist according to the statistical deviation amount. In the application, test data with 21932 points was segmented by separating it into 16 points in total with four different planes and four corner points per plane. Surfaces with four different depths were obtained as the result of the research. Each of them included four points. These segmented surfaces consisting of four points will facilitate integrated data production by integrating vectorial terrestrial laser scanner data into raster camera data, without the need to conventional measurements that accelerate particularly documentation and modeling in the fields of historical indoor areas. & 2014 Elsevier Ltd. All rights reserved.
Keywords: Segmentation Terrestrial laser scanning Computer programming Integration
1. Introduction Terrestrial laser scanners, which are important parts of today's measurement technology, have an area of usage in documentation and restoration studies on historical monuments. When we compare this new method with the previous photogrammetric measurement technique, we can see that the obtained final products have several advantages and disadvantages in terms of architectural usage. When architectural products are examined, geometrical accuracy and image reality of the photographic product are essential factors. Thus, photogrammetric method in architectural studies has not been completely abandoned. It is obvious that when fast geometric measurement speed coming from terrestrial laser scanner is combined with photogrammetry, there will be an effective measurement method. The aim of this study is to produce a filtering function algorithm for the purpose
E-mail address: [email protected] Tel.: þ90 262 6051810; fax: þ 90 (262) 6530675.
1
http://dx.doi.org/10.1016/j.optlaseng.2014.07.007 0143-8166/& 2014 Elsevier Ltd. All rights reserved.
of segmentation of laser data which is the most significant stage for providing integrated data production by integrating vectorial terrestrial laser scanner data into raster camera data, without the need to conventional measurements that accelerate particularly documentation and modeling in the fields of historical indoor areas and also, the aim of the study is to test that algorithm with the sample data. The digital photogrammetry and laser scanning technique are two of the most frequently used techniques in acquisition of three dimensional digital models [1]. The biggest disadvantage of terrestrial laser scanners is their inability to provide sufficient color precision of the scanned objects for a realistic model. This fact is emphasized frequently in the literature. Laser scanning can produce intense three dimensional point-cloud data which are required to establish high-definition geometric models, but the quality of color information is sometimes lower than the requirement [2]. Because of this scanner inability, in the literature, it is suggested that digital photogrammetry technique and laser scanning technique are combined for a successful three dimensional model. There are also distinctions in terms of accuracy, reliability,
24
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
and detail acquisition ability and automation level between the techniques. Thus, mostly it is necessary to combine the data from different technologies in order to model complex architectures. Image based approaches and scanner based methods are generally complimentary to each other [3]. Photo-realistic modeling is a technique which produces three dimensional virtual copies of an object in high quality textures, by integrating geometrical data from surface textures of the object and from terrestrial laser scanner [4]. As a result, in most of the cases, the best solution would be a combination of the above mentioned methods considering their benefits [5–7]. Today, there are several studies on indoor areas in the context of combining terrestrial laser scanners and photogrammetry studies. [8] are presenting an accuracy assessment of 3D point clouds of complex interiors produced with a fully automated open source photogrammetric software developed within the IGN (French Mapping Agency) [8]. Laser scanners allow millions of points to be recorded in a few minutes. Because of their practicality and versatility, these kinds of instruments are today widely used in the field of architectural, archeological and environmental surveying [9]. Today, detailed, complete and exact 3D models with photo-realistic textures are increasingly demanded for numerous applications in architecture and archaeology [10]. The integration of several technologies in heritage documentation is discussed every time a new technology appears. Photogrammetry has always been technology driven. TLS technology is more recent and changed the documentation approach from vector based models to point cloud based approaches ([11]). For a long time, laser scanning was the main solution to produce dense 3D point-clouds allowing high resolution geometric models, while photogrammetry was more suited to produce high resolution 3D textured models of small objects [12]. TLS techniques have been widely adopted for cultural heritage documentation and therefore many papers have tried to investigate the advantages and disadvantages of TLS and manual close range photogrammetry ([2,7,13,14]). The general conclusion is that due to the complexity of the scenes and of the materials used the choice of the method is heavily correlated with the scene that has to be modeled and hence a combination of the two methods could be interesting in various cases [8]. Because of the rapid changes in technology, some new concepts such as indoor space maps and Historic Building Information Modeling (HBIM) are introduced to us. The HBIM process begins with remote collection of survey data using a terrestrial laser scanner combined with digital photo modeling. The final HBIM product is the creation of full 3D models including detail behind the object's surface concerning its methods of construction and material make-up. The resultant HBIM can automatically create cut sections, details and schedules in addition to the orthographic projections and 3D models (wire frame or textured) for both the analysis and conservation of historic objects, structures and environments [15].
Terrestrial laser scanners can be considered as highly automatic motorized total stations. Unlike total stations however, where the operator directly chooses the points to be surveyed, laser scanners randomly acquire a dense set of points. The operator only selects the portion of the object he wishes to acquire and the density of the points he desires in the scan (usually the angular step of the scan in vertical and horizontal planes can be selected by the operator). Once these initial values have been chosen, the acquisition is completely automatic. The result of the laser survey is a very dense points cloud (also called DDSM – Dense Digital Surface Model). For each point of the model the X, Y, and Z coordinates and the reflectivity value are known. As this set of points is acquired in a completely arbitrary way, with the exception of the parameters imposed by the operator, it is necessary to manage this data in a critical and reasonable way. Particular attention must be paid to the quality of the original data [9]. Intense laser point cloud segmentation is one of the most bust academic study areas related to terrestrial laser scanners. The use of terrestrial laser scanners is increasing in the field of cultural heritage recording due to their high data acquisition rate, relatively high accuracy and high spatial data density. The main problem related to this new technique is the treatment of the collected data. [16] describes an automatic approach in laser scanning point clouds for architectural modeling. The aim of the algorithm is to obtain real surfaces of the scanned object and reduce the data volume [16]. Automatic modeling is applicable for buildings with relatively simple structure, such as rectangular solid, but it is not applicable for buildings with intricate structure. The goal is to model Japanese traditional houses and buildings in a district using TLS and photogrammetry for the sake of disaster simulation. In the proposed methodology, users are requested to define the procedure to estimate planes, edges and vertices [17]. Segmentation is the most important step in the feature extraction process. In practice, most segmentation approaches use geometrical information to segment the 3D point cloud [18]. The segmentation process is the essential step in obtaining surfaces, since the extraction of features of the different building elements basically depends on the accuracy of the lsegmentation step ([19]) [18]. As man-made structures are dominated by planar surfaces, many attempts have been made to segment planar surfaces from point clouds [18]. When segmentation is employed as a preprocessing step before the application of filtering algorithms, it is called segmentation-based filtering ([20]). Therefore, the segmentation processes for planar surfaces on man-made objects can be considered as a first step in the creation of 3D model documentation with a best fit to reality directly from 3D point clouds. However, although segmentation is one of the ;main processing steps, it is far from being solved even for planar features ([21]) [18]. In the past decade, many algorithms have been designed to extract planar surfaces from point clouds using segmentation methods. Usually one of three distinct methods is employed for segmenting points: region growing ([22–24]),
Fig. 1. Segmentation method.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
clustering of features ([16,25–28]) or the model fitting method ([29–32]). While region-growing and feature-clustering methods are based on geometrical criteria for grouping homogeneous
25
regions that are present in the point cloud data, the model fitting algorithms are based on fitting geometric primitive shapes [18]. 1. The region-growing method: ([24]) proposed an approach to automatically extract planar surfaces from TLS point clouds following the region-growing segmentation method of ([33]). The region-growing method does suffer from the main and difficult disadvantage of having to define the correct seed surface, because if the definition of the seed surface is wrong (particularly in the case of large noisy data-sets) the error will grow and all processes will fail [18]. 2. The method based on clustering features: ([28]) proposed an unsupervised clustering approach based on fuzzy methods. Both the Fuzzy C-Means (FCM) algorithm and the Possibilistic C-Means (PCM) mode-seeking algorithm are used incombination with a similarity-driven cluster merging method [18]. 3. The model fitting method: The model fitting method is based on fitting geometric primitive shapes, which can be represented mathematically as planar surfaces; then points are conformed by the mathematical representation that would group them as one segment. Two widely known algorithms in line with model fitting methods are RANSAC (introduced by ([34])) and the Hough transform introduced by ([35]) [18]. By the way, the suggested algorithm in this study has been included model fitting segmentation methods.
Fig. 2. Point cloud data structure in 3D coordinate system.
Fig. 3. Distance of a point to a plane.
A new data integration model is suggested for the operating processes in architectural documentation and cultural heritage studies, with the terrestrial laser scanning that uses new data production methods of photogrammetry [36]. When coordinate information for terrestrial laser scanner point cloud data is documented, it is seen that there is no regular order and classification for the data [36]. Points with known three dimensional coordinates, which are used in image rectification, should be picked out from all the TLS data. In order to classify point location data of surfaces with different depths, raw data obtained with scanner is filtered and data is classified with the help of a mathematical algorithm. By the help of filtering function, it is aimed to automatically produce the points to be used in photogrammetric image rectification without any operation on point cloud data forming TLS data which requires the presence of any operator. Sample point cloud data is classified in Matlab environment by the algorithm of filtering function which has been developed for this purpose. In the second part of the study, the algorithm is described. Third part describes sample application; fourth part focuses on the results and discussion. The fifth part is the conclusion part.
Fig. 4. Matrix form of segmentation algorithm in Matlab software.
26
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
2. Algorithm
Fig. 5. Four corners of the surfaces obtained by algorithm.
Algorithm of this study which aims at filtering laser point cloud data of parallel surfaces in indoor areas by the help of filtering function is shown in Fig. 1. The first step of filtering function algorithm is primarily choosing one of the parallel indoor surfaces as the reference plane by an operator. The point cloud data that is shown in Fig. 2 is defined by distinct surfaces. Indoor areas are usually defined by plane surfaces such as walls. Fig. 2 shows the point cloud data and different surface construction in three dimensional coordinate system. Mathematical function that represents plane surface is given in Eq. (1) [37]. Ax þ By þCz þD ¼ 0
Fig. 6. Application site.
ð1Þ
As shown in Eq. (1), surface function has four parameters, namely A, B, C, and D. Calculation of these four parameters is essential to mathematically express the chosen reference plane surface. For the sake of determining parameters of the chosen reference plane surface, point coordinates which are chosen should be read and entered manually by an operator from the reference plane. A plane is mathematically defined in four parameters. Thus, the only solution is to define the parameters of reference plane with four points. For determining the reference plane parameters in the adjustment computation, choosing more than four points which belong to same reference plane by an operator over the plane is made possible. The second value that
Fig. 7. Application data. (Four indoor plane surfaces with different depths).
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
operator will enter manually is the threshold value. Threshold value is the minimum difference of depth that the filter will be operated. Apart from these two stages in algorithm, all the operations are performed automatically by software developed in Matlab. The second step of algorithm is to calculate parameters of the chosen reference plane. After five or more points are chosen manually by an operator in the first step of algorithm, four parameters that represent reference plane are determined in the adjustment computation automatically by interface which is developed in Matlab. So, adjusted reference plane is generated in this step of algorithm. The third step of algorithm is to calculate the distance of all points within laser point cloud to reference plane with parameters determined in the adjustment computation. Fig. 3 shows the distance of a point to a plane. With the given in Eq. (2) [37], distances of all points within laser point cloud to the adjusted reference plane is calculated. This equation is the filtering
27
function chosen for segmentation. h¼
jAx1 þ By1 þCz1 þ Dj pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi A2 þB2 þ C 2
ð2Þ
Where h is the distance of a laser point to the reference plane, A, B, C, and D are the parameters of the adjusted reference plane and x1, y1, z1 are the three dimensional coordinate of each laser point. These distances are added to segmentation matrix S as a column in vector form. Distances of points into reference plane in segmentation filtering are calculated separately for each point. Matlab operation environment is defined in segmentation matrix form. It is proper to call this as “segmentation matrix”. However, calculations are made with column algorithm and final column vector is used for classification of points into surfaces. According to that, first column in segmentation matrix shown in Fig. 4 is X value, the second column is Y value, the third column is Z value of points in terrestrial laser scanning point cloud data. The fourth column is the distance of points into reference plane, the fifth
Fig. 8. Application data in Matlab software.
28
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
column shows statistical differences of the distances, the sixth column shows exponential values of those differences, and the seventh column illustrates which surface the point belongs to. Fourth step of algorithm is segmentation (classification) part. It is used for statistical change of the vector created for this study. It is the detection of how many different surfaces are in the vector, depending on the amount of statistical deviation. Statistical analysis is made based on standard deviation values. The comparison value for standard deviation is the threshold value. So, the amount of statistical deviation is determined with reference to the searched minimum difference of depth. Thus, points with the less minimum difference of depths are considered on the same surface. In statistical analysis (5th Column) the number of different surfaces in total are detected, and all the distances are shifted into a positive value with exponential function (6th Column). So, raw data is obtained for classification. Exponential of the found value is taken since there are some conditions in which the distance of point to plane is negative and that the point remains in the other side of the plane. Thus the points between two surfaces are assigned to the surface with the smaller absolute value. All points within laser scanning data are assigned to some surface based on this value and to the matrix that belongs to the surface (7th Column). The last step of segmentation algorithm is the stage that defines four limit points of surface coming from laser scanning points which is assigned to matrix of the surface. Minimum and maximum x, y plane coordinate values of the points within surface matrix are used for this purpose. The edge points in each segment of segmented original laser point cloud data are detected according to minX–minY, minX–maxY, maxY–minX, maxX–maxY values. Thus, both the limit points of surface and x, y plane coordinates are determined. However, original values are not taken as the height value for the points with defined plane coordinates in the algorithm. If we created a surface with the Z value obtained from laser scanning data of four points which creates the surface, then every surface would not be parallel to each other. So, when assigning height value to four edge points, average of the Z (height) value for all the points which are assigned for that surface as the result of segmentation, is assigned as the height value. Thus, Z value of four edge points that are segmented for each surface is the mean Z value and it is the same value for each of the four points. This is shown in Fig. 5.
As the result of segmentation, each of the surfaces with thousands of points is converted into planes which include only four points and which take the average of heights of all the points within a segment.
3. Application Laser point cloud of a class, scanned with Leica HDS 3000 on GYTE (shown in Fig. 6) in order to test filtering algorithm. Scanning frequency is 5 mm. While obtaining the determined data in txt format from CYCLONE (Leice) software, affine transformation is performed in a way to give Z data depth for point cloud data. Thus, determining the reference surface will be easier for operator. An original data set containing four different surfaces and 21,932 points was selected. There are various differences in depth ranging from 1 cm to 20 cm between the surfaces of doors, borders, walls and columns. In the indoor space shown in Fig. 7, surface 1 is accepted as reference plane in order to test the filtering function from four plane surfaces which are parallel to each other. In Fig. 7, Surface 1 is the backmost surface and surface 4 is the foremost surface. Five points on reference surface are chosen by an operator in Cyclone screen for the calculation of plane equation parameters. Threshold value for test data (minimum difference of depth) is determined as 1 cm. When the software developed in Matlab is started, 21,932 three dimensional laser point clouds are classified into four separate surfaces. This is shown in Fig. 8. Fig. 9 illustrates the graphics of exponential values. In the first segment of the original 21,932 laser scanning data, divided into four different classes, there are 3906 points, in its second segment 6588 points, in its third segment 1951 points, in its four segment 9487 points (with 5 points taken from surface1). Fig. 10 shows the surfaces passing through each of the laser point cloud data, which are divided into four classes. As shown in Fig. 10, distance threshold value between second and third plane surfaces is around 1 cm and the filter satisfyingly segmented these two different surfaces. All points on laser point cloud are assigned into a plane surface and they are recorded into their appropriate surface matrix. X, y plane coordinate values of four edge points of plane surface are
Fig. 9. Segmantation graphic.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
found by the help of points within the matrix of each surface. Then, only one Z value is assigned to four points for this plane surface.
4. Results and discussion Here, the crucial point is that there is no limitation in choosing the surface which is defined as reference plane by operator. In the suggested algorithm, reference plane can be chosen by operator as any surface. Operator may determine the closest or the farthest plane surface or any surface between these surfaces as the reference surface. This is the reason of using exponential values during creating segmentation matrix which is defined as matrix S. Distances of the points to the reference plane might be negative or positive and this does not effect segmentation and classification results since exponential values are used. So, negative or positive distances to the reference plane (showing which side they stay
29
according to the plane and not affecting the result) do not cause any error in segmentation. Thus, when reference plane is determined as the foremost surface (surface 4) and five points are chosen from that surface, there are 3906 points in the first segment, 6542 in the second segment, 1997 in the third segment, 9487 in the fourth segment (with 5 points taken from surface 4). So, first and fourth surfaces with surface points which are out of the algorithm threshold value give 100% accuracy. And, 8542 points on two parallel surfaces with a distance which is similar to the threshold value give approx. 99% accuracy. Table 1 shows segmented amount of points based on the information that both of the surfaces are reference planes. In this situation, one might question whether faulty edge points are determined on the surfaces close to the threshold values during determination of limit values of the surface; but in fact, this question is irrelevant because x, y plane coordinates of the surface are determined from the matrix points that are assigned to the surface.
Fig. 10. The result of segmentation (Four segmentated surfaces which are different depths).
30
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31
Table 1 Segmentation according to two different reference planes. Reference surface
Amount of points
Total amount of points
1. Surface 2. Surface 3. Surface 4. Surface Surface 1 (backmost) 3906 Surface 4 (foremost) 3906 Difference in point 0 amounts
6588 6542 46
1951 1997 46
9487 9487 0
21932 21932 92
Images of four surfaces with different depths which are formed of four corner points obtained by segmentation, taken by terrestrial laser scanner cameras or digital cameras, are textured over those surfaces and this provides simplicity and convenience, compared to other multi-point surfaces. Thus, by the help of planes created in this way, point cloud data is much more easily processed by both the hardware and the operator. This segmentation study provides a data structure which is much more appropriate for texture mapping than point cloud data.
5. Conclusion Considering the core software as a result of the research, a general advantage–disadvantage list is shown below for the functional model. 5.1. Advantages of suggested algorithm Any of the surfaces with difference of depth can be chosen as the reference surface by the operator. In determination of the reference surface, there is no limitation such as choosing the nearest or farthest one for operator. The original aspect of the study is that four corner points within original laser data for each surface are determined with the average of Z values of all the points that are segmented for that surface, for the sake of creating a texture map. So, the walls are made surfaces which are really parallel to each other. If a surface which passes through all the points in segmentation in original laser scanning data segmented into four separate surfaces, this surface obtained by the difference of Z (height values) will not be the plane surface. By the help of filtering function, the points to be used in photogrammetric image rectification are produced automatically, without any operation on point cloud data forming TLS data which requires the presence of any operator. 5.2. Disadvantages of suggested algorithm Difference of depth is determined as the threshold value. If, for example, the difference of depth is determined as 5 cm, then differences of depth less than 5 cm will be considered as one single surface by the filtering function, instead of a separate surface. This is a disadvantage. Thus, rectification points that are most appropriate for parallel surfaces are detected for the surface texture model, by simplifying with a geometric object based filtering function. This geometry is formed by surfaces which have different depths and are parallel to each other. These kind of surfaces are frequently seen in indoor spaces for terrestrial laser scanning with a dense point cloud data. Then, in the integration study for obtaining a true scale surface texture model, these points are detected from laser scanner data. Also, image coordinates of the points are detected by reducing with the help of “projection over image” equations.
With the study that will be made after this point, data obtained by camera equipment in indoor spaces and terrestrial laser scanner data will be integrated and texture models that can be defined only with three dimensionally located and scaled raster data will be obtained. Models created with this integration are much smaller than the laser scanning data in terms of file size. Thus, file sizes decrease in modeling. So, depending on the power of hardware, memory and hard disk problems that we encounter during editing three dimensional model files with software will be resolved. Additionally, since converting them into VRML format is made by the help of texturing, these kind of models will be more preferred in virtual tour studies. Terrestrial laser scanning data which gives three dimensional information directly in terms of product but does not have a high radiometric resolution and photographic image which is necessary for surface texture will complement each other. Combination of these two data source will increase speed and reliability of documentation. References [1] Abdelhafiz A. (Ph.D. Thesis). Integrating digital photogrammetry and terretrial laser scanning. TU, Braunsvhweig, Germany: Institute of Geodesy and Photogrammetry; 2009. [2] Kadobayashi R, Kochi N, Otani H, Furukawa R. Comparision and evaluation of laser scanning and photogrammetry and their combined use for digital recording of cultural heritage. In: Proceedings of the XXth ISPRS congress, Istanbul, Turkey; 12–23 July 2004. p. 401–6. [3] Gonzo L, Voltolini F, Girardi S, Rizzi A, Remondino F, El-Hakim SF. Multiple techniques approach to the 3D virtual reconstructions of cultural heitage. In: Proceedings of the 5th eurographics italian chapter conference, Trento: Italy; 14–16 February 2007. [4] Ortiz P, ve Sanchez H. Virtual city models combining cartography and photorealistic texture mapped range data. In: Proceedings of the XXIIth international cartography conferance, Coruna: Spain; 9–16 July 2005. [5] Fuchs A, Alby E, Begriche R, Grussenmeyer P, Perrin JP. Confrontation du Relevé Laser 3D Aux Techniques de Relevé Conventionnelles et Développement D'outils Numériques Pour la Restitution Architecturale. Revue de la Société Française de Photogrammétrie et de Télédétection n1 173/174 (2004-1/ 2), 2004. p. 36–47. [6] Gonzo L, El-Hakim SF, Picard M, Girardi S, Whiting E. Photo-realistic 3-D reconstruction of castles with multiple-sources image-based techniques. In: Proceedings of the international archives of the photogrammetry, remote sensing and spatial information sciences, vol. 35 (B5); 2004. p. 120–5. [7] Grussenmeyer P, Landes T, Voegtle T, Ringle K. Comparision methods of terrestrial laser scanning photogrammetry and tacheometry data For recording of cultural heritage buildings. In: Proceedings of the international archives of the photogrammetry, remote sensing and spatial information science, XXXVII (B5), Beijing, China; 3–11 July, 2008. [8] Georgantas A, Bredif M, Desseilligny MP. An accuracy assessment of automated photogrammetric techniques for 3D modeling of complex interiors. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIXB3, Melbourne, Australia; 25 August–01 September 2012, 2012. [9] Bornaz L, Rinaudo F. Terrestrial aser scanner data processing. In: Turkey V, editor. Proceedings of the XXth ISPRS congress technical commission Orhan Altan ISPRS Archives – vol. XXXV Part B5, Istanbul: Turkey; 12–23 July 2004. p. 514–9. [10] Kersten TP, Stallmann D. Automatic texture mapping of architectural and archaeological 3D models. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIX-B5, , Melbourne, Australia; 25 August–01 September 2012. [11] Grussenmeyer P, Hanke K. Cultural heritage applications. Airborne and terrestrial laser scanning. Whittles Publishing; 2010; 271–90. [12] Grussenmeyer P, Alby E, Landes T, Koehl M, Guillemin S, Hullo JF. et al. Recording approach of heritage sites based on merging point clouds from high resolution photogrammetry and terrestrial laser scanning. In: Proceedings of the XXII ISPRS congress on international archives of the photogrammetry, international archives of the photogrammetry, remote sensing and spatial information sciences, XXXIX-B5, Melbourne, Australia; 25 August–01 September; 2012. [13] Alshawabkeh Y, Haala N. Integration of digital photogrammetry and laser scanning for heritage documentation. In: Proceedings of the XXth international society for photogrammetry and remote sensing congress, Comission V, Istanbul, Turkey, 12–23 July; 2004. p. 424–9. [14] Boehler W, Marbs A. 3D scanning and photogrammetry for heritage recording: a comparison. In: Proceedings of the 12th international conference on geoinformatics, University of Gavle, Sweden; 2004. p. 291–8.
C. Sahin / Optics and Lasers in Engineering 64 (2015) 23–31 [15] Murphy M, McGovern E, Pavia S. Historic building information modelling – adding intelligence to laser and image based surveys of European classical architecture. ISPRS J Photogramm Remote Sens 2013;76:89–102. [16] Lerma JL, Biosca JM. Segmentation and ifltering of laser scanner data for cultural heritage. In: Proceedings of the CIPA XX international symposium. Torino, Italy; 26 September–01 October; 2005. [17] Susaki J. Modeling of building from terrestrial laser scanning considering popology for occlusion. In: Proceedings of the 22nd CIPA symposium. Kyoto, Japan; October, 11–15; 2009. [18] Awwad TM, Zhu Q, Du z, Zhang Y. An improved segmentation approach for planar surfaces from unstructed 3D point clouds. Photogramm Rec 2010;25 (129):5–23. [19] Rodriguez-Gonzalvez P, Gonzalez-Aguilera D, Gomez-Lahoz J. From point cloud to surface: modeling structures in laser scanner point clouds. In: Proceedings of International archives of the photogrammetry, remote sensing and spatial information sciences, Workshop on Laser Scanning and Silvilaser 2007, Espoo, Finland; vol. 36(3/W52), September 12–14; 2007. p. 338–43. [20] Tovari D, Pfeifer N. Segmentation based robust interpolation—a new approach to laser data filtering. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Laser Scanning 2005, Enschede, The Netherlands; vol. 36(3/W19), September 12–14; 2005. p. 79–84. [21] Hoover A, Jean-Baptiste G, Jiang Xiaoyi, Flynn PJ, Bunke H, Goldgof D. et al. Range image segmentation: the user's dilemma. In: Proceedings of IEEE international symposium on computer vision, Florida, USA; 1995. p. 323–8. [22] Hofmann AD, Maas HG, Streilein A. Knowledge-based building detection based on laser scanner data and topographic map information. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Graz, Austria; vol. 34(3/A); 2002. p. 169–74. [23] Dold C, Brenner C. Automatic matching of terrestrial scan data as a basis for the generation of detailed 3D city models. International archives of photogrammetry, remote sensing and spatial information sciences, Istanbul, Turkey; vol. 35(B3); 2004. p. 1091–6. [24] Pu S, Vosselman G. Automatic extraction of building features from terrestrial laser scanning. International archives of the photogrammetry, remote sensing and spatial information sciences, 25-27 September; vol. 36(5); 2006. p. 5. [25] Filin S. Surface clustering from airborne laser scanning data. International Archives of photogrammetry, remote sensing and spatial information sciences. vol. 34(3/A); 2002. p. 119–124.
31
[26] Hofmann AD. Analysis of TIN-structure parameter spaces in airborne laser scanner data for 3-D building model generation. International archives of photogrammetry, remote sensing and spatial information sciences. vol. 35 (B3); 2004. p. 302–307. [27] Filin S, Pfeifer N. Segmentation of airborne laser scanning data using a slope adaptive neighborhood. ISPRS J Photogramm Remote Sens 2006;60(2):71–80. [28] Biosca JM, Lerma JL. Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods. ISPRS J Photogramm Remote Sens 2008;63(1):84–98. [29] Bauer J, Karner K, Schindler K, Klaus A, Zach C. Segmentation of building models from dense 3D point-clouds. In: Proceedings of 27th workshop of the austrian association for pattern recognition, Laxenburg; 2003. p. 253–8. [30] Bretar F, Roux M. Extraction of 3D planar primitives from raw airborne laser data: a normal driven RANSAC approach. In: Proceedings of IAPR conference on machine vision applications, Tsukuba Science City: Japan; 2005. p. 452–5. [31] Boulaassal H, Landes T, Grussenmeyer P, Tarsha-Kurdi F. Automatic segmentation of building facades using terrestrial laser data. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Workshop on Laser Scanning 2007 and SilviLaser 2007, Espoo, Finland; vol. 36(3/W52); 2007. p. 65–70. [32] Boulaassal H, Landes T, Grussenmeyer P. Automatic extraction of planar clusters and their contours on building fac¸ades recorded by terrestrial laser scanner. Int J Archit Comput 2009;7(1):1–20. [33] Vosselman G, Gorte BGH, Sithole G, Rabbani T. Recognising structure in laser scanner point clouds. International archives of photogrammetry, remote sensing and spatial information sciences. vol. 36(8/W2); 2004. p. 33–38. [34] Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981;24(6):381–95. [35] Hough PVC. Method and means for recognizing complex patterns. U.S. Patent 3,069,654. 1962. [36] Ergun B. A novel 3D geometric object filtering function for application in indoor area with terrestrial laser scanning data. Opt Laser Technol 2010;42:799–804. [37] Balci M. General mathematics, vol. 2. Balci Publishing; 2000.