Advanced Engineering Informatics 34 (2017) 114–124
Contents lists available at ScienceDirect
Advanced Engineering Informatics journal homepage: www.elsevier.com/locate/aei
Semantic as-built 3D modeling of structural elements of buildings based on local concavity and convexity Hyojoo Son, Changwan Kim
MARK
⁎
Department of Architectural Engineering, Chung-Aug University, Seoul 06974, Republic of Korea
A R T I C L E I N F O
A B S T R A C T
Keywords: As-built BIM Laser-scan data Local convexity Scan-to-BIM Structural works
The aim of this study is to propose a method for generating as-built BIMs from laser-scan data obtained during the construction phase, particularly during ongoing structural works. The proposed method consists of three steps: region-of-interest detection to distinguish the 3D points that are part of the structural elements to be modeled, scene segmentation to partition the 3D points into meaningful parts comprising different types of elements (e.g., floors, columns, walls, girders, beams, and slabs) using local concave and convex properties between structural elements, and volumetric representation. The proposed method was tested in field experiments by acquiring and processing laser-scan data from construction sites. The performance of the proposed method was evaluated by quantitatively measuring how accurately each of the structural elements was recognized as its functional semantics. Overall, 139 elements of the 141 structural elements (99%) in the two construction sites combined were recognized and modeled according to their actual functional semantics. As the experimental results imply, the proposed method can be used for as-built BIMs without any prior information from as-planned models.
1. Introduction Interest has been growing regarding how best to model and represent building facilities in as-is conditions. For project management purposes, the composing building elements need to be represented in volume-enclosing and solid shapes that have three-dimensional (3D) forms at the individual object level; they also need to have their own information regarding functional semantics in as-built building information models (or BIMs) [1–7]. In BIMs, building elements consist of structural and nonstructural elements. Structural elements comprise floors, columns, walls, girders, beams, slabs, and foundations; nonstructural elements include architectural elements (e.g., freestanding walls, canopies, interior partitions, stairways, ceilings, and exterior walls), mechanical, electrical, and plumbing (MEP) elements (e.g., electrical equipment, piping, ductwork, and mechanical equipment); and other finishes (e.g., windows and doors). All building elements are networked with neighboring elements to perform their respective functions. For example, two or more columns can be connected with a girder for support. In BIMs, the 3D shapes of such elements, their functional semantics, and the connectivity relations between them need to be described. Accordingly, when generating as-built BIMs, it is necessary to recognize each of the semantic elements to be modeled and to identify the connectivity relations among the elements [8].
⁎
Corresponding author. E-mail address:
[email protected] (C. Kim).
http://dx.doi.org/10.1016/j.aei.2017.10.001 Received 21 May 2017; Received in revised form 29 August 2017; Accepted 3 October 2017 1474-0346/ © 2017 Elsevier Ltd. All rights reserved.
Additionally, a building is constructed by following a plan that includes all work activities during the construction phase. As that phase progresses, most of the elements (particularly the structural elements) are covered by other objects, such as architectural elements and finishes, which are installed late in the process. For this reason, it is necessary to generate and update as-built BIMs during or after each major construction activity (e.g., the structural, MEP, architectural, and finish steps) to ensure that the BIMs include all of the composing building elements. Although designers first create the 3D models of building facilities in an office environment, discrepancies (in terms of, for example, dimension, location, orientation, and shape) certainly exist between asbuilt and as-planned models [9–11]. Such discrepancies can become wider as the construction phase progresses due to manual layouts and on-site guesswork. Thus, as-planned BIMs can become inaccurate or even obsolete if considerable revisions are not made to reflect the construction process. In spite of the existence of early design in the form of 3D models, such a revision process—which should incorporate the design changes and construction errors that take place during the construction phase—is intricate. To close this gap, it is necessary to generate as-built BIMs based on real-world information at every stage during construction [12,13]. Nowadays, a highly accurate laser-scanning survey can be utilized
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
phase results in an accurate assessment of the as-built status of the structural elements. These methods mainly rely on point-recognition metrics that identify correspondences between the as-built data and either the asplanned model or a 3D CAD database to recognize individual elements and retrieve 3D elements from the as-planned model or a 3D CAD database. For this reason, these methods can cause recognition errors when discrepancies—in terms of dimension, location, orientation, or shape—are observed between as-built conditions and as-planned models. Although a recent study by Bosché et al. [9,10] tackled this issue by integrating the scan-versus-BIM and scan-to-BIM approaches in the as-built modeling of MEP work, this integrated method also primarily depends on the as-planned model. Other researchers (e.g., [3,20–22,4,23–29]) have recognized that, the exposed parts of structural and architectural elements can be recognized and/or modeled without an as-planned model or a 3D CAD database. This can be accomplished through the extraction of planar patches, the segmentation and classification of patches, and the creation of a model that includes all the building elements that have planar surfaces. Pu and Vosselman [3] proposed a method to generate building-façade models from 3D point cloud. Façade elements (e.g., walls and roofs) are distinguished as features. To recognize these features from segmented 3D point cloud, information about the features’ sizes, positions, orientations, and topologies is used. Then, a polygon is created for each feature—using least-squares fitting, convex-hull fitting, or concave-polygon fitting, according to the size of each feature. Information from the created polygons is used to hypothesize about the occluded parts. Finally, a building-façade model is generated from the polygons and the hypothesized parts. Truong-Hong et al. [21,22] proposed an approach to generate 3D façade models from 3D point cloud. Their approach is capable of detecting the building façade’s boundary points, features, and determining the openings. For this purpose, the flying-voxel method is used to determine whether each point belongs to the building’s façade. The approach is then used to generate surface models of the building façade using an octree representation. Xiong et al. [4] proposed an approach to generate 3D indoor building models from 3D point cloud. Their method is capable of identifying and modeling the main structural elements of an indoor environment (e.g., walls, floors, ceilings, windows, and doorways), even in the presence of clutter and occlusion. Their proposed approach begins with the extraction of planar patches from a voxelized 3D point cloud; the features of various types of surfaces and the contextual relationships between them are then identified, and each patch is labeled as one of various types of main structural elements. After that, openings such as windows and doorways are located via visibility reasoning. Next, a learning algorithm is used to estimate the shape of these window and doorway openings, and occluded surface regions are filled in using a 3D inpainting algorithm. As the basis of these methods is the process of planar patch extraction, each of the elements—such as walls, doors, and roofs in an outdoor environment or walls, floors, ceilings, windows, and doorways in an indoor environment—is represented in the form of a planar (that is, non-volumetric) surface. To be more specific, in previous methods (e.g., [29]), the objects are not in the form of 3D, solid, volume-enclosing shapes; instead, each element is represented by only a single surface or by a few separate surfaces. For example, these methods represent a wall as a planar surface, but the wall would best be represented as a single, volumetric object with multiple surfaces and adjacency relationships between it and other elements in its BIM [32]. Thus, these methods require additional processes so that nonplanar volumetric elements or those that consist of several planes—such as columns, walls, girders, and beams—can be recognized [2,30,31]. For example, a column can be recognized by using additional processes to identify the various planes that comprise the column; it is then necessary to combine them into a unified entity. Some researchers (e.g., Xiao and Furukawa [33]; Stambler and Huber [34]) have proposed a method
to capture real-world information for generating as-built BIMs of building facilities that are under construction. Performing a laserscanning survey during major construction activities (whether ongoing or completed) allows us to generate and update as-built 3D BIMs based on highly accurate, as-built point clouds. To generate as-built BIMs from such accurate and dense point clouds, it is necessary to recognize all the semantic building elements and to separate them from one another. Few studies have attempted to solve this problem. Previous studies can be largely divided into those that considered an as-planned model during the recognition process and those that did not. Some researchers (e.g., [14–19]) have attempted to model structural elements by aligning as-built data with computer-aided design (CAD) models, recognizing individual elements, and then retrieving 3D elements from as-planned models or a 3D CAD database. However, the as-planned model does not always depict the same elements that are present at the scene during construction; thus, the recognition of individual elements ends up being incomplete. Such discrepancies between the as-built and as-planned conditions may be due to (human) construction errors or changes made in the field regarding constructability issues. Other studies (e.g., [3,20–22,4,23–29]) have recognized that, during the occupancy phase, the exposed parts of the structural and architectural elements can be recognized and/or modeled without an as-planned model or a 3D CAD database. However, as the basis of these methods is planar patch extraction, these methods require additional processes to identify the nonplanar volumetric elements or those that consist of several planes, including columns, walls, girders, and beams [2,30,31]. In summary, to our knowledge, there are limited methods available to generate an as-built 3D BIM during the construction phase without an as-planned model. The ultimate goal of this study is to develop a method for generating and updating as-built BIMs during or after each major construction activity, including structural, MEP, architectural, and finish work; this method could serve the purposes of project management. To realize this goal, this study proposes a method for generating as-built BIMs from laser-scan data obtained during the construction phase; this method would be especially useful for structural work. 2. Related works Several research studies have been conducted to attempt to automate the generation of 3D as-built models of structural elements of buildings. Some studies (e.g., [14–19]) have proposed methods to recognize and model structural elements from as-built data by utilizing as-planned models or a 3D CAD database. In the methods proposed by Bosché et al. [14], Bosché [15], and Turkan et al. [19], 3D point clouds are registered with an as-planned model in the same coordinate system with an iterative closest point. Once the 3D CAD models of structural elements and 3D point clouds are registered, the as-built objects are recognized. Then, the structural elements are modeled by retrieving matched 3D CAD models. Son and Kim [16] proposed methods to recognize and model structural elements from as-built data by utilizing a 3D CAD database. In their method, the data processing is initiated by extracting 3D point cloud that corresponds to structural elements from the as-built data based on those elements’ color features. They also modeled the as-built objects by aligning the as-built data with the CAD model, recognizing individual elements, and retrieving 3D elements from the as-planned model or from a 3D CAD database. Kim et al. [18] proposed a method to recognize and model structural elements from as-built data by utilizing a four-dimensional (4D) BIM. Once the as-built data is aligned with the as-planned model and matched to the information in the BIM, the construction sequence—defined as the sequence-of-activity execution specified in the BIM—is examined to help identify the inaccurate aspects of the as-built status. Then, the topological relationships among the structural elements—defined as the connectivity between elements, as specified in the BIM—are examined. The as-built status-revision 115
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
main steps of the proposed method. The details of the proposed method are as follows.
for modeling walls as solids consisting of volumetric primitives. Xiao and Furukawa [33] proposed a 3D inverse constructive solid geometry algorithm to generate 3D wall models made up of primitives by merging or discarding the candidate 3D primitives that were generated from each 2D primitive. Stambler and Huber [34] proposed an algorithm to segment surfaces, classify them, and optimize them using 3D wall models by introducing room-, floor-, and building-level reasoning based on the assumption that a complete building is presented in each registered 3D point cloud. Although they also aimed at reconstructing structural elements’ volumetric representations from registered 3D point cloud, these methods are applicable only to the reconstruction of thin structures such as walls [24,26,28,33,34]. In summary, to our knowledge, few limited methods are available to generate as-built 3D models of structural elements of buildings during the construction phase without an as-planned model.
3.1. Region-of-Interest (ROI) detection The laser-scan data acquired during the construction phase comprise 3D (x, y, z) point coordinates and the color values (R, G, B) of the corresponding 3D points. To distinguish between the 3D points that are part of the building elements to be modeled, structural elements in this case, the ensemble classifier that Son et al. [36] proposed was employed. This method uses a color value as a key feature to distinguish the ROI from laser-scan data. The color values in the red, green, and blue (RGB) color space are converted to the hue, saturation, and intensity (HSI) color space to reduce the effect of illumination variations on material colors in outdoor environments. The ensemble classifier uses a voting scheme that internally combines base classifiers (support vector machine, C4.5, and k-nearest neighbors) to produce better results. The ensemble classifier is applied to classify the 3D points corresponding to the structural elements of construction materials, such as concrete, steel, or wood. This study employed the ensemble classifier for the concrete to distinguish 3D points corresponding to concrete structures from the registered 3D point cloud. Through this process, not only the geometry and connectivity relations of the building elements but also the attribute (here, information about the materials) can also be modeled [8]. Details on the classification of construction materials using an ensemble classifier can be found in Son et al. [36].
3. Methodology The laser-scan data acquired during the construction phase contain the building elements to be modeled as well as other objects, such as stacked materials. For this reason, the proposed method begins with a region-of-interest (ROI) detection process to distinguish the 3D points that are part of the structural elements to be modeled. Here, the ROI refers to the 3D points in the registered 3D point cloud that are part of the objects to be modeled. The ROI detection process consists of two steps: color space conversion and the classification of construction materials. Then, scene segmentation is followed; this is a process in which the laser-scan data acquired during the construction phase are partitioned into meaningful parts that consist of different types of building elements (e.g., floors, columns, walls, girders, beams, and slabs). The scene segmentation process consists of two steps: segmentation according to local convexity and segmentation refinement. The method proposed in this study is an extension of the method used in our previous study [35], which enabled the segmentation of the structural elements with respect to their own functional semantics. The next step of the proposed method is volumetric representation; this consists of shape classification and 3D parameter calculation. Fig. 1 illustrates the
3.2. Scene segmentation The basic principle of the scene segmentation step is that most objects are composed of connected and enclosed convex surfaces; these objects are separated from other objects by concave boundaries [37]. From this point onward, each structural element can be recognized by merging the areas that have convex properties enclosed by concave boundaries. Two adjacent surfaces of different objects have a concave property, as shown in red in Fig. 2(a). On the other hand, concave and convex properties can be simply differentiated as follows. Such concave and convex properties have been proven to be unique and powerful features for scene segmentation from 3D point clouds [38,37]. The scene is approximated using an adjacency graph, which utilizes spatially connected surface patches. In the adjacency graph, the concave and convex properties of the edges are computed based on local criteria (see Section 3.2.1.1). Then, locally connected subgraphs with convex properties are identified through the region-growing process to represent the building elements (see Section 3.2.1.2). Finally, the results of the segmentation by local convexity are refined by discarding small segments, further segmenting the under-segmented parts and merging the over-segmented parts (see Section 3.2.2).
As-Built Data
Color Space Conversion Classification of Construction Materials 3.1. ROI Detection
3.2.1. Segmentation by local convexity 3.2.1.1. Supervoxel representation. After the ROI detection is completed, 3D coordinates (x, y, z) are used to segment the resulting data into semantic parts that consist of different types of building elements. For the first step, a graph of connected linear patches is generated. This step is based on the supervoxel algorithm proposed by Papon et al. [39]. The supervoxel algorithm is designed for edge-preserving segmentation in which each of the supervoxels in an adjacency graph is regarded as a linear patch and their normal vectors are computed. Here, a normal vector is defined as one that faces outward following the right-hand rule, which means that the vertices comprising each face are arranged counter-clockwise. This approach specifies that the normal vector is unique. The supervoxel algorithm has the advantage of efficiently processing the huge 3D point cloud by achieving substantial data reduction [37]. When employing this algorithm, the adjacency graph is built from the results of the ROI detection (i.e., the neighboring voxels are those
Segmentation by Local Convexity
Segmentation Refinement 3.2. Scene Segmentation Shape Classification
3D Parameter Calculation 3.3. Volumetric Representation As-Built BIM Fig. 1. Proposed method for generating as-built BIM.
116
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
Fig. 2. (a) Concave areas (in red); (b) convex areas (in blue). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
compensate for errors by classifying those connections,
that share a face, edge, or vertex). This is computed within the octree structure by searching and determining the neighboring leaves in the voxel grid. Using this representation, we can reduce the computational complexity by processing at a supervoxel level rather than a point level, thereby speeding up the process of handling the huge amount of laserscan data. The details of the supervoxel algorithm can be found in Papon et al. [39], Stein et al. [37], and Stein et al. [40]. Once a graph of connected linear patches is built from the ROI detection result, a convexity graph is created by classifying the edges of the graph of the connected linear patches. To determine whether a connection between patches has convex or concave properties, the two criteria that Stein et al. [37] proposed are employed: convexity and ⎯→ ⎯ ⎯→ ⎯ → sanity. Given two adjacent supervoxels p1 and p2 with centroids x1 and → ⎯→ ⎯ ⎯→ ⎯ x2 and normal vectors n1 and n2 , the convexity between two adjacent supervoxels can be defined using the relationship of the surface normals → → → of the vector (d = x1− x2 ) joining their centroids. The angle of the vector’s normals can be calculated by obtaining the dot product be⎯→ ⎯ ⎯→ ⎯ ⎯→ ⎯ ⎯→ ⎯ tween α1 and α2 . For concave relations (Fig. 3(a)), α1 is larger than α2 and can be addressed as follows:
⎯ ·d − ⎯ ·d ̂ < 0, ̂ ⎯→ α1 > α2 ⇔ ⎯→ n n 1 2
⎯ , ⎯→ ⎯ ⎯ ⎯→ ⎯ −1 ⎯→ β = ∠ ( ⎯→ n 1 n2 ) = |α1−α2 | = cos ( n1 · n2 ) < βThresh,
as convex when the adjacent patches have a normal difference less than the specified angle threshold βThresh . By setting the threshold, the convexity criterion is defined as follows:
⎯→ ⎯ ⎯→ ⎯ ̂ CC (→ pi ,→ pj ) ≔ ⎧ true ( n1 − n2 )·d > −∨ (β < βThresh ) ⎨ false otherwise. ⎩
(4)
In addition to the convexity criterion, the sanity is defined to identify connections for which adjacent patches are connected at only a single point. For this purpose, the relationship of the surface normals of → → → the vector (d = x1− x2 ) joining their centroids and the direction of the → → → intersection ( s = x1 × x2 ) of the planes are considered by approximating the supervoxel’s surface. Singular configurations can be de→ → termined by observing the angle θ between d and s . If the directions of two patches are parallel (θ = 0°), they are only joined at a single point, which indicates that a singular configuration is reached. Otherwise, if the two adjacent patches share one side of the boundary, the directions of two patches are orthogonal (θ = 90°). Here, the angle θ is defined to be the minimum between the two directions, since the orientation of → s can be arbitrary:
(1)
⎯→ ⎯ ⎯→ ⎯ where . Likewise, for convex relations (Fig. 3(b)), α1 is smaller than α2 and can be addressed as follows:
⎯ ·d − ⎯ ·d ̂ > 0. ̂ ⎯→ α1 < α2 ⇔ cos(α1)−cos(α2) > 0 ⇔ ⎯→ n n 1 2
(3)
→→ → → →→ →→ ⎯ ,⎯→ ⎯ θ (⎯→ p 1 p2 ) = min(∠ ( d , s ),∠ ( d ,− s )) = min(∠ ( d , s ),180°−∠ ( d , s ))
(2)
(5)
Adjacent patches with small concave properties are ignored to
For the sanity criterion, adjacent patches with similar normals are
Fig. 3. Illustration of how to measure the convexity criterion (a) Concave; (b) convex.
117
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
Fig. 4. (a) Concave areas (in red) and convex areas (in blue); (b) segmentation of convexity graph. (The convex-connected patches are found around the concave edges.) (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
filter size is defined by the number of supervoxels consisting of a segment and the bounding dimension. In the tested cases, the number of these supervoxels varied, and the following values were determined experimentally, with the small segments and segments corresponding to the structural elements being separated. If a segment consists of fewer than 5 supervoxels, the segment is considered to be discarded. Or, if the two values of the length, width, and height of the segment’s bounding box are less than 100 mm, the segment is also considered to be discarded. This filtering step is repeated until no segments smaller than the filter size are present in the 3D point cloud. The proposed method uses three distinctive features—the Lalonde feature [41], each segment’s direction, and connectivity—that yield an accurate classification of the individual segments of the 3D data set into the four semantic classes of structural elements: column, beam/girder, floor/slab/wall, and other. The Lalonde feature [41] is a 3D vector that is appropriate for capturing the linearness, surfaceness, and scatterness of a 3D data set [42,43]. Thus, this feature is useful for distinguishing linear objects (columns and beams/girders), surface objects (floors/ slabs/walls), and scattered items (other). To extract the Lalonde feature of each segment, the eigenvalues of the covariance matrix for that segment are computed. The covariance matrix for a given segment is defined as follows:
also ignored to compensate for errors by classifying those connections using the angle between the normals θ Thresh (β ) and a sigmoid function: max θ Thresh (β ) = θThresh ·(1 + exp[−α·(β−βoff )])−1
(6)
with βoff and α set to 25° and 0.25, respectively, as experimented with by Stein et al. [40,37]. By setting the threshold, the sanity criterion is defined as follows:
⎯ , ⎯→ ⎯ true θ(→ pi −→ pj ) > θ Thresh (β ( ⎯→ n 1 n2 )) SC (→ pi ,→ pj ): =⎧ ⎨ false otherwise. ⎩
(7)
3.2.1.2. Convexity graph segmentation. The created convexity graph is used to find each of the elements connected by convex edges. This process considers the way elements are connected as well as the consequences of the connectivity between patches. Convexity graph segmentation is achieved using a region-growing process. Selecting an ⎯ ,N ) begins the region-growing pi = (→ x i , ⎯→ n arbitrary seed supervoxel → i i → ⎯ , and edges of adjacent process with centroid x i , normal vector ⎯→ n i supervoxels e ∈ Ni , which propagates the seed-patch to those patches that have convex edges until a concave property is reached. Fig. 4(a) shows that the differentiated concave and convex properties. When two adjacent supervoxels have a convex property, the results were illustrated in blue. While two adjacent supervoxels have a concave property, the results were illustrated in red. For the selected seed supervoxel and propagated supervoxels, segment label is assigned. Then, the region-growing process is started again with another seed supervoxel that has not been labeled. This process continues until segment labels have been assigned to all the supervoxels. Through this region-growing process, the convex-connected patches are found around the concave edges. As a result, it leads to an enclosing concave boundary that segments the convexity graph (see Fig. 4(b)).
Covariance Matrix =
1 N
N
∑ i=1
(Xi −X )(Xi −X )T ,
(8)
where N is the number of 3D coordinates in the segment, Xi is the N coordinate vector of point i , and X = (1/ N ) ∑i = 1 Xi . After the covar→ → e3 and the iance matrix is calculated, the eigenvectors of e1, e2 , and → corresponding eigenvalues, λ1, λ2 , and λ3 , where λ1 ⩾ λ2 ⩾ λ3 , are extracted. The components of the Lalonde feature for that segment are L1 = λ1, L2 = λ1−λ2 , and L3 = λ2−λ3 [43,44]. Then, L1, L2 , and L3 are normalized to be scaled from zero to one. The second feature is each segment’s direction. The segments corresponding to columns have directions along the z-axis, whereas those corresponding to beams/girders have directions along the x- and y-axes. Thus, each segment can be classified using its direction into either a column or a beam/girder. The direction of a given segment is computed using the covariance matrix. In the covariance matrix, the eigenvector that has the largest eigenvalue represents the principal axis of the segment, and this axis represents the direction of the given segment. Using the features described above, an support vector machine (SVM) classifier—which is known to be the most reliable means of classifying 3D data nonlinearly—is used to solve the nonlinear classification problem [43–45]. In this study, SVM with a radial basis function (RBF) as a kernel function is employed to solve nonlinear problems by mapping
3.2.2. Segmentation refinement The purpose of this step is to discard small segments that can be found at the edges of each structural element in the resulting segmented 3D point cloud. The purpose is also to further segment the under-segmented parts and to merge the over-segmented parts with adjacent elements once more. This step is done while considering these elements’ connectivity and geometric properties to separate each segment into columns, beams/girders, floors/slabs, and walls. The resulting small segments are caused by normal vector estimation, which is based on the local neighborhood. In the first step of the segmentation refinement, such small segments are discarded, with filters as minimal segments. Here, a simple filter is implemented using the predefined filter size. The 118
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
and minimum bounding circle are calculated, the ratios between the convex hull’s area to the minimum bounding rectangle’s area and the minimum bounding circle’s area are calculated, respectively. If there is only one shape for which the calculated value is over 90% for all three cross-sections, then this shape is indicated to be the expected shape. If the indicated shapes with the highest calculated value are different among the three cross-sections, then the majority shape is indicated to be the expected shape.
the input feature vectors into a high-dimensional feature space and finding a linear separating hyperplane that maximizes the margins between the classes. Each segment is classified according to the category that the trained SVM model predicts. For training and testing, a total of 204 sets of “columns,” 225 sets of “beams/girders,” 142 sets of “floors/slabs/walls,” and 174 sets of “other” classes were collected from different sites and used. As a result, each of the columns is segmented from the other structural elements. However, the beams and girders are under-segmented because the bottoms of the connected beams and girders do not have concave properties. In addition, floors and slabs are over-segmented due to the holes that stacked materials produce in floors and because each part of the slabs is connected to adjacent walls, beams, and girders. Therefore, the second step of the segmentation refinement is intended to further segment the under-segmented parts and to merge the over-segmented parts with adjacent elements once more considering these elements’ connectivity and geometric properties. Connectivity is used as a criterion to separate each of the beams and girders. These connectivity relations can be identified at the common face where one element shares the other element(s) [38]. In building facilities, certain elements are only located in certain positions and have nesting and connectivity relations with their neighboring elements [3,8]. Common connectivity relations include “attached/connected-to” and “supported-by [38].” If supervoxels belonging to different elements are adjacent, the two elements are considered to be attached to each other. Also, if there is a convexity between the supervoxels of the other element adjacent to and above the column, it is considered that the other element is supported by the column. For example, girders are attached to or supported by a column or a wall, beams are attached to girders or a wall, and columns and walls are attached to or supported by floors. The connectivity relations can be computed with the proximity and directions between (at least) two or more neighboring structural elements from the sets of the segmented 3D point cloud [8]. By using these connectivity criteria for the column and girder, each girder is segmented first, and each remaining beam is then clustered. The next step is to merge the over-segmented walls, floors, and slabs based on their 3D geometric properties. For the floors and slabs, the heights on which the planar segments lie are used as a geometric property. The over-segmented floors and slabs are merged again by combining segments with similar heights. For the walls, if the planar segments corresponding to the walls share a plane within a certain range to the plane, then over-segmented walls are merged again in the same context.
3.3.2. 3D parameter calculation Once each segment has been classified according to type of shape, they can be used to define the position and orientation of the central axis of each type. For the different types of functional semantics (columns, beams/girders, floors/slabs, and walls) and shapes of the structural elements, the parameters that should be determined are as follows. For elements that have rectangular cross-sections, the length and width are defined by the size of the minimum bounding rectangle (see Fig. 6(a)), whereas for elements that have circular cross-sections, the radius is defined by the size of the minimum bounding circle (see Fig. 6(b)). The height of the vertical or horizontal elements—such as columns, beams/girders, and walls—is determined by calculating the vertical or horizontal distance between the minimum and maximum points along the directional axis. In the case of columns, data for some parts at the bottom of the columns may not be obtained where materials are stacked around the columns during the construction phase. For this reason, this study considers the connectivity relations between columns and floors, girders, or slabs again. By comparing the spacing between floors and slabs (or girders) and the height of the calculated columns, this study has prevented errors that would otherwise result from the models of the columns that are not attached to the floors. In addition, since only data for one side can be obtained for the floors and the slabs due to the nature of data acquisition, the floors and slabs were modeled with a default thickness of 200 mm. Using the information on each segment’s direction obtained in Section 3.2.2 and 3D coordinates, the position and orientation of the central axis of each segment are defined. After calculating the parameter set, the process for surface model generation can be considered as parametric modeling using geometric primitives [47]. A model of each rectangular column, beam/girder, and wall is generated by using the calculated length, width, and height values. A model of a circular column and beam/girder is generated by using the calculated radius and height. The 3D parameter calculation and volumetric shape representation of a curved wall were not considered in this study. The positions of the column, beam/girder, and wall are computed using the 3D coordinates of each segment. Also, the orientations of the directional axes of the column, beam/girder, and wall are determined by considering the orientation of the segments to which they are connected. The model is exported in a user-defined format, which is a structured ASCII text file. It follows the Standard for Exchange of Product Model Data (STEP) physical file format to be mapped in the International Foundation Classes (IFC) scheme using the EXPRESS data definition language. For columns, beams/girders, floors/slabs, and walls, IFC entity types, such as IfcColumn, IfcBeam, IfcSlab, IfcWall, IfcWallStandardCase, IfcBulidingElementProxy, IfcRelDecomposes and IfcRelConnects, are defined with their geometric and connectivity properties following the IFC scheme. In this way, an as-built 3D model of the structural elements of buildings is automatically modeled from the laser-scan data.
3.3. Volumetric representation 3.3.1. Shape classification The final step of the proposed process is volumetric shape modeling. By determining the type of shape of the cross-section and calculating a set of parameters for the modeling of the individual segments, 3D models of segmented parts can be generated. Two types of shapes are considered in this study: rectangular and circular. To determine the shape of the individual segments, the following are considered in this study. At any 3D coordinates in the individual segments, three points are randomly selected (see Fig. 5(a)). The slices of a certain thickness (100 mm), with the slicing plane perpendicular to the segment’s axis, are extracted from each of the three points, and are projected onto the slicing plane. Then, each of the three cross-sections is fitted with its convex hull, so-called minimum bounding rectangle and minimum bounding circle (see Fig. 5(b)). Fitting the minimum bounding rectangle results in a rectangle that has the smallest area. For this purpose, the method proposed by Freeman and Shapira [46] was employed, ensuring that the side of a minimum bounding rectangle is collinear with the side of the convex hull. The minimum bounding circle is approximated using the center of the projected points and the longest distance between the projected points and the center. Once the areas of the convex hull, minimum bounding rectangle,
4. Results and discussion In order to evaluate the capability of the proposed method, as-built data were acquired during structural works using ScanStation C10 by Leica Geosystems, Ag. For this purpose, laser scanning was performed from 16 scan positions at the 1 st floor of a new building construction 119
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
Fig. 5. (a) Selecting three points (in red) and extracting slices (in blue) from each of the three point; (b) shape classification by comparing fitted geometries (blue: projected 3D points; red: convex hull; orange: minimum bounding rectangle; green: minimum bounding circle). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 6. (a) Parameters for elements that have rectangular cross-sections; (b) parameters for elements that have circular cross-sections.
Fig. 7. (a) Acquired as-built laser-scan data (56,686,889 points); (b) result of ROI detection.
points corresponding to concrete structures were targeted and detected by employing the ensemble classifier proposed by Son et al. [36]. Through this process, the stacked materials and fence having different color values from the concrete structures, were distinguished and removed. In order to efficiently show the inside, only part of the ROI detection result is cropped and presented in Fig. 7(b). Fig. 8 shows the result of the graph building of connected linear
project. Following laser scan data collection, all 16 scan point clouds were registered in a common coordinate system. Fig. 7(a) shows the registered 3D point cloud. In this scene, there are 2 floors (each of having different heights), 9 columns, 2 walls, 13 beams, 18 girders, and 2 slabs of having different heights. Fig. 7(b) shows the result of the region-of-interest (ROI) detection from the registered 3D point cloud shown in Fig. 7(a). In this case, 3D 120
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
Fig. 8. (a) Result of convexity graph segmentation and filtering; (b) Magnified portion of (a).
similar height values, 21 slabs were merged into 2 slabs (see Fig. 10). As a result, a total of 45 structural elements were successfully segmented with respect to their own functional semantics. The final step of the proposed process is volumetric shape modeling. By calculating dimensions, such as the length, width, and height, each column can be modeled as shown in Fig. 11(a). And as such, walls, girders, beams, and slabs can be modeled as shown in Fig. 11(b). The proposed approach described above was applied to another building facility, which is a new building construction project in ChungAng University, with more complex structural elements. The laser-scan data was acquired in June 2015. And a single scan was processed from region-of-interest detection to segmentation following local convexity steps. As Fig. 12 shows, the proposed method has an advantage of automatically segmenting as-built point clouds into a set of volumetric building elements. When the previous methods without as-planned model is to recognize a column, additional processes are required; first, finding several different planes with the column, and next, combining them into a unified entity. Furthermore, the proposed method does not require any assumptions about the shape of building elements. The proposed method can segment free-form shaped walls, cylindrical and boxed columns, and other complex-shaped building elements. The final step of the proposed process is volumetric modeling. By calculating dimensions, such as the length, width, and height, each column, beams, and girders can be modeled as shown in Fig. 13. As explained in Section 3.3.2, only data for one side can be obtained for the floors and the slabs due to the nature of data acquisition. Hence, the floors and slabs were modeled with a default thickness of 200 mm. To
patches, convexity graph creation, convexity graph segmentation, and filtering processes. In this figure, different segments are displayed using different colors. This figure shows the proposed method based on the concavity and convexity properties, dividing the 3D point cloud between floors, columns, beams and girders, and slabs. The segmented elements are as follows: 3 floors, 9 columns, 2 walls, 3 beams, 6 girders, 5 beam-girder composites, and 21 slabs connected by walls, beams, and girders. Although each of the columns is segmented from other structural elements, 5 beam-girder composites are under-segmented because the bottoms of the connected beams and girders do not have concave properties, here shown as the 3D points colored in purple in Fig. 8. Therefore, segmentation refinement was performed to separate each of the beams and girders using a connectivity property with adjacent columns and walls. Fig. 9 shows the result of the segmentation between beams and girders. The 3D points colored in purple (see Fig. 8) were segmented into different beams and girders, as shown in Fig. 9. As a result, 5 beamgirder composites were re-segmented into 10 beams and 12 girders. Finally, the last step of the segmentation refinement was performed on the result shown in Fig. 9. As shown in Figs. 8 and 9, floors and slabs were over-segmented. In the case of the floors, they were over-segmented due to holes produced by the stacked materials on the floors were distinguished and discarded in the ROI detection process. By merging the segments having similar height values, 3 floors were merged into 2 floors. In addition, in the case of the slabs, they were over-segmented because each part of the slabs was connected by adjacent walls, beams, and girders. By merging the segments having
Fig. 9. (a) Result of the segmentation between beams and girders; (b) magnified portion of (a).
121
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
Fig. 10. (a) Result of merging over-segmented parts; (b) magnified portion of (a).
Fig. 11. Case 01: Result of volumetric modeling of (a) Columns; (b) Adding walls, beams/girders, and slabs.
Fig. 12. (a) Acquired as-built laser-scan data (3,602,979 points); (b) result of convexity graph segmentation and filtering (1,891,536 points).
acquired at this stage, and the method of data acquisition for the corresponding surfaces will be explored in future research. The performance of the individual semantic element recognition was evaluated by quantitatively measuring how accurately each of the structural elements was recognized as its functional semantics. The individual semantic element recognition results show that the proposed method can be used for modeling that includes not only the geometry of
more effectively display the modeling results for columns, walls, beams, and girders in two cases, the models of floors and slabs were excluded from the illustration. For example, only the model of the slab in Case 01 was shown in Fig. 11(a). If the bottom and top surfaces or lateral surfaces of floors or slabs can be obtained in the data acquisition, the thickness could be estimated in the same way as the thickness of the walls. However, the data of the corresponding surfaces cannot be 122
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
of the structural elements, the proposed method can be used for segmenting laser-scan data into semantic building elements and for modeling different types of building elements, especially structural works. The capability of the proposed method was evaluated with laser-scan data acquired from the field experiments. The experimental results showed that the proposed method could be used for as-built BIM without any prior information from an as-planned model. In addition, computational efficiency was achieved by employing the supervoxel algorithm. The novelty of the method could be summarized as follows. The proposed feature has proven to be effective to segment 3D point cloud corresponding structural elements into element units according to their actual functional semantics (columns, beams/girders, floors/slabs, and walls). Additionally, the generated model includes semantic information and is interoperable so that it can be read in existing commercial BIM software. Although this study defined and exported some instances for the entities required to map to the IFC scheme, the development of a mapping method for exporting the generated as-built models to the complete IFC file format, which contains all instances for the entities of the IFC specification, will be covered in future research. Also, future research will be devoted to the quantitative evaluation of the proposed method in various building facilities. Moreover, the proposed method will be expanded further to develop a method for generating and updating as-built BIM during or after each major construction activity—not only structural works including curved elements but also MEP, architectural, and finishing works—for project management purposes.
Fig. 13. Case 02: Result of volumetric modeling of columns and beams/girders.
Table 1 Results of the individual semantic element recognition. Columns
Beams/ Girders
Floors/ Slabs
Walls
Total
Case 01
9/9 (100%) 25/25 (100%) 34/34 (100%)
4/4 (100%) 2/2 (100%) 6/6 (100%)
2/2 (100%)
Case 02
31/31 (100%) 62/64 (97%) 93/95 (98%)
46/46 (100%) 93/95 (98%)
Total
4/4 (100%) 6/6 (100%)
139/141 (99%)
Acknowledgments This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2013R1A1A2A10058175).
individual structural elements but also the their functional semantics. Obtaining high recognition performance is important because the method proposed was to generate an as-built model of structural elements. To measure the performance of the individual semantic element recognition, the ratio of the number of elements of each type (columns, beams/girders, floors/slabs, and walls) that were correctly recognized was compared to the total number of elements of that type. The results of the evaluation of the individual semantic element recognition are summarized in Table 1. As shown in the table, the individual semantic element recognition results are satisfactory, with 100% and 98% of the structural elements in cases 01 and 02, respectively, being correctly recognized. Moreover, overall, 99% of the structural elements in the two cases combined were recognized and modeled according to their actual functional semantics. This study also sought to measure the processing time. The timing began when the acquired as-built laser-scan data first started to be processed and the data processing from the region-of-interest detection, scene segmentation, and volumetric representation was performed. Each test was repeated 30 times for the two cases of test beds. The proposed method successfully modeled the constituents of as-built building elements within about 4.44 h from 56,686,889 points in Case 01 and within 0.42 h from 3,602,979 points in Case 02. As this study mainly focused on how accurately each of the structural elements was recognized as its functional semantics and how to automate the process, the computational time of within hours is acceptable, as the proposed method results in substantial performance improvements with respect to the completeness of the generated as-built BIMs in regard to achieving automation of the process. Regarding the computational time, there is room for improvement in the proposed process, which will be the subject of future work.
References [1] A. Stadler, T.H. Kolbe, Spatio-semantic coherence in the integration of 3D city models, in: Proc. 5th Int. Symp. on Spatial Data Quality, Enschede, The Netherlands, June 2007. [2] C. Nagel, A. Stadler, T.H. Kolbe, Conceptual requirements for the automatic reconstruction of building information models from uninterpreted 3D models, in: Proc. Int. Archives of Photogrammetry, Remote Sensing and Spatial Information Science, Vancouver, Canada, July 2009, pp. 46–53. [3] S. Pu, G. Vosselman, Knowledge based reconstruction of building models from terrestrial laser scanning data, ISPRS J. Photogram. Remote Sensing 64 (6) (2009) 575–584. [4] X. Xiong, A. Adan, B. Akinci, D. Huber, Automatic creation of semantically rich 3D building models from laser scanner data, Autom. Constr. 31 (2013) 325–337. [5] M.F. Ahmed, C.T. Haas, R. Haas, Automatic detection of cylindrical objects in built facilities, J. Comput. Civil Eng. 28 (3) (2014) 04014009-1-04014009-11. [6] A. Hanna, M. Yeutter, D. Aoun, State of practice of building information modeling in the electrical construction industry, J. Constr. Eng. Manage. 140 (12) (2014) 05014011-1–05014011-11. [7] J. Jung, S. Hong, S. Jeong, S. Kim, H. Cho, S. Hong, J. Heo, Productive modeling for development of as-built BIM of existing indoor structures, Autom. Constr. 42 (2014) 68–77. [8] V. Pătrăucean, I. Armeni, M. Nahangi, J. Yeung, I. Brilakis, C. Haas, State of research in automatic as-built modelling, Adv. Eng. Inform. 29 (2015) 162–171. [9] F. Bosché, A. Guillemet, Y. Turkan, C.T. Haas, R. Haas, Tracking the built status of MEP works: assessing the value of a scan-vs-BIM system, J. Comput. Civil Eng. 28 (4) (2014) 05014004-1–05014004-13. [10] F. Bosché, M. Ahmed, Y. Trukan, C.T. Haas, R. Haas, The value of integrating Scanto-BIM and Scan-vs-BIM techniques for construction monitoring using laser scanning and BIM: the case of cylindrical MEP components, Autom. Constr. 49 (Part B) (2015) 201–213. [11] T. Gao, S. Ergan, B. Akinci, J. Garrett, Evaluation of different features for matching point clouds to building information models, J. Comput. Civil Eng. 30 (1) (2016) 04014107-1–04014107-13. [12] H. Hajian, B. Becerik-Gerber, A research outlook for real-time project information management by integrating advanced field data acquisition systems and building information modeling, Proc. Computing in Civil Engineering, Austin, TX, June 2009, pp. 83–94. [13] T. Randall, Construction engineering requirements for integrating laser scanning technology and building information modeling, J. Constr. Eng. Manage. 137 (2011) 797–805. [14] F. Bosché, C.T. Haas, B. Akinci, Automated recognition of 3D CAD objects in site
5. Conclusion This study presented a method for scene segmentation and the generation of volumetric building elements from laser-scan data acquired during the construction phase. By using loc concave properties 123
Advanced Engineering Informatics 34 (2017) 114–124
H. Son, C. Kim
[15]
[16]
[17] [18] [19] [20]
[21]
[22]
[23]
[24] [25]
[26]
[27]
[28] [29]
[30]
[31]
(2014) 86–97. [32] P. Tang, D. Huber, B. Akinci, R. Lipman, A. Lytle, Automatic reconstruction of asbuilt building information models from laser-scanned point clouds: a review of related techniques, Autom. Constr. 19 (7) (2010) 829–843. [33] J. Xiao, Y. Furukawa, Reconstructing the world’s museums, Int. J. Comput. Vision 10 (3) (2012) 668–681. [34] A. Stambler, D. Huber, Building modeling through enclosure reasoning, in: Proc. 2014 2nd Int. Conf. on 3D Vision, Tokyo, Japan, December 2014, pp. 118–125. [35] H. Son, J. Na, C. Kim, Semantic as-built 3D modeling of buildings under construction from laser-scan data based on local convexity without an as-planned model, in: Proc., Int. Symp. on Automation and Robotics in Construction, Oulu, Finland, June 2015, pp. 1–6. [36] H. Son, C. Kim, N. Hwang, C. Kim, Y. Kang, Classification of major construction materials in construction environments using ensemble classifiers, Adv. Eng. Inform. 28 (1) (2014) 1–10. [37] S.C. Stein, F. Wörgötter, M. Schoeler, J. Papon, T. Kulvicius, Convexity based object partitioning for robot applications, in: Proc., 2014 IEEE Int. Conf. on Robotics and Automation, Hong Kong, China, May – June 2014, pp. 3213–3220. [38] T.-H. Nguyen, A.A. Oloufa, K. Nassar, Algorithms for automated deduction of topological information, Autom. Constr. 14 (1) (2005) 59–70. [39] J. Papon, A. Abramov, M.Schoeler, F. Wörgötter, Voxel cloud connectivity segmentation – Supervoxels for point clouds, in: Proc., 2013 IEEE Conf. on Computer Vision and Pattern Recognition, Portland, OR, June 2013, pp. 2027–2034. [40] S.C. Stein, M. Schoeler, J. Papon, F. Wörgötter, Object partitioning using local convexity, in: Proc. 2014 IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, OH, June 2014, pp. 304–311. [41] J.-F. Lalonde, N. Vandapel, D.F. Huber, M. Hebert, Natural terrain classification using three-dimensional ladar data for ground robot mobility, J. Field Robot. 23 (1) (2006) 839–861. [42] E.H. Lim, D. Suter, Conditional random field for 3D point clouds with adaptive data reduction, in: Proc. Int. Conf. on Cyberworlds, Hannover, Germany, October 2007, pp. 404–408. [43] M. Himmelsbach, T. Luettel, H.-J. Wuensche, Real-time object classification in 3D point clouds using point feature histograms, in: Proc. 2009 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, St. Louis, October 2009, pp. 994–1000. [44] D.V. Nguyen, L. Kuhnert, J. Schlemper, K.-D. Kuhnert, Terrain classification based on structure for autonomous navigation in complex environments, in: Proc. 2010 Third Int. Conf. on Communications and Electronics, Nha Trang, Vietnam, August 2010, pp. 163–168. [45] L. Shi, S. Kodagoda, R. Ranasinghe, Fast indoor scene classification using 3D point clouds, in: Proc. Australasian Conf. on Robotics and Automation, Melbourne, Australia, December 2011. [46] H. Freeman, R. Shapira, Determining the minimum-area encasing rectangle for an arbitrary closed curve, Commun. ACM 18 (7) (1975) 409–413. [47] Y.-K. Cho, C.T. Haas, Rapid geometric modeling for unstructured construction workspaces, Comput.-Aided Civil Infrastruct. Eng. 18 (4) (2003) 242–253.
laser scans for project 3D status visualization and performance control, J. Comput. Civil Eng. 23 (2009) 311–318. F. Bosché, Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction, Adv. Eng. Inform. 24 (1) (2010) 107–118. H. Son, C. Kim, 3D structural component recognition and modeling method using color and 3D data for construction progress monitoring, Autom. Constr. 19 (7) (2010) 844–854. Y. Turkan, F. Bosché, C.T. Haas, R. Haas, Automated progress tracking using 4D schedule and 3D sensing technologies, Autom. Constr. 22 (2012) 414–421. C. Kim, H. Son, C. Kim, Automated construction progress measurement using a 4D building information model and 3D data, Autom. Constr. 31 (2013) 75–82. Y. Turkan, F. Bosché, C.T. Haas, R. Haas, Toward automated earned value tracking using 3D imaging tools, J. Constr. Eng. Manage. 139 (4) (2013) 423–433. X. Xiong, D. Huber, Using context to create semantic 3D models of indoor environment, in: Proc. British Machine Vision Conf., Aberystwyth, Ceredigion, August – September 2010, pp. 45.1–45.11. L. Truong-Hong, D. Laefer, T. Hinks, H. Carr, Flying voxel method with Delaunay triangulation criterion for facade/feature detection for computation, J. Comput. Civil Eng. 26 (6) (2012) 691–707. L. Truong-Hong, D. Laefer, T. Hinks, H. Carr, Combining an angle criterion with voxelization and the flying voxel method in reconstructing building models from lidar data, Comput.-Aided Civil Infrastruct. Eng. 28 (2) (2013) 112–129. A. Dimitrov, M. Golparvar-Fard, Segmentation of building point cloud models including detailed architectural/structural features and MEP systems, Autom. Constr. 51 (2015) 32–45. C. Thomson, J. Boehm, Automatic geometry generation from point clouds for BIM, Remote Sensing 7 (9) (2015) 11753–11775. G. Zhang, P.A. Vela, P. Karasev, I. Brilakis, A sparsity-inducing optimization-based algorithm for planar patches extraction from noisy point-cloud data, Comput.-Aided Civil Infrastruct. Eng. 30 (2) (2015) 85–102. I. Anagnostopoulos, V. Pătrăucean, I. Brilakis, P. Vela, Detection of walls, floors, and ceilings in point cloud data, in: Proc. Construction Research Congress 2016, San Juan, Puerto Rico, May–June 2016. I. Armeni, O. Sener, A.R. Zamir, H. Jiang, I. Brilakis, M. Fischer, S. Savarese, 3D semantic parsing of large-scale indoor spaces, in: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, June–July 2016. S. Ochmann, R. Vock, R. Wessel, R. Klein, Automatic reconstruction of parametric building models from indoor point clouds, Comput. Graphics 54 (2016) 94–103. E. Valero, A. Adán, F. Bosché, Semantic 3D reconstruction of furnished interiors using laser scanning and RFID technology, J. Comput. Civil Eng. 30 (4) (2016) 04015053-1–04015053-19. D. Holz, S. Behnke, Approximate triangulation and region growing for efficient segmentation and smoothing of range images, Robot. Autonom. Syst. 62 (9) (2014) 1282–1293. R. Hulik, M. Spanel, P. Smrz, Z. Matema, Continuous plane detection in point-cloud data based on 3D Hough transform, J. Visual Commun. Image Represent. 25 (1)
124