Use Cases for Subcontractors and Fabricators

Chapter 7

Use Cases for Subcontractors and Fabricators Mohammad Nahangi1, Minkoo Kim2 1 Advanced Building Systems, KATERRA, Toronto, Canada; 2Building and Real Estate, The Hong Kong Polytechnic University, Kowloon, Hong Kong

Chapter outline 7.1 Executive summary 7.2 The case for ICV from the subcontractors’ perspective 7.2.1 Roles of subcontractors and fabricators 7.2.2 Brief relevant studies 7.2.2.1 Monitoring of the production, storage, delivery to the site, installation, and quality control of components using noncontact sensing technologies 7.2.2.2 Supporting the assembly or erection of components and quality control using sensing technologies 7.2.2.3 Geometric quality control practices in the manufacturing/ installation stage using sensing technologies

316 317 317 319

319

320

7.3 Production monitoring 7.3.1 Off-site production monitoring 7.3.2 On-site production monitoring 7.3.2.1 Identification of critical interface points in local coordinate system 7.3.2.2 Calculation of transformations for matching segments 7.3.2.3 Overview of the optimization strategies 7.3.2.4 Example 1: smallscale steel bridge segments 7.3.2.5 Example 2: fullscale concrete panels and floor decks 7.4 Fabrication control 7.4.1 Compatibility between tolerances and process capabilities

322 322 327

328

329

330

332

336 339

340

321

Infrastructure Computer Vision. https://doi.org/10.1016/B978-0-12-815503-5.00007-3 Copyright © 2020 Elsevier Inc. All rights reserved.

315

316 Infrastructure Computer Vision 7.4.2 Existing methods for dimensional variation control in construction 7.4.2.1 Deviation mapping for dimensional variation analysis 7.4.2.2 Kinematics chain ebased DVA 7.5 As-built surveying, modeling, alignment, and fitting for off-site fabrication 7.5.1 Model versus built comparison for automated discrepancy quantification

341

342 344

346

7.5.1.1 Preprocessing 7.5.1.2 Registration 7.5.1.3 Local discrepancy quantification 7.5.2 Realignment calculation 7.6 Replacing the tape measure 7.6.1 Summary from the examples provided 7.6.2 The potential of adopting the technologies in fabricators’ practice 7.6.3 Time-effectiveness 7.7 Discussions and future directions References

347 348 348 349 351 351

352 355 357 358

347

7.1 Executive summary This chapter presents some opportunities that infrastructure computer vision (ICV) has provided for the subcontractors and fabricators. Subcontractors and fabricators need ICV to measure the progress of the work completed, to better understand and capture the status of the built environment, and to measure the quality of the fabricated components and modules. Using digital twins of the components and modules at the design phase and the construction phase, subcontractors and fabricators are capable of tracking the required information. ICV combined with some further technologies have opened some unprecedented opportunities to improve the fabrication, installation, and maintenance procedures. ICV is first investigated from the subcontractors’ perspectives. Various perspectives for different trades including steel fabricators, formwork fabricators, reinforcing steel fabricators, etc., are discussed to better frame the ICV solutions for the subcontractors and fabricators in Section 7.2. In this section, a brief summary on the state-of-the-art research and practice is also presented. Section 7.3 then evaluates production monitoring at two different stages: (1) off-site production monitoring and (2) on-site production monitoring. Next, fabrication control methods and the existing technologies are then thoroughly discussed in Section 7.4. In Section 7.5, vision-based techniques and visual analytics for as-built status assessment and discrepancy quantification are discussed. Furthermore, a systematic realignment strategy for construction components is presented. Finally, a summary is provided to assess the visionbased approaches for geometric quality control (QC) and as-built status modeling in terms of accuracy and time-effectiveness. In Section 7.6, the potential of adopting the sensing technologies in subcontractors’ and

Use Cases for Subcontractors and Fabricators Chapter | 7

317

fabricators’ practice is provided with the aim of replacing the current tape measurement practice. Finally, discussions and future directions of the ICV for subcontractors and fabricators are presented at the end of this chapter.

7.2 The case for ICV from the subcontractors’ perspective Subcontractors are normally hired by a general contractor, and in many cases they sign a contract to perform part of the tasks for the project. They perform a very wide range of specialized tasks contracted by general contractors. Fabricators are a primary subcontractor in the construction industry, and they create small- and large-scale components manufactured in a well-controlled environment. They may be considered to form part of the engineered materials managed as part of a construction project supply chain. There are five different types of primary fabricators in the construction industry.

7.2.1 Roles of subcontractors and fabricators Structural steel fabricatorsdThey are responsible for fabrication and erection of structural steel components and assemblies, and they have primary responsibilities including (1) fabricating steel components; (2) connecting steel components; and (3) quality inspection of the assembly of steel components. The main tasks involving in structural steel products include 1) cutting, 2) punching and bending, 3) fitting (bolting, welding, and assembly), 4) cleaning and finishing, 5) QC, and 6) shipping and delivering to the project site for installation/erection. Among the processes, the task of geometric QC is essential for the success of structural steel production. Dimensional quality check is normally conducted at stages before, during and after the manufacturing process. Dimensional marking is checked before cutting, bending, and assembly. After part or final assembling, all dimensional marks are reconfirmed for accuracy before further works such as reforming, etc., at which stage the dimensional inspection is conducted. In addition, quality inspection of welding is important in structural steel products because a severe structural failure can occur if welding is improperly executed. Checklists for welding contain defects such as undercut, crack, blow hole, and fused spatter. Formwork fabricatorsdFormwork is inspected before the reinforcing steel is in place to ensure that the dimensions and location of the concrete conform to design drawings. Formwork fabricators specialize in producing either temporary or permanent molds to give a solid shape to concrete solutions. Their main tasks include 1) erecting wooden, steel, or aluminum formwork safely in a particular location; 2) monitoring the formwork until the concrete mix cures adequately; and 3) inspecting the completed formwork and position of reinforcing steel. During the production process, quality inspection, which investigates the dimensions and defects, is an important process in formwork production. Normally, fabricators inspect the completed formwork and

318 Infrastructure Computer Vision

position of steel reinforcements after the production, and the primary checklists include 1) cover which is the amount of clear space between the form surface and the closed reinforcement bar; 2) layout and overall dimensions; and 3) surface defects. Reinforcing steel fabricatorsdWork associated with reinforcing steel (generally called “rebar”) consumes relatively little project time, but it requires close coordination with the formwork subcontractors to ensure an efficient operation. The key role of rebar fabricators is to place rebars in the right place according to the placing drawings, and inspection of the position of concrete reinforcing bars is required to be implemented before placing concrete. Precast concrete fabricatorsdPrecast and prefabricated construction elements are manufactured in factories, before they are assembled or installed on construction sites. Precast concrete contractors are in charge of 1) ensuring the completeness of precast molds and their assembly, 2) inspecting (e.g., dimensions and locations) of manufactured reinforcing steel positioned in the precast concrete formwork and precast concrete components themselves, and 3) transporting the produced precast components to construction sites. In the fabrication of a precast concrete mold, checking the level, dimensions, and flatness of the base of the mold is an important task before assembling the mold. In the placing and fixing of reinforcing steel in the mold, rebar size, spacing, and lap length are the main checklists to be inspected. Quality assessment of precast and prefabricated elements must be performed to ensure smooth on-site assembly and installation as well as the high quality of final construction. Mechanical, electrical, and plumbing (MEP) fabricatorsdMEP fabricators are mainly involved in producing and fabricating air conditioning systems, water supply and drainage systems, fire protection systems, electrical power and lighting systems, security access, surveillance systems, and building management systems. At the construction phase, MEP contractors oversee MEP service installation to identify any poor quality elements and assemblies. Given the dynamic environment in the construction industry, failure to achieve adequate quality levels of construction components manufactured by subcontractors and fabricators has been an obstacle to the delivery of projects on time and within cost. Current quality assurance processes being conducted for construction components primarily use human operations and paper-forms. For example, for quality inspection of precast concrete elements, quality checklists including dimensions (length, width, and thickness), squareness, and flatness of precast concrete components are currently monitored based on measurement tapes, levels, and straightedges. In addition, current practices for reinforcement position inspection rely on measuring tapes to measure the required checklists of reinforcements. Inspectors often take photos to create a visual record of the as-built status. This visual inspection based on contacttype measurement devices is laborious, inaccurate, and time-consuming. In addition, current approaches for measurement QC on off-site construction

Use Cases for Subcontractors and Fabricators Chapter | 7

319

components are not as effective as they could be in identifying manufacturing defects early in the manufacturing process. As a result, defects can be detected in later phases of construction or in the maintenance phase, resulting in a large rework cost for the project. Furthermore, traditional means of storing and managing the inspection results rely on the paper-based format, which is ineffective and inefficient for communication with others. Addressing defects caused by fabrication errors and during shipping requires automated and continuous quality inspection and control. Visual sensing data captured by digital cameras and laser scanners can meet many of the requirements for the QC of such products. Such technologies make it possible to automate the quality inspection and control tasks. Automating these tasks will reduce fabrication errors and thus decrease rework cost. Although many successful applications of these sensing technologies have been achieved, mainly on food and industrial products, there has been lack of applications of sensing technologies to the tasks being implemented in construction component manufacturing sites. Nevertheless, there are increasing efforts and studies in the research community to improve and solve the limitations of the current quality inspection in the manufacturing and construction fields. In addition, widespread implementation already exists in piping and module fabrication.

7.2.2 Brief relevant studies There has been recent research adopting sensing technologies to automate and improve the quality inspection process:

7.2.2.1 Monitoring of the production, storage, delivery to the site, installation, and quality control of components using noncontact sensing technologies There have been few examples of applying sensing technologies at the manufacturing and assembly stages. Regarding dimensional quality assessment, the previous literature can be categorized into mainly two research areas conducted by two research groups. The first batch of research aims to conduct quality assessment for welds, particularly focusing on the measurement of misalignment between two welded plates. Rodrı´guez-Martı´n et al. (Rodrı´guezMartı´n et al., 2016) utilized macrophotogrammetry technique to generate point cloud data for welding specimens. Then, principal component analysis (PCA) plane fitting was applied to the point cloud data to find the planes of two welded plates for misalignment measurement. Furthermore, Rodrı´guezGonza´lvez et al. (Rodrı´guez-Gonza´lvez et al., 2017) leveraged both macrophotogrammetry and 3D laser scanning to generate point cloud data of the welding specimens. Both point-based analysis and feature-based analysis were adopted to evaluate the accuracy of point cloud data generated from the two

320 Infrastructure Computer Vision

different approaches. The second batch of research is focused on the dimensional quality assessment of precast concrete elements. Kim et al. (Kim et al., 2014a) proposed an algorithm for automatic dimension estimation for precast concrete bridge deck panels using terrestrial laser scanning. Later, they (Kim et al., 2015a) proposed a quality assessment framework combining building information model (BIM) and laser scanning. This framework was further improved and validated on full-scale precast concrete panels in a following work (Kim et al., 2016). Wang et al. (Wang et al., 2016b) further developed a technique that can estimate the dimensions for precast concrete bridge deck panels with arbitrary outer boundaries and complex structural features. In addition, Kim et al. (Kim et al. 2019b) proposed a mirror-aided laser scanning system that can scan the side surfaces of precast concrete element in case the side surfaces are not visible from the laser scanner located above the center of a targeted precast concrete element. In addition to the dimension estimation of concrete, another research by Wang et al. (Wang et al., 2017) was conducted to estimate the locations of rebars on reinforced precast concrete elements based on colored laser scanning data and machine learning algorithms. On the other hand, only a few research works were reported on the surface quality assessment in manufacturing stage. Kim et al. (Kim et al., 2014b) developed a technique for simultaneous localization and quantification of spalling defects on precast concrete surfaces using terrestrial laser scanning. Lo´pez et al. (Lo´pez et al., 2010) developed a hybrid 3D-2D laser scanning system containing a 3D laser scanner and a 2D camera for the characterization of slate slabs. Wang et al. (Wang et al., 2016b) developed techniques of surface flatness and distortion inspection for precast concrete elements based on terrestrial laser scanning data.

7.2.2.2 Supporting the assembly or erection of components and quality control using sensing technologies Regarding the research on the assembly of prefabricated components, the research team from University of Waterloo has conducted a series of research works that look at the dimensional discrepancy of prefabricated construction assemblies to assist the realignment of the assemblies. Safa et al. (Safa et al., 2015) developed a three-station (production QA, postproduction QA, and onsite QA) quality management model for prefabricated pipe spools, in which photogrammetry and handheld laser scanning were adopted for 3D data acquisition. Nahangi et al. (2015b) developed an automated framework for realignment of defective construction assemblies. Laser scanning was used to obtain the as-built status of pipe spools and quantify the discrepancies while inverse kinematics analogy was adopted to calculate the necessary realignment plans. In another work, Nahangi et al. (Nahangi et al., 2015c) developed a forward kinematics model to analyze the local discrepancy and quantify the deviations of prefabricated assemblies based on laser scan data. The deviation

Use Cases for Subcontractors and Fabricators Chapter | 7

321

information then becomes a discrepancy feedback for automated refitting and realignment of prefabricated assemblies. In addition to the abovementioned works, Yoon et al. (2018). also leveraged laser scanning for the assembly of precast elements. They assessed the dimensional quality of precast bridge deck slabs and precast girders. Based on the as-built dimensions of both slabs and girders, the optimal placement of slabs with respect to girders was found by minimizing the mismatches between shear pockets on slabs and shear connectors on girders. Details of opportunities and limitations of these studies proposed for QC of prefabricated components for supporting the assembly/ erection are discussed in detail in Sections 7.3 and 7.4.

7.2.2.3 Geometric quality control practices in the manufacturing/ installation stage using sensing technologies On top of the academic studies, there have been practical studies based on sensing technologies in the manufacturing/installation process of prefabricated components, particularly in the United States and Canada. For instance, an engineering and consulting company, TruePoint (TruePoint 2019), conducted several case studies using laser scanning to improve the productivity in fabrication and assembly of prefabricated components. The company captured flange information of gas pipes including bolt hole orientation, elevation information, and pitch of flanges as shown in Fig. 7.1. For the case study, 100 natural gas separator units were scanned to help the client decrease the installation time of the separate units. As the output of the sensing data captured at the site, the client received Recap files for use in Autodesk MEP as well as completed drawings that provide the information of identified bolt hole patterns, centerline locations, elevation of features, connection points, and pitch of inlets and outlet flanges. Dimensional control on fabricated panels was also conducted by the same company to capture the dimensions of an existing building fac¸ade for the installation of prefabricated paneling. However, there are some limitations in the current practice implemented in the industry. Currently, as-built modeling of prefabricated components is mostly performed manually by the user or consultant. They normally use commercial software programs such as Trimble

FIGURE 7.1 Example of quality control practice in the manufacturing/installation stage (A) 100 natural gas separator units to be connected. (B) Colorized point cloud data of the inlet of a separator unit and (C) Measurement of bolt hole orientation captured by laser scanning. Images from TruePoint (2019)

322 Infrastructure Computer Vision

RealWorks and Leica Cyclone to generate 3D layouts of the as-built components. During the layout generation of pipeline structures, the users manually segment the complicated pipelines into individual pipelines and normally use the cylinder-fitting function to generate the layout of the various segments, which is time-consuming and tedious. In addition, the current practice is mostly focused on the dimensional control and as-built layout generation of pipeline components. Therefore, automation in the layout generation of as-built prefabricated components and extension of application to other prefabricated components except for pipeline components are necessary.

7.3 Production monitoring Despite growing technological and process-related advancements in off-site construction, geometric variability is inevitable and often creates challenges for assembly aggregation. Production monitoring becomes very critical to identify the geometric variabilities of components being aggregated either offsite or on-site. Although the use of automated production processes can be very accurate, component geometry can change from its nominal design status as the result of manufacturing processes such as welding and from flexing and warping due to handling, transporting, and installation (Lawson et al., 2014). When components deviate excessively from their designed geometry, misfits can occur, resulting in delays, disruption, waste, and rework. Ultimately, the negative effects of component misfit can compound into uncontrollable project cost increases and a loss in client satisfaction and confidence (Gibb, 1999). While the traditional approach to managing component misfit issues in stickbuilt construction has been to “custom-cut and fit at the job site” (Ballast, 2007), the approach often taken in off-site construction has been to employ trialand-error strategies utilizing shim plates, cut-off lengths, trimming andand cutting, etc. (Shahtaheri et al., 2017). Trial-and-error strategies can be timeconsuming, resource intensive, and do not effectively address the risk of rework.

7.3.1 Off-site production monitoring Off-site production is being more widely adopted. As high-quality and highspeed construction are increasingly demanded in the current construction industry, monitoring the quality of products manufactured off-site is essential for the success of a project. Precast concrete elements are a representative example of off-site production. There are some recent studies related to the inspection of precast concrete elements. Kim et al. (2014a) proposed an automatic dimension estimation technique for precast concrete bridge deck panels using laser scanning as shown in Fig. 7.2. In their study, a new edge and corner extraction algorithm was developed and a dimension accuracy of around 2 mm was achieved. The scope of their study was further improved and validated on full-scale precast concrete panels in a following work (Kim et al.,

Use Cases for Subcontractors and Fabricators Chapter | 7

323

FIGURE 7.2 Dimensional quality assessment of PC elements: (A) Lab-scale precast concrete element and (B) Extracted features leading to automated dimensional quality assessment (Kim et al. 2014a).

FIGURE 7.3 Full-scale precast slab and visualization of the dimensional quality assessment results by comparing the DT and as-built geometries (Kim et al., 2016).

2016). Two types of full-scale precast slabs tests were conducted, and the representative results are shown in Fig. 7.3. A meaningful finding was made based on the full-scale field tests that extensive dimensional deviations greater than allowable tolerances often occur in reality, indicating the potential use of the dimensional quality assessment technique in the precast manufacturing industry.

324 Infrastructure Computer Vision

In addition, in the same research group, Wang et al. (2016a) further developed a technique that can estimate the dimensions for precast concrete bridge deck panels with arbitrary outer boundaries and complex structural features. In addition to bridge deck panels, the dimensional quality of a precast bridge girder was also inspected (Yoon et al., 2018). The quality inspection of the precast bridge girder was focused on the locations and orientations of steel studs, which were supposed to be precisely inserted into shear pockets on deck panels to form girder-deck connections. Laboratory testing on small-scale specimens of girders showed an accuracy of 1.7 mm in stud location estimation and 0.5 degrees for stud orientation estimation. In the study, a field test was conducted on a real precast bridge girder as shown in Fig. 7.4. The field test shows that the average location error of all shear connectors is about 13.24 mm. Because of dimensional errors of precast girders that occurred in

FIGURE 7.4 Full-scale precast girder and visualization of the dimensional (location) quality assessment results: (A) Test configuration of the full-scale girder scanning; (B) Location estimation result of shear connectors on the full-scale precast girders by comparing the DT and as-built geometries (Yoon et al. 2018).

Use Cases for Subcontractors and Fabricators Chapter | 7

325

the fabrication stage, deformation of precast girders, and incorrect welding location of shear connectors, large location errors were experienced. Furthermore, Wang et al. (2017) proposed a new technique that estimates the locations of rebar on reinforced precast concrete elements based on colored laser scanning data. A one-class SVM classifier and a rebar recognition algorithm were developed for extracting rebar based on both geometric and collar features of scan points. However, those previous techniques developed have two difficulties for real-world applications. First, the side surfaces of precast elements are hard to be scanned because of limited space between precast concrete elements that are stacked for storage. Second, to scan the whole side surface of a precast concrete element with a large size, multiple scanner locations are required, which is time- and cost demanding. To tackle the limitations, Kim et al. (2019b) proposed a mirror-aided scanning system that can scan the side surfaces of precast concrete element for geometric quality inspection in case the side surface is not visible from the laser scanner. The scheme of the mirror-aided moving scanning system is illustrated in Fig. 7.5. The proof-of-concept laboratory test results show an average dimension estimation accuracy of 2.5 mm, demonstrating the potential for geometric quality inspection of side surfaces of precast concrete elements without changing the location of laser scanner. In summary, the previous extensive studies on dimensional quality assessment of prefabricated elements using point cloud data present very accurate (less than 3 mm) results, proving the feasibility of those techniques for automating the dimensional assessment of prefabricated concrete elements. However, existing research on geometry quality inspection in the off-site production during the fabrication phase has two major limitations that are 1) most studies were focused on precast concrete bridge elements and 2) those dimensional inspection are conducted for postfabrication stage where precast concrete or prefabricated elements are already fabricated. Therefore, future

FIGURE 7.5 Overview of the mirror-aided moving scanning system: (A) Mirror with frame for rotation; (B) Scanning configuration of the side surface inspection with the moving mirror (Kim et al. 2019b).

326 Infrastructure Computer Vision

research is needed for the geometry quality inspection of other types of off-site production components such as precast bathroom unit and prefabricated and prefinished volumetric construction which are popularly adopted in Hong Kong and Singapore. In addition, future research should be conducted to extend the geometry quality inspection from postfabrication stage to prefabrication and fabrication state. On the other hand, some research work was conducted on the surface quality assessment of prefabricated off-site components in the manufacturing stage. For example, Kim et al. (2015b) proposed a surface quality assessment technique for simultaneous localization and quantification of spalling defects on precast concrete surfaces using terrestrial laser scanning. The method used the angle and distance deviations as defect-sensitive features to improve the localization and quantification of spalling defects on the surface of precast concrete elements. A defect classifier is developed to automatically diagnose whether the investigated surface region is damaged, where the defect is located, and how large it is. Simulation and experimental test results show that the method provides (1) autonomous and simultaneous spalling defect localization and volume estimation and (2) improved defect localization and quantification with the combination of complementary defect-sensitive features. Fig. 7.6 shows the surface defect localization and quantification results

FIGURE 7.6 Visualization of surface defect localization and quantification on precast concrete panel: (A) angle deviation feature; (B) distance deviation feature; (C) combination of the two features; and (D) defect classification (the solid lines in red color (gray in print version) indicate the boundaries of the actual defect area, and the regions in white color are the detected defect regions) (Kim et al., 2015b).

Use Cases for Subcontractors and Fabricators Chapter | 7

327

FIGURE 7.7 Surface flatness and distortion inspection: (A) experimental setup on a full-scale precast slab; (B) dimensions of the tested full-scale precast slab; and (C) the bowing result of the panel (Wang et al., 2016b).

on the precast concrete panel manufactured and tested. It can be seen that the localization performance was improved with combination of the two defect indices. The test results show that the detection rate for spallings was 90% and the accuracy is 2 mm, which is promising for real-world application to manufacturing sites. In addition, in the study, it was found that the incident angle is the most critical parameter affecting the localization and volume estimation accuracy. Wang et al. (2016b) developed algorithms for automatic measurement of surface flatness and distortion inspection for precast concrete elements based on terrestrial laser scanning data. The surface flatness was measured with the floor flatness (FF) number, and surface distortion was measured using warping, bowing, and differential elevation. Experiments on laboratory specimens provide a surface quality assessment of 1.5 for FF number measurement and 2 mm for distortion measurement. Fig. 7.7 shows the field test setup and the bowing result, indicating that laser scanning technology can be a promising solution for surface flatness and distortion inspection. Moreover, Bosche and Guenet (2014) proposed an approach that integrated laser scanning and BIM for surface flatness control. Two standard flatness control techniques, namely straightedge and f-numbers, were used to measure the flatness of floors, and validation experiments were conducted on two actual concrete floor slabs. The test results show that the straightedge measurements from laser scanning has an average difference of 1 mm or less compared with manual measurement.

7.3.2 On-site production monitoring Despite growing technological and process-related advancements in off-site construction, geometric variability is inevitable and often creates challenges for assembly aggregation. Measuring geometric deviations and component alignment on project sites is a challenging task that needs to be performed to monitor geometric tolerance compliance and to control excessive geometric

328 Infrastructure Computer Vision

variability. Traditional methods for tolerance measurement are prone to error and lack sufficient level of automation. As such, 3D imaging in construction is emerging as a powerful tool for geometric quality monitoring and discrepancy quantification (Nahangi et al. 2015a, 2015c). For measuring geometric data, 3D image (point cloud) registration is a solution for comparing the as-built status with the designed geometric status. While the as-designed status of modular components can be obtained through a computer-aided design (CAD) model and integrated with the BIM, the as-built status is obtained through the use of laser scanning or structure-from-motion systems (also known as stereo vision). The comparison of as-built and as-designed statuses is useful not only for tracking the built status of interchangeable components but also for planning the optimum aggregation sequence of interchangeable components in an assembly. This section presents an example of using 3D imaging and computer vision for optimum assembly planning of modular construction segments. Addressing the challenges related to the minimization of rework in off-site facilities and on construction sites is the ultimate goal of this section. When interchangeable modular segments are being installed and erected on construction sites, there are multiple ways to assemble the components. Furthermore, due to inevitable component geometric variability, tracking the as-built status and updating the assembly plan becomes even more challenging. Finding an assembly plan with minimum geometric deviation from the as-designed status is the key to minimizing rework related to geometric variability. In this way, the geometric deviations are systematically controlled, and therefore the rework associated with such deviations is minimized, which is the key contribution of this work. For automated and optimum assembly planning, employing 3D imaging is one reliable and accurate technique for the representation of the as-built status. For this example, laser scanning is used for data acquisition in the form of point clouds which are then imported as an input to the processing framework for calculating the optimum assembly plan. As seen in Fig. 7.8, the framework has three primary steps: (1) analyzing modular segments locally, (2) matching the segments globally, and (3) optimizing the assembly plan by minimizing the resulting geometric deviations. By minimizing geometric deviations, aggregation and erection costs are saved and schedule delays are minimized.

7.3.2.1 Identification of critical interface points in local coordinate system The global control points are first initialized from the design model. Then, critical interface points are extracted from the as-built status (point clouds) manually or automatically from the acquired point clouds. As previously explained, critical points change based on the type of module being investigated. In this framework, the extraction of critical interface points is the only manual process. However, the key objective is to automatically plan the

Use Cases for Subcontractors and Fabricators Chapter | 7

Input: 3D point cloud of modular segments

329

Optimization strategy

Calculate the coordination elements of segments locally

Approach I : Final geometric deviation minimization

Model and construct potential assembly plans

Output : Optimum assembly plan

Approach II : Rework avoidance

FIGURE 7.8 Optimum assembly framework to be used for on-site assembly optimization.

optimum aggregation of modular components because this is difficult or impossible to do manually. After capturing the point cloud, critical points are extracted in the form of the local coordinates system in which they were scanned. These points are stored in an array for further manipulation and required calculations.

7.3.2.2 Calculation of transformations for matching segments Once tie-in and control points for each segment are identified, the required transformation from local to global coordinate system must be calculated. A similar approach suggested by (Kim et al., 2013; Nahangi and Haas, 2014b) is used here for calculating this transformation. This transformation from local to G global coordinate system is denoted by ½G l T, as shown in Fig. 7.9. ½l T is then applied as follows: fPgG ¼ ½G l TfPi gl

(7.1)

In which, fPi gl is the point set that represents the tie-in points in the local coordinate system, and fPgG is the point set in the global coordinate system that matches fPi gl. ! As a homogeneous transformation, ½G l T consists of a rotational ( R ) and a ! translational ( T ) part. For calculating the rotation and translation matrix, PCA

:local system Tie-in points

Global system (Assembly coordinates)

v

Control points

Transformation from local to global coordinate system …

v

FIGURE 7.9 Illustration of the transformation to match the point clouds of locally acquired segments in the global assembly coordinate system.

330 Infrastructure Computer Vision

is used. PCA aligns the points by aligning the principal axes of the point sets. For the PCA alignment, the first step is to calculate the covariance matrix as follows: n n  T o 1X K ¼ covðpÞ ¼ pi  p pi  p (7.2) n i¼1 Where, p is the centroid of the point set p, and n is the number of points in the point set. Once the covariance is calculated, its eigenvector is calculated using single value decomposition as follows: K ¼ USUT

(7.3)

where S is a diagonal matrix, and U is the eigenvector. The eigenvector for both point sets in the local and global coordinate systems is calculated using Eqs. (7.2), (7.3). Ul and UG are the eigenvectors for the point sets in the local and global coordinate systems, respectively. The ! rotation matrix R is therefore calculated as follows: R ¼ UG  U1 l

(7.4)

T ¼ PG  R  Pl

(7.5)

! and the required translation T is calculated as follows:

In which PG and Pl are the centroids of the tie-in points in the global and local coordinate systems, respectively.

7.3.2.3 Overview of the optimization strategies The next step for optimum assembly planning is to find the best order for aggregating and erecting modular components. For this purpose, two strategies can be used: 1. minimizing rework of the final assembly by finding the sequence of components for each slot that minimizes the geometric deviation at the end point of assembly (Fig. 7.10). 2. avoiding rework that finds the best component for each segment that minimizes the geometric deviation of critical points for each slot (Fig. 7.11). For a serialeparallel assembly, both optimization strategies are applicable depending on the critical metric measured for minimizing the geometric deviation and variability. However, for a volumetric assembly, Approach I is not applicable. The reason is that in a volumetric assembly, the variability of components in various slots is independent; while the dependency of the mating parts (i.e., adjacent assemblies) is the key factor that relates the slots in the sequence of the assembly plan.

Use Cases for Subcontractors and Fabricators Chapter | 7

Model+allowable tolerances in

“tie-in” points

Calculate for nonaggregated segments

Update

Update the assembly plan by adding the identified segment

Assembly completed

No

331

Critical points in

Yes

Pick the assembly plan with minimum deviation

Report the assembly plan

FIGURE 7.10 Optimum assembly plan for minimizing the resulting final deviation and the required components.

Model+allowable tolerances in

“tie-in” points

Calculate for unassembled segments

Critical points in

Pick the segment with minimum deviation Update

Update the assembly plan by adding the identified segment

No

Assembly completed

Yes

Report the assembly plan

FIGURE 7.11 Optimum assembly plan for minimizing the resulting final deviation and the required components.

7.3.2.3.1 Approach I: minimization of overall assembly geometric deviation The first strategy takes the critical points for the assembly from the designed drawings, existing in the BIM and returns the best combination that minimizes the geometric deviation at the end of the assembly. As shown in Fig. 7.12, all possible combinations of assembly plans are evaluated. For each assembly plan, the components are matched from the start assembly point using the algorithm explained in the previous section. The corresponding assembly for each slot is transformed to the global coordinate system by the same transformation calculated previously. The tie-in points for the next slot are updated as the critical points from the previous component matched to the previous slot. This procedure is continuously performed for all of the slots until the assembly is completed. Finally, the geometric deviation is identified by calculating the deviation between the as-built and as-designed statuses. This geometric deviation is stored in the same array that the assembly is stored. Once the geometric deviation is calculated for all possible assembly plans, the

332 Infrastructure Computer Vision Comp 1

Comp n

Comp 2 …

3D point clouds of segments are captured in their local coordinate systems Designed

Resulting Deviation Built Potentially

…

…

…

…

…

… Slot 1

Slot 2

Slot n

FIGURE 7.12 Hypothetical example for optimizing the assembly plan using Approach I (minimizing final rework).

assembly plan associated with the minimum geometric deviation is extracted as the optimum assembly plan. This way, the rework associated with aligning the final critical region is minimized. Therefore, the related labor and equipment costs are minimized, assuming the rework and geometric deviation are directly proportional. 7.3.2.3.2 Approach II: rework avoidance This strategy finds the most suitable components for each slot by minimizing the incurred geometric deviation. First, the transformations for matching the components to the as-designed status are calculated for the slot being investigated. As shown in Fig. 7.13, the most suitable component (closest to the asdesigned status) is assigned to the current slot. The as-built status is then updated by calculating and updating the tie-in points for the next slot. This procedure is performed until the assembly is completed. Rather than comparing the final critical component to the as-designed status and minimizing the incurred error for that region, this strategy avoids error accumulation as the components are aggregated. In the approach explained, allowable geometric tolerances can also be controlled at each critical location (i.e., the tie-in points), and if any components are identified as being “out-of-tolerance,” it will be marked for realignment or replacement. Required realignment actions can be calculated using the approach presented by Nahangi et al. (2015a). Two case studies are provided in the following sections to show the application of optimum assembly framework for on-site assembly monitoring.

7.3.2.4 Example 1: small-scale steel bridge segments A small-scale bridge (illustrated in Fig. 7.14) was originally designed and built by an undergraduate team at the University of Waterloo. The bridge is approximately 6 m long and is comprised of assemblies which are 305 mm by

Use Cases for Subcontractors and Fabricators Chapter | 7

Comp 1

333

Comp N

Comp 2 …

…

Built Designed

FIGURE 7.13 Hypothetical example for optimizing the assembly plan using Approach II (rework avoidance).

(A)

(B)

Type A

Type A

Type A

Type B

Type A

Type A

Type A

(D)

(C)

Module type A

Module type B

FIGURE 7.14 Example 1. (A) actual image, (B) DT of the assembled bridge, geometric dimensions of modules Type A (C) and Type B (D). Dimensions are in mm.

152 mm by 102 mm. The bridge was designed in three types of modules (nine modules in total) that are bolted together into a parallel system. The bridge contains six Type A modules (Fig. 7.14C) and one Type B module (Fig. 7.14D). The third module type (legs at the ends) was not considered in this example for simplicity. The bridge was aggregated into Type A and Type B modules as shown in Fig. 7.15. Although the six Type A modules should be theoretically interchangeable, fabrication process capabilities introduced geometric variability, impacting the degree of interchangeability. As a result, depending on the

334 Infrastructure Computer Vision

(A)

(B)

(C)

FIGURE 7.15 Assembly planning for Example 1. (A) Six interchangeable modules of Type A are scanned and critical points are extracted. (B) Critical points in the DTs are extracted at each slot. (C) Module Type B is installed in the Slot 4, which makes a constraint in the assembly planning and the optimization problem involved.

specific module assembly plan, gaps can be introduced between interfaces, causing the bridge to have different overall lengths. Point clouds of the entire assemblies are acquired and all critical points are extracted from the 3D point clouds. On the other hand, the coordinates of the critical interface points from the as-designed status are also extracted for calculating the geometric deviation from the as-built status and planning for the optimum assembly. Fig. 7.16 shows typical examples for the extraction of critical points for assembly Type A in this example. Once the critical points are extracted, the implemented optimization Approaches I and II (explained in the methodology) are applied on the modular bridge components. Both approaches are applicable on the modular bridge as the geometric deviation at each slot or the final segment may be critical. The results of the optimization Approach I are shown in Fig. 7.17. 720 possible

(A)

(B)

: local coordinate system

: reference frame for assembly

FIGURE 7.16 Critical points extraction for the DT (A), and laser scanned point cloud representing the as-built status (B).

Use Cases for Subcontractors and Fabricators Chapter | 7 All assembly plans

Acceptable plans

335

Allowable tolerance limit

80

Deviation (mm)

70 60 50 40 30 20 10 0 0

60

120

180

240

300

360

420

480

540

600

660

720

Assemply plan #

FIGURE 7.17 The geometric deviation for all feasible assembly plans for Example I. Number of possible assembly plans equals the permutation of six interchangeable modules (6! ¼ 720). The threshold value for acceptable deviation is considered 15 mm, and all 51 assembly plans are identified as red points. The best five assembly plans are reported in Table 7.1.

assembly plans are considered, and the resultant deviation at the final segment is measured by comparing the resulting critical points to the design drawings. As seen in Fig. 7.17, the geometric deviation changes from 12.9 to 68.7 mm for various assembly plans. A threshold value of 15 mm is considered as the acceptable variation limit (tolerance). As such only 51 assembly plans are deemed acceptable based on the variation limit. Of these 51 assembly plans, the 5 best plans are reported in Table 7.1, where the most suitable assembly plan can be chosen based on existing constraints in the fabrication plant which were not considered in the optimization step. The most suitable assembly plan using Approach II for the optimization step is also reported in Table 7.1.

TABLE 7.1 Summary of the result for assembly planning applied on Example 1. Optimization strategy

Assembly plan

Deviation (mm)

Processing time (sec)

Approach I

AP 1 : f6; 1; 2; 4; 7; 5; 3g

12.88

80.54

AP 2 : f5; 6; 2; 4; 1; 3; 7g

12.95

AP 3 : f6; 5; 2; 4; 7; 3; 1g

12.97

AP 4 : f6; 3; 2; 4; 1; 5; 7g

12.99

AP 5 : f6; 2; 1; 4; 7; 5; 3g

13.15

AP : f6; 2; 1; 4; 7; 5; 3g

13.15

Approach II

0.51

336 Infrastructure Computer Vision

TABLE 7.2 Summary of the processing time at each step for example 1. Data collection

w15e20 minutes to scan each module w1e2 minutes to probe the critical points of each module

Processing time

w80 seconds for Approach I <1 second for Approach II

Total

w15 minutesa

a

Probed points are used for critical interface points.

One notable observation is the time-effectiveness of the optimization Approach II. As seen in Table 7.1, the processing time for the optimization Approach II is 0.51 seconds which is significantly faster than the total processing time for the Approach I (80.54 s). If it is preferable to reduce the overall geometric deviation of the bridge by choosing an assembly plan which has the least amount of gaps between components, and an overall bridge length which matches the design length best, then Approach I should be employed. If, however, the overall geometric deviation of the bridge is less important as minimizing the overall amount of rework associated with aggregation, then Approach II should be employed. Time-related aspects of the Example 1 are summarized in Table 7.2. The benefit of using either optimization Approach I or II can be expressed in terms of a minimization in rework. Assuming that the overall deviation associated with aggregating the components in the bridge is required to be less than 15 mm without incurring rework, then the probability of rework is equal to 92% (because 669 of the total 720 assembly plans result in an overall deviation greater than 15 mm). Furthermore, assuming the average amount of rework to be 4 hours for cutting, grinding, and rewelding of components to ensure adequate assembly geometry, then the time savings is approximately equal to 3.4 hours, or 86% of the total time required for rework.

7.3.2.5 Example 2: full-scale concrete panels and floor decks This example covers the installation of precast concrete panels in steel frames, as part of the floor system for a modular data center. The precast concrete panels were cast into 16 light-gage steel frames (Fig. 7.18B) and then aggregated into a steel floor frame (Fig. 7.18C). Five different sizes of concrete panels were used; each 102 mm thick, 2559 mm long, and vary in width. The dimensions of the concrete panels used in this example are summarized in Table 7.3. The floor frame contains 11 slots for concrete panels with cross bracing to support the sides of each panel. The panel types must be placed in the order specified in Fig. 7.18A, for the panels to be properly supported on all sides. The designers accounted for anticipated geometric deviations by specifying a

Use Cases for Subcontractors and Fabricators Chapter | 7

337

(A)

(B)

(C)

(E)

(D)

FIGURE 7.18 Example 2: ull-scale concrete panels. (A) Assembly. floor frame (B), and the stack of the concrete panels (C). Laser scan of the floor frame (D), and the stack of concrete panels (E).

TABLE 7.3 Dimensional properties of the concrete panel types used for Case II (illustrated in Fig. 7.18). Concrete panel type

Dimensions (mm)

Type a

13672559

Type b

14972559

Type c

13932559

Type d

10582559

Type e

11622559

3 mm gap between all the panels and the frame. However, as the result of fabrication process capabilities, geometric deviations occasionally resulted in panels being too large or too small to easily fit into the steel frame. In the case of panel misfit, additional work was required to correct geometry. One correction strategy could have been to substitute interchangeable panels with each other to optimally match panel tie-in points with the frame control points. For each steel frame, there are 288 distinct assembly plans which could have been explored, thereby increasing the aggregation flexibility in ensuring adequate panel fit.

338 Infrastructure Computer Vision

TABLE 7.4 Summary of the results for assembly planning of the concrete panels used in example 2. Assembly type

Assembly plan

Processing time

Average deviation at each critical point

Type a

a-1, a-3, a-2

2.23 seconds

1.3 mm

Type b

b-3, b-2, b-4, b-1

Type c

c-1*

Type d

d-2, d-1

Type e

e-1*

As mentioned earlier, volumetric assemblies utilize optimization Approach II only. Each assembly type is investigated for the associated slots in the design. The geometric deviation of the resulting plan from the as-designed status is minimized for each assembly type, with the constraint that each assembly has an allowable variation limit (tolerance) of 3 mm. In case that any assembly plan results in greater values than the allowable threshold, the assembly plan is ignored. The explained procedure is implemented and modified for the concrete panels and the floor frame investigated in this example. Key results are reported in Table 7.4. As seen in Table 7.4, the procedure for assembly planning is time-effective and the average deviation at each critical point is less than the acceptable threshold (3 mm). The processing time including data collection, manual manipulation, and assembly planning optimization for this case study is summarized in Table 7.5.

TABLE 7.5 Summary of the processing time at each step for the example 2. Data collection

w28 minutes for the floor frame (three scans) w16 minutes for the panels (two scans)

Preprocessing (critical interface point extraction)

w20 minutes for both the panels and the frames

Processing

w2e3 seconds

Total

w1 hour

Use Cases for Subcontractors and Fabricators Chapter | 7

339

The benefit of using optimization Approach II in this example can be expressed in terms of a minimization of rework. While the time required to optimally plan the assembly is on the order of 1 hour for an entire frame of 11 concrete panels, the time associated with rework is challenging to calculate precisely. However, from observing the fabrication crew, it can be reasonably estimated that the impact of rework is low to medium with a schedule impact on the order of 5 man-hours per concrete panel. If only a single panel requires rework, the time savings is equal to approximately 4 hours or 80% of the total time required for rework. Because of the high quantity of floor frames in this building (16 frames and 176 concrete panels), utilization of the proposed optimum assembly planning framework can be extremely beneficial for reducing rework associated with geometric variability.

7.4 Fabrication control As new technologies emerge and progress within the construction industry, it is becoming favorable to overcome technical challenges through the application of virtual tools and simulation techniques. The use of BIM, for instance, can be used for making vast construction process improvements (Azhar et al., 2015). While there are often large upfront costs associated with the acquisition and use of virtual tools and simulation techniques, they are beneficial for analyzing large amounts of data and simulating expected interactions and outcomes at a fraction of the cost and time of traditional methods (AbouRizk, 2010). These large cost and schedule savings are largely the result of approaching the design and construction process in a proactive manner, where the impact of encountering logistical conflicts and challenges is far less than encountering them during construction. A current challenge faced by many contractors whose work involves prefabrication is how to mitigate challenges associated with dimensional variability (DV). It is rather common to see contractors in the commercial and industrial construction sectors who frequently face geometric and tolerancerelated conflicts associated with fabrication, assembly, fit-up, and erection of prefabricated assemblies. Rausch et al. (2017) provide several detailed examples of tolerance-related challenges faced on projects involving prefabrication and modularization. These examples, along with other research (Milberg and Tommelein, 2009), demonstrate the ongoing challenge within construction for addressing DV in a systematic way. To approach the current state of DV management in construction, researchers are exploring how analytical tools and methods primarily used in the manufacturing industry can be applied to industrialized construction. Dimensional variation analysis (DVA) has emerged as a promising method for mitigating the impact of DV. While dimensional tolerances for prefabricated structures should comply with values outlined in governing standards (AISC, 2010), a range of

340 Infrastructure Computer Vision

additional resources are often relied on, because baseline tolerances from standards may not be strict enough in some cases for ensuring adequate alignment between prefabricated assemblies. Engineers and designers have to rely on tacit knowledge, libraries of case-specific tolerances, and/or ad-hoc strategies to derive tolerances for prefabricated structures. As these resources do not always produce adequate tolerances, assembly geometry is still corrected during construction rather than being proactively addressed during design (AISC, 2005). The current state of tolerance specification can be described as an inefficient and reactive process, which is further compounded by the fact that associated rework typically delays activities along the critical path of a project.

7.4.1 Compatibility between tolerances and process capabilities Process capabilities define the expected variation of a given process (e.g., steel frame welding, rebar placement, concrete pouring, component alignment, etc.) which in turn can be used to determine its probability of not exceeding required tolerances. Compatibility between processes and tolerances is important for ensuring that an assembly can be fabricated, assembled, and installed on-site correctly. For instance, if the length of a steel beam must have a tolerance of 3 mm (1/800 ) to fit on-site properly, the processes affecting the length of that beam (e.g., cutting, measuring, grinding, etc.) must cumulatively have a variation less than 3 mm (1/800 ). In this case, the tolerance can be divided (or absorbed) between compounding processes; however, because each process has its own specific capability (i.e., DV), the net variation of processes must be less than the specified tolerance for the length of the beam (Fig. 7.19A). This example describes a design approach referred to as tolerance allocation, where overall assembly tolerances are distributed to the underlying processes of the assembly. The reverse design approach is referred to

FIGURE 7.19 Examples of tolerance design approaches: (A) tolerance allocation for the variability on the size of a steel beam and (B) tolerance analysis for the design of a connection for a curtain wall system.

Use Cases for Subcontractors and Fabricators Chapter | 7

341

as tolerance analysis and occurs where process capabilities are analyzed to derive a suitable overall assembly tolerance. For instance, to determine the amount of adjustability (i.e., tolerance) required for the connection of a prefabricated curtain wall system, the variability of the underlying building substrate as well as positional variability of the curtain wall must be analyzed to derive suitable tolerances (Fig. 7.19B). Both cases of tolerance analysis and tolerance allocation require some knowledge about the capabilities of processes in terms of their DV. While it may be difficult to determine the DV of construction processes, Milberg and Tommelein (Milberg et al., 2002) demonstrate that failure to consider process capabilities can result in severe conflicts during installation on-site.

7.4.2 Existing methods for dimensional variation control in construction Current methods for analyzing, detecting, and controlling DV in construction can be done throughout the life cycle of a project (i.e., during design, fabrication, or assembly on-site). Although variation control during fabrication and assembly on-site can utilize proactive 3D analysis techniques such as spatial change analysis (Tang et al., 2016) or automated compliance checking (Nahangi and Haas, 2014a), the majority of variation control techniques during construction are performed in a reactive manner (i.e., problems related to DV are solved only once they have occurred). Proactive methods for DV control are typically only considered during the design stage through the use of BIM. Clash detection is an example of a proactive approach for detecting and resolving DV conflicts, as discussed earlier. Proactively resolving dimensional conflicts is superior to reactive methods, which is a large reason why approximately 90% of commercial contractors are currently using BIM-based clash detection on projects which utilize a BIM (Farnsworth et al., 2015). In a study by Leite et al. (Leite et al., 2011), it was shown that field-detected clashes are more costly than the extra time spent upfront in modeling a more detailed BIM. For this reason, spending more time during the design to detect and avoid clashes can offset the cost associated with field rework associated with DVs. Within clash detection, there are three types of clashes (or dimensional conflicts): (1) hard clashes, where two components occupy the same space, (2) soft clashes, where there is limited or insufficient space for access, and (3) logical clashes, which include constructability problems (Eastman et al., 2011). While the automatic detection of dimensional conflicts in a construction assembly is extremely powerful and can save upward of millions of dollars on a given project (Vico Software, 2016), BIM-based clash detection does not instruct construction crews on how to resolve dimensional conflicts. Accordingly, contractors often avoid dimensional conflicts by leaving adequate clearance envelopes around components in the BIM using standardized tolerances. This approach requires that

342 Infrastructure Computer Vision

contractors use their experience or a priori knowledge to know how to adequately specify acceptable tolerances. As such, this approach can be problematic for the assembly of construction components, which often require direct contact between components, with little to no gaps. In this section, two major methods for DVA in the construction are discussed: (1) deviation mapping and (2) kinematics chainebased DVA.

7.4.2.1 Deviation mapping for dimensional variation analysis Tolerance mapping can be very challenging to use for analyzing the effect that DVs have on a construction assembly. In this section, the overall concept of tolerance mapping is presented in the context of a DV analysis to demonstrate how it can be tedious and complex to use. For this purpose, it is herein referred to as deviation mapping because the deviations of component features and components are analyzed rather than their allowable tolerances; however, it follows the exact same approach as used in tolerance mapping. Deviation maps are challenging because without proper selection of only critical component features and critical components in an assembly, they become extremely detailed and unnecessarily large. Secondly, the notation used in deviation maps requires an extensive knowledge about geometric dimensioning and tolerancing (GD&T), which is not intuitive for engineers and designers in the construction industry. The general procedure for creating a deviation map can be described in three steps: (1) create an assembly network to define how all parts are geometrically related to each other in an assembly, (2) create component diagrams to define how all component features are geometrically related to each other in each component, and (3) amalgamate all component diagrams in the assembly network to create the overall deviation map. To demonstrate the creation of a deviation map, a simple example is shown, which outlines the geometric relationships of all component features for a steel component in a modular steel bridge (Fig. 7.20). Three variation categories are employed in deviation mapping to define the relationship between component features and components within an assembly. These categories are based on GD&T notation: (1) orientation and location variations, which define a component feature’s spatial state, (2) form variation, which defines how straight, flat, or round a component feature is, and (3) size variations which define two-point measurements of a component feature. For the assembly diagram, typically only orientation and location tolerances are used, because the assembly diagram defines how the subcomponents or parts are spatially related (Fig. 7.21). The creation of component diagrams and the overall deviation map (Fig. 7.22) follows the same approach taken for the assembly diagram (i.e., component features are geometrically related using GD&T notation). Details related to these figures can be found in (Rausch, 2016).

Use Cases for Subcontractors and Fabricators Chapter | 7

343

(A)

(B)

(C)

FIGURE 7.20 Structural component used as an example to demonstrate complexity of deviation mapping. (a) Structural assembly of component. (B), (C) Location of component in overall modular bridge.

(A)

(B)

(C) Xo Yo Zo X L YL Z L

Form Tolerance

F X

Location & Orientation Tolerances

Y

Z

Size Tolerances

FIGURE 7.21 Example of steps involved with creating an assembly diagram for a single structural assembly. (A) A dimensioned drawing for an assembly is broken down into its subcomponents using (B) geometric dimensioning and tolerancing notation to create (C) an assembly diagram.

As shown Fig. 7.22, the use of deviation mapping as a form of DVA is not practical because it is tedious to setup, and without a proper selection of critical DVs, it becomes extremely detailed, even for simple structural components. This justifies the need for a more systematic and generalized approach for modeling DVs in construction assemblies.

344 Infrastructure Computer Vision

1. Assembly Diagram

3. Deviation Map

2. Component Diagrams

FIGURE 7.22 Example of steps involved for creating a deviation map: (1) create the assembly diagram, (2) create diagrams for each component. and (3) amalgamate all component diagrams into the assembly diagram to obtain the overall deviation map.

7.4.2.2 Kinematics chainebased DVA Using robotics concepts and theories has opened up a wide and efficient range of solutions in engineering problems. For example, kinematics theories have been used for state modeling and sensing of construction equipment such as pipe manipulators and excavators (Cho et al., 2004). A specific pose of the end effector (i.e., the end of a kinematics chain, which is the critical feature of interest) can then be modeled using related inverse and forward kinematics. Robotics concepts, and more specifically kinematics theories, can also be used for automating tasks associated with a high level of repetition or harsh tasks that are performed by workers in hazardous areas. For instance, a machine visioneassisted system was developed by (Feng et al., 2015) for automating the task of bricklaying assembly in a prefabricated environment. Comparatively, kinematics theory was used for modeling the geometry of construction assemblies as a mathematical function (Nahangi et al., 2015c). Discrepancies of the as-built state of construction assemblies are therefore quantified (via forward kinematics) and required corrective actions are then calculated (via inverse kinematics) (Nahangi et al. 2015a, 2015b). This is going to be discussed in Section 7.5. Using kinematics chains for identifying the geometric relationships has been found to be very effective for integrating parametric models for systematic and electronic monitoring of civil infrastructure. For modeling the geometric relationships of different segments of an assembly, the kinematics chain is developed using the analogy of robotics discussed earlier. For developing the kinematics chain, a similar approach to Nahangi et al. (2015c) is employed. Transformations are then derived using the DenaviteHartenberg (D-H) convention (Denavit and Hartenberg, 1955).

Use Cases for Subcontractors and Fabricators Chapter | 7

y Axis i-1

345

Axis i z x

Link i-1

z

y

Link i

z

x

y x

FIGURE 7.23 D-H convention and parameters for geometry modeling. D-H parameters are used to relate the geometric relationships of an assembly.

While it is possible to use any consistent convention for the derivation of the transformations, the D-H convention is a systematic method that can be programmed and integrated with other components of the proposed framework. D-H parameters represent any homogeneous transformation as a combination of four transformations, as illustrated in Fig. 7.23. Of these four transformations (illustrated in Fig. 7.23), two are rotational and two are translational transformations as follows:      Ti ¼ Rotz;qi Transz;di Transx;ai Rotx;ai 32 32 2 1 0 0 0 1 cqi sqi 0 0 76 76 6 6 sqi cqi 0 0 76 0 1 0 0 76 0 76 76 6 ¼6 76 76 76 0 0 1 di 76 0 6 0 0 1 0 54 54 4 2

0 cqi

6 6 sqi 6 ¼6 6 0 4 0

0 0 1 0 0 0 1 3 sqi cai sqi sai ai cqi 7 cqi cai cqi sai ai sqi 7 7 7 sai cai di 7 5 0

0

0

0 0 ai 1 0 0 1 0 0

32

1

0

76 6 07 76 0 cai 76 6 07 54 0 sai 1 0 0

0 sai cai 0

0

3

7 07 7 7 07 5 1

1 (7.6)

in which qi , di , ai , and ai are parameters associated with link i and joint i (Fig. 7.24). cb and sb denote cos b and sin b, respectively. The four parameters qi , di , ai , and ai are also known as “link length,” “link twist,” “link offset,” and “joint angle,” respectively. Generally, two types of joints can define the characteristics of an assembly connection (Fig. 7.24): 1. Rotational joints: these are considered where DV can occur in the form of rotation. Rotational joints are also known as revolute joints.

346 Infrastructure Computer Vision Combined joint: 1 translational joint + 1 rotational joint

FIGURE 7.24 Schematic of a hypothetical joint. The joint is comprised of one translational joint and one rotational. The value di is variable for translational connections, and qi is variable for rotational connections.

2. Translational joints: these are considered where DV can occur in the form of translation or offset. Such joints are also known as prismatic joints.

7.5 As-built surveying, modeling, alignment, and fitting for off-site fabrication As-built modeling is a process by which dimensions of an existing assembly are measured and input into an as-built model for reporting and analyzing the products’ compliance with contractual requirements (Patraucean et al., 2015). Currently, the input data for this process are collected predominantly using manual direct contact metrology. Conventionally, QC personnel use measuring tapes, calipers, custom gauges, squares, and straightedges to collect data that then feeds into an evaluation of whether assemblies are compliant with design specifications. The effectiveness of these tools decreases as the assembly’s geometrical complexity increases because manual measurement is subjective, prone to error, time-consuming, costly, and discontinuous. New tools have emerged for encoding dimensional information about visible building and spatial elements in a way that is reliable, accurate, and continuous (Bhatla et al., 2012; Dai et al., 2013). These tools or 3D reconstruction methods include imaging (camera) systems for photogrammetry (Jahanshahi et al., 2009; Liu et al., 2014) or from (time-of-flight (TOF) or phase-shift) laser scanners and create datasets in the forms of point clouds or points endowed with 3D Cartesian coordinates (Son et al., 2015; Tang et al., 2010). Recent studies on MEP components, in particular, include automated detection (Ahmed et al., 2014), progress tracking (Bosche´ et al., 2014), segmentation (Dimitrov and Golparvar-Fard, 2015), and dimensional analysis (Nahangi and Haas, 2014b). In this section, a framework developed by Nahangi et al. (2015a) for (1) automated comparison of as-built status (3D images acquired) with as-designed status (3D CAD models integrated with BIM) and (2) automated calculation of

Use Cases for Subcontractors and Fabricators Chapter | 7

DOF

Kinematics chain

BIM

As-built

Detected discrepancies

Localized & quantified discrepancies

∑

Corrective actions

347

Plumbed structure

Repeat Step 1

FIGURE 7.25 Framework for realignment of defective construction elements and assemblies on construction sites.

the required realignment is presented. Fig. 7.25 shows the framework for automated discrepancy quantification and realignment of defective assemblies proposed to be used by fabricators during assembly or construction.

7.5.1 Model versus built comparison for automated discrepancy quantification As mentioned earlier, the first step for a control system is to capture the initial geometric arrangement of that system. In construction, the current geometric arrangement is also known as the as-built status. For identifying the as-built status of the assemblies and structural frames, a combination of 3D imaging and robotics theories is used. In summary, the as-built status identification includes preprocessing, registration, and local discrepancy identification.

7.5.1.1 Preprocessing The algorithm presented in this approach takes a 3D image and DT of an assembly as inputs, quantifies the discrepancies incurred, and characterizes the initial status. Preparation of the inputs for the primary processing engine is generally called preprocessing. In particular, this framework requires the following inputs and the corresponding preprocessing steps: 1. As-built status: The framework requires a 3D point cloud representing the as-fabricated status of the assemblies. The point cloud (3D image) is acquired using one of the many 3D imaging technologies such as laser scanning, photo/videogrammetry, range imaging, etc. 2. Building information model (BIM): CAD models that are integrated with BIMs are also required to identify the desired design state of the assemblies being investigated. 3D CAD models are usually in the form of solid objects and therefore should be converted into a point cloud (e.g., *.STL format in most cases) for performing the comparison required for capturing the initial state. To have a resampled point cloud that appropriately represents the DT, an evenly distributed point cloud should be generated. 3. Degrees of freedom (DOFs): For actuation and realignment planning, a set of DOFs are required. The DOFs are defined as joints where manipulations can be applied in the form of either a twist for rotational or lengthening/

348 Infrastructure Computer Vision

shortening for translational DOFs (joints). The potential DOFs are identified based on the geometry, feasible realignment, and feasible applicable actions. Then the analogy of robotics is applied and the kinematics chain is developed. The development of the kinematics chain is based on D-H convention (Denavit and Hartenberg, 1955).

7.5.1.2 Registration For the purpose of comparing the point clouds representing the as-built and asdesigned states, a modified iterative closest point (ICP) algorithm is used. In principle, the modified ICP algorithm used here is similar to the methods explained in Nahangi et al. (2015c). In contrast with the modified ICP algorithm used for serial manipulators, the registration approach used in this work considers the possibility of existing multiple origins. The summary of the modified ICP algorithm is shown in Fig. 7.26, where S is the as-built point cloud and M is the model point cloud. Typical results of the constrained registration compared with regular registration are shown in Fig. 7.27. Rather than the point cloud format which is required for the described registration method, solid object format is used in Fig. 7.27 for better presentation of the results. 7.5.1.3 Local discrepancy quantification Once correct registration is applied considering the local positions and orientations, discrepancies can be reliably quantified. For quantifying the discrepancies incurred, a local registration is performed. The local discrepancy quantification procedure is illustrated in Fig. 7.28. While the local ICP is performed on the subclusters defined by each cube position, the discrepancies are measured in the global frame. The rotational and translational discrepancies are transformed and measured in the local frames using a transformation chain. The transformation chain will then be used in the calculation of the eventual required joint manipulations as well.

Apply T on M and S

Regular ICP on M and S

Define the origin frame (s)

Identify m and s

Find T such that registers s and m

Locally aligned M and S

Size of neighbourhood region

FIGURE 7.26 The modified iterative closest point (ICP) for registering as-designed and as-built point clouds.

Use Cases for Subcontractors and Fabricators Chapter | 7

(A)

Gray: Target state (designed)

Discrepancy

(B)

Red: As-built state

349

Discrepancy

Refine the registration

Connecting platform

Fixed frames as the origin platform

FIGURE 7.27 Steps to perform modified registration. Regular registration is initially performed (A) The origin frames (origin platform) is fixed, based on the consideration that the origin platform is connecting to the adjacent module (B) Discrepancies resulting from two registration steps are compared.

(A)

(B)

(C)

FIGURE 7.28 Local discrepancy quantification. (A) A 3D cube moves along the assembly; (B) identifies the contained points from Model M and Scene; (C) the local discrepancies are transformed to the local frames.

7.5.2 Realignment calculation The system of equations resulting from defining the DOFs (G), fixed platform, and target points has no general and closed form solution (Tsai, 1999). Numerical approaches should be employed for solving the resulting nonlinear system of equations. The general approach to solve the resulting nonlinear systems of equations is similar to the approach for serial systems developed in Nahangi et al. (2015b). The linearization approach makes use of Taylor ! seriesebased expansion of the kinematics chain equation P ¼ FðGÞ. A new function E is defined for the discrepancy function as follows: EðGÞ ¼ FðGÞ  Pg ;

(7.7)

350 Infrastructure Computer Vision

in which Pg is the desired configuration that contains the desired target positions at each target point defined. The inverse kinematics problem is then simplified to solving the system of equations EðGÞ ¼ 0. For solving this nonlinear system, the discrepancy function is approximated using linearization by the Taylor-based expansion employed. An initial start point is required to start the iterative solution. The equation is then iteratively solved for the un! known vector which is G . The procedure is also known as the Newtone Raphson method (Broyden, 1967). A form of linearization in a multidimensional space suitable for ultrageneral cases is used in this chapter. The numerical solution for the inverse kinematics problem is summarized below. The kinematics function defining the target positions is linearized using Taylor expansion as follows: EðGiþ1 Þ ¼ EðGi Þ þ

vE ðGiþ1  Gi Þ ¼ 0 vG

(7.8)

In which, Gi is the initial state of the assembly in iteration i. Substituting vE=vG ¼ vF=vG with J, where J is the Jacobian matrix, and rewriting Eq. (7.8) results in E ¼  J  DG

(7.9)

Premultiplying both sides of Eq. (7.9) with the pseudoinverse of the Jacobian matrix (Jþ ) results in DG ¼  Jþ  E

(7.10)

Among existing forms and solution for pseudoinverse matrices, the MooreePenrose pseudoinverse (Penrose, 1955) is used here to implement the mathematical solution of the inverse kinematics problem. The MooreePenrose pseudoinverse is defined as follows: 1

Jþ ¼ ðJT JÞ JT

(7.11)

in which T is the transposition operator. More detail and explanation on this discussion and pseudoinverse matrices can be found in Penrose (1955), Weisstein (2014). As mentioned earlier, the explained procedure is also known as NewtoneRaphson method, in general. In the ultrageneral cases where the Jacobian matrix is very likely to be noninvertible (i.e., nonsquare), a pseudoinverse is used rather than the regular inverse in the NewtoneRaphson method. In general cases, for solving the resulting nonlinear system of equations being investigated, the method is called the Quasi-NewtoneRaphson (QNR) method (Broyden, 1967; Dennis and More´, 1977). Other methods to solve the nonlinear system of equations EðGÞ ¼ 0 include the “continuation” and “homotopy” methods. For more detail on these methods and solving the nonlinear system of equations resulting from the inverse kinematics problem, see Garcia and Li (1980), Li (1997), Sommese et al. (2004), Tsai (1999),

Use Cases for Subcontractors and Fabricators Chapter | 7

Capture the initial state

Calculate the realigned state

Calculate the initial configuration

Calculate the desired change

Calculate the realigned configuration

+

351

Calculate the required change

Report the total changes required +

FIGURE 7.29 Flowchart to solve the inverse kinematics problem EðGÞ ¼ 0 for parallel systems.

Uchida and McPhee (2011), Wampler et al. (1990). Fig. 7.29 shows a flowchart for solving the resulting equation for the inverse kinematics problem using the QNR method. As a mathematical note, one obvious solution of Eq. (7) is ðG0 Þ, when Pg equals the originally design state input based on the discrepancies calculated. In other words, the required change of a system (both serial and parallel) to achieve the original design state equals the reverse of the initial discrepancies ðG0 Þ calculated based on the comparison with the original design state. However, the desired state and thereby desired configuration are continually getting updated based on the changes in the interfaces involved. Such an unavoidable and continual change motivates the challenging concepts of “asbuilt modeling” and “as-built BIM” in construction and therefore necessitates continual maintenance and monitoring of construction components. Hence, it is unlikely that the specific case explained here exists in real projects; however, the mathematical solution explained can handle that case as well. A practical application of the preceding method would be to steel erection. Initial bolt-up results usually in a nonsquare frame. Squaring the frame is a time-consuming process that typically takes 25% of the total erection time, and it is an iterative, intuitive process in practice, like leveling a total station with a set of screws, but much more complicated in that it uses cables, comealongs, and turnbuckles. Being able to scan a frame, and then quickly calculate the actions required to square the frame, could save substantial time. The framework described is also useful in robotic alignment, fitting, and assembly.

7.6 Replacing the tape measure 7.6.1 Summary from the examples provided This chapter introduces possible solutions for automating the tasks of fabrication and assembly. Section 7.3 presents some of the most recently developed production monitoring techniques using laser scanning, photogrammetry, and 2D images for assemblies being aggregated both off-site and on-site. For offsite production monitoring, it is more focused on studies that employ laser scanning technique to capture and assess the dimensional as-built state of

352 Infrastructure Computer Vision

individual construction components including precast slabs, steel girders. For on-site production monitoring, an optimum strategy for the assembly of modular components is explained. Optimization of the assembly plan will help the contractors to stay as close as possible to the designed drawings and specifications to avoid falling out of tolerance and therefore incurring huge costs. Section 7.4 presents existing approaches for dimensional assessment of construction components. Traditional approach (deviation mapping) and a vision-based approach combined with robotics theories for identifying the DV of construction components are explained and examples are provided. Section 7.5 streamlines an automated framework for discrepancy detection and quantification of defective industrial assemblies. Realignment calculation is then explained using the quantified discrepancies and devising an inverse kinematics analogy for fixing the defective assemblies.

7.6.2 The potential of adopting the technologies in fabricators’ practice The ultimate goal of adopting sensing technologies in manufacturing sites is to eliminate the use of error-prone tools such as tape measurement for the process of quality inspection. To investigate the possibility of the replacement for the primary tasks of fabricators, identification of sensing technologies suitable for the main tasks is necessary. Table 7.6 illustrates typical technical specifications of currently available 3D measurement sensors in the construction industry. 3D laser scanners have a high measurement accuracy, and its measurement range is large. Compared to the other 3D measurement technologies including scanarm laser scanner, photogrammetry, and TOF camera, laser scanning approach can cover a bigger area with high measurement resolution, indicating that it is a versatile solution for the quality inspection and control of fabrication and assembly tasks. A practical outcome of this is that large, off-shore oil platforms are fabricated using 3D laser scanning and/or scan-arm laser scanning for dimensional control. In literature, previous studies (Kim et al. 2019a; Dai and Lu, 2010; Dai et al., 2013; Golparvar-Fard et al., 2011) conducted a performance comparison on the as-built dimensions of construction components between laser scanning methods and vision-based approaches. The results of those studies show that image-based methods including photogrammetry and videogrammetry have a measurement accuracy of 1e10 cm, which is not acceptable for most fabrication tasks requiring an accuracy of less than 1 cm. In addition, it was found that the performance of the vision-based approaches normally deteriorates due to the quality of the photos and poor lighting conditions (Dai and Lu, 2010). Based on these technical specifications and literature findings, it can be concluded that laser scanning approaches are more suitable than vision-based methods from the perspectives of speed, coverage, and accuracy. For some tasks of fabricators such as welding inspection which requires a high measurement accuracy less

Property

3D laser scanner (E.g., FARO Focus-3D)

ScanArm laser scanner (E.g., FaroArm)

Stereo camera (E.g., Pointgrey bumblebee2)

Time-of-flight camera (E.g., MESA SR4000)

Accuracy

2 mm @ 20 m

0.024 mm @ 1.8 m

2 mm @ 2 m

20 mm @ 5 m

Measurement range

0.6e120 m

0.1e4 m

w4 m

0.1e10 m

Measurement angle

Horizontal 360 degrees/ vertical 310 degrees

-

Horizontal 66 degrees/43 degrees

Horizontal 44 degrees/35 degrees

Measurement speed

960,000 points/seconds

560,000 points/ seconds

48 frames/seconds

30 frames/seconds

Resolution

Depends on angular resolution

Minimum point spacingd40 mm

Pixels 648  488

Pixels 176  144

Use Cases for Subcontractors and Fabricators Chapter | 7

TABLE 7.6 Comparison of technical specifications of 3D measurement sensors.

353

354 Infrastructure Computer Vision

than 1 mm, this technology, however, may not be used. A recent study (Rodrı´guez-Gonza´lvez et al., 2017) shows that microphotogrammetry technology can provide an accuracy of less than 1 mm for welding inspection, which can be an alternative solution for the primary tasks of fabricators. Table 7.7 summarizes the potential sensing technologies that can be adopted in the manufacturing and fabrication stage based on the examples and findings discussed in Sections 7.3e7.5. For the analysis, allowable tolerances of the tasks conducted by fabricators are identified based on standard tolerance documents (Ballast, 2007) for potential evaluation. For formwork fabricators, the tolerances for the primary inspection tasks such as dimensions and positions of an object are within a range of  5e10 mm. Although there have been few previous studies (Akula et al., 2013; Golparvar-Fard et al., 2011) on the dimensional inspection of formworks, no measurement accuracy is provided in those studies. However, based on the examples where laser scanning technology is adopted, it can be expected that laser scanning is a feasible solution

TABLE 7.7 Potential sensing technologies for fabricators/assemblers. Fabricators assemblers

Target for quality control

Tolerance required

Potential sensors to replace tape measurement

Formwork fabricators

Dimension of complete formwork

6 mm

Laser scanner

Dimension of penetration/opening for services

10 mm

Clear(cover) distance to side forms

5 mm

Distance between reinforcement

6 mm

Dimension of complete PC elements

3e10 mm

Position of opening

6 mm

Warping

1.5/300 mm

Welding

Class B e  2 mm Class Cd  3 mm

High-resolution microphotogrammetry Or a fixed-based laser scanner such as the FARO gage arm

Cross section

5 mm

Laser scanner

Precast concrete fabricators

Structural steel fabricators

Laser scanner

Use Cases for Subcontractors and Fabricators Chapter | 7

355

because around 2 mm accuracy that is far within the allowable tolerances for each dimension checklist was achieved in the similar previous studies (Kim et al. 2014a; Wang et al., 2017). In addition, based on a comparison test conducted in a previous study (Akula et al., 2013), the laser scanning technology provides a better reinforcement bar recognition than image-based photogrammetry technology. Therefore, it can be concluded that laser scanning is the best option against the other sensing technologies for the formwork dimension QC. By adopting laser scanning systems, subcontractors and fabricators can automate the geometric QC process and increase the productivity and reliability in the manufacturing process. Laser scanning can be also a promising solution for precast concrete fabricators. Based on the previous studies (Kim et al. 2014a; Kim et al., 2016 and Wang et al., 2016a), laser scanning can provide an accuracy of better than 2.0 mm for the dimension and position estimation of precast concrete panels. In addition, according to the sequential studies (Wang et al., 2016a and Kim et al. 2019b), it was found that laser scanning approach can achieve a dimensional estimation accuracy of less than 2.0 mm on precast concrete elements with more complex geometry that is required for the connection with adjacent elements. Furthermore, surface flatness and distortion on precast concrete elements were successfully measured with an accuracy of 1.5 mm (Wang et al., 2016b). These results indicate the potential of using laser scanners for the automation of dimensional inspection of precast concrete elements. Structural steel fabricators can benefit from photogrammetry and laser scanning technologies. For the welding tasks on structural steel components, photogrammetry will be a promising option based on a recent study (Rodrı´guez-Gonza´lvez et al., 2017) that provides an accuracy of less than 1 mm for welding inspection under the condition that the dual sensors are fixed at a close distance from the welding area. Scan-arm laser scanners such as Faro Gage Arm (FAROArm, 2019) are also a possible option for the welding due to high accuracy in measuring dimensions of welding area. However, due to the limitation in measurement range and inspection time, it is not a favorable option for the dimensional inspection of structural steel components. As for the dimensional inspection of structural steel products, laser scanning is the most feasible option based on the fact that structural steel products are likely to be lengthy and their shape may be more complex. To scan the complex shape and long products, multiple scans at different locations are required for the entire surface inspection of the products.

7.6.3 Time-effectiveness The potential sensing technologies discussed in the previous section can significantly reduce inspection time. Taking the laser scanning technology as an example, it provides a remarkable saving in time compared with conventional

356 Infrastructure Computer Vision

FIGURE 7.30 Deviation analysis between the DT model and as-built model of a full-scale precast slab.

inspection methods that use commercial software for the dimensional control of prefabricated components. For dimensional control of prefabricated components, deviation analysis is a common method to calculate the dimensional discrepancies between the DT model and the as-built model. Fig. 7.30 shows the deviation analysis between the DT model and the as-built model of a full-scale precast slab. For the deviation analysis, point clouds of 10 different position scans were used to generate a 3D as-built model. Commercial software, Cyclone (Leica 2019) and Geomagic Studio (Geomagic 2019), were used for the registration of the point clouds and the estimation of dimension discrepancies between two models of the precast slab. Table 7.8 summarizes the comparison of time consumed for the dimension estimation of the top surface of the fullscale precast slab among three different inspection methods. It shows that laser scanning technology significantly reduces the inspection time compared to the two conventional methods that are based on 1) commercial data processing and model software and 2) manual inspection using contact-type measurement tapes. On the other hand, during the assembly stage of fabricated components, employing 3D imaging could accomplish reliable and accurate results for the representation of the as-built status. As described in Section 7.3, the laser scanning approach with optimization algorithms could minimize the rework

TABLE 7.8 Comparison of time consumed for dimensional inspection of a full-scale precast slab among three different methods. Time (minutes) Method

Scanning

Modeling

Measuring

Total

Deviation analysis using commercial software

30

60

10

100

Manual inspection

e

e

90

90

Laser scanning technique (Kim et al., 2016)

5

e

0.5

5.5

Use Cases for Subcontractors and Fabricators Chapter | 7

357

with a significant time saving compared to current practice for rework of assembly. Moreover, in the previous study (Rausch et al., 2017) conducted for assembly of precast concrete panels, a significant time saving of approximately 4 hours or 80% of the total time required for rework was observed under the condition that only a single panel requires rework.

7.7 Discussions and future directions There are still remaining challenges in automated dimensional QCs for manufacturing and fabrication, which need to be addressed in the near future as follows: 1) improvement of inspection accuracydthe accuracy for dimensional quality inspection based on noncontact sensing technologies needs further improvement because prefabricated products manufactured in off-site facilities require a tighter dimensional tolerance. Based on the understanding of measurement error sources of laser scanners, error prediction models such as the mixed-pixel removal method (Wang et al. 2016c) and the dimensional compensation method (Tang et al. 2009) could be used to improve the performance of dimensional quality inspection of prefabricated components. For example, Wang et al. (Wang et al. 2016c) found that the dimension estimation error could be reduced by more than 60% after removing mixed pixels from raw scan data. On the other hand, improvement in the accuracy of photogrammetry could be a potential research direction. This is because due to the substantial advantages of low cost and better portability, photogrammetry can be a better choice for geometric QC once range measurement accuracy can be improved. In particular, recently developed computer vision techniques that are robust to lighting conditions and provide a long measurement range could be adopted in the manufacturing stage to increase the accuracy of geometric QC; 2) necessity of dimensional control before the completion of prefabricated components and scope extension to various types of prefabricated componentsedmost research in the manufacturing stage has been conducted after prefabricated components are manufactured. Future research could extend this scope of study to encompass the stage before the completion of prefabricated products. This is because even though geometric quality problems of prefabricated components are identified from sensing data, it is already too late to resolve the problem. For example, it is important to assess the concrete cover, position, and spacing of reinforcing bars before concrete casting. On the other hand, the current scope of study on geometric quality inspection of prefabricated components/structures in the assembly phase is limited to pipe spools. Further study should be followed to extend the scope of study to various types of prefabricated components such as vertical/horizontal alignments/assemblies of structural components and other types of MEP components; and 3) Improvement in the applicability of the current techniquesdmost of the existing geometric quality inspection techniques are applicable only under certain assumptions. For example, most dimensional

358 Infrastructure Computer Vision

quality inspection methods are applicable only to elements with planar and regular geometries. Similarly, many research on detecting surface defects of prefabricated components simply assume that the normal status of surfaces is planar. However, existing methods may not be applicable to nonplanar elements/structures. For example, in case that the shape of external fac¸ade panels is curved and each fac¸ade panel has different dimensions and shapes, the existing dimensional and surface inspection methods become inapplicable. Therefore, generic and robust techniques/methods with fewer assumptions are needed to be developed. Moreover, many previous attempts of using sensing technologies for automating the manufacturing process have been tested in a laboratory environment, which may not perform well in real-world scenarios. To tackle this limitation, future research should be conducted to adopt the current research to real-world scenarios to handle complex conditions that may appear on manufacturing sites.

References AbouRizk, S., 2010. Role of simulation in construction engineering and management. J. Constr. Eng. Manage. 136 (10), 1140e1153. Ahmed, M.F., Haas, C.T., Haas, R., 2014. Automatic detection of cylindrical objects in built facilities. Journal of Computing in Civil Engineering 28 (3), 04014009. AISC, 2010. Code of Standard Practice for Steel Buildings and Bridges (AISC 303-10). AISC, 2005. Code of Standard Practice for Steel Buildings and Bridges. American Institute of Steel Construction, Chicago, Illinois. Akula, M., Lipman, R.R., Franaszek, M., Saidi, K.S., Cheok, G.S., Kamat, V.R., 2013. Real-time drill monitoring and control using building information models augmented with 3D imaging data. Autom. Constr. 36, 1e15. Azhar, S., Khalfan, M., Maqsood, T., 2015. Building information modelling (BIM): now and beyond. Construction Economics and Building 12 (4), 15e28. Ballast, D.K., 2007. Handbook of Construction Tolerances. John Wiley & Sons. Bhatla, A., Choe, S.Y., Fierro, O., Leite, F., 2012. Evaluation of accuracy of as-built 3D modeling from photos taken by handheld digital cameras. Autom. Constr. 28, 116e127. Bosche, F., Guenet, E., 2014. Automating surface flatness control using terrestrial laser scanning and building information models. Autom. Constr. 44, 212e226. Bosche´, F., Guillemet, A., Turkan, Y., Haas, C.T., Haas, R., 2014. Tracking the built status of MEP works: assessing the value of a scan-vs-BIM system. J. Comput. Civ. Eng. 28 (4). Broyden, C.G., 1967. Quasi-Newton methods and their application to function minimisation. Math. Comput. 368e381. Cho, Y.K., Haas, C.T., Sreenivasan, S.V., Liapi, K., 2004. Position error modeling for automated construction manipulators. J. Constr. Eng. Manage. 130 (1), 50e58. Dai, F., Lu, M., 2010. Assessing the accuracy of applying photogrammetry to take geometric measurements on building products. J. Constr. Eng. Manage. 136 (2), 242e250. Dai, F., Rashidi, A., Brilakis, I., Vela, P., 2013. Comparison of image-based and time-of-flightbased technologies for three-dimensional reconstruction of infrastructure. J. Constr. Eng. Manage. 139 (1), 69e79.

Use Cases for Subcontractors and Fabricators Chapter | 7

359

Denavit, J., Hartenberg, R.S., 1955. A kinematic notation for lower-pair mechanisms based on matrices. J. Appl. Mech. 215e221. Dennis, J.,J., More´, J., 1977. Quasi-Newton methods, motivation and theory. SIAM Rev. 19 (1), 46e89. Dimitrov, A., Golparvar-Fard, M., 2015. Segmentation of building point cloud models including detailed architectural/structural features and MEP systems. Autom. Constr. 51, 32e45. Eastman, C., Eastman, C.M., Teicholz, P., Sacks, R., 2011. BIM Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers and Contractors. John Wiley & Sons. Farnsworth, C.B., Beveridge, S., Miller, K.R., Christofferson, J.P., 2015. Application, advantages, and methods associated with using BIM in commercial construction. Int J Construct Educ Res 11 (3), 218e236. FAROArm, 2019. https://www.faro.com/products/factory-metrology/faroarm/. Feng, C., Xiao, Y., Willette, A., McGee, W., Kamat, V.R., 2015. Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites. Autom. Constr. 59, 128e138. Garcia, C., Li, T., 1980. On the number of solutions to polynomial systems of equations. SIAM J. Numer. Anal. 17 (4), 540e546. Geomagic Studio, 2019. Geomagic Inc (May 7, 2019). http://support1.geomagic.com/Support/ 5605/5668/en-US/Article/Folder/336/Geomagic-Studio. Gibb, A.G., 1999. Off-site Fabrication: Prefabrication, Pre-assembly and Modularization. John Wiley & Sons. Golparvar-Fard, M., Bohn, J., Teizer, J., Savarese, S., Pen˜a-Mora, F., 2011. Evaluation of imagebased modeling and laser scanning accuracy for emerging automated performance monitoring techniques. Autom. Constr. 20 (8), 1143e1155. Jahanshahi, M.R., Kelly, J.S., Masri, S.F., Sukhatme, G.S., 2009. A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures. Struct. Infrastruct. E. 5 (6), 455e486. Kim, M., Wang, Q., Park, J., Cheng, J.C., Sohn, H., Chang, C., 2016. Automated Dimensional Quality Assurance of Full-Scale Precast Concrete Elements using Laser Scanning and BIM. Autom. Constr 72, 102e114. Kim, M., Cheng, J.C., Sohn, H., Chang, C., 2015a. A framework for dimensional and surface quality assessment of precast concrete elements using BIM and 3D laser scanning. Autom. Constr. 49, 225e238. Kim, M., Sohn, H., Chang, C., 2015b. Localization and Quantification of Concrete Spalling Defects using Terrestrial Laser Scanning. J. Comput. Civ. Eng. 29 (6), 04014086. Kim, M., Sohn, H., Chang, C., 2014a. Automated dimensional quality assessment of precast concrete panels using terrestrial laser scanning. Autom. Constr. 45, 163e177. Kim, M., Sohn, H., Chang, C., 2014b. Localization and quantification of concrete spalling defects using terrestrial laser scanning. J. Comput. Civ. Eng. 29 (6), 04014086. Kim, C., Son, H., Kim, C., 2013. Fully automated registration of 3D data to a 3D CAD model for project progress monitoring. Autom. Constr. 35, 587e594. Kim, M., Wang, Q., Li, H., 2019a. Non-contact sensing based geometric quality assessment of buildings and civil structures: A review. Autom. Constr. 100, 163e179. Kim, M., Wang, Q., Yoon, S., Sohn, H., 2019b. A mirror-aided laser scanning system for geometric quality inspection of side surfaces of precast concrete elements. Measurement 141, 420e428. Lawson, M., Ogden, R., Goodier, C., 2014. Design in Modular Construction. CRC Press.

360 Infrastructure Computer Vision Leica Cyclone, 2019. Leica Geosystems Inc (May 7, 2019). http://hds.leica-geosystems.com/en/ Leica-Cyclone_6515.htm. Leite, F., Akcamete, A., Akinci, B., Atasoy, G., Kiziltas, S., 2011. Analysis of modeling effort and impact of different levels of detail in building information models. Autom. Constr. 20 (5), 601e609. Li, T., 1997. Numerical solution of multivariate polynomial systems by homotopy continuation methods. Acta. Numer. 6, 399e436. Liu, Y.F., Cho, S., Spencer Jr., B.F., Fan, J.S., 2014. Concrete crack assessment using digital image processing and 3D scene reconstruction. Journal of Computing in Civil Engineering 30 (1), 04014124. Lo´pez, M., Martı´nez, J., Matı´as, J.M., Vila´n, J.A., Taboada, J., 2010. Application of a hybrid 3D2D laser scanning system to the characterization of slate slabs. Sensors 10 (6), 5949e5961. Milberg, C., Tommelein, I.D., Alves, T., 2002. Improving design fitness through tolerance analysis and tolerance allocation. In: Proc., 3rd International Conference on Concurrent Engineering in Construction. University of California, Berkeley. Milberg, C.T., Tommelein, I.D., 2009. Tolerance and constructability of soldier piles in slurry walls. J. Perform. Constr. Facil. 24 (2), 120e127. Nahangi, M., Czerniawski, T., Haas, C.T., Walbridge, S., West, J., 2015a. Parallel systems and structural frames realignment planning and actuation strategy. J. Comput. Civ. Eng. 04015067. Nahangi, M., Haas, C.T., 2014a. Automated 3D compliance checking in pipe spool fabrication. Adv. Eng. Inf. 28 (4), 360e369. Nahangi, M., Haas, C.T., West, J., Walbridge, S., 2015b. Automatic Realignment of Defective Assemblies using an Inverse Kinematics Analogy. ASCE Journal of Computing in Civil Engineering 30 (2), 04015008. Nahangi, M., Yeung, J., Haas, C.T., Walbridge, S., West, J., 2015c. Automated assembly discrepancy feedback using 3D imaging and forward kinematics. Autom. ConStruct. 56, 36e46. Nahangi, M., Haas, C.T., 2014b. Automated 3D compliance checking in pipe spool fabrication. Adv. Eng. Inf. 28 (4), 360e369. Patraucean, V., Armeni, I., Nahangi, M., Yeung, J., Brilakis, I., Haas, C., 2015. State of research in automatic as-built modelling. Adv. Eng. Inf. Penrose, R., 1955. A generalized inverse for matrices. In: Proc., Mathematical Proceedings of the Cambridge Philosophical Society. Cambridge Univ Press, pp. 406e413. Rausch, C., 2016. Development of a Framework for the Strategic Management of Dimensional Variability of Structures in Modular Construction. Rausch, C., Nahangi, M., Perreault, M., Haas, C.T., West, J., 2017. Optimum assembly planning for modular construction components. J. Comput. Civ. Eng. 31 (1). Rodrı´guez-Gonza´lvez, P., Rodrı´guez-Martı´n, M., Ramos, L.F., Gonza´lez-Aguilera, D., 2017. 3D reconstruction methods and quality assessment for visual inspection of welds. Autom. Constr. 79, 49e58. Rodrı´guez-Martı´n, M., Rodrı´guez-Gonza´lvez, P., Lagu¨ela, S., Gonza´lez-Aguilera, D., 2016. Macro-photogrammetry as a tool for the accurate measurement of three-dimensional misalignment in welding. Autom. Constr. 71, 189e197. Safa, M., Shahi, A., Nahangi, M., Haas, C., Noori, H., 2015. Automating measurement process to improve quality management for piping fabrication. In: Proc., Structures. Elsevier, pp. 71e80. Shahtaheri, Y., Rausch, C., West, J., Haas, C., Nahangi, M., 2017. Managing risk in modular construction using dimensional and geometric tolerance strategies. Autom. Constr. 83, 303e315, 2017.

Use Cases for Subcontractors and Fabricators Chapter | 7

361

Sommese, A.J., Verschelde, J., Wampler, C.W., 2004. Advances in polynomial continuation for solving problems in kinematics. J. Mech. Des. 126 (2), 262e268. Son, H., Bosche´, F., Kim, C., 2015. As-built data acquisition and its use in production monitoring and automated layout of civil infrastructure: a survey. Adv. Eng. Inf. Tang, P., Akinci, B., Huber, D., 2009. Quantification of edge loss of laser scanned data at spatial discontinuities. Autom. Constr. 18 (8), 1070e1108. Tang, P., Chen, G., Shen, Z., Ganapathy, R., 2016. A spatial-context-based approach for automated spatial change analysis of piece-wise linear building elements. Comput. Aided Civ. Infrastruct. Eng. 31 (1), 65e80. Tang, P., Huber, D., Akinci, B., Lipman, R., Lytle, A., 2010. Automatic reconstruction of as-built building information models from laser-scanned point clouds: a review of related techniques. Autom. Constr. 19 (7), 829e843. TruePoint, 2019. http://www.truepointscanning.com/case-studies. Tsai, L., 1999. Robot Analysis: The Mechanics of Serial and Parallel Manipulators. John Wiley & Sons. Uchida, T., McPhee, J., 2011. Triangularizing kinematic constraint equations using gro¨bner bases for real-time dynamic simulation. Multibody Syst. Dyn. 25 (3), 335e356. Vico Software, 2016. Vico Guest Blogger Series: How Swinerton Builders Deliver Virtual Design and Construction. http://www.vicosoftware.com/vico-blogs/guest-blogger/tabid/88454/bid/ 12659/How-Swinerton-Builders-Deliver-Virtual-Design-and-Construction.aspx. Wampler, C., Morgan, A., Sommese, A., 1990. Numerical continuation methods for solving polynomial systems arising in kinematics. J. Mech. Des. 112 (1), 59e68. Wang, Q., Cheng, J.C., Sohn, H., 2017. Automated Estimation of Reinforced Precast Concrete Rebar Positions using Colored Laser Scan Data. Computer-Aided Civil and Infrastructure Engineering 32 (9), 787e802. Wang, Q., Kim, M., Cheng, J.C., Sohn, H., 2016a. Automated quality assessment of precast concrete elements with geometry irregularities using terrestrial laser scanning. Autom. Constr. 68, 170e182. Wang, Q., Kim, M., Sohn, H., Cheng, J.C., 2016b. Surface flatness and distortion inspection of precast concrete elements using laser scanning technology. Smart. Struct. Syst. 18 (3), 601. Wang, Q., Sohn, H., Cheng, J.C., 2016c. Development of a mixed pixel filter for improved dimension estimation using AMCW laser scanner. ISPRS J. Photogramm. Remote Sens. 119, 246e258. Weisstein, E.W., 2014. Moore-penrose Matrix Inverse. http://mathworld.wolfram.com/MoorePenroseMatrixInverse.html. Yoon, S., Wang, Q., Sohn, H., 2018. Optimal placement of precast bridge deck slabs with respect to precast girders using 3D laser scanning. Autom. Constr. 86, 81e98.