- Email: [email protected]
Robust registration for removing vibrations in 3D reconstruction of web material
Optics and Lasers in Engineering 68 (2015) 135–148
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Robust registration for removing vibrations in 3D reconstruction of web material Rubén Usamentiaga n, Daniel F. Garcia Department of Computer Science and Engineering, University of Oviedo, Campus de Viesques, 33204 Gijón, Asturias, Spain
art ic l e i nf o
a b s t r a c t
Article history: Received 22 July 2014 Received in revised form 28 October 2014 Accepted 22 November 2014
Vibrations are a major challenge in laser-based 3D reconstruction of web material. In uncontrolled environments, the movement of web material forward along a track is inevitably affected by vibrations. These oscillations significantly degrade the performance of the 3D reconstruction system, as they are incorrectly interpreted as irregularities on the surface of the material, leading to an erroneous reconstruction of the 3D surface. This work proposes a method to estimate and remove these vibrations based on a robust registration procedure. Registration is used to estimate vibrations and a rigid transformation is used to compensate the movements, removing the effects of vibrations on 3D reconstruction. The proposed method is applied to an extensive dataset, both synthetic and real, with very good results. & 2015 Elsevier Ltd. All rights reserved.
Keywords: Structured light 3D reconstruction Vibrations
1. Introduction 3D reconstruction of web material, such as steel or paper, is generally carried out using laser light sectioning sensors [1]. A laser projects a light plane onto the material while it moves forward along a track. The projected laser stripe is observed by a camera, and the coordinates of the stripe are extracted from the image and converted to real world units using a previously calibrated map [2]. The resulting signal is a height profile that provides information about the height of a section of the material. This process is repeated while the objects move forward, resulting in a 3D map of the surface of the material. The main advantages of this 3D reconstruction method are its accuracy and resilience to changes in illumination [3]. For this reason, this is the most widely used method in industrial environments [4]. Vibrations are a major challenge in laser-based 3D reconstruction: periodic or aperiodic movements while the material is moving forward along a track result in incorrect estimations of height profiles, which are propagated to the 3D reconstruction. In controlled conditions, vibrations can be avoided or reduced using various techniques. For example, they can be absorbed by a pneumatic system [5]. However, in uncontrolled environments, such as the industrial field, vibrations are very difficult to avoid [6]. Therefore, in order to obtain an accurate 3D reconstruction of an object, it is necessary to apply a method that can be used to estimate and remove the effects of vibrations on 3D reconstruction.
n
Corresponding author. Tel.: þ 34 985 182626; fax: þ 34 985 181986. E-mail address: [email protected] (R. Usamentiaga).
http://dx.doi.org/10.1016/j.optlaseng.2014.11.019 0143-8166/& 2015 Elsevier Ltd. All rights reserved.
When there are known invariants between consecutive height profiles, vibrations can be estimated and removed relatively easily. For example, when an object is placed on a flat conveyor belt, vibrations affect both the object and the conveyor belt. The known shape of the conveyor belt can be used to estimate the transformation between consecutive height profiles caused by vibrations. Similarly, when the shape of the object has some known features, such as straight lines connected by a curved corner [7], they can be detected and used to identify the transformation caused by vibrations. In the case of road surfaces, the invariant is the assumption of the planarity and smoothness of the road surfaces [8]. Another option is based on applying previous information about the type of vibrations, for example the frequency. Then, a band-pass filter can be used to accurately remove vibrations from the 3D reconstruction [9]. In other types of applications, the 3D reconstruction is not the focus, but rather the measurement of vibrations. In this case invariants, such as a constant shape, are used to estimate vibrations [10,11]. When there are no invariants, more than one laser stripe can be used [12]. Based on this approach, the height profile of the same section of an object is calculated as many times as the number of laser stripes projected onto the object. The redundant information obtained using this procedure is used to estimate and remove the effects of vibrations on 3D reconstructions. Two laser stripes are required to estimate simple vibrations. Three laser stripes provide enough information to estimate more complex vibrations. The disadvantages of this approach are, that additional laser projectors can significantly increase the cost of a 3D reconstruction system, and that calibration becomes much more complex, as the lasers must be aligned very accurately. Additional information about the shape of the object to compensate for vibrations can also be obtained using structured light
136
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 1. Laser-based 3D reconstruction of an object with changing surface normals. (a) Surface without vibrations. (b) Laser scanning of the surface without vibrations. (c) Laser scanning of the surface with vibrations. (d) Same surface with vibrations.
sensors [13]. A light pattern could be projected on the object in order to obtain a point cloud with height information about the shape of the object [14–16]. Image registration techniques could then be used to align two consecutive scans and compensate for vibrations [17]. However, 3D sensors based on structured-light patterns are sensitive to ambient light and reflectance, they cannot be used with shiny surfaces such as steel, have a much more complex calibration, and require that the projector is installed close to the object. Moreover the frame rate of these sensors is usually low, thus they cannot be applied to very fast moving objects. Registration is also problematic when the object does not have distinguishing features that can be used to align consecutive scans, such as a flat object. This work presents a method to estimate and remove the effects of vibrations in laser-based 3D reconstruction: robust registration between consecutive height profiles. Two consecutive height profiles are generally acquired from two very close sections of an object. Thus, the positions of some points do not change between consecutive height profiles. These points, inliers from the point of view of the registration, create an invariant between consecutive height profiles that can be exploited to estimate and remove vibrations. In order to apply this procedure correctly, inliers must be detected robustly without previous information. This work proposes a robust registration based on the median distance and the surface normals of the height profiles. The result is a robust alignment of height profiles that removes vibrations from the final 3D reconstruction. Surface registration from different angles of the same object is a method used by many commercial 3D laser scanners [18]. This process requires views to partially overlap [19]: one part of the point cloud describing the surface of an object from a specific angle overlaps the same part of the object in another point cloud. These sets of points observed from different angles are the key to registering and merging the surfaces. The invariant that is used for registration are the overlapping points. In this work the same principle is applied, however, the goal is to find the points that can be used between consecutive height profiles to detect the geometric transformation caused by vibrations. The main contribution of this work is the application of registration techniques to the estimation and removal of vibrations, and the optimization of these techniques for the particular case of height profiles.
2. Registration of height profiles Laser-based 3D reconstruction involves projecting a laser stripe onto the object. The laser stripe is observed by a camera, and the
coordinates of the laser points in the image are extracted. These coordinates are translated to world coordinates resulting in a height profile, i.e., the height of a section of the object. This technique is called laser triangulation because the laser points, the camera and the laser projector form a triangle. A set of consecutive height profiles can be used to reconstruct the whole surface of the object. Fig. 1a shows an example of a surface: a pyramid. Laserbased reconstruction would produce an approximation of the surface, as can be seen in Fig. 1b. Increasing the acquisition rate produces a denser reconstruction, more similar to the real surface. 3D reconstruction based on laser projection requires a relative movement between the camera and laser projector, and the object. This movement can be affected by vibrations, which produce variations in the position of the object perpendicular to its movement. Vibrations while the object moves forward along a track modify the position where the laser stripe is projected. Thus, they are interpreted as height variations. For example, if the surface in Fig. 1a is affected by vibrations the calculated height profiles could be similar to Fig. 1c. In this example, it is considered that the movement of the object is affected by random vertical translations and by rotations, two common types of vibrations. The resulting 3D reconstruction of the surface would be similar to Fig. 1d. As can be seen, the reconstruction is completely inaccurate because it has been corrupted by the vibrations. The method proposed in this work to estimate and remove vibrations is based on a robust registration between consecutive height profiles. The general goal of registration is to find the best alignment between two objects or instances of the same object [20]. The shapes of the objects do not need to be identical, a partial overlap is enough. Registration can be divided into several types, depending on the kind of data used for alignment [21]. Data can be acquired from different viewpoints of the same scene, or from different sensors. It is also common to perform registration between a view of the object and a reference model. This latter type of registration is used when data is acquired at different times. In this case, the aim is to find changes in the scene that appear between the consecutive acquisitions. This is the type of registration proposed for height profiles. Registration is generally applied to images or cloud points. In this work, registration is applied to height profiles, which are a particular case of 2D cloud points. The registration process is based on the assumption that alignment is possible between height profiles, that is, that the positions of some points do not change much. Vibrations can be modeled using a rigid transformation and the object is assumed to be a rigid material. Therefore, vibrations do not cause a deformation of the object. When vibrations affect
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
137
Fig. 2. Registration of height profiles with changing surface normals. (a) Two consecutive height profiles. (b) Correspondences between the two profiles. (c) Initial transformation. (d) Normals of the first height profile. (e) Normals of the second height profile. (f) Correspondences of points when the similarity of normals are considered. (g) Correspondences of points when both the similarity of normals and distance are considered. (h) Final transformation.
the height profile, the positions of all points vary. However, there are some points where the change of position is mostly caused by vibrations, not by a change in the shape of the object. This assumption is true for web material, but can also be true for many other types of objects. These inliers can be used to estimate and remove vibrations. The challenge is to detect these points robustly without previous information. The proposed registration procedure is based on an adaptation of the Iterative Closest Point (ICP) algorithm [22]. The ICP algorithm is the state-of-the-art solution for registration [23]. The overall aim of the algorithm is to estimate a rigid transformation that minimizes a metric distance between the two datasets, such as the average distance. The estimation of the transformation improves with each iteration of the ICP algorithm. The algorithm is proved to converge monotonically to a local solution of the problem, given a proper initial alignment [24]. The proposed algorithm for the registration of height profiles is as follows: (1) Initial transformation: From one height profile to the next, the width can be considered constant in the case of web material. Thus, a direct correspondence between the points in the two profiles can be assumed. Therefore, an initial transformation is performed by considering all the points in the two profiles. Fig. 2a shows two consecutive height profiles acquired from the surface in Fig. 1c. The shape of the object has changed from one height profile to the next. In addition, vibrations have translated
the second profile upwards and rotated it. A direct correspondence between the points in the two profiles is carried out, as can be seen in Fig. 2b. Then, a rigid transformation between the two sets of points is calculated using the least squares method [25]. This transformation is applied to the second scan, resulting in Fig. 2c. (b) Detection of inliers: Inliers in the two height profiles are detected as the points that meet two criteria: similar surface normals and similar distances. First, the normals of the two height profiles are calculated. The calculation of the surface normal for a point p involves fitting a window of points around p to a line. The normal for this point is the normal of the fitted line, or plane in the case of 3D points [26]. Another very efficient approach to estimate the normals is based on the covariance matrix [27]. This process is repeated for each point of the two profiles. The results can be seen in Fig. 2d and e. In order to avoid orientation inconsistency, all normals are considered with a positive y axis. A similarity score, sc, for the normals in the two height profiles, n1 and n2, is calculated as sc ¼ n1 n2
ð1Þ
A high value of sc means similar surface normals [28]. Therefore, a threshold can be applied to identify surface normal similarity. Fig. 2f shows the points in the two height profiles with similar surface normals. This set of points are potential inliers. As can be seen, there is an outlier in the peak that has the same surface normal in the two height profiles.
138
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 3. Laser-based 3D reconstruction of an object with similar surface normals. (a) Surface without vibrations. (b) Laser scanning of the surface without vibrations. (c) Laser scanning of the surface with vibrations. (d) Same surface with vibrations.
Next, the median distance between the potential inliers is calculated. Only those inlier candidates whose distance is lower than k times the median distance are considered valid inliers [29]. The resulting inliers can be seen in Fig. 2g. As can be seen, the outlier in the peak has been removed. Another popular method to reject outliers is the X84 rule [30], which uses robust estimates for location and scale to set a rejection threshold. However, this method cannot be applied for the considered problem because it assumes an underlying Gaussian distribution of the residuals, that is, the distances. The method is based on the calculation of the median absolute deviation, which is similar to the interquartile range. The X84 rule prescribes rejecting values that are more than k median absolute deviations away from the median. In the example of Fig. 2f, this value is zero, thus it would reject all points. (3) Refined transformation: A new rigid transformation between height profiles is calculated considering only the inliers. The result of this transformation can be seen in Fig. 2h. The result of this transformation has removed vibrations completely. Height profiles are perfectly aligned, even when the shape of the object has changed partially. (4) Iterate: In the height profiles used in the example, one iteration was enough to compensate for the vibrations. More complex vibrations could require more than one iteration. Thus, steps 2 and 3 are repeated until the change between two successive iterations is lower than an established threshold. In the example shown in Fig. 1 most inliers are detected due to the surface normals. However, changes in the shape of the object can keep normals intact. This is the case shown in Fig. 3, which has been corrupted with the same vibrations as in the example in Fig. 1. The registration process in this case is shown in Fig. 4. As can be seen, the rejection due to differences in surface normals does not work with this surface. Fig. 4d and e shows that all points in the two profiles have the same surface normals. However, the rejection based on the median distance detects inliers correctly, as can be seen in Fig. 4g. This is exploited to align height profiles and remove vibrations. The result is a perfect alignment between the two profiles, shown in Fig. 4h. The registration process, R, is applied to two consecutive height profiles, Hi and H i þ 1 . The result is a modified version of the second profile aligned with the first one, H 0i þ 1 . This process can be expressed as H 0i þ 1 ¼ RðH i ; H i þ 1 Þ
ð2Þ
The registration of height profiles is repeated for all consecutive height profiles, starting with the first two profiles. Thus, for a generic profile j the registration is carried out using (3). The registration of height profile j is carried out against the previously aligned profile. The first height profile does not have a previous profile, thus (2) starts with profile number 2. Profile 1 can be aligned against a horizontal profile. Applying this process to the surfaces shown in Figs. 1d and 3d produces the surfaces without vibrations as shown in Figs. 1a and 3a, respectively. H j ¼ RðH 0j 1 ; H j Þ
8 j ¼ 2…n
ð3Þ
2.1. Comparison with RanSac A different approach to achieve robust registration between consecutive height profiles is the RanSac model estimator [31]. The RanSac algorithm is an iterative procedure to estimate models with data that includes outliers. It works by selecting small subsets of the available data, fitting the model and calculating the percentage of the total data that fits the model. Despite its simplicity, it works very well even with a high percentage of outliers. In this case, the model is a rigid transformation between height profiles. The score of the model is the number of points in the two profiles that match after the rigid transformation. The results applying the RanSac algorithm show excellent performance. In the examples shown in Figs. 2 and 4 the results are similar to those obtained using the proposed procedure: perfect alignment. However, the proposed procedure clearly outperforms the RanSac algorithm when white noise is present in the height profiles. White noise is not produced by vibrations, but by other causes, such as inaccurate calibration or rounding errors. In order to compare the performance of the proposed procedure with the RanSac algorithm, a registration test has been performed. Two consecutive height profiles are corrupted with random white noise. The noise is uniformly distributed with a magnitude equal to 10% of the maximum height. Fig. 5a shows the original height profiles, and Fig. 5b shows the corrupted height profiles. Fig. 5c and d shows the worst alignment obtained using the proposed procedure and the RanSac algorithm, respectively, in 100 tests. In each test the profiles were corrupted with a different pattern with equal distribution and magnitude. As can be seen, the proposed procedure aligns the height profiles correctly even in the presence of white noise. On the other hand, with the RanSac algorithm fails the registration test due to the noise: only the right part of the profiles is aligned.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
139
Fig. 4. Registration of height profiles with similar surface normals. (a) Two consecutive height profiles. (b) Correspondences between the two profiles. (c) Initial transformation. (d) Normals of the first height profile. (e) Normals of the second height profile. (f) Correspondences of points when the similarity of normals are considered. (g) Correspondences of points when both the similarity of normals and distance are considered. (h) Final transformation.
Fig. 5. Registration of height profiles with noise. (a) Original consecutive height profiles. (b) Height profiles with noise. (c) Worst alignment with the proposed procedure. (d) Worst alignment with the RanSac algorithm.
The results are compared numerically using the normalized mean distance. This metric is calculated using (4), where n is the number of points in the height profile, Hij is point j of profile i after registration, Oij is point j of profile i in the original profile with neither white noise nor vibrations, and d is the Euclidean distance. The normalized mean distance is a percentage of error over the maximum height of the object, independently of the specific
height of the object: 1 Pn j ¼ 1 d H ij Oij n Error ¼ 100 maxðOi Þ minðOi Þ
ð4Þ
Fig. 6 shows the histograms of the normalized mean distance for all the tests. Fig. 6a shows the histogram for the proposed
140
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
3.1. Synthetic data
procedure. In 100% of the cases the error is around 10%, that is, the same error with which the height profiles were corrupted. Fig. 6b shows the results for the RanSac algorithm. In this case, the error increases: 20% of the tests obtain an error of 20% of the height of the profile. This 20% correspond to cases similar to Fig. 5d, where the RanSac algorithm does not align the height profiles correctly because they are affected by white noise. The RanSac algorithm can be controlled with a parameter that indicates the threshold distance between points to be considered inliers. This parameter can be relaxed to work better with noise. However, this also results in worse performance when noise is not present in the data. The proposed procedure works equally well in both cases: with or without noise. The reason why the proposed procedure outperforms the RanSac algorithm in the presence of noise is that considering only the distance between points does not provide robust results. The RanSac algorithm works very well with a high percentage of outliers, but only when the other points are good matches without random noise.
In order to assess the proposed procedure, several experiments have been carried out. Four different synthetic objects were created (Fig. 7). The first two (Fig. 7a and b) are the objects used in Section 2: a pyramid and a triangular prism. These two objects have flat surfaces. Two more objects were created with curved surfaces (Fig. 7c and d). The object in Fig. 7c represents a curved object with some constant points on both sides, that is, inliers. Fig. 7d is a curved object with no constant points, that is, all points vary from one height profile to the next. This object does not meet the required assumption to apply the proposed procedure, where some inliers between consecutive height profiles are required. However, this object will be used to verify the performance of the proposed procedure with this undesirable situation. Four types of vertical translations are applied to the objects (Fig. 8). These translation models are based on plausible vibrations that can affect the movement of the object. The sinusoidal pattern (Fig. 8a) assumes a periodic movement of the object while it moves forward, upwards and downwards, without actually touching the roll that moves the object. The damped sinusoidal pattern (Fig. 8c) assumes that vibrations lose energy faster than it is being supplied. Semi-sinusoidal translations (Fig. 8c) assume that the object hits the roll periodically producing only half a sinusoidal
3. Results and discussion
100
100
80
80
Percentage of tests
Percentage of tests
In this section, the proposed procedure is applied to synthetic and real data to test performance.
60 40 20 0
0
10
20
30
40
50 Error
60
70
80
90
100
60 40 20 0
0
10
20
30
40
50
60
70
80
90
100
Error
Fig. 6. Histograms of the normalized mean distance. (a) Proposed procedure. (b) RanSac algorithm.
Fig. 7. Synthetic objects used for tests. (a) Pyramid. (b) Triangular prism. (c) Curved surface with inliers. (b) Curved surface with no inliers.
Fig. 8. Types of translations considered. (a) Sinusoidal translation pattern. (b) Damped sinusoidal translation pattern. (c) Semi-sinusoidal translation pattern. (b) Random translation pattern.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
movement. Random translations (Fig. 8d) assume movements without pattern, defined using a random uniform distribution. Another type of vibration is rotations that produce semi-circular movements of the object around a pivot position, similar to rocking. A sinusoidal pattern is assumed for rotations while the object moves forward. All these types of vibrations are common in industrial applications. Four vibration patterns are used to corrupt the shape of the considered synthetic objects: (a) Vertical translations following a sinusoidal pattern. (b) A combination of vertical translations following a damped sinusoidal pattern and rotations around a pivot point on the edge of the object. (c) A combination of vertical translations following a random pattern and rotations around a pivot point in the middle of the object. (d) A combination of vertical translations following a semisinusoidal pattern and rotations around a pivot point on the middle of the object. The resulting object has been corrupted with additional random white noise. The noise is uniformly
141
distributed with a magnitude equal to 10% of the maximum height. This combination of vibrations and white noise can be considered an extreme case, used to test the proposed procedure under very exceptional conditions. The proposed robust registration procedure for removing the effects of vibrations on 3D reconstruction has been applied to the four synthetic objects corrupted with these four vibrations patterns. The results can be seen in Figs. 9–12. The first column of these figures shows the corrupted surfaces with vibrations, and the second column shows the resulting surface after applying the proposed procedure. The results with the pyramid and the triangular prims can be seen in Figs. 9 and 10, respectively. The first three cases are combinations of different types of vibrations. As can be seen, the proposed procedure removes these vibrations completely, resulting in an accurate 3D reconstruction of the surface of the original object. In these three cases, the error measured using (4) provides a value of zero, that is, the surface is reconstructed perfectly. When the proposed procedure is applied when white noise is added after
Fig. 9. Proposed procedure applied to the pyramid. (a) Pyramid with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Pyramid with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Pyramid with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Pyramid with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
142
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 10. Proposed procedure applied to the triangular prism. (a) Triangular prism with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Triangular prism with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Triangular prism with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Triangular prism with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
vibrations, it does not provide a perfect result, as can be seen in Figs. 9h and 10h. The white noise used to corrupt the surface of the object is present in the resulting 3D map. The proposed procedure is not designed to remove this type of noise, only vibrations affecting the height profiles. The error in the resulting map is around 10%, similar to the magnitude of the white error added to the synthetic object. The registration procedure is also slightly affected by the white noise. When noise is higher than the variation between height profiles, some points are incorrectly considered as inliers. This causes an erroneous estimation of the vibrations, resulting in a slightly modified 3D map. This issue can be seen in Fig. 9h where a small curvature can be observed along the surface. The curved surface with inliers provides similar results to the two previous examples, as can be seen in Fig. 11. The curved surface does not affect the performance of the proposed procedure. Inliers are correctly detected and the resulting maps are accurate reconstructions of the object. White noise affects the proposed procedure, producing an error with similar magnitude to the white noise. The curved surface without inliers is a great challenge for the proposed procedure. The registration requires inliers between
consecutive height profiles. Therefore, the removal of vibrations from this object is difficult. The results can be seen in Fig. 12. In the first three cases, vibrations are not detected correctly. Parts of the height profile in the middle are considered as inliers. Thus, the transformation considering these points removes part of the variation in that zone and increases the variation on the edges. In these three cases, the problem results in an error of around 5%. When white noise is added this error increases up to 10%. The results are inaccurate 3D reconstructions of the object. However, the results are still much better than the reconstruction obtained without filtering. Moreover, it does not result in a spurious transformation. Therefore, even in this extreme case the results can be considered positive. The results shown in Figs. 9g, 10g, 11g, and 12g indicate that the proposed procedure is affected by white noise. The experiments were repeated with the addition of a standard smoothing low-pass filter on the surface of the object before using the proposed method. The resulting error decreased sharply: to 0.3% on average. Clearly, when white noise is present, it is advisable to use this type of filter.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
143
Fig. 11. Proposed procedure applied to the curved surface with inliers. (a) Surface with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Surface with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Surface with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Surface with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
Fig. 13 shows the results using the RanSac algorithm. The first vibration pattern corresponding to vertical translations following a sinusoidal pattern is applied to the pyramid and the curved surface without inliers. The main parameter of the RanSac algorithm is the threshold distance between points to be considered inliers. Three configurations are used: 1e-2, 1e-4, and 1e-6. The best configuration for the pyramid is a threshold distance of 1e-6. However, this same configuration leads to an incorrect result when applied to the curved surface. The best configuration for this model is a threshold distance of 1e-2. The best results are the same as those obtained with the proposed procedure. However, the RanSac algorithm requires different configurations for different shapes. A possible solution is a different robust estimator: the Least Median of Squares [32]. The advantage compared with RanSac is that no threshold configuration is required. Tests with this method provided very good results in some cases, but when the number on inliers was above 50%, the results were worse than RanSac, and worse than the proposed procedure. This is the case of the pyramid object. The problem is that this method always assumes that a least 50% of the points are good correspondences.
The RanSac algorithm has also been applied when white noise is added. Fig. 14 shows the results for the pyramid corrupted with vibrations and random white noise (vibration pattern 4). The RanSac algorithm is applied with the same three configurations, and is executed twice for each of them. The results show one major drawback of the RanSac algorithm with white noise: unpredictability. Not only are the results much worse than the proposed procedure, the results are different for each execution. This tests confirms previous results, and clearly indicates that the proposed procedure offers much better robustness and accuracy than the RanSac algorithm. 3.2. Real data The proposed procedure has been applied to real data obtained from the 3D reconstruction of steel strips. This type of web material is generally 2 m wide and 10 km long. 3D reconstruction is required for quality control, such as flatness inspection [33]. Data about the surface of the strips is obtained using a laser-sectioning sensor. Fig. 15a, c, and e shows the data obtained from this sensor for three different steel strips. Fig. 15a shows a steel strip with a defect called
144
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 12. Proposed procedure applied to the curved surface with no inliers. (a) Surface with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Surface with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Surface with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Surface with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
wavy edges, caused by over-rolling. The strip in Fig. 15c has a curved surface, and the strip in Fig. 15e has a flat surface. The surfaces of these strips are corrupted with vibrations of different frequencies, vertical translations and rotations. The results after applying the proposed procedure can be seen in Fig. 15b, d, and f. These results are compatible with the shape of the strips. Clearly, vibrations have been removed, producing a clean 3D reconstruction of the surface of the strips. These results indicate that the proposed procedure greatly improves the resulting 3D reconstruction, producing a surface equivalent to that which would be obtained if the movement were not affected by vibrations. In order to assess the proposed procedure quantitatively, a manual experiment was carried out. The objective is to compare the results with those obtained using a laser-sectioning sensor with the proposed filtering activated. A steel strip that has wavy edges with a sinusoidal shape was used. The technicians measured the wave length and wave height at the edges of the strip. These type of defects are periodic, thus wave length and wave height can be used to estimate the length of the strip at that edge. These results can be compared with those obtained using a lasersectioning sensor, with or without the proposed procedure.
A common parameter used for the quantitative evaluation of the strip shape is the I-unit. The I-unit index is calculated using (5), where Li is the length at position i, or wave length, and Lref is the length of a reference flat fiber: I unitðiÞ ¼
Li Lref 5 ΔLi 5 10 ¼ 10 Lref Lref
ð5Þ
The results of the experiments can be seen in Fig. 16. This bar chart show the I-units measured using the manual procedure and the I-units calculated using a laser light-sectioning sensor with and without the filter activated. The strips are around 700 m long, and their I-units are measured in four different zones (Z1–Z4). The results obtained in manual measurements are similar to the results obtained when the proposed filter is activated. Without the filter, vibrations greatly modify the calculated I-units producing spurious results. The results with the filter activated are qualitatively comparable to manual measurements, which indicate that the proposed procedure for filtering vibrations is working correctly. Manual experiments are not a viable option for extended validation because they are time-consuming operations that require stopping the production line, which reduces productivity sharply.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
145
Fig. 13. Results when using the RanSac algorithm with sinusoidal vibrations. (a) Pyramid with RanSac 1e-2. (b) Curved surface with RanSac 1e-2. (c) Pyramid with RanSac 1e4. (d) Curved surface with RanSac 1e-4. (e) Pyramid with RanSac 1e-6. (f) Curved surface with RanSac 1e-6.
Therefore, in order to provide additional quantitative results of different defects with different vibration patterns a test piece made of aluminum and plastic was used. A set of artificial defects were added to the flat surface consisting of waves (on the left edge, 1221 I-units), peaks (on the center, 610 I-units) and a flat surface (on the right edge, 0 I-units). The test piece is moved using a roll table to reproduce the industrial conditions, providing the correct movement and vibrations required to assess the performance of the proposed procedure. Fig. 17 shows a photograph of the test piece and the laser stripe projection while the piece is moved by the rolls. The results of the experiments can be seen in Fig. 18. In the figure, the scale in the Z axis is not the same as in the X or Y in order to appreciate translations more clearly. Three vibrations patterns are considered: random translations, a combination of random translations and rotations on one side, and a combination of random translations and rotations on both sides. Rotations are introduced by creating artificial defects in the rolls that move the test piece. As can be seen, there is a huge improvement between the raw reconstruction of the surface and the result using the proposed procedure. The results when the filter is activated are vibration-free 3D reconstructions. Table 1 shows the measured I-units of the test piece. Table 2 shows the same data but in this case compared with the real values in terms of percentage error. The resulting error when the proposed procedure is applied is very low. However, the raw data provides a large error, in some cases the measure value is more than two times the real value. These results indicate that the proposed method makes it possible to estimate and remove vibrations from the surface of 3D reconstruction of the objects. Although vibration cannot be avoided in industrial environments, the proposed registration procedure greatly improves the 3D reconstruction results provided by standard laser-sectioning sensors. Not only does it provide robust results, but it provides them quickly. Many types of applications, such as industrial inspection systems, require fast processing
algorithms in order to detect problems and apply corrective measures as quickly as possible. In order to test the speed of the proposed procedure, the time necessary to filter standard 3D maps has been measured. The experiments were performed using an Intel Core i7 4770 running at 3.4 GHz with 16 GB of RAM. The algorithm was coded in Cþ þ . The average measured execution time for 3D maps consisting of 2000 height profiles of 100 points each is 30 ms. This low computational cost is a great advantage, as the system can be applied in real time at a high frame rate.
4. Conclusion This paper proposes a robust registration procedure to estimate and remove vibrations in 3D reconstruction sensors based on laser light sectioning. The extracted height profiles, while the object moves forward along a track, provide enough information to approximate the surface of the object accurately. In uncontrolled conditions, vibrations affect the movement of the object, corrupting the extracted height profiles and leading to inaccurate 3D reconstructions. The proposed method is based on the registration of consecutive height profiles to estimate and compensate for the geometric transformation between the profiles caused by vibrations. The proposed method is based on the assumption that the position of some points between consecutive height profiles only varies due to vibrations, i.e., these points are not affected by a change in the shape of the object. This requirement is met in different types of objects and materials, notably in web material. An extensive dataset, both synthetic and real, has been used to verify the performance of the proposed procedure, with excellent results. The results indicate that a wide range of synthetic objects can be faithfully reconstructed in 3D maps in spite of many different types of vibrations. Moreover, comparisons with the RanSac algorithm indicate that the proposed procedure provides much better results. Under extreme conditions, with white noise
146
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 14. Results using the RanSac algorithm with the pyramid corrupted with white noise. (a) First execution with RanSac 1e-2. (b) Second execution with RanSac 1e-2. (c) First execution with RanSac 1e-4. (d) Second execution with RanSac 1e-4. (e) First execution with RanSac 1e-6. (f) Second execution with RanSac 1e-6.
Fig. 15. Proposed procedure applied to the 3D reconstruction of steel strips. (a) Strip 1. (b) Result after removing vibrations in strip 1. (c) Strip 2. (d) Result after removing vibrations in strip 2. (e) Strip 3. (f) Result after removing vibrations in strip 3. (g) Strip 4. (h) Result after removing vibrations in strip 4.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
corrupting the surfaces, a low pass filter is required to keep a low reconstruction error. Tests with real data show very good results, qualitatively comparable to the original surfaces. These experiments with real data show how vibrations affect the movement of the material to a greater or lesser extent. It also proves the success of the proposed procedure in real conditions and in real-time.
1500 Manual Without filter With filter
I−units
1000
500
0
Z1
Z2
147
Z3
Z4
Measurement position
Fig. 16. Comparison between manual measurement and a laser-sectioning sensor with the proposed procedure.
Table 1 Quantitative comparison with test piece in I-unit. Vibrations
Trans. Trans. and rot. (one side) Trans. and rot. (both sides)
Without filter (I-units)
With filter (I-units)
Wave
Peak
Flat
Wave
Peak
Flat
1726 3059 2177
850 2235 821
195 0 485
1224 1204 1209
619 627 618
0 0 0
Table 2 Quantitative comparison with test piece in percentage error. Vibrations
Trans. Trans. and rot. (one side) Trans. and rot. (both sides) Fig. 17. Test piece used for quantitative evaluation.
Without filter (%)
With filter (%)
Wave
Peak
Flat
Wave
Peak
Flat
41.35 150.53 78.29
39.34 266.39 34.59
NA 0 NA
0.27 1.33 0.98
1.53 2.78 1.36
0 0 0
Fig. 18. Proposed procedure applied to the test piece. (a) Raw reconstruction of the test piece with random translations. (b) Result after removing vibrations in (a). (c) Raw reconstruction of the test piece with random translations and rotations on once side. (d) Result after removing vibrations in (b). (e) Raw reconstruction of the test piece with random translations and rotations on both sided. (f) Result after removing vibrations in (e).
148
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Acknowledgments This work has been partially funded by the project TIN200124903 of the Spanish National Plan for Research, Development and Innovation. The authors would like to thank the technicians and engineers of ArcelorMittal for their help during tests and for their gracious time. The authors are grateful to anonymous reviewers for constructive feedback and insightful suggestions that greatly improved this article. References
[15]
[16]
[17] [18]
[19] [20]
[1] Bernardini F, Rushmeier H. The 3d model acquisition pipeline. In: Computer graphics forum, vol. 21, no. 2. Wiley Online Library; 2002. p. 149–72. [2] Zhang G, Wei Z. A novel calibration approach to structured light 3d vision inspection. Opt Laser Technol 2002;34(5):373–80. [3] Sansoni G, Trebeschi M, Docchio F. State-of-the-art and applications of 3d imaging sensors in industry, cultural heritage, medicine, and criminal investigation. Sensors 2009;9(1):568–601. [4] Molleda J, Usamentiaga R, García DF, Bulnes FG, Espina A, Dieye B, et al. An improved 3d imaging system for dimensional quality inspection of rolled products in the metal industry. Comput Ind 2013;64(9):1186–200. [5] Magalhães DS, Serra RL, Vannucci AL, Moreno AB, Li LM. Glasses-free 3d viewing systems for medical imaging. Opt Laser Technol 2012;44(3):650–5. [6] Randall RB. Vibration-based condition monitoring: industrial, aerospace and automotive applications. West Sussex, United Kingdom: John Wiley & Sons; 2011. [7] Pernkopf F. 3d surface acquisition and reconstruction for inspection of raw steel products. Comput Ind 2005;56(8):876–85. [8] Yu S-J, Sukumar SR, Koschan AF, Page DL, Abidi MA. 3d reconstruction of road surfaces using an integrated multi-sensory approach. Opt Lasers Eng 2007;45 (7):808–18. [9] Usamentiaga R, Garcia DF, Molleda J, Bulnes FG, Bonet G. Vibrations in steel strips: effects on flatness measurement and filtering. IEEE Trans Ind Appl 2014;50(5):3103-3112, http://dx.doi.org/10.1109/TIA.2014.2308359. [10] Doignon C, Knittel D. A structured light vision system for out-of-plane vibration frequencies location of a moving web. Mach Vis Appl 2005;16 (5):289–97. [11] Kindt P, Sas P, Desmet W. Measurement and analysis of rolling tire vibrations. Opt Lasers Eng 2009;47(3):443–53. [12] Usamentiaga R, Molleda J, Garcia DF, Bulnes FG. Removing vibrations in 3d reconstruction using multiple laser stripes. Opt Lasers Eng 2014;53:51–9. [13] Salvi J, Pages J, Batlle J. Pattern codification strategies in structured light systems. Pattern Recognit 2004;37(4):827–49. [14] Zhang L, Curless B, Seitz SM. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Proceedings of first
[21] [22] [23] [24]
[25] [26]
[27]
[28] [29]
[30] [31]
[32] [33]
international symposium on 3D data processing visualization and transmission, 2002. IEEE; 2002. p. 24–36. Kawasaki H, Furukawa R, Sagawa R, Yagi Y. Dynamic scene shape reconstruction using a single structured light pattern. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE; 2008. p. 1–8. Xu J, Xi N, Zhang C, Shi Q, Gregory J. Real-time 3d shape inspection system of automotive parts based on structured light pattern. Opt Laser Technol 2011;43 (1):1–8. Je C, Park H-M. Optimized hierarchical block matching for fast and accurate image registration. Signal Process: Image Commun 2013;28(7):779–91. Winkelbach S, Molkenstruck S, Wahl FM. Low-Cost laser range scanner and fast surface registration approach. In: Franke K, Müller K-R, Nickolay B, Schäfer R, editors. Pattern Recognition, Lecture Notes in Computer Science, 4174. Berlin, Heidelberg: Springer; 2006. p. 718–8. http://dx.doi.org/10.1007/ 11861898_72. He B, Lin Z, Li YF. An automatic registration algorithm for the scattered point clouds based on the curvature feature. Opt Laser Technol 2013;46:53–60. Castellani U, Bartoli A. 3d shape registration. In: 3D Imaging, analysis and applications. London: Springer; 2012. p. 221–64. Zitova B, Flusser J. Image registration methods: a survey. Image Vis Comput 2003;21(11):977–1000. Besl PJ, McKay ND. A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 1992;14(Feb. (2)):239–56. Segal A, Haehnel D, Thrun S. Generalized-ICP. In: Robotics: science and systems, vol. 2, no. 4; 2009. Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. In: Proceedings of third international conference on 3-D digital imaging and modeling, 2001. IEEE; 2001. p. 145–52. Forsyth DA, Ponce J. Computer vision: a modern approach. New Jersey, USA: Prentice Hall Professional Technical Reference; 2002. Klasing K, Althoff D, Wollherr D, Buss M. Comparison of surface normal estimation methods for range sensing applications. In: IEEE international conference on robotics and automation, 2009. ICRA'09. IEEE; 2009. p. 3206–11. Berkmann J, Caelli T. Computation of surface geometry and segmentation using covariance techniques. IEEE Trans Pattern Anal Mach Intell 1994;16 (11):1114–6. Hoppe H, DeRose T, Duchamp T, McDonald J, Stuetzle W. Surface reconstruction from unorganized points, vol. 26, no. 2, ACM; 1992. Diebel J, Reutersward K, Thrun S, Davis J, Gupta R. Simultaneous localization and mapping with active stereo vision. In: Proceedings of 2004 IEEE/RSJ international conference on intelligent robots and systems, 2004 (IROS 2004), vol. 4. IEEE; 2004. p. 3436–43. Fusiello A, Trucco E, Tommasini T, Roberto V. Improving feature tracking with robust statistics. Pattern Anal Appl 1999;2(4):312–20. Fischler M, Bolles R. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981;24(6):381–95. Ong EP, Spann M. Robust optical flow computation based on least-median-ofsquares regression. Int J Comput Vis 1999;31(1):51–82. Usamentiaga R, Molleda J, García DF, Bulnes FG. Machine vision system for flatness control feedback. In: Second international conference on machine vision, 2009. ICMV'09. IEEE; 2009. p. 105–110.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Robust registration for removing vibrations in 3D reconstruction of web material Rubén Usamentiaga n, Daniel F. Garcia Department of Computer Science and Engineering, University of Oviedo, Campus de Viesques, 33204 Gijón, Asturias, Spain
art ic l e i nf o
a b s t r a c t
Article history: Received 22 July 2014 Received in revised form 28 October 2014 Accepted 22 November 2014
Vibrations are a major challenge in laser-based 3D reconstruction of web material. In uncontrolled environments, the movement of web material forward along a track is inevitably affected by vibrations. These oscillations significantly degrade the performance of the 3D reconstruction system, as they are incorrectly interpreted as irregularities on the surface of the material, leading to an erroneous reconstruction of the 3D surface. This work proposes a method to estimate and remove these vibrations based on a robust registration procedure. Registration is used to estimate vibrations and a rigid transformation is used to compensate the movements, removing the effects of vibrations on 3D reconstruction. The proposed method is applied to an extensive dataset, both synthetic and real, with very good results. & 2015 Elsevier Ltd. All rights reserved.
Keywords: Structured light 3D reconstruction Vibrations
1. Introduction 3D reconstruction of web material, such as steel or paper, is generally carried out using laser light sectioning sensors [1]. A laser projects a light plane onto the material while it moves forward along a track. The projected laser stripe is observed by a camera, and the coordinates of the stripe are extracted from the image and converted to real world units using a previously calibrated map [2]. The resulting signal is a height profile that provides information about the height of a section of the material. This process is repeated while the objects move forward, resulting in a 3D map of the surface of the material. The main advantages of this 3D reconstruction method are its accuracy and resilience to changes in illumination [3]. For this reason, this is the most widely used method in industrial environments [4]. Vibrations are a major challenge in laser-based 3D reconstruction: periodic or aperiodic movements while the material is moving forward along a track result in incorrect estimations of height profiles, which are propagated to the 3D reconstruction. In controlled conditions, vibrations can be avoided or reduced using various techniques. For example, they can be absorbed by a pneumatic system [5]. However, in uncontrolled environments, such as the industrial field, vibrations are very difficult to avoid [6]. Therefore, in order to obtain an accurate 3D reconstruction of an object, it is necessary to apply a method that can be used to estimate and remove the effects of vibrations on 3D reconstruction.
n
Corresponding author. Tel.: þ 34 985 182626; fax: þ 34 985 181986. E-mail address: [email protected] (R. Usamentiaga).
http://dx.doi.org/10.1016/j.optlaseng.2014.11.019 0143-8166/& 2015 Elsevier Ltd. All rights reserved.
When there are known invariants between consecutive height profiles, vibrations can be estimated and removed relatively easily. For example, when an object is placed on a flat conveyor belt, vibrations affect both the object and the conveyor belt. The known shape of the conveyor belt can be used to estimate the transformation between consecutive height profiles caused by vibrations. Similarly, when the shape of the object has some known features, such as straight lines connected by a curved corner [7], they can be detected and used to identify the transformation caused by vibrations. In the case of road surfaces, the invariant is the assumption of the planarity and smoothness of the road surfaces [8]. Another option is based on applying previous information about the type of vibrations, for example the frequency. Then, a band-pass filter can be used to accurately remove vibrations from the 3D reconstruction [9]. In other types of applications, the 3D reconstruction is not the focus, but rather the measurement of vibrations. In this case invariants, such as a constant shape, are used to estimate vibrations [10,11]. When there are no invariants, more than one laser stripe can be used [12]. Based on this approach, the height profile of the same section of an object is calculated as many times as the number of laser stripes projected onto the object. The redundant information obtained using this procedure is used to estimate and remove the effects of vibrations on 3D reconstructions. Two laser stripes are required to estimate simple vibrations. Three laser stripes provide enough information to estimate more complex vibrations. The disadvantages of this approach are, that additional laser projectors can significantly increase the cost of a 3D reconstruction system, and that calibration becomes much more complex, as the lasers must be aligned very accurately. Additional information about the shape of the object to compensate for vibrations can also be obtained using structured light
136
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 1. Laser-based 3D reconstruction of an object with changing surface normals. (a) Surface without vibrations. (b) Laser scanning of the surface without vibrations. (c) Laser scanning of the surface with vibrations. (d) Same surface with vibrations.
sensors [13]. A light pattern could be projected on the object in order to obtain a point cloud with height information about the shape of the object [14–16]. Image registration techniques could then be used to align two consecutive scans and compensate for vibrations [17]. However, 3D sensors based on structured-light patterns are sensitive to ambient light and reflectance, they cannot be used with shiny surfaces such as steel, have a much more complex calibration, and require that the projector is installed close to the object. Moreover the frame rate of these sensors is usually low, thus they cannot be applied to very fast moving objects. Registration is also problematic when the object does not have distinguishing features that can be used to align consecutive scans, such as a flat object. This work presents a method to estimate and remove the effects of vibrations in laser-based 3D reconstruction: robust registration between consecutive height profiles. Two consecutive height profiles are generally acquired from two very close sections of an object. Thus, the positions of some points do not change between consecutive height profiles. These points, inliers from the point of view of the registration, create an invariant between consecutive height profiles that can be exploited to estimate and remove vibrations. In order to apply this procedure correctly, inliers must be detected robustly without previous information. This work proposes a robust registration based on the median distance and the surface normals of the height profiles. The result is a robust alignment of height profiles that removes vibrations from the final 3D reconstruction. Surface registration from different angles of the same object is a method used by many commercial 3D laser scanners [18]. This process requires views to partially overlap [19]: one part of the point cloud describing the surface of an object from a specific angle overlaps the same part of the object in another point cloud. These sets of points observed from different angles are the key to registering and merging the surfaces. The invariant that is used for registration are the overlapping points. In this work the same principle is applied, however, the goal is to find the points that can be used between consecutive height profiles to detect the geometric transformation caused by vibrations. The main contribution of this work is the application of registration techniques to the estimation and removal of vibrations, and the optimization of these techniques for the particular case of height profiles.
2. Registration of height profiles Laser-based 3D reconstruction involves projecting a laser stripe onto the object. The laser stripe is observed by a camera, and the
coordinates of the laser points in the image are extracted. These coordinates are translated to world coordinates resulting in a height profile, i.e., the height of a section of the object. This technique is called laser triangulation because the laser points, the camera and the laser projector form a triangle. A set of consecutive height profiles can be used to reconstruct the whole surface of the object. Fig. 1a shows an example of a surface: a pyramid. Laserbased reconstruction would produce an approximation of the surface, as can be seen in Fig. 1b. Increasing the acquisition rate produces a denser reconstruction, more similar to the real surface. 3D reconstruction based on laser projection requires a relative movement between the camera and laser projector, and the object. This movement can be affected by vibrations, which produce variations in the position of the object perpendicular to its movement. Vibrations while the object moves forward along a track modify the position where the laser stripe is projected. Thus, they are interpreted as height variations. For example, if the surface in Fig. 1a is affected by vibrations the calculated height profiles could be similar to Fig. 1c. In this example, it is considered that the movement of the object is affected by random vertical translations and by rotations, two common types of vibrations. The resulting 3D reconstruction of the surface would be similar to Fig. 1d. As can be seen, the reconstruction is completely inaccurate because it has been corrupted by the vibrations. The method proposed in this work to estimate and remove vibrations is based on a robust registration between consecutive height profiles. The general goal of registration is to find the best alignment between two objects or instances of the same object [20]. The shapes of the objects do not need to be identical, a partial overlap is enough. Registration can be divided into several types, depending on the kind of data used for alignment [21]. Data can be acquired from different viewpoints of the same scene, or from different sensors. It is also common to perform registration between a view of the object and a reference model. This latter type of registration is used when data is acquired at different times. In this case, the aim is to find changes in the scene that appear between the consecutive acquisitions. This is the type of registration proposed for height profiles. Registration is generally applied to images or cloud points. In this work, registration is applied to height profiles, which are a particular case of 2D cloud points. The registration process is based on the assumption that alignment is possible between height profiles, that is, that the positions of some points do not change much. Vibrations can be modeled using a rigid transformation and the object is assumed to be a rigid material. Therefore, vibrations do not cause a deformation of the object. When vibrations affect
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
137
Fig. 2. Registration of height profiles with changing surface normals. (a) Two consecutive height profiles. (b) Correspondences between the two profiles. (c) Initial transformation. (d) Normals of the first height profile. (e) Normals of the second height profile. (f) Correspondences of points when the similarity of normals are considered. (g) Correspondences of points when both the similarity of normals and distance are considered. (h) Final transformation.
the height profile, the positions of all points vary. However, there are some points where the change of position is mostly caused by vibrations, not by a change in the shape of the object. This assumption is true for web material, but can also be true for many other types of objects. These inliers can be used to estimate and remove vibrations. The challenge is to detect these points robustly without previous information. The proposed registration procedure is based on an adaptation of the Iterative Closest Point (ICP) algorithm [22]. The ICP algorithm is the state-of-the-art solution for registration [23]. The overall aim of the algorithm is to estimate a rigid transformation that minimizes a metric distance between the two datasets, such as the average distance. The estimation of the transformation improves with each iteration of the ICP algorithm. The algorithm is proved to converge monotonically to a local solution of the problem, given a proper initial alignment [24]. The proposed algorithm for the registration of height profiles is as follows: (1) Initial transformation: From one height profile to the next, the width can be considered constant in the case of web material. Thus, a direct correspondence between the points in the two profiles can be assumed. Therefore, an initial transformation is performed by considering all the points in the two profiles. Fig. 2a shows two consecutive height profiles acquired from the surface in Fig. 1c. The shape of the object has changed from one height profile to the next. In addition, vibrations have translated
the second profile upwards and rotated it. A direct correspondence between the points in the two profiles is carried out, as can be seen in Fig. 2b. Then, a rigid transformation between the two sets of points is calculated using the least squares method [25]. This transformation is applied to the second scan, resulting in Fig. 2c. (b) Detection of inliers: Inliers in the two height profiles are detected as the points that meet two criteria: similar surface normals and similar distances. First, the normals of the two height profiles are calculated. The calculation of the surface normal for a point p involves fitting a window of points around p to a line. The normal for this point is the normal of the fitted line, or plane in the case of 3D points [26]. Another very efficient approach to estimate the normals is based on the covariance matrix [27]. This process is repeated for each point of the two profiles. The results can be seen in Fig. 2d and e. In order to avoid orientation inconsistency, all normals are considered with a positive y axis. A similarity score, sc, for the normals in the two height profiles, n1 and n2, is calculated as sc ¼ n1 n2
ð1Þ
A high value of sc means similar surface normals [28]. Therefore, a threshold can be applied to identify surface normal similarity. Fig. 2f shows the points in the two height profiles with similar surface normals. This set of points are potential inliers. As can be seen, there is an outlier in the peak that has the same surface normal in the two height profiles.
138
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 3. Laser-based 3D reconstruction of an object with similar surface normals. (a) Surface without vibrations. (b) Laser scanning of the surface without vibrations. (c) Laser scanning of the surface with vibrations. (d) Same surface with vibrations.
Next, the median distance between the potential inliers is calculated. Only those inlier candidates whose distance is lower than k times the median distance are considered valid inliers [29]. The resulting inliers can be seen in Fig. 2g. As can be seen, the outlier in the peak has been removed. Another popular method to reject outliers is the X84 rule [30], which uses robust estimates for location and scale to set a rejection threshold. However, this method cannot be applied for the considered problem because it assumes an underlying Gaussian distribution of the residuals, that is, the distances. The method is based on the calculation of the median absolute deviation, which is similar to the interquartile range. The X84 rule prescribes rejecting values that are more than k median absolute deviations away from the median. In the example of Fig. 2f, this value is zero, thus it would reject all points. (3) Refined transformation: A new rigid transformation between height profiles is calculated considering only the inliers. The result of this transformation can be seen in Fig. 2h. The result of this transformation has removed vibrations completely. Height profiles are perfectly aligned, even when the shape of the object has changed partially. (4) Iterate: In the height profiles used in the example, one iteration was enough to compensate for the vibrations. More complex vibrations could require more than one iteration. Thus, steps 2 and 3 are repeated until the change between two successive iterations is lower than an established threshold. In the example shown in Fig. 1 most inliers are detected due to the surface normals. However, changes in the shape of the object can keep normals intact. This is the case shown in Fig. 3, which has been corrupted with the same vibrations as in the example in Fig. 1. The registration process in this case is shown in Fig. 4. As can be seen, the rejection due to differences in surface normals does not work with this surface. Fig. 4d and e shows that all points in the two profiles have the same surface normals. However, the rejection based on the median distance detects inliers correctly, as can be seen in Fig. 4g. This is exploited to align height profiles and remove vibrations. The result is a perfect alignment between the two profiles, shown in Fig. 4h. The registration process, R, is applied to two consecutive height profiles, Hi and H i þ 1 . The result is a modified version of the second profile aligned with the first one, H 0i þ 1 . This process can be expressed as H 0i þ 1 ¼ RðH i ; H i þ 1 Þ
ð2Þ
The registration of height profiles is repeated for all consecutive height profiles, starting with the first two profiles. Thus, for a generic profile j the registration is carried out using (3). The registration of height profile j is carried out against the previously aligned profile. The first height profile does not have a previous profile, thus (2) starts with profile number 2. Profile 1 can be aligned against a horizontal profile. Applying this process to the surfaces shown in Figs. 1d and 3d produces the surfaces without vibrations as shown in Figs. 1a and 3a, respectively. H j ¼ RðH 0j 1 ; H j Þ
8 j ¼ 2…n
ð3Þ
2.1. Comparison with RanSac A different approach to achieve robust registration between consecutive height profiles is the RanSac model estimator [31]. The RanSac algorithm is an iterative procedure to estimate models with data that includes outliers. It works by selecting small subsets of the available data, fitting the model and calculating the percentage of the total data that fits the model. Despite its simplicity, it works very well even with a high percentage of outliers. In this case, the model is a rigid transformation between height profiles. The score of the model is the number of points in the two profiles that match after the rigid transformation. The results applying the RanSac algorithm show excellent performance. In the examples shown in Figs. 2 and 4 the results are similar to those obtained using the proposed procedure: perfect alignment. However, the proposed procedure clearly outperforms the RanSac algorithm when white noise is present in the height profiles. White noise is not produced by vibrations, but by other causes, such as inaccurate calibration or rounding errors. In order to compare the performance of the proposed procedure with the RanSac algorithm, a registration test has been performed. Two consecutive height profiles are corrupted with random white noise. The noise is uniformly distributed with a magnitude equal to 10% of the maximum height. Fig. 5a shows the original height profiles, and Fig. 5b shows the corrupted height profiles. Fig. 5c and d shows the worst alignment obtained using the proposed procedure and the RanSac algorithm, respectively, in 100 tests. In each test the profiles were corrupted with a different pattern with equal distribution and magnitude. As can be seen, the proposed procedure aligns the height profiles correctly even in the presence of white noise. On the other hand, with the RanSac algorithm fails the registration test due to the noise: only the right part of the profiles is aligned.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
139
Fig. 4. Registration of height profiles with similar surface normals. (a) Two consecutive height profiles. (b) Correspondences between the two profiles. (c) Initial transformation. (d) Normals of the first height profile. (e) Normals of the second height profile. (f) Correspondences of points when the similarity of normals are considered. (g) Correspondences of points when both the similarity of normals and distance are considered. (h) Final transformation.
Fig. 5. Registration of height profiles with noise. (a) Original consecutive height profiles. (b) Height profiles with noise. (c) Worst alignment with the proposed procedure. (d) Worst alignment with the RanSac algorithm.
The results are compared numerically using the normalized mean distance. This metric is calculated using (4), where n is the number of points in the height profile, Hij is point j of profile i after registration, Oij is point j of profile i in the original profile with neither white noise nor vibrations, and d is the Euclidean distance. The normalized mean distance is a percentage of error over the maximum height of the object, independently of the specific
height of the object: 1 Pn j ¼ 1 d H ij Oij n Error ¼ 100 maxðOi Þ minðOi Þ
ð4Þ
Fig. 6 shows the histograms of the normalized mean distance for all the tests. Fig. 6a shows the histogram for the proposed
140
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
3.1. Synthetic data
procedure. In 100% of the cases the error is around 10%, that is, the same error with which the height profiles were corrupted. Fig. 6b shows the results for the RanSac algorithm. In this case, the error increases: 20% of the tests obtain an error of 20% of the height of the profile. This 20% correspond to cases similar to Fig. 5d, where the RanSac algorithm does not align the height profiles correctly because they are affected by white noise. The RanSac algorithm can be controlled with a parameter that indicates the threshold distance between points to be considered inliers. This parameter can be relaxed to work better with noise. However, this also results in worse performance when noise is not present in the data. The proposed procedure works equally well in both cases: with or without noise. The reason why the proposed procedure outperforms the RanSac algorithm in the presence of noise is that considering only the distance between points does not provide robust results. The RanSac algorithm works very well with a high percentage of outliers, but only when the other points are good matches without random noise.
In order to assess the proposed procedure, several experiments have been carried out. Four different synthetic objects were created (Fig. 7). The first two (Fig. 7a and b) are the objects used in Section 2: a pyramid and a triangular prism. These two objects have flat surfaces. Two more objects were created with curved surfaces (Fig. 7c and d). The object in Fig. 7c represents a curved object with some constant points on both sides, that is, inliers. Fig. 7d is a curved object with no constant points, that is, all points vary from one height profile to the next. This object does not meet the required assumption to apply the proposed procedure, where some inliers between consecutive height profiles are required. However, this object will be used to verify the performance of the proposed procedure with this undesirable situation. Four types of vertical translations are applied to the objects (Fig. 8). These translation models are based on plausible vibrations that can affect the movement of the object. The sinusoidal pattern (Fig. 8a) assumes a periodic movement of the object while it moves forward, upwards and downwards, without actually touching the roll that moves the object. The damped sinusoidal pattern (Fig. 8c) assumes that vibrations lose energy faster than it is being supplied. Semi-sinusoidal translations (Fig. 8c) assume that the object hits the roll periodically producing only half a sinusoidal
3. Results and discussion
100
100
80
80
Percentage of tests
Percentage of tests
In this section, the proposed procedure is applied to synthetic and real data to test performance.
60 40 20 0
0
10
20
30
40
50 Error
60
70
80
90
100
60 40 20 0
0
10
20
30
40
50
60
70
80
90
100
Error
Fig. 6. Histograms of the normalized mean distance. (a) Proposed procedure. (b) RanSac algorithm.
Fig. 7. Synthetic objects used for tests. (a) Pyramid. (b) Triangular prism. (c) Curved surface with inliers. (b) Curved surface with no inliers.
Fig. 8. Types of translations considered. (a) Sinusoidal translation pattern. (b) Damped sinusoidal translation pattern. (c) Semi-sinusoidal translation pattern. (b) Random translation pattern.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
movement. Random translations (Fig. 8d) assume movements without pattern, defined using a random uniform distribution. Another type of vibration is rotations that produce semi-circular movements of the object around a pivot position, similar to rocking. A sinusoidal pattern is assumed for rotations while the object moves forward. All these types of vibrations are common in industrial applications. Four vibration patterns are used to corrupt the shape of the considered synthetic objects: (a) Vertical translations following a sinusoidal pattern. (b) A combination of vertical translations following a damped sinusoidal pattern and rotations around a pivot point on the edge of the object. (c) A combination of vertical translations following a random pattern and rotations around a pivot point in the middle of the object. (d) A combination of vertical translations following a semisinusoidal pattern and rotations around a pivot point on the middle of the object. The resulting object has been corrupted with additional random white noise. The noise is uniformly
141
distributed with a magnitude equal to 10% of the maximum height. This combination of vibrations and white noise can be considered an extreme case, used to test the proposed procedure under very exceptional conditions. The proposed robust registration procedure for removing the effects of vibrations on 3D reconstruction has been applied to the four synthetic objects corrupted with these four vibrations patterns. The results can be seen in Figs. 9–12. The first column of these figures shows the corrupted surfaces with vibrations, and the second column shows the resulting surface after applying the proposed procedure. The results with the pyramid and the triangular prims can be seen in Figs. 9 and 10, respectively. The first three cases are combinations of different types of vibrations. As can be seen, the proposed procedure removes these vibrations completely, resulting in an accurate 3D reconstruction of the surface of the original object. In these three cases, the error measured using (4) provides a value of zero, that is, the surface is reconstructed perfectly. When the proposed procedure is applied when white noise is added after
Fig. 9. Proposed procedure applied to the pyramid. (a) Pyramid with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Pyramid with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Pyramid with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Pyramid with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
142
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 10. Proposed procedure applied to the triangular prism. (a) Triangular prism with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Triangular prism with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Triangular prism with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Triangular prism with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
vibrations, it does not provide a perfect result, as can be seen in Figs. 9h and 10h. The white noise used to corrupt the surface of the object is present in the resulting 3D map. The proposed procedure is not designed to remove this type of noise, only vibrations affecting the height profiles. The error in the resulting map is around 10%, similar to the magnitude of the white error added to the synthetic object. The registration procedure is also slightly affected by the white noise. When noise is higher than the variation between height profiles, some points are incorrectly considered as inliers. This causes an erroneous estimation of the vibrations, resulting in a slightly modified 3D map. This issue can be seen in Fig. 9h where a small curvature can be observed along the surface. The curved surface with inliers provides similar results to the two previous examples, as can be seen in Fig. 11. The curved surface does not affect the performance of the proposed procedure. Inliers are correctly detected and the resulting maps are accurate reconstructions of the object. White noise affects the proposed procedure, producing an error with similar magnitude to the white noise. The curved surface without inliers is a great challenge for the proposed procedure. The registration requires inliers between
consecutive height profiles. Therefore, the removal of vibrations from this object is difficult. The results can be seen in Fig. 12. In the first three cases, vibrations are not detected correctly. Parts of the height profile in the middle are considered as inliers. Thus, the transformation considering these points removes part of the variation in that zone and increases the variation on the edges. In these three cases, the problem results in an error of around 5%. When white noise is added this error increases up to 10%. The results are inaccurate 3D reconstructions of the object. However, the results are still much better than the reconstruction obtained without filtering. Moreover, it does not result in a spurious transformation. Therefore, even in this extreme case the results can be considered positive. The results shown in Figs. 9g, 10g, 11g, and 12g indicate that the proposed procedure is affected by white noise. The experiments were repeated with the addition of a standard smoothing low-pass filter on the surface of the object before using the proposed method. The resulting error decreased sharply: to 0.3% on average. Clearly, when white noise is present, it is advisable to use this type of filter.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
143
Fig. 11. Proposed procedure applied to the curved surface with inliers. (a) Surface with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Surface with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Surface with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Surface with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
Fig. 13 shows the results using the RanSac algorithm. The first vibration pattern corresponding to vertical translations following a sinusoidal pattern is applied to the pyramid and the curved surface without inliers. The main parameter of the RanSac algorithm is the threshold distance between points to be considered inliers. Three configurations are used: 1e-2, 1e-4, and 1e-6. The best configuration for the pyramid is a threshold distance of 1e-6. However, this same configuration leads to an incorrect result when applied to the curved surface. The best configuration for this model is a threshold distance of 1e-2. The best results are the same as those obtained with the proposed procedure. However, the RanSac algorithm requires different configurations for different shapes. A possible solution is a different robust estimator: the Least Median of Squares [32]. The advantage compared with RanSac is that no threshold configuration is required. Tests with this method provided very good results in some cases, but when the number on inliers was above 50%, the results were worse than RanSac, and worse than the proposed procedure. This is the case of the pyramid object. The problem is that this method always assumes that a least 50% of the points are good correspondences.
The RanSac algorithm has also been applied when white noise is added. Fig. 14 shows the results for the pyramid corrupted with vibrations and random white noise (vibration pattern 4). The RanSac algorithm is applied with the same three configurations, and is executed twice for each of them. The results show one major drawback of the RanSac algorithm with white noise: unpredictability. Not only are the results much worse than the proposed procedure, the results are different for each execution. This tests confirms previous results, and clearly indicates that the proposed procedure offers much better robustness and accuracy than the RanSac algorithm. 3.2. Real data The proposed procedure has been applied to real data obtained from the 3D reconstruction of steel strips. This type of web material is generally 2 m wide and 10 km long. 3D reconstruction is required for quality control, such as flatness inspection [33]. Data about the surface of the strips is obtained using a laser-sectioning sensor. Fig. 15a, c, and e shows the data obtained from this sensor for three different steel strips. Fig. 15a shows a steel strip with a defect called
144
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 12. Proposed procedure applied to the curved surface with no inliers. (a) Surface with sinusoidal vibrations. (b) Result after removing sinusoidal vibrations. (c) Surface with damped sinusoidal and rotation vibrations. (d) Result after removing damped sinusoidal and rotation vibrations. (e) Surface with random and rotation vibrations. (f) Result after removing random and rotation vibrations. (g) Surface with semi-sinusoidal and rotation vibrations plus white noise. (h) Result after removing semi-sinusoidal and rotation vibrations plus white noise.
wavy edges, caused by over-rolling. The strip in Fig. 15c has a curved surface, and the strip in Fig. 15e has a flat surface. The surfaces of these strips are corrupted with vibrations of different frequencies, vertical translations and rotations. The results after applying the proposed procedure can be seen in Fig. 15b, d, and f. These results are compatible with the shape of the strips. Clearly, vibrations have been removed, producing a clean 3D reconstruction of the surface of the strips. These results indicate that the proposed procedure greatly improves the resulting 3D reconstruction, producing a surface equivalent to that which would be obtained if the movement were not affected by vibrations. In order to assess the proposed procedure quantitatively, a manual experiment was carried out. The objective is to compare the results with those obtained using a laser-sectioning sensor with the proposed filtering activated. A steel strip that has wavy edges with a sinusoidal shape was used. The technicians measured the wave length and wave height at the edges of the strip. These type of defects are periodic, thus wave length and wave height can be used to estimate the length of the strip at that edge. These results can be compared with those obtained using a lasersectioning sensor, with or without the proposed procedure.
A common parameter used for the quantitative evaluation of the strip shape is the I-unit. The I-unit index is calculated using (5), where Li is the length at position i, or wave length, and Lref is the length of a reference flat fiber: I unitðiÞ ¼
Li Lref 5 ΔLi 5 10 ¼ 10 Lref Lref
ð5Þ
The results of the experiments can be seen in Fig. 16. This bar chart show the I-units measured using the manual procedure and the I-units calculated using a laser light-sectioning sensor with and without the filter activated. The strips are around 700 m long, and their I-units are measured in four different zones (Z1–Z4). The results obtained in manual measurements are similar to the results obtained when the proposed filter is activated. Without the filter, vibrations greatly modify the calculated I-units producing spurious results. The results with the filter activated are qualitatively comparable to manual measurements, which indicate that the proposed procedure for filtering vibrations is working correctly. Manual experiments are not a viable option for extended validation because they are time-consuming operations that require stopping the production line, which reduces productivity sharply.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
145
Fig. 13. Results when using the RanSac algorithm with sinusoidal vibrations. (a) Pyramid with RanSac 1e-2. (b) Curved surface with RanSac 1e-2. (c) Pyramid with RanSac 1e4. (d) Curved surface with RanSac 1e-4. (e) Pyramid with RanSac 1e-6. (f) Curved surface with RanSac 1e-6.
Therefore, in order to provide additional quantitative results of different defects with different vibration patterns a test piece made of aluminum and plastic was used. A set of artificial defects were added to the flat surface consisting of waves (on the left edge, 1221 I-units), peaks (on the center, 610 I-units) and a flat surface (on the right edge, 0 I-units). The test piece is moved using a roll table to reproduce the industrial conditions, providing the correct movement and vibrations required to assess the performance of the proposed procedure. Fig. 17 shows a photograph of the test piece and the laser stripe projection while the piece is moved by the rolls. The results of the experiments can be seen in Fig. 18. In the figure, the scale in the Z axis is not the same as in the X or Y in order to appreciate translations more clearly. Three vibrations patterns are considered: random translations, a combination of random translations and rotations on one side, and a combination of random translations and rotations on both sides. Rotations are introduced by creating artificial defects in the rolls that move the test piece. As can be seen, there is a huge improvement between the raw reconstruction of the surface and the result using the proposed procedure. The results when the filter is activated are vibration-free 3D reconstructions. Table 1 shows the measured I-units of the test piece. Table 2 shows the same data but in this case compared with the real values in terms of percentage error. The resulting error when the proposed procedure is applied is very low. However, the raw data provides a large error, in some cases the measure value is more than two times the real value. These results indicate that the proposed method makes it possible to estimate and remove vibrations from the surface of 3D reconstruction of the objects. Although vibration cannot be avoided in industrial environments, the proposed registration procedure greatly improves the 3D reconstruction results provided by standard laser-sectioning sensors. Not only does it provide robust results, but it provides them quickly. Many types of applications, such as industrial inspection systems, require fast processing
algorithms in order to detect problems and apply corrective measures as quickly as possible. In order to test the speed of the proposed procedure, the time necessary to filter standard 3D maps has been measured. The experiments were performed using an Intel Core i7 4770 running at 3.4 GHz with 16 GB of RAM. The algorithm was coded in Cþ þ . The average measured execution time for 3D maps consisting of 2000 height profiles of 100 points each is 30 ms. This low computational cost is a great advantage, as the system can be applied in real time at a high frame rate.
4. Conclusion This paper proposes a robust registration procedure to estimate and remove vibrations in 3D reconstruction sensors based on laser light sectioning. The extracted height profiles, while the object moves forward along a track, provide enough information to approximate the surface of the object accurately. In uncontrolled conditions, vibrations affect the movement of the object, corrupting the extracted height profiles and leading to inaccurate 3D reconstructions. The proposed method is based on the registration of consecutive height profiles to estimate and compensate for the geometric transformation between the profiles caused by vibrations. The proposed method is based on the assumption that the position of some points between consecutive height profiles only varies due to vibrations, i.e., these points are not affected by a change in the shape of the object. This requirement is met in different types of objects and materials, notably in web material. An extensive dataset, both synthetic and real, has been used to verify the performance of the proposed procedure, with excellent results. The results indicate that a wide range of synthetic objects can be faithfully reconstructed in 3D maps in spite of many different types of vibrations. Moreover, comparisons with the RanSac algorithm indicate that the proposed procedure provides much better results. Under extreme conditions, with white noise
146
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Fig. 14. Results using the RanSac algorithm with the pyramid corrupted with white noise. (a) First execution with RanSac 1e-2. (b) Second execution with RanSac 1e-2. (c) First execution with RanSac 1e-4. (d) Second execution with RanSac 1e-4. (e) First execution with RanSac 1e-6. (f) Second execution with RanSac 1e-6.
Fig. 15. Proposed procedure applied to the 3D reconstruction of steel strips. (a) Strip 1. (b) Result after removing vibrations in strip 1. (c) Strip 2. (d) Result after removing vibrations in strip 2. (e) Strip 3. (f) Result after removing vibrations in strip 3. (g) Strip 4. (h) Result after removing vibrations in strip 4.
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
corrupting the surfaces, a low pass filter is required to keep a low reconstruction error. Tests with real data show very good results, qualitatively comparable to the original surfaces. These experiments with real data show how vibrations affect the movement of the material to a greater or lesser extent. It also proves the success of the proposed procedure in real conditions and in real-time.
1500 Manual Without filter With filter
I−units
1000
500
0
Z1
Z2
147
Z3
Z4
Measurement position
Fig. 16. Comparison between manual measurement and a laser-sectioning sensor with the proposed procedure.
Table 1 Quantitative comparison with test piece in I-unit. Vibrations
Trans. Trans. and rot. (one side) Trans. and rot. (both sides)
Without filter (I-units)
With filter (I-units)
Wave
Peak
Flat
Wave
Peak
Flat
1726 3059 2177
850 2235 821
195 0 485
1224 1204 1209
619 627 618
0 0 0
Table 2 Quantitative comparison with test piece in percentage error. Vibrations
Trans. Trans. and rot. (one side) Trans. and rot. (both sides) Fig. 17. Test piece used for quantitative evaluation.
Without filter (%)
With filter (%)
Wave
Peak
Flat
Wave
Peak
Flat
41.35 150.53 78.29
39.34 266.39 34.59
NA 0 NA
0.27 1.33 0.98
1.53 2.78 1.36
0 0 0
Fig. 18. Proposed procedure applied to the test piece. (a) Raw reconstruction of the test piece with random translations. (b) Result after removing vibrations in (a). (c) Raw reconstruction of the test piece with random translations and rotations on once side. (d) Result after removing vibrations in (b). (e) Raw reconstruction of the test piece with random translations and rotations on both sided. (f) Result after removing vibrations in (e).
148
R. Usamentiaga, D.F. Garcia / Optics and Lasers in Engineering 68 (2015) 135–148
Acknowledgments This work has been partially funded by the project TIN200124903 of the Spanish National Plan for Research, Development and Innovation. The authors would like to thank the technicians and engineers of ArcelorMittal for their help during tests and for their gracious time. The authors are grateful to anonymous reviewers for constructive feedback and insightful suggestions that greatly improved this article. References
[15]
[16]
[17] [18]
[19] [20]
[1] Bernardini F, Rushmeier H. The 3d model acquisition pipeline. In: Computer graphics forum, vol. 21, no. 2. Wiley Online Library; 2002. p. 149–72. [2] Zhang G, Wei Z. A novel calibration approach to structured light 3d vision inspection. Opt Laser Technol 2002;34(5):373–80. [3] Sansoni G, Trebeschi M, Docchio F. State-of-the-art and applications of 3d imaging sensors in industry, cultural heritage, medicine, and criminal investigation. Sensors 2009;9(1):568–601. [4] Molleda J, Usamentiaga R, García DF, Bulnes FG, Espina A, Dieye B, et al. An improved 3d imaging system for dimensional quality inspection of rolled products in the metal industry. Comput Ind 2013;64(9):1186–200. [5] Magalhães DS, Serra RL, Vannucci AL, Moreno AB, Li LM. Glasses-free 3d viewing systems for medical imaging. Opt Laser Technol 2012;44(3):650–5. [6] Randall RB. Vibration-based condition monitoring: industrial, aerospace and automotive applications. West Sussex, United Kingdom: John Wiley & Sons; 2011. [7] Pernkopf F. 3d surface acquisition and reconstruction for inspection of raw steel products. Comput Ind 2005;56(8):876–85. [8] Yu S-J, Sukumar SR, Koschan AF, Page DL, Abidi MA. 3d reconstruction of road surfaces using an integrated multi-sensory approach. Opt Lasers Eng 2007;45 (7):808–18. [9] Usamentiaga R, Garcia DF, Molleda J, Bulnes FG, Bonet G. Vibrations in steel strips: effects on flatness measurement and filtering. IEEE Trans Ind Appl 2014;50(5):3103-3112, http://dx.doi.org/10.1109/TIA.2014.2308359. [10] Doignon C, Knittel D. A structured light vision system for out-of-plane vibration frequencies location of a moving web. Mach Vis Appl 2005;16 (5):289–97. [11] Kindt P, Sas P, Desmet W. Measurement and analysis of rolling tire vibrations. Opt Lasers Eng 2009;47(3):443–53. [12] Usamentiaga R, Molleda J, Garcia DF, Bulnes FG. Removing vibrations in 3d reconstruction using multiple laser stripes. Opt Lasers Eng 2014;53:51–9. [13] Salvi J, Pages J, Batlle J. Pattern codification strategies in structured light systems. Pattern Recognit 2004;37(4):827–49. [14] Zhang L, Curless B, Seitz SM. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Proceedings of first
[21] [22] [23] [24]
[25] [26]
[27]
[28] [29]
[30] [31]
[32] [33]
international symposium on 3D data processing visualization and transmission, 2002. IEEE; 2002. p. 24–36. Kawasaki H, Furukawa R, Sagawa R, Yagi Y. Dynamic scene shape reconstruction using a single structured light pattern. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE; 2008. p. 1–8. Xu J, Xi N, Zhang C, Shi Q, Gregory J. Real-time 3d shape inspection system of automotive parts based on structured light pattern. Opt Laser Technol 2011;43 (1):1–8. Je C, Park H-M. Optimized hierarchical block matching for fast and accurate image registration. Signal Process: Image Commun 2013;28(7):779–91. Winkelbach S, Molkenstruck S, Wahl FM. Low-Cost laser range scanner and fast surface registration approach. In: Franke K, Müller K-R, Nickolay B, Schäfer R, editors. Pattern Recognition, Lecture Notes in Computer Science, 4174. Berlin, Heidelberg: Springer; 2006. p. 718–8. http://dx.doi.org/10.1007/ 11861898_72. He B, Lin Z, Li YF. An automatic registration algorithm for the scattered point clouds based on the curvature feature. Opt Laser Technol 2013;46:53–60. Castellani U, Bartoli A. 3d shape registration. In: 3D Imaging, analysis and applications. London: Springer; 2012. p. 221–64. Zitova B, Flusser J. Image registration methods: a survey. Image Vis Comput 2003;21(11):977–1000. Besl PJ, McKay ND. A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 1992;14(Feb. (2)):239–56. Segal A, Haehnel D, Thrun S. Generalized-ICP. In: Robotics: science and systems, vol. 2, no. 4; 2009. Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. In: Proceedings of third international conference on 3-D digital imaging and modeling, 2001. IEEE; 2001. p. 145–52. Forsyth DA, Ponce J. Computer vision: a modern approach. New Jersey, USA: Prentice Hall Professional Technical Reference; 2002. Klasing K, Althoff D, Wollherr D, Buss M. Comparison of surface normal estimation methods for range sensing applications. In: IEEE international conference on robotics and automation, 2009. ICRA'09. IEEE; 2009. p. 3206–11. Berkmann J, Caelli T. Computation of surface geometry and segmentation using covariance techniques. IEEE Trans Pattern Anal Mach Intell 1994;16 (11):1114–6. Hoppe H, DeRose T, Duchamp T, McDonald J, Stuetzle W. Surface reconstruction from unorganized points, vol. 26, no. 2, ACM; 1992. Diebel J, Reutersward K, Thrun S, Davis J, Gupta R. Simultaneous localization and mapping with active stereo vision. In: Proceedings of 2004 IEEE/RSJ international conference on intelligent robots and systems, 2004 (IROS 2004), vol. 4. IEEE; 2004. p. 3436–43. Fusiello A, Trucco E, Tommasini T, Roberto V. Improving feature tracking with robust statistics. Pattern Anal Appl 1999;2(4):312–20. Fischler M, Bolles R. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 1981;24(6):381–95. Ong EP, Spann M. Robust optical flow computation based on least-median-ofsquares regression. Int J Comput Vis 1999;31(1):51–82. Usamentiaga R, Molleda J, García DF, Bulnes FG. Machine vision system for flatness control feedback. In: Second international conference on machine vision, 2009. ICMV'09. IEEE; 2009. p. 105–110.