- Email: [email protected]
Volumetric Reconstruction of Objects and Scenes Using Range Images
Digital Signal Processing 9, 120–135 (1999) Article ID dspr.1999.0339, available online at http://www.idealibrary.com on
Volumetric Reconstruction of Objects and Scenes Using Range Images D. L. Elsner, R. T. Whitaker, and M. A. Abidi Department of Electrical Engineering, University of Tennessee, Knoxville, Tennessee 37996 E-mail: [email protected]
This paper reviews volumetric methods for fusing sets of range images to create 3D models of objects or scenes. It also presents a new reconstruction method, which is a hybrid that combines several desirable aspects of techniques discussed in the literature. The proposed reconstruction method projects each point, or voxel, within a volumetric grid back onto a collection of range images. Each voxel value represents the degree of certainty that the point is inside the sensed object. The certainty value is a function of the distance from the grid point to the range image, as well as the sensor’s noise characteristics. The super-Bayesian combination formula is used to fuse the data created from the individual range images into an overall volumetric grid. We obtain the object model by extracting an isosurface from the volumetric data using a version of the marching cubes algorithm. Results are shown from simulations and real range finders. r1999 Academic Press Key Words: range imaging; range scanners; data fusion; surface reconstruction; scene reconstruction; occupancy grids.
1. INTRODUCTION The creation of 3D models from range data is important in a variety of applications, including reverse engineering, object recognition, inspection and quality control, medical imaging, computer graphics, and remote mapping. Depending on the application, there are a number of different sensors and strategies that can be employed. This paper addresses the problem of 3D reconstruction using range data from a time-of-flight laser range finder also called LADAR (light amplitude detection and ranging). Range measurements give distances to object surfaces, and a scanning mechanism provides a 2D array of such measurements, i.e., a range map or range image, taken from a certain point of view (Fig. 1a). Range maps give what is commonly referred to as 1051-2004/99 $30.00 Copyright r 1999 by Academic Press All rights of reproduction in any form reserved.
120
FIG. 1. (a) A range scanner produces a 2D array of distance measurements from a single point of view. (b) The occupancy approach casts the probability of occupancy from each reading into the volume and then combines those readings.
2.5D information about a scene. They provide depth information from one point of view but they do not distinguish between objects that occlude one another and they do not provide a full 3D description of an object—i.e., not all surfaces of an object can be seen by a single range map. To obtain a full 3D description of an object, multiple range maps from several viewpoints must be combined. This paper deals with low-level approaches to 3D reconstruction, specifically those approaches that embed the surfaces of 3D objects within a volume. Alternately high-level approaches try to fit specific object models or geometric primitives to the range data. Low-level approaches make very few assumptions about the objects being modeled and avoid complicated issues of surface parameterization. However, volumetric reconstruction methods provide very little semantic information about a scene, and therefore, they are best suited to applications where the resulting models are viewed by a user, such as telerobotics or telepresence. Such reconstructions are also useful for robot navigation and as a preprocessing step to subsequent high-level processing. The reconstruction problem is not as simple as it first appears. There are several challenges that must be overcome. A laser rangefinder’s output is an array of 3D points, but they must be combined using a single representation to produce a 3D model. A collection of range maps consists of multiple viewpoints; these views must be properly aligned in order to combine the information from the different range maps. Noise in the scanning process means that range maps are inherently uncertain; data from different viewpoints must be fused to create a single, reliable 3D model. This paper describes a 3D volumetric modeling algorithm that combines several useful aspects of techniques discussed in the literature. This hybrid method exhibits some new capabilities and promises to be useful for creating 3D models of either single objects or entire scenes. The alignment problem,
121
commonly known as registration, is considered as a separate problem [1–3], which is beyond the scope of this paper. The remainder of this paper is as follows. Section 2 gives an overview of work in 3D modeling from range data. Section 3 describes the theory behind our implementation. Section 4 presents the results of our method applied to synthetic data, as well as data from real scenes using several different laser range finders. Section 5 summarizes the work with some conclusions.
2. BACKGROUND Historically, much of the work with volumetric modeling has been done with binary data. This entails storing a 1 or a 0 in each cell of the volumetric grid. A 1 means that the voxel (volume element or grid point) is part of the object while a 0 means that it is not. Sakaguchi et al. [4] use this type of representation for modeling parts to be used with CAD systems. Croteau et al. [5] use a binary volumetric grid to model the workspace in which a mobile robot works. Tarbox and Gottschlich [6] use a binary volumetric model in a process to inspect machined parts. Using 3D binary data to represent objects severely limits the resolution of the model and causes 3D aliasing. With increasing processing speed and memory, it is feasible to use 3D greyscale data to obtain greater accuracy. One method for incorporating greyscale or graded information into volumes is to use occupancy grids. Several authors [7–11] have proposed combining 3D range data by first creating a grid that encodes the probability-of-occupancy at each grid point. Each range measurement has an associated sensor model, a graded, 3D occupancy function that projects from the sensor along the line of sight to form a kind of cone in 3D (Fig. 1b). Typically occupancy-grid algorithms combine the probability-of-occupancy values from individual range readings in a voxel-by-voxel manner using the super-Bayesian combination formula. Variations on this approach include methods for storing multiple relevant data values at each grid point [12] and different fusion operators for combining the individual range readings [13–19]. There are several drawbacks to the occupancy approach. First, in order to reconstruct a surface, probability values are usually thresholded to find regions that are most likely to be occupied. Unfortunately, this approach does not produce the most likely surface—i.e., the resulting surface estimate is biased toward the sensor. Thus, while the occupancy approach is well-suited for some applications, such as path planning in robot navigation, it does not lend itself to the recovery of 3D object shape. The other drawback of the occupancy approach is that it combines range readings one at a time. This raises two issues. One is computation speed. For high-resolution sensors (such as LADAR) the computation at each grid point is affected by a large number of range readings (assumed to be far greater than the number of individual range maps).
122
Thus for high-resolution reconstructions, the computational time becomes very large. Also, the resampling of the sensor model onto the 3D grid introduces sampling artifacts. This is especially true if the sensor model has highfrequency components (i.e. accurately defined boundaries) as is also the case with LADAR. Generally, the occupancy-grid approach seems best suited for low-resolution, low-accuracy scanning devices such as sonar—the medium for which it was originally developed. More recently, researchers have begun to address the problem of extracting high-fidelity models from large amounts of laser range data. Much of the work has been designed for use with structured-light or triangulating laser range finders, which work best in confined spaces and offer very good (e.g. submillimeter) accuracy. While structured-light scanners are best suited for small objects or workspaces, LADAR scanners are useful for applications such as scene modeling which require larger areas to be scanned. Using a structured light scanner, Curless and Levoy [20] create 3D models by a volumetric method that reconstructs surfaces from multiple range maps. Each voxel of the grid has two values associated with it for each range map. First is the implicit function, which is the signed distance transform along the line of sight. Second is the weight function, a kind of confidence value that is used when combining the implicit functions generated by different views. The surface is located where the implicit function is zero. Data from multiple views are combined by a weighted average to create an overall implicit function and weighting function. A related implicit-function approach is proposed by Hoppe et al. [21] for object modeling with a set of unorganized points. They use the signed distance function to the range measurement as the distance from a point to the true surface. They estimate the signed distance function in a 3D region near the individual data points and extract a surface, where the signed distance function is equal to zero, using a variation of the marching cubes algorithm [22]. Hilton et al. [23] have done similar work using multiple 2.5D images. They use both the weighted sums of signed distance functions, as well as a set of rules that account for special circumstances such as surfaces on opposite sides of a thin object. All of these methods are based on heuristics for constructing implicit functions from multiple range maps, and they typically lack any consistent statistical framework for showing that surface estimates are true to the data. For the most part results in these papers are derived from very accurate data of individual, well-defined objects. Whitaker [24] has proposed a statistical approach to reconstructing surfaces from range data. He shows that for range data that is corrupted by noise, the optimal surface estimate can be expressed as a combination of two terms: one resembling the implicit functions proposed in the literature and the other representing the prior information about the objects being reconstructed. The result is an iterative, surface-fitting process—implemented in a volume— which is optimal but somewhat time-consuming. The technique presented in this paper employs aspects of the occupancy grids, as well as those methods based on implicit functions. The result is a
123
method that is well suited for reconstructing scenes (multiple objects) from time-of-flight laser range scanners with significant amounts of noise. Like the occupancy grid method, we create a certainty function that is based on the characteristics of the scanner; however, our function is designed to facilitate surface extraction. We map the certainty function into the volumetric grid in a manner similar to the way implicit functions are mapped into volumes. This avoids some drawbacks, especially the resampling problems, that we noticed with the occupancy grid methods [25]. We combine views using the superBayesian combination formula to get an overall certainty function. Using the statistically based certainty function and the super-Bayesian combination formula allows us to handle the noise commonly present in the scanners that are used in robotic systems.
3. VOLUMETRIC MODELING TECHNIQUE The volumetric method that we have developed is intended to overcome several of the drawbacks that we have noticed with both occupancy grid formulations and the implicit-function methods. With the occupancy grids our biggest concerns are the aliasing and computation time associated with the strategy of using individual sensor readings, as well as the inability to extract the most-likely surface. For the implicit-function methods the most important issues are the lack of statistical methodology and the inability to handle complex scenes and noise. The volumetric reconstruction process consists of three distinct steps. First is the model of the range data for a single range map. Second is a method to combine multiple range maps into a single representation. Third is a method for extracting surfaces from the volumetric data. These steps are interrelated, and the entire process must be considered when developing each step. Individual sensor readings are corrupted by noise. From a noise model, we can generate a one-dimensional probability density function along the line of sight. Suppose that the sensor gives readings that are corrupted with zeromean, uniform, additive noise of width 2⑀. The probability density function (pdf) of a measurement conditional on a surface position r is as shown in Fig. 2b. Taking the integral of the pdf gives the cumulative distribution function shown in Fig. 2c.
FIG. 2. (a) A sensor reading returns a distance r to an object. (b) The density function arising from uniform noise with a width of 2⑀. (c) The distribution function arising from uniform noise with a width of 2⑀.
124
FIG. 3. (a) The density function arising from uniform noise with a width of 2⑀. (b) The distribution function arising from uniform noise with a width of 2⑀.
The cumulative distribution function gives the probability that the measurement is between the sensor (located at x ⫽ 0) and the surface. The distribution function can be interpreted as the likelihood that a position x is occluded from the sensor. This is because if the surface is located at x, everything to the right of x must be occluded. There are two additional items for which we must account: outliers and the possibility of other surfaces. A range measurement with a fixed percentage of outliers, lying in the range [r ⫺ ␦, r ⫹ ␦], would have a pdf with the shape shown in Fig. 3a. Keeping the percentage of outliers constant and letting the width of the outlier distribution go to infinity, we get the cumulative distribution shown graphically in Fig. 3b. The last consideration is additional surfaces. Figure 2b is a good certainty-ofocclusion graph for a single sensor from a single location. However, the general goal is to combine multiple sensor readings from different locations. Figure 4a shows such a configuration. As we move along the line of sight from the scanner we first pass through the region where the only possibility of occupancy is that the range measurement is an outlier. As we pass through the region defined by the uniform distribution, then probability of occupancy increases, passing 21 at the position of the range reading. Once we have passed through the surface, we are behind the surface that was measured by the range reading, and there is a possibility that we could pass out of that object and into another one and so
FIG. 4. An altered distribution function that takes into account the possibility of other surfaces that may be seen from other directions. The slope of the line segments is determined by the noise level. The location of the peak value ⑀ determines the distance between surfaces that can be distinguished.
125
forth. Thus, the probability of occupancy gradually falls to 21 at a rate that is commensurate with the average size of the objects in the scene. Expressing the certainty as a function of the distance to the range reading (i.e., ␣ ⫽ r ⫺ x) gives
C(␣) ⫽
5
, ⫹
1/2 ⫺ ⑀
(1 ⫺ ) ⫹
␣ ⬍ ⫺⑀, (␣ ⫹ ⑀),
⫺ 1/2 ⑀
⫺⑀ ⱕ ␣ ⱕ ⑀,
(␣ ⫺ ⑀), ⑀ ⬍ ␣ ⱕ 2⑀,
1 ,
(1)
2⑀ ⬍ ␣.
2
Figure 4b shows the graph that results from this falloff that accounts for additional surfaces. We have assumed that falloff to complete uncertainty happens over a distance ⑀. Note that this shape is similar to the shape that Elfes [26, 27] derived for the one-dimensional occupancy grid. There are two important differences. Ours does not drop completely to zero in front of the sensor; this is to account for outliers. In our graph the most likely surface lies at a value of 21; this will make the surface extraction step easier. Another difference is that certainty function from Eq. (1) is piecewise linear, while the sensor model for occupancy grids is typically continuous and nonlinear. The linearity in the certainty function is due to the uniform distribution used for the sensor model. The use of the piecewise linear certainty function has some advantages. First, it makes the derivation of the certainty function easier to formulate and implement. The piecewise linear certainty function that results from a uniform distribution is faster to calculate than the erf-based certainty function that would result from using a Gaussian pdf. A Gaussian pdf, however, would probably be a better model for true error characteristics of a range finder; we have experimented with both and have found very little difference in the final results. The graph in Fig. 4b shows the 1D certainty function that applies to each sensor reading along the line of sight associated with that reading. The goal is to extend this certainty function to a 3D grid. A range map is a 2D array of distance measurements, ru,v, which can be expressed as a continuous function, r(, ): R2 6 R, where the continuity is achieved by a bilinear interpolation between points on the grid. If we assume that the lines of sight emanating from the scanner do not cross (as is the case with spherical, cylindrical, or planar-projective geometries that most systems employ) the geometry of the scanner induces a transformation from Cartesian coordinates to the coordinates of the scanner,
121 2 R
(x)
⫽ (x) , (x)
126
(2)
where x 僆 R3. Thus, for a single range map, we can express the certainty for any point in 3D, C(x) ⫽ C((x ) ⫺ r((x), (x))).
(3)
Equation (3) describes how to compute the certainty for each vertex in the volumetric grid by projecting a line onto to the sensor location through the range map. The point where the line intersects the range map is (, ) and the range measurement for that point is found by interpolating nearby range measurements. The distance from the sensor a voxel describes leads to an estimate between that 3D location and the range measurement. The 1D certainty function assigns a certainty value to the voxel based on its distance from the range measurement along the projected line. Figure 5 shows this graphically. The difference between this approach and the occupancy method (Figs 1b and 5) is that the proposed method projects backward from each voxel to each individual range map, rather than from each individual range measurement into the grid. Thus, each range image is treated as a single entity with a topology (necessary for the interpolation) and a unique inverse mapping. To extend this analysis to multiple range maps, we let rj(, ) represent the jth range map, whose pose is described by a transformation Tj relative to some world coordinate system (e.g. the volume itself). For a point x in world coordinates the certainty value associated with the range map rj is Cj(x) ⫽ C((Tj x) ⫺ rj((Tj x), (Tj x))).
(4)
The next step in the reconstruction algorithm is to combine the certainty values from the individual range maps. Using super-Bayesian reasoning to combine two independent probabilities of occupancy, a and b, gives the
FIG. 5. The proposed volumetric reconstruction method maps each 3D grid point back (along the lines of sight) to the various range maps.
127
combination rule [28] f (a, b) ⫽
ab ab ⫹ (1 ⫺ a)(1 ⫺ b)
.
(5)
This fusion operator has several desirable attributes. First it is commutative and associative, and thus, it does not make arbitrary distinctions about the order in which we combine the range data. Also, it is monotonic; larger certainty values can only increase the overall certainty of the result. Finally, it is symmetric; there is no distinction between modeling the certain of vacancy rather than the certainty of occupancy. Note that the super-Bayesian fusion operator also has a convenient identity at the value of 21. That is, f (a, 21) ⫽ a. Thus, 21 can be used as the ‘‘no information’’ value, meaning that we are completely unsure if the voxel is occluded or not. If we let Cj(x) be the value of the certainty value generated from the jth range map along the line of sight associated with the point x in the volume, then the total certainty from all M range maps is M
兿 C (x) j
f (x) ⫽
j⫽1
M
.
M
(6)
兿 C (x) ⫹ 兿 (1 ⫺ C (x)) j
j⫽1
j
j⫽1
Once all the range maps have been combined into a single volumetric grid, the surface model associated with the scene or object is extracted. For this we use a form of the marching cubes algorithm proposed by Cline and Lorensen [22] to visualize medical data. The super-Bayesian combination formula that we use for the fusion step tends to push the voxel values toward 1 or 0. This can lead to large quantization errors if we simply interpolate linearly between voxel vertices to find the surface location as is done in the standard marching cubes algorithm. Therefore we subsample the vertex edges that pass through the surface and use the original range data to find the certainty values at each new subsample. This additional information provides a more accurate estimate of position of the surface-edge intersection (which determines the locations of the triangle vertices in the final surface mesh) than a mere linear interpolation.
4. RESULTS AND DISCUSSION This chapter shows how our volumetric technique generates 3D models from sequences of registered range maps. We start by showing results from simulated data. Then we demonstrate how our method reconstructs objects from images taken with a Perceptron P5000 scanner. Finally, we use range maps taken with a Coleman Coherent Laser Radar scanner to reconstruct a small scene. Both the Perceptron and Coleman scanners operate on the time-of-flight principle.
128
FIG. 6.
(a) A 3D rendering of a CAD model. (b) A simulated range image of that model.
The general steps of the algorithm are: 1. Acquire registered (3D transformation to world coordinates known) range maps. 2. Apply a 5 ⫻ 5 mean filter to each range map (helps reduce outliers). 3. Initialize a 3D grid to ‘‘unknown’’ value of 21.
FIG. 7. (a) Noiseless data as in Fig. 6 shows the accuracy of the volumetric reconstruction is limited only by the resolution of the volume. (b) With noise, the results are less accurate. (c) With sufficient input data (120 views) the reconstruction algorithm is able to overcome the noise by taking advantage of redundant information between views.
129
4. For each voxel in the grid: (a) Compute the projection back to each range map, and from the range reading and the sensor error compute a certainty value at that location (Eq. (4)). (b) Fuse the certainty values from all of the range maps (Eq. (6)). 5. Extract a surface at the certainty value of 21. Figure 6a shows a rendering of a 3D model of a robotic manipulator and Fig. 6b shows an ideal, simulated range map (darker indicates a smaller range value) from one viewpoint. We have constructed 12 such simulated range maps of that manipulator from different points of view. Figure 7a shows the reconstruction of the 3D model, done on a 80 ⫻ 80 ⫻ 80 voxel grid. Notice that the volumetric approach handles the complicated shapes and topologies of that object. These experiments also demonstrate that the algorithm deals properly with noise. Figures 7b–c show the reconstructions that result from range maps that were corrupted by additive, uniform noise noise with a half width that is
FIG. 8. Range maps taken of a test object. (a–b) Two out of 12 images taken from different points of view. (c–d) are surface plots of those images that show the nature of the noise— is the range, and and are the positions of the mirrors.
130
FIG. 9.
The 3D model generated from the range data from Fig. 8 on an 80 ⫻ 80 ⫻ 80 grid.
13% of the typical object distance. Figure 7b, constructed from 12 images, shows that the addition of noise has clearly made the objects rougher; however, they are still recognizable. Figure 7c shows the reconstruction that results from 120 simulated, noisy, range maps. The algorithm is able to discern more details by taking advantage of redundant information. Figures 8a–b show two (out of a collection of 12) range maps of some stacks of paper and wooden blocks, taken with a Perceptron P5000 scanner. These images have been registered (roughly) by keeping track of the position of the scanner; some fine adjustments to the registration were done by hand. Figures 8c–d, which show surface plots of those same images, demonstrate the amount of noise present in this data. Figure 9 shows the reconstructions on an 80 ⫻ 80 ⫻ 80 volume that result from the 12 images of the object shown in Fig. 8. Figure 10 shows three range maps of a scene containing a traffic cone and
FIG. 10.
Three range maps taken from a Coleman scanner.
131
FIG. 11. The reconstruction of the cone scene using two of the three range maps on a 200 ⫻ 200 ⫻ 200 grid. Regions that are occluded from both viewpoints are evident in the shadow of objects and between the cone and the blocks.
several concrete blocks, taken with a Coleman Coherent Laser Radar scanner. The scans have been registration and calibrated using software that is available with that scanner. Except for the outliers (particularly bad in the third image) that scanner obtains very high accuracy (approximate 1 mm). Figure 11 shows the reconstruction on a 200 ⫻ 200 ⫻ 200 grid of the scene using the two least noisy images, Figs 10(a) and 10(b). The low noise level in the images allows our method to capture such details as the handles and vents on the lockers along the back wall. Regions that are occluded from both views are evident as the shadow-like effect behind objects. The odd shape between the cone and the blocks is also caused by occlusion. Figure 12 shows the reconstruction of the cone scene on a 200 ⫻ 200 ⫻ 200 grid using all three range images. The third range map contains more noise than the others; hence, we applied a 3 ⫻ 3 median filter to it before we added it to our model. The higher noise level caused the ‘‘grooves’’ that are especially noticeable along the wall behind the blocks. Also notice that there are small ridges on the traffic cone. This is the stair-step effect that results from the
FIG. 12. The reconstruction of the cone scene using all three of range maps on a 200 ⫻ 200 ⫻ 200 grid. The additional range map contains more noise than the other two. This caused the artifacts on the back wall. The third image has filled in some of the occluded regions.
132
discrete nature of the range map. There are two solutions to the stair-step problem. One can select the voxel size to be large enough to average across several pixels. The other approach is to apply a step-edge detector to the range map and ignore voxels that map to the edges. The second solution could be problematic for noisy range maps, but may be feasible in other situations.
5. CONCLUSIONS The use of volumes for fusing multiple range maps offers a low-level approach to 3D model reconstruction. We have reviewed the literature and proposed a new approach which combines many of the best properties of two different strategies documented in the literature. Most other methods use a linear implicit functions for each view and combine them using a weighted sum. Our method uses a certainty function whose shape can be tailored to model the error characteristics of the sensor. We combine views using the super-Bayesian combination formula which yields an updated certainty function. Alternatively, the occupancy-grid approach uses a sensor model and super-Bayesian update, but which projects sensor readings from the sensor onto the grid, causing sampling artifacts and slow updates. Other nonbinary methods typically use information from only those points on the object’s surface and ignore background points. Our method uses background points to carve away free space around an object. This is especially useful when working with data with large errors; carving away free space can help remove effects of outlier (noise ‘‘spikes’’) on the surface. Our method labels each point in the grid as either outside or inside; the surface extraction creates a continuous surface between the two. Some other methods (at times intentionally) leave holes where there is no sensed data.
ACKNOWLEDGMENTS This work was supported by the U.S. Department of Energy’s University Research Program in Robotics under Grant DOE-DE-FG02-86NE37968, Mechanical Technology Incorporated with the Federal Energy Technology Center under Grant DE-AR21-95MC32093, and the Office of Naval Research under Grant N00014-97-1-0227. The Coleman laser data was provided by the Oak Ridge National Laboratory.
REFERENCES 1. Besl, P., and McKay, N. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14 (1992), 239–256. 2. Chen, Y., and Medioni, G. Object modelling by registration of multiple range images. J. Image Vision Comput. 10 No. 3 (1992), 145–155. 3. Zhang, Z. Iterative point matching for registration of free-form curves and surfaces, Int. J. Comput. Vision, 13 (1994), 119–152. 4. Sakaguchi, Y., Kato, H., Sato, K., and Inokuchi, S. Acquisition of Entire Surface Data Based on Fusion of Range Data. IEICE Trans. E74 (1991), 3417–3422.
133
5. Croteau, A., Laurendeau, D., and Lessard, J. Building a space occupancy model for a powerline maintenance robot using a range data sensor. In Intelligent Robots and Computer Vision X: Algorithms and Techniques 1991. 6. Tarbox, G. H., and Gottschlich, S. N. Ivis: An integrated volumetric inspection system, Computer Vision and Image Understanding, 61, No. 3 (1995), 430–444. 7. Matthies, L., and Elfes, A. Integration of sonar and stereo range data using a grid-based representation. In Proceedings of the 1988 IEEE International Conference on Robotics and Automation. April 1988, Vol. 2, pp. 727–733. 8. Elfes, A., and Matthies, L. Sensor integration for robot navigation: Combining sonar and range data in a grid-based representation, in Proceedings of the 26th IEEE Conference on Decision and Control, 3 (1987), 1802–1807. 9. Lim, J. H., and Cho, D. W. Specular reflection probability in the certainty grid representation. Trans. ASME. J. Dyn. Systems, Measurement & Control 116 (1994), 512–520. 10. Moravec, H. P. Sensor fusion in certainty grids for mobile robots, AI Mag. 9 (1988), 61–77. 11. Martin, M. C., and Moravec, H. P. Robot Evidence Grids. Tech. Rep. CMU-RI-TR-96-06. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, March 1996. 12. Auran, P. G., and Silven, O. Ideas for underwater 3d sonar range sensing and environmental modelling,’’ in Proceedings of the IFAC CAMS ’95 (Control Applications in Marine Systems) Workshop, 1995. 13. Moreno, L., Salichs, M. A., and Gachet, D. Fusion of proximity data in certainty grids, Parallel and Distributed Computing in Engineering systems: Proceedings of the IMACS/IFAC International Symposium on Parallel and Distributed Computing in Engineering Systems, pp. 269–274, 1992. 14. Tirumalai, A. P., Schunck, B. G., and Jain, R. C. Evidential reasoning for building environment maps. IEEE Transactions on Systems, Man, and Cybernetics, 25 (1995), 10–20. 15. Puente, E. A., Moreno, L., Salichs, M. A., and Gachet, D. Analysis of data fusion methods in certainty grids application to collision danger monitoring. In Proceedings IECON ’91. 1991 International Conference on Industrial Electronics, Control and Instrumentation, 2 (1991), 1133–1137. 16. Auran, P. G., and Malvig, K. 3d sonar based sensing for auvs: Realtime experiments within the uncertainty grid framework. Accepted for presentation at IFAC ’96. 17. Poloni, M., Ulivi, G., and Vendittelli, M. Fuzzy logic and autonomous vehicle: Experiments in ultrasonic vision. Fuzzy Sets and Systems (Netherlands), 69 (1995), 15–27. 18. Oriolo, G., Ulivi, G., and Vendittelli, M. Fuzzy maps: A new tool for mobile robot perception and planning. Journal of Robotic Systems 14, No. 1, (1997), 179–197. 19. Santos, V., Gonclaves, J. G. M., and Vaz, F. Perception maps for the local navigation if a mobile robot: a neural network approach. In Proceedings of the 1994 IEEE International Conference on Robotics and Automation 3, (1993), 2193–2198. 20. Curless, B., and Levoy, M. A Volumetric Method for Building Complex Models from Range Images. In SIGGRAPH ’96, 1996. 21. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., and Stuetzle, W. Surface reconstruction from unorganized points. Computer Graphics 26, No. 2, (1992), 71–78. 22. Lorensen, W. E., and Cline, H. E. Marching cubes: A high resolution 3d surface construction algorithm. Computer Graphics 21, No. 4, (1987), 163–169. 23. Hilton, A., Stoddart, A. J., Illingworth, J., and Windeatt, T. Reliable surface reconstruction from multiple range images. In ECCV Springer-Verlag, 1996. 24. Whitaker, R. T. A level-set approach to 3D reconstruction from range data. Int. J. Comput. Vision October, No. 3 (1998), 203–231. 25. Elsner, D. L. Volumetric Modeling through Fusion of Multiple Range Images with Confidence Estimate, Master’s thesis, University of Tennessee, Knoxville, Knoxville, TN 37996-2100, December 1997. 26. Elfes, A. Using Occupancy Grids for Mobile Robot Perception and Navigation. Computer 22 (1989), 46–57. 27. Elfes, A. Occupancy Grids: A Probabilistic Framework for Robot Perception and Navigation. Ph.D. thesis. Carnegie Mellon University, Pittsburgh, PA, May 1989. 28. Berger, J. O. Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer-Verlag, Berlin, 1985.
134
DAVID ELSNER received his B.S. in electrical engineering from the University of NebraskaLincoln in 1991. He worked at Johnson Engineering Corporation as a design engineer until 1996, when he began his master’s studies at the University of Tennessee, Knoxville. He earned an M.S. in Electrical Engineering in 1997 and now works for Harris Information Systems Division in Melbourne, Florida. ROSS WHITAKER received his B.S. in electrical engineering and computer science from Princeton University in 1986. In 1993 he earned his Ph.D. in computer science from the University of North Carolina. In 1994 he joined ECRC in Munich, Germany as a research scientist in the User Interaction and Visualization Group. In 1996 he joined the Department of Electrical Engineering at the University of Tennessee as an assistant professor. He teaches image processing and pattern recognition and 3D computer vision. His research interests include nonlinear methods for image processing and 3D reconstruction by deformable models. MONGI ABIDI received his M.S. in electrical engineering in 1985 and the Ph.D. in electrical engineering in 1987, both from the University of Tennessee, Knoxville. In 1986, he joined the Department of Electrical and Computer Engineering at the University of Tennessee, Knoxville, where he is now a professor. His interests are image processing, multisensor processing, fuzzy logic, and robot sensing. He is co-editor of the book Data Fusion in Robotics and Machine Intelligence, Academic Press, 1992.
135
Volumetric Reconstruction of Objects and Scenes Using Range Images D. L. Elsner, R. T. Whitaker, and M. A. Abidi Department of Electrical Engineering, University of Tennessee, Knoxville, Tennessee 37996 E-mail: [email protected]
This paper reviews volumetric methods for fusing sets of range images to create 3D models of objects or scenes. It also presents a new reconstruction method, which is a hybrid that combines several desirable aspects of techniques discussed in the literature. The proposed reconstruction method projects each point, or voxel, within a volumetric grid back onto a collection of range images. Each voxel value represents the degree of certainty that the point is inside the sensed object. The certainty value is a function of the distance from the grid point to the range image, as well as the sensor’s noise characteristics. The super-Bayesian combination formula is used to fuse the data created from the individual range images into an overall volumetric grid. We obtain the object model by extracting an isosurface from the volumetric data using a version of the marching cubes algorithm. Results are shown from simulations and real range finders. r1999 Academic Press Key Words: range imaging; range scanners; data fusion; surface reconstruction; scene reconstruction; occupancy grids.
1. INTRODUCTION The creation of 3D models from range data is important in a variety of applications, including reverse engineering, object recognition, inspection and quality control, medical imaging, computer graphics, and remote mapping. Depending on the application, there are a number of different sensors and strategies that can be employed. This paper addresses the problem of 3D reconstruction using range data from a time-of-flight laser range finder also called LADAR (light amplitude detection and ranging). Range measurements give distances to object surfaces, and a scanning mechanism provides a 2D array of such measurements, i.e., a range map or range image, taken from a certain point of view (Fig. 1a). Range maps give what is commonly referred to as 1051-2004/99 $30.00 Copyright r 1999 by Academic Press All rights of reproduction in any form reserved.
120
FIG. 1. (a) A range scanner produces a 2D array of distance measurements from a single point of view. (b) The occupancy approach casts the probability of occupancy from each reading into the volume and then combines those readings.
2.5D information about a scene. They provide depth information from one point of view but they do not distinguish between objects that occlude one another and they do not provide a full 3D description of an object—i.e., not all surfaces of an object can be seen by a single range map. To obtain a full 3D description of an object, multiple range maps from several viewpoints must be combined. This paper deals with low-level approaches to 3D reconstruction, specifically those approaches that embed the surfaces of 3D objects within a volume. Alternately high-level approaches try to fit specific object models or geometric primitives to the range data. Low-level approaches make very few assumptions about the objects being modeled and avoid complicated issues of surface parameterization. However, volumetric reconstruction methods provide very little semantic information about a scene, and therefore, they are best suited to applications where the resulting models are viewed by a user, such as telerobotics or telepresence. Such reconstructions are also useful for robot navigation and as a preprocessing step to subsequent high-level processing. The reconstruction problem is not as simple as it first appears. There are several challenges that must be overcome. A laser rangefinder’s output is an array of 3D points, but they must be combined using a single representation to produce a 3D model. A collection of range maps consists of multiple viewpoints; these views must be properly aligned in order to combine the information from the different range maps. Noise in the scanning process means that range maps are inherently uncertain; data from different viewpoints must be fused to create a single, reliable 3D model. This paper describes a 3D volumetric modeling algorithm that combines several useful aspects of techniques discussed in the literature. This hybrid method exhibits some new capabilities and promises to be useful for creating 3D models of either single objects or entire scenes. The alignment problem,
121
commonly known as registration, is considered as a separate problem [1–3], which is beyond the scope of this paper. The remainder of this paper is as follows. Section 2 gives an overview of work in 3D modeling from range data. Section 3 describes the theory behind our implementation. Section 4 presents the results of our method applied to synthetic data, as well as data from real scenes using several different laser range finders. Section 5 summarizes the work with some conclusions.
2. BACKGROUND Historically, much of the work with volumetric modeling has been done with binary data. This entails storing a 1 or a 0 in each cell of the volumetric grid. A 1 means that the voxel (volume element or grid point) is part of the object while a 0 means that it is not. Sakaguchi et al. [4] use this type of representation for modeling parts to be used with CAD systems. Croteau et al. [5] use a binary volumetric grid to model the workspace in which a mobile robot works. Tarbox and Gottschlich [6] use a binary volumetric model in a process to inspect machined parts. Using 3D binary data to represent objects severely limits the resolution of the model and causes 3D aliasing. With increasing processing speed and memory, it is feasible to use 3D greyscale data to obtain greater accuracy. One method for incorporating greyscale or graded information into volumes is to use occupancy grids. Several authors [7–11] have proposed combining 3D range data by first creating a grid that encodes the probability-of-occupancy at each grid point. Each range measurement has an associated sensor model, a graded, 3D occupancy function that projects from the sensor along the line of sight to form a kind of cone in 3D (Fig. 1b). Typically occupancy-grid algorithms combine the probability-of-occupancy values from individual range readings in a voxel-by-voxel manner using the super-Bayesian combination formula. Variations on this approach include methods for storing multiple relevant data values at each grid point [12] and different fusion operators for combining the individual range readings [13–19]. There are several drawbacks to the occupancy approach. First, in order to reconstruct a surface, probability values are usually thresholded to find regions that are most likely to be occupied. Unfortunately, this approach does not produce the most likely surface—i.e., the resulting surface estimate is biased toward the sensor. Thus, while the occupancy approach is well-suited for some applications, such as path planning in robot navigation, it does not lend itself to the recovery of 3D object shape. The other drawback of the occupancy approach is that it combines range readings one at a time. This raises two issues. One is computation speed. For high-resolution sensors (such as LADAR) the computation at each grid point is affected by a large number of range readings (assumed to be far greater than the number of individual range maps).
122
Thus for high-resolution reconstructions, the computational time becomes very large. Also, the resampling of the sensor model onto the 3D grid introduces sampling artifacts. This is especially true if the sensor model has highfrequency components (i.e. accurately defined boundaries) as is also the case with LADAR. Generally, the occupancy-grid approach seems best suited for low-resolution, low-accuracy scanning devices such as sonar—the medium for which it was originally developed. More recently, researchers have begun to address the problem of extracting high-fidelity models from large amounts of laser range data. Much of the work has been designed for use with structured-light or triangulating laser range finders, which work best in confined spaces and offer very good (e.g. submillimeter) accuracy. While structured-light scanners are best suited for small objects or workspaces, LADAR scanners are useful for applications such as scene modeling which require larger areas to be scanned. Using a structured light scanner, Curless and Levoy [20] create 3D models by a volumetric method that reconstructs surfaces from multiple range maps. Each voxel of the grid has two values associated with it for each range map. First is the implicit function, which is the signed distance transform along the line of sight. Second is the weight function, a kind of confidence value that is used when combining the implicit functions generated by different views. The surface is located where the implicit function is zero. Data from multiple views are combined by a weighted average to create an overall implicit function and weighting function. A related implicit-function approach is proposed by Hoppe et al. [21] for object modeling with a set of unorganized points. They use the signed distance function to the range measurement as the distance from a point to the true surface. They estimate the signed distance function in a 3D region near the individual data points and extract a surface, where the signed distance function is equal to zero, using a variation of the marching cubes algorithm [22]. Hilton et al. [23] have done similar work using multiple 2.5D images. They use both the weighted sums of signed distance functions, as well as a set of rules that account for special circumstances such as surfaces on opposite sides of a thin object. All of these methods are based on heuristics for constructing implicit functions from multiple range maps, and they typically lack any consistent statistical framework for showing that surface estimates are true to the data. For the most part results in these papers are derived from very accurate data of individual, well-defined objects. Whitaker [24] has proposed a statistical approach to reconstructing surfaces from range data. He shows that for range data that is corrupted by noise, the optimal surface estimate can be expressed as a combination of two terms: one resembling the implicit functions proposed in the literature and the other representing the prior information about the objects being reconstructed. The result is an iterative, surface-fitting process—implemented in a volume— which is optimal but somewhat time-consuming. The technique presented in this paper employs aspects of the occupancy grids, as well as those methods based on implicit functions. The result is a
123
method that is well suited for reconstructing scenes (multiple objects) from time-of-flight laser range scanners with significant amounts of noise. Like the occupancy grid method, we create a certainty function that is based on the characteristics of the scanner; however, our function is designed to facilitate surface extraction. We map the certainty function into the volumetric grid in a manner similar to the way implicit functions are mapped into volumes. This avoids some drawbacks, especially the resampling problems, that we noticed with the occupancy grid methods [25]. We combine views using the superBayesian combination formula to get an overall certainty function. Using the statistically based certainty function and the super-Bayesian combination formula allows us to handle the noise commonly present in the scanners that are used in robotic systems.
3. VOLUMETRIC MODELING TECHNIQUE The volumetric method that we have developed is intended to overcome several of the drawbacks that we have noticed with both occupancy grid formulations and the implicit-function methods. With the occupancy grids our biggest concerns are the aliasing and computation time associated with the strategy of using individual sensor readings, as well as the inability to extract the most-likely surface. For the implicit-function methods the most important issues are the lack of statistical methodology and the inability to handle complex scenes and noise. The volumetric reconstruction process consists of three distinct steps. First is the model of the range data for a single range map. Second is a method to combine multiple range maps into a single representation. Third is a method for extracting surfaces from the volumetric data. These steps are interrelated, and the entire process must be considered when developing each step. Individual sensor readings are corrupted by noise. From a noise model, we can generate a one-dimensional probability density function along the line of sight. Suppose that the sensor gives readings that are corrupted with zeromean, uniform, additive noise of width 2⑀. The probability density function (pdf) of a measurement conditional on a surface position r is as shown in Fig. 2b. Taking the integral of the pdf gives the cumulative distribution function shown in Fig. 2c.
FIG. 2. (a) A sensor reading returns a distance r to an object. (b) The density function arising from uniform noise with a width of 2⑀. (c) The distribution function arising from uniform noise with a width of 2⑀.
124
FIG. 3. (a) The density function arising from uniform noise with a width of 2⑀. (b) The distribution function arising from uniform noise with a width of 2⑀.
The cumulative distribution function gives the probability that the measurement is between the sensor (located at x ⫽ 0) and the surface. The distribution function can be interpreted as the likelihood that a position x is occluded from the sensor. This is because if the surface is located at x, everything to the right of x must be occluded. There are two additional items for which we must account: outliers and the possibility of other surfaces. A range measurement with a fixed percentage of outliers, lying in the range [r ⫺ ␦, r ⫹ ␦], would have a pdf with the shape shown in Fig. 3a. Keeping the percentage of outliers constant and letting the width of the outlier distribution go to infinity, we get the cumulative distribution shown graphically in Fig. 3b. The last consideration is additional surfaces. Figure 2b is a good certainty-ofocclusion graph for a single sensor from a single location. However, the general goal is to combine multiple sensor readings from different locations. Figure 4a shows such a configuration. As we move along the line of sight from the scanner we first pass through the region where the only possibility of occupancy is that the range measurement is an outlier. As we pass through the region defined by the uniform distribution, then probability of occupancy increases, passing 21 at the position of the range reading. Once we have passed through the surface, we are behind the surface that was measured by the range reading, and there is a possibility that we could pass out of that object and into another one and so
FIG. 4. An altered distribution function that takes into account the possibility of other surfaces that may be seen from other directions. The slope of the line segments is determined by the noise level. The location of the peak value ⑀ determines the distance between surfaces that can be distinguished.
125
forth. Thus, the probability of occupancy gradually falls to 21 at a rate that is commensurate with the average size of the objects in the scene. Expressing the certainty as a function of the distance to the range reading (i.e., ␣ ⫽ r ⫺ x) gives
C(␣) ⫽
5
, ⫹
1/2 ⫺ ⑀
(1 ⫺ ) ⫹
␣ ⬍ ⫺⑀, (␣ ⫹ ⑀),
⫺ 1/2 ⑀
⫺⑀ ⱕ ␣ ⱕ ⑀,
(␣ ⫺ ⑀), ⑀ ⬍ ␣ ⱕ 2⑀,
1 ,
(1)
2⑀ ⬍ ␣.
2
Figure 4b shows the graph that results from this falloff that accounts for additional surfaces. We have assumed that falloff to complete uncertainty happens over a distance ⑀. Note that this shape is similar to the shape that Elfes [26, 27] derived for the one-dimensional occupancy grid. There are two important differences. Ours does not drop completely to zero in front of the sensor; this is to account for outliers. In our graph the most likely surface lies at a value of 21; this will make the surface extraction step easier. Another difference is that certainty function from Eq. (1) is piecewise linear, while the sensor model for occupancy grids is typically continuous and nonlinear. The linearity in the certainty function is due to the uniform distribution used for the sensor model. The use of the piecewise linear certainty function has some advantages. First, it makes the derivation of the certainty function easier to formulate and implement. The piecewise linear certainty function that results from a uniform distribution is faster to calculate than the erf-based certainty function that would result from using a Gaussian pdf. A Gaussian pdf, however, would probably be a better model for true error characteristics of a range finder; we have experimented with both and have found very little difference in the final results. The graph in Fig. 4b shows the 1D certainty function that applies to each sensor reading along the line of sight associated with that reading. The goal is to extend this certainty function to a 3D grid. A range map is a 2D array of distance measurements, ru,v, which can be expressed as a continuous function, r(, ): R2 6 R, where the continuity is achieved by a bilinear interpolation between points on the grid. If we assume that the lines of sight emanating from the scanner do not cross (as is the case with spherical, cylindrical, or planar-projective geometries that most systems employ) the geometry of the scanner induces a transformation from Cartesian coordinates to the coordinates of the scanner,
121 2 R
(x)
⫽ (x) , (x)
126
(2)
where x 僆 R3. Thus, for a single range map, we can express the certainty for any point in 3D, C(x) ⫽ C((x ) ⫺ r((x), (x))).
(3)
Equation (3) describes how to compute the certainty for each vertex in the volumetric grid by projecting a line onto to the sensor location through the range map. The point where the line intersects the range map is (, ) and the range measurement for that point is found by interpolating nearby range measurements. The distance from the sensor a voxel describes leads to an estimate between that 3D location and the range measurement. The 1D certainty function assigns a certainty value to the voxel based on its distance from the range measurement along the projected line. Figure 5 shows this graphically. The difference between this approach and the occupancy method (Figs 1b and 5) is that the proposed method projects backward from each voxel to each individual range map, rather than from each individual range measurement into the grid. Thus, each range image is treated as a single entity with a topology (necessary for the interpolation) and a unique inverse mapping. To extend this analysis to multiple range maps, we let rj(, ) represent the jth range map, whose pose is described by a transformation Tj relative to some world coordinate system (e.g. the volume itself). For a point x in world coordinates the certainty value associated with the range map rj is Cj(x) ⫽ C((Tj x) ⫺ rj((Tj x), (Tj x))).
(4)
The next step in the reconstruction algorithm is to combine the certainty values from the individual range maps. Using super-Bayesian reasoning to combine two independent probabilities of occupancy, a and b, gives the
FIG. 5. The proposed volumetric reconstruction method maps each 3D grid point back (along the lines of sight) to the various range maps.
127
combination rule [28] f (a, b) ⫽
ab ab ⫹ (1 ⫺ a)(1 ⫺ b)
.
(5)
This fusion operator has several desirable attributes. First it is commutative and associative, and thus, it does not make arbitrary distinctions about the order in which we combine the range data. Also, it is monotonic; larger certainty values can only increase the overall certainty of the result. Finally, it is symmetric; there is no distinction between modeling the certain of vacancy rather than the certainty of occupancy. Note that the super-Bayesian fusion operator also has a convenient identity at the value of 21. That is, f (a, 21) ⫽ a. Thus, 21 can be used as the ‘‘no information’’ value, meaning that we are completely unsure if the voxel is occluded or not. If we let Cj(x) be the value of the certainty value generated from the jth range map along the line of sight associated with the point x in the volume, then the total certainty from all M range maps is M
兿 C (x) j
f (x) ⫽
j⫽1
M
.
M
(6)
兿 C (x) ⫹ 兿 (1 ⫺ C (x)) j
j⫽1
j
j⫽1
Once all the range maps have been combined into a single volumetric grid, the surface model associated with the scene or object is extracted. For this we use a form of the marching cubes algorithm proposed by Cline and Lorensen [22] to visualize medical data. The super-Bayesian combination formula that we use for the fusion step tends to push the voxel values toward 1 or 0. This can lead to large quantization errors if we simply interpolate linearly between voxel vertices to find the surface location as is done in the standard marching cubes algorithm. Therefore we subsample the vertex edges that pass through the surface and use the original range data to find the certainty values at each new subsample. This additional information provides a more accurate estimate of position of the surface-edge intersection (which determines the locations of the triangle vertices in the final surface mesh) than a mere linear interpolation.
4. RESULTS AND DISCUSSION This chapter shows how our volumetric technique generates 3D models from sequences of registered range maps. We start by showing results from simulated data. Then we demonstrate how our method reconstructs objects from images taken with a Perceptron P5000 scanner. Finally, we use range maps taken with a Coleman Coherent Laser Radar scanner to reconstruct a small scene. Both the Perceptron and Coleman scanners operate on the time-of-flight principle.
128
FIG. 6.
(a) A 3D rendering of a CAD model. (b) A simulated range image of that model.
The general steps of the algorithm are: 1. Acquire registered (3D transformation to world coordinates known) range maps. 2. Apply a 5 ⫻ 5 mean filter to each range map (helps reduce outliers). 3. Initialize a 3D grid to ‘‘unknown’’ value of 21.
FIG. 7. (a) Noiseless data as in Fig. 6 shows the accuracy of the volumetric reconstruction is limited only by the resolution of the volume. (b) With noise, the results are less accurate. (c) With sufficient input data (120 views) the reconstruction algorithm is able to overcome the noise by taking advantage of redundant information between views.
129
4. For each voxel in the grid: (a) Compute the projection back to each range map, and from the range reading and the sensor error compute a certainty value at that location (Eq. (4)). (b) Fuse the certainty values from all of the range maps (Eq. (6)). 5. Extract a surface at the certainty value of 21. Figure 6a shows a rendering of a 3D model of a robotic manipulator and Fig. 6b shows an ideal, simulated range map (darker indicates a smaller range value) from one viewpoint. We have constructed 12 such simulated range maps of that manipulator from different points of view. Figure 7a shows the reconstruction of the 3D model, done on a 80 ⫻ 80 ⫻ 80 voxel grid. Notice that the volumetric approach handles the complicated shapes and topologies of that object. These experiments also demonstrate that the algorithm deals properly with noise. Figures 7b–c show the reconstructions that result from range maps that were corrupted by additive, uniform noise noise with a half width that is
FIG. 8. Range maps taken of a test object. (a–b) Two out of 12 images taken from different points of view. (c–d) are surface plots of those images that show the nature of the noise— is the range, and and are the positions of the mirrors.
130
FIG. 9.
The 3D model generated from the range data from Fig. 8 on an 80 ⫻ 80 ⫻ 80 grid.
13% of the typical object distance. Figure 7b, constructed from 12 images, shows that the addition of noise has clearly made the objects rougher; however, they are still recognizable. Figure 7c shows the reconstruction that results from 120 simulated, noisy, range maps. The algorithm is able to discern more details by taking advantage of redundant information. Figures 8a–b show two (out of a collection of 12) range maps of some stacks of paper and wooden blocks, taken with a Perceptron P5000 scanner. These images have been registered (roughly) by keeping track of the position of the scanner; some fine adjustments to the registration were done by hand. Figures 8c–d, which show surface plots of those same images, demonstrate the amount of noise present in this data. Figure 9 shows the reconstructions on an 80 ⫻ 80 ⫻ 80 volume that result from the 12 images of the object shown in Fig. 8. Figure 10 shows three range maps of a scene containing a traffic cone and
FIG. 10.
Three range maps taken from a Coleman scanner.
131
FIG. 11. The reconstruction of the cone scene using two of the three range maps on a 200 ⫻ 200 ⫻ 200 grid. Regions that are occluded from both viewpoints are evident in the shadow of objects and between the cone and the blocks.
several concrete blocks, taken with a Coleman Coherent Laser Radar scanner. The scans have been registration and calibrated using software that is available with that scanner. Except for the outliers (particularly bad in the third image) that scanner obtains very high accuracy (approximate 1 mm). Figure 11 shows the reconstruction on a 200 ⫻ 200 ⫻ 200 grid of the scene using the two least noisy images, Figs 10(a) and 10(b). The low noise level in the images allows our method to capture such details as the handles and vents on the lockers along the back wall. Regions that are occluded from both views are evident as the shadow-like effect behind objects. The odd shape between the cone and the blocks is also caused by occlusion. Figure 12 shows the reconstruction of the cone scene on a 200 ⫻ 200 ⫻ 200 grid using all three range images. The third range map contains more noise than the others; hence, we applied a 3 ⫻ 3 median filter to it before we added it to our model. The higher noise level caused the ‘‘grooves’’ that are especially noticeable along the wall behind the blocks. Also notice that there are small ridges on the traffic cone. This is the stair-step effect that results from the
FIG. 12. The reconstruction of the cone scene using all three of range maps on a 200 ⫻ 200 ⫻ 200 grid. The additional range map contains more noise than the other two. This caused the artifacts on the back wall. The third image has filled in some of the occluded regions.
132
discrete nature of the range map. There are two solutions to the stair-step problem. One can select the voxel size to be large enough to average across several pixels. The other approach is to apply a step-edge detector to the range map and ignore voxels that map to the edges. The second solution could be problematic for noisy range maps, but may be feasible in other situations.
5. CONCLUSIONS The use of volumes for fusing multiple range maps offers a low-level approach to 3D model reconstruction. We have reviewed the literature and proposed a new approach which combines many of the best properties of two different strategies documented in the literature. Most other methods use a linear implicit functions for each view and combine them using a weighted sum. Our method uses a certainty function whose shape can be tailored to model the error characteristics of the sensor. We combine views using the super-Bayesian combination formula which yields an updated certainty function. Alternatively, the occupancy-grid approach uses a sensor model and super-Bayesian update, but which projects sensor readings from the sensor onto the grid, causing sampling artifacts and slow updates. Other nonbinary methods typically use information from only those points on the object’s surface and ignore background points. Our method uses background points to carve away free space around an object. This is especially useful when working with data with large errors; carving away free space can help remove effects of outlier (noise ‘‘spikes’’) on the surface. Our method labels each point in the grid as either outside or inside; the surface extraction creates a continuous surface between the two. Some other methods (at times intentionally) leave holes where there is no sensed data.
ACKNOWLEDGMENTS This work was supported by the U.S. Department of Energy’s University Research Program in Robotics under Grant DOE-DE-FG02-86NE37968, Mechanical Technology Incorporated with the Federal Energy Technology Center under Grant DE-AR21-95MC32093, and the Office of Naval Research under Grant N00014-97-1-0227. The Coleman laser data was provided by the Oak Ridge National Laboratory.
REFERENCES 1. Besl, P., and McKay, N. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14 (1992), 239–256. 2. Chen, Y., and Medioni, G. Object modelling by registration of multiple range images. J. Image Vision Comput. 10 No. 3 (1992), 145–155. 3. Zhang, Z. Iterative point matching for registration of free-form curves and surfaces, Int. J. Comput. Vision, 13 (1994), 119–152. 4. Sakaguchi, Y., Kato, H., Sato, K., and Inokuchi, S. Acquisition of Entire Surface Data Based on Fusion of Range Data. IEICE Trans. E74 (1991), 3417–3422.
133
5. Croteau, A., Laurendeau, D., and Lessard, J. Building a space occupancy model for a powerline maintenance robot using a range data sensor. In Intelligent Robots and Computer Vision X: Algorithms and Techniques 1991. 6. Tarbox, G. H., and Gottschlich, S. N. Ivis: An integrated volumetric inspection system, Computer Vision and Image Understanding, 61, No. 3 (1995), 430–444. 7. Matthies, L., and Elfes, A. Integration of sonar and stereo range data using a grid-based representation. In Proceedings of the 1988 IEEE International Conference on Robotics and Automation. April 1988, Vol. 2, pp. 727–733. 8. Elfes, A., and Matthies, L. Sensor integration for robot navigation: Combining sonar and range data in a grid-based representation, in Proceedings of the 26th IEEE Conference on Decision and Control, 3 (1987), 1802–1807. 9. Lim, J. H., and Cho, D. W. Specular reflection probability in the certainty grid representation. Trans. ASME. J. Dyn. Systems, Measurement & Control 116 (1994), 512–520. 10. Moravec, H. P. Sensor fusion in certainty grids for mobile robots, AI Mag. 9 (1988), 61–77. 11. Martin, M. C., and Moravec, H. P. Robot Evidence Grids. Tech. Rep. CMU-RI-TR-96-06. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, March 1996. 12. Auran, P. G., and Silven, O. Ideas for underwater 3d sonar range sensing and environmental modelling,’’ in Proceedings of the IFAC CAMS ’95 (Control Applications in Marine Systems) Workshop, 1995. 13. Moreno, L., Salichs, M. A., and Gachet, D. Fusion of proximity data in certainty grids, Parallel and Distributed Computing in Engineering systems: Proceedings of the IMACS/IFAC International Symposium on Parallel and Distributed Computing in Engineering Systems, pp. 269–274, 1992. 14. Tirumalai, A. P., Schunck, B. G., and Jain, R. C. Evidential reasoning for building environment maps. IEEE Transactions on Systems, Man, and Cybernetics, 25 (1995), 10–20. 15. Puente, E. A., Moreno, L., Salichs, M. A., and Gachet, D. Analysis of data fusion methods in certainty grids application to collision danger monitoring. In Proceedings IECON ’91. 1991 International Conference on Industrial Electronics, Control and Instrumentation, 2 (1991), 1133–1137. 16. Auran, P. G., and Malvig, K. 3d sonar based sensing for auvs: Realtime experiments within the uncertainty grid framework. Accepted for presentation at IFAC ’96. 17. Poloni, M., Ulivi, G., and Vendittelli, M. Fuzzy logic and autonomous vehicle: Experiments in ultrasonic vision. Fuzzy Sets and Systems (Netherlands), 69 (1995), 15–27. 18. Oriolo, G., Ulivi, G., and Vendittelli, M. Fuzzy maps: A new tool for mobile robot perception and planning. Journal of Robotic Systems 14, No. 1, (1997), 179–197. 19. Santos, V., Gonclaves, J. G. M., and Vaz, F. Perception maps for the local navigation if a mobile robot: a neural network approach. In Proceedings of the 1994 IEEE International Conference on Robotics and Automation 3, (1993), 2193–2198. 20. Curless, B., and Levoy, M. A Volumetric Method for Building Complex Models from Range Images. In SIGGRAPH ’96, 1996. 21. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., and Stuetzle, W. Surface reconstruction from unorganized points. Computer Graphics 26, No. 2, (1992), 71–78. 22. Lorensen, W. E., and Cline, H. E. Marching cubes: A high resolution 3d surface construction algorithm. Computer Graphics 21, No. 4, (1987), 163–169. 23. Hilton, A., Stoddart, A. J., Illingworth, J., and Windeatt, T. Reliable surface reconstruction from multiple range images. In ECCV Springer-Verlag, 1996. 24. Whitaker, R. T. A level-set approach to 3D reconstruction from range data. Int. J. Comput. Vision October, No. 3 (1998), 203–231. 25. Elsner, D. L. Volumetric Modeling through Fusion of Multiple Range Images with Confidence Estimate, Master’s thesis, University of Tennessee, Knoxville, Knoxville, TN 37996-2100, December 1997. 26. Elfes, A. Using Occupancy Grids for Mobile Robot Perception and Navigation. Computer 22 (1989), 46–57. 27. Elfes, A. Occupancy Grids: A Probabilistic Framework for Robot Perception and Navigation. Ph.D. thesis. Carnegie Mellon University, Pittsburgh, PA, May 1989. 28. Berger, J. O. Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer-Verlag, Berlin, 1985.
134
DAVID ELSNER received his B.S. in electrical engineering from the University of NebraskaLincoln in 1991. He worked at Johnson Engineering Corporation as a design engineer until 1996, when he began his master’s studies at the University of Tennessee, Knoxville. He earned an M.S. in Electrical Engineering in 1997 and now works for Harris Information Systems Division in Melbourne, Florida. ROSS WHITAKER received his B.S. in electrical engineering and computer science from Princeton University in 1986. In 1993 he earned his Ph.D. in computer science from the University of North Carolina. In 1994 he joined ECRC in Munich, Germany as a research scientist in the User Interaction and Visualization Group. In 1996 he joined the Department of Electrical Engineering at the University of Tennessee as an assistant professor. He teaches image processing and pattern recognition and 3D computer vision. His research interests include nonlinear methods for image processing and 3D reconstruction by deformable models. MONGI ABIDI received his M.S. in electrical engineering in 1985 and the Ph.D. in electrical engineering in 1987, both from the University of Tennessee, Knoxville. In 1986, he joined the Department of Electrical and Computer Engineering at the University of Tennessee, Knoxville, where he is now a professor. His interests are image processing, multisensor processing, fuzzy logic, and robot sensing. He is co-editor of the book Data Fusion in Robotics and Machine Intelligence, Academic Press, 1992.
135