- Email: [email protected]
Computing stable poses of piecewise smooth objects
CVGIP:
IMAGE
UNDERSTANDING
Vol. 55, No. 2, March, pp. 109-118, 1992
Computing
Stable Poses of Piecewise Smooth Objects DAVID J. KRIEGMAN
Center for Systems Science, Department
of Electrical
Engineering,
Yale University,
New Haven,
Connecticut
06.520-1968
Received August 19, 1991; accepted November 13, 1991
When a three dimensional object is known to be lying on a planar surface, its pose is restricted from six to three degreesof freedom. Computer vision algorithms can exploit the few stable posesof modeled objects to simplify sceneinterpretation and more accurately determine object location. This paper presents necessary and sufficient conditions for the pose of a piecewise smooth curved three-dimensional object to be stable. For objects whose surfaces are represented by implicit algebraic equations, these conditions can be expressed as systems of polynomial equations that are readily solved by homotopy continuation. Examples from the implemented algorithm are presented. o 1992 hdemic PW, IW.
1. INTRODUCTION
lndependent of the task, “CAD-Based” vision algorithms should exploit all available knowledge of object geometry, lighting conditions, viewpoint constraints, and material characteristics. Consider the prototypical task of determining the pose (position and orientation) and identity of a three-dimensional object traveling along a conveyor belt into a robotic workcell. Geometric models are readily available from computer aided design (CAD) systems, a supporting surface constrains object pose, and the location of the observer relative to the support is known. If the scene is static, the object must be lying in a stable position, a local minimum of its potential energy. In general, an object only assumes a finite number of stable poses, and for each one, the unknown spatial relationship of an observer and the object is reduced to three degrees of freedom; i.e., an object’s pose p is restricted to lw x S0(2).’ When a camera can be located overhead with viewing conditions approximating orthographic projection, measures that are invariant to planar rigid body transformations can be used for recognition and to determine p. Such measures include 2D moments, distances between pairs of feature points, angles between the legs of triangles formed by triplets of features, and contour curvature. There is a wide body of literature that addresses the ’ Surfaces of revolution are exceptions, where the object is free to rotate about its axis, and the potential energy will remain unchanged.
use of these measures (see for example [Gri89, Hor86, SS87]). When the camera axis is not orthogonal to the supporting plane, these measures are no longer invariant under planar motion, and alternatives are required. One particularly appealing approach to “CAD based” vision is the use of viewer centered representations of an object’s appearance such as aspect graphs [KV79]. Whereas the view space of a general orthographic projection aspect graph is two-dimensional, the viewing direction is confined to a circle on the view sphere for each stable pose. Similarly, the dimension of a perspective projection aspect graph is reduced from three to two. Not only is their construction simplified, but the number of stable views is correspondingly reduced, aiding scene interpretation. In his thesis, Chakravarty used the stable poses of objects to reduce the number of characteristic views that were found by tessellating the lower-dimensional view space [Cha82]. The preferred orientations of objects with respect to the world were also used in ACRONYM to simplify recognition. In particular, the initial experiments involved overhead views of airport scenes [Bro81] while in later extensions, explicit constraints for the stable poses of objects were introduced [CLC84]. In each case, the stable poses were specified by a person rather than being derived from the available CAD models. It is well known that the stable poses can be found from the convex hull of the modeled object, and this is feasible for the case of polyhedral objects since many convex hull algorithms are available [PS85]. However, such algorithms are unavailable when objects have curved surfaces, though there has been recent progress [SW87, DS90]. In this paper, we explore an implemented algorithm for determining the stable poses of objects bounded by piecewise smooth curved surfaces. Object models can be described either by Boolean combinations of volumetric primitives through constructive solid geometry (CSG) or by a boundary representation (BREP) of surface splines. Necessary and sufficient conditions for a stable pose are derived that relate the differential geometry of the object at the contact points and the center of mass. In the implementation, surfaces are represented as implicit algebraic equations, and the necessary conditions can be formu-
109 1049-9660/92 $3.00 Copyright 0 1992 by Academic Press, Inc. All rights of reproduction in any form reserved.
110
DAVID J. KRIEGMAN
lated as finding the roots of systems of polynomial equations. These roots are found reliably by the numerical method of homotopy continuation. An algorithm for determining the stable poses of an object is (a) Solve the systems of equations corresponding to the necessary conditions for stability yielding a set S of candidate contact points. (b) For each element of b, compute the object’s pose. (c) Remove from S those elements that do not satisfy the presented sufficient conditions. (d) Since the conditions are only local, remove from S those poses where the object intersects the hypothesized supporting plane at some distal point. An outline of the rest of the paper is as follows. In Section 2, the simplified two-dimensional case is introduced, and it is shown how these results apply to certain classes of three-dimensional objects, namely solids of revolution, cones, and cylinders. These results are extended in Section 3 to more general three dimensional objects. In Section 4, the implementation is represented, and the stable poses of complex objects are shown. Finally, the paper is concluded in Section 5 with a few brief remarks on future directions of this algorithm and its applicability to CAD based vision. 2. TWO-DIMENSIONAL
with the supporting line, otherwise the object would fall. Without friction, the supporting line must also be aligned with i, otherwise the object would slide freely along the supporting line. Without any further loss in generality, the origin o is considered to lie on the supporting line. The object is still free to move along the supporting line at a fixed orientation while maintaining contact, and U will remain unchanged. So, computing the stable poses is reduced to finding the object’s orientation 8 and distance yo from c to the supporting line; the transformation of coordinates in 6 to W is then given by
We now consider conditions for a pose to be stable. In particular, points of contact between the object and the supporting line are found by expressions that relate local curve geometry and the center of mass. Once the contact points are found for a specific object, the pose can be computed easily. In general, these conditions hold in the neighborhood of the contact points, and it must also be determined whether the object intersects the supporting line elsewhere.
OBJECTS
2.1. Before discussing the three-dimensional case, we consider the stable poses of two-dimensional objects bound by piecewise smooth curves in a planar world with gravity. Let [o, i, j] be a basis for a right-handed Euclidean coordinate system called the world frame “Mr,where j is aligned with the gravity vector g, and i is orthogonal to j. The bounding curve can be partitioned into C2 branches which join at vertices. Without loss of generality, each branch is parameterized by arc length and given by xi: I+ [w2with respect to some object frame 6, where Z is an open interval of [w. The coordinates of the center of mass c = [x,, y,lt in the object frame are given by
m
c,l- Ix=-x Iy=-” li
II X
Ph, Y)
dxdy,
(1)
If the only contact point is a vertex, the pose is clearly unstable since the object will simply topple over. Now, consider contact of a smooth curve branch and the supporting halfplane. If the object does not intersect the halfplane at the contact point, the tangent to the bounding curve i(s) = d%(s)lds must be aligned with the supporting line. Since j = g, we have i(s) * g = 0.
(4)
This defines the condition for contact of a smooth curve branch with the supporting line, and consequently, the pose can be parameterized by s.~ Additionally, the potential energy for contact at x(s) is given by
Y
where p is the density of the object at a point (x, y) and m is its mass. The object’s pose p E [w2x SO(2) represents the rigid body transformation between the 6 and W. We are interested in those poses where the object does not intersect the halfplane bounded by a supporting line. An object’s potential energy I/ is defined as U=j-(c-o)+
One Contact Point
Uo,
(2)
where U0 is some constant. A pose is considered stable if it is a local minimum of U with respect to changes in p. Clearly, a pose is stable only if the object is in contact
U(s) = (c - S(s)) * g = (c - k(s)) - ii(s)
(5)
since the curve normal must be aligned with g. A pose is stable if it is a local minimum of U, and a necessary condition is dUlds = 0. Thus, by differentiating (5) and using (4), a necessary condition for stable contact at one point is (c - x(s)) ’ t(s) = 0.
2 Technically, this is called two point contact IBG831.
(6)
111
STABLE POSES OF OBJECTS
.C lILLx-* (4
lc
PI
porting line must lie between the two contact points (p, and p2), or
lc Cd)
(4
0<
FIG. 1. Local conditions for a stable pose of a planar object: (a) one point of contact; (b) contact with two vertices; (c) contact with two smooth curve points; (d) contact with a vertex and a curve point.
For values of s satisfying (6), the pose is then found from (3) and (4). Note that (6) is only a necessary condition for U(S) to be a local minimum. In general, it is sufficient to show that d*CJlds* > 0 at these points. In the case of one contact point, it can be shown from the Frenet-Set-ret equations that this corresponds to [dC76] 1 Ic - x(s)1 < k(s)’
(7)
In other words, the center of mass must be closer to the contact point than the center of the osculating circle. Thus, (6) and (7) are necessary and sufficient local conditions for a stable pose, and an example of a contact point satisfying these conditions is illustrated in Fig. l(a). 2.2.
Two Contact Points
The pose of a two-dimensional object will also be stable if there are two points of contact with the supporting line, and the projection of the center of mass onto the supporting line lies between the contacts. The contact points may either be vertices or lie on smooth curve branches. When both points are vertices as in Fig. l(b), the supporting line passes through the two vertices, and the pose is determined easily. As shown in Fig. l(c), a necessary condition for stable support by two points that lie on smooth curve branches is
1 (XI(SI)
-
x2Cs2N
. nh)
=
0
MSI)
-
x2(~2))
* n2b2)
=
0,
- PI) ’ cc - PI> <
lp2
(10)
- PII*.
If the projection of c falls outside of this interval, the object will “roll” about one of the contact points, hence the pose would be unstable. Figure 2 shows stable and unstable poses of a two-dimensional object bounded by a star shaped curve given by [x(0>, y(0)] = [r(e) cos 8, r(d) sin 01, where r(e) = 4 cos* 0 + 3 cos 8 + 3. The poses in Fig. 2 satisfy the necessary conditions for stability, however, Figs. 2(c) and 2(d) do not satisfy the sufficient condition (7). While the stability of two-dimensional objects may seem irrelevant to three-dimensional computer vision, these results are useful for computing stable poses of three important classes of objects whose bounding surfaces are cylinders, cones, and surfaces of revolution. A cylinder is a surface formed by moving a line (ruling) along a planar generating curve while keeping it perpendicular to that plane; a cone is formed similarly except the rulings pass through a common point. For solids bounded by cylindrical or conical surfaces, contact with the supporting plane occurs along rulings rather than at isolated points. Each ruling corresponds to one point on the generating curve, so stable poses of cylinders and cones can be determined by finding the stable poses of the generating curve in a plane. A surface of revolution is formed by revolving a planar generating curve about an axis lying in the same plane. The solid bounded by the
I
I
(8) a.
where n(s) is the unnormalized curve normal at s and is given by n(s) = dt(s)lds. This system of two equations can be solved for s1 and s2 to obtain a set of pairs of contact points. When one point p is a vertex and the other point lies on a smooth curve branch, the necessary condition for a stable pose is (p - x(s)) . n(s) = 0
(p2
(9)
which can be solved to find points on the curve branch. A sufficient condition for such a pose to be stable is that the projection of the center of mass c onto the sup-
lia
b.
&hi
/
C. 1 FIG. 2. Poses (a) and (b) are stable and satisfy (8). While poses (c) and (d) satisfy (6), they are unstable since they do not satisfy (7). If the center of mass in (d) were shifted downward (perhaps by embedding a lead weight), one can imagine the pose becoming stable, much like an inflatable toy clown.
DAVID J. KRIEGMAN
112
FIG. 3. The stable pose of a vase shaped object formed by revolving the planar curve defined by r(z) = .124z3 - .872z + ,145, z E t-2.5, 2.51, about the z axis. Of course, there are two other stable poses where the vase rests on its top and bottom.
surface of revolution will be stable when its planar generating curve is stable. Note that the pose is also stable when contact occurs along the circle where the tangent to the generating curve is perpendicular to the axis; consider the top and bottom of a torus. Figure 3 shows a stable pose of a tipped-over vase. 3. THREE-DIMENSIONAL
OBJECTS
Conditions for stable poses of three-dimensional curved objects are similar to those for stable poses of two-dimensional objects. In this case, the supporting line becomes a supporting plane, and the pose is a parameterization of R3 x SO(3). For stable poses to exist, the supporting plane must be orthogonal to the gravity vector, and the object must be in contact with the plane. In a stable pose, the object still has three degrees of freedom (rigid motion of the contact points in the supporting plane) where the potential energy remains constant. This leaves three degrees of freedom to describe a stable pose. A constructive solids geometry (CSG) approach is used to model objects; however, it is trivial to extend these results to parametric surface patches [Hof89]. Object models are composed of Boolean combinations (union, intersection, difference) of “primitive” halfspaces. The smooth surfaces bounding the halfspaces are given by some implicit equation S&X,y, z) = 0, where i is the ith surface in the model. The direction of the surface normal at a point p = (x, y, z) is given by ni(p) = Vsi(p). An edge is the intersection between two surfaces and is given implicitly by fi(x, y, z) = 0 and gi(X, y, z) = 0 for edge i. We assume that surfaces intersect transversally so the tangent direction of the edge at a point p is ti = Vh(p) x Vgi(p). With some care in an implementation, the necessity for transversal intersections can be lifted easily Finally, the intersection of three surfaces at a point is named a vertex. As Koenderink recently noted, the convex hull of a generic smooth object corresponds to contact of a plane with the object at one, two, or three points [Koe90]. Sim-
ilarly, the stable poses correspond to contact of the convex hull with the supporting plane; however, the convex hull does not need to be explicitly computed. Below, systems of equations are presented that characterize necessary conditions on the local object geometry for a pose to be stable. The systems of equations can be solved, yielding a set of contact points. From these points, the pose can be computed, and it must be determined if some other part of the object intersects the supporting halfspace. For piecewise smooth objects, contact can occur at regular surface points, along edges, or at vertices. Because the combinatorics of these three contact types leads to the consideration of many cases, only contacts involving edges and regular surface points are presented. Conditions for contact of vertices are simple to derive based on what is presented below. 3.1. One Contact Point Since contact between the supporting plane and a single vertex or edge point is always unstable, only contact at regular surface points will be considered. If you are unconvinced that an object’s pose can be stable with only one contact point, consider an M&M candy resting on a table. Following a similar line of argument as in the 2D case, the necessary condition for a pose to be stable with one contact point p is that the line between p and the center of mass is orthogonal to the tangent plane at p. This can be expressed as3
s(p) = 0 i (p - c) x Vs(p) = 0.
(11)
Points satisfying (11) are critical points of the potential energy function U, and for the pose to be stable, U must be a local minimum. Also note that the contact point must not be hyperbolic since the object would necessarily intersect the supporting halfplane; i.e., the surface crosses the tangent plane at hyperbolic points. Now, any perturbation of the contact point from p amounts to moving in some direction on the surface. Since the normal curvature at p lies between the two principal curvatures kl and kZ, the pose will be stable when lc - PI
3.2.
(
1 1 k,,~.
1
(12)
Two Contact Points
Like the runners on a rocking horse, an object can contact the supporting plane at two points (p, and pZ) in a
3 Note that the cross-product gives three equations, but only two of these are independent.
113
STABLE POSES OF OBJECTS
stable pose. Excluding vertices, there are three possible pairings of contact points, namely surface-surface, surface-edge, and edge-edge. In these cases, the line between the contact points must lie in the tangent space at the contact points. As in the two-dimensional case, the projection of the center of mass in the direction of gravity onto the line defined by the contact points must lie between these points. Otherwise, the object would topple over one of the contact points. Additionally, there is a relationship between center of mass and the curvature at the contact points ensuring that the object is not top heavy which would cause it to topple sideways. The necessary conditions for stability with two points of contact are detailed in the appendix. First consider contact at two regular surface points where the surface normals are aligned, the line between p1 and p2 lies in the tangent planes, and the center of mass lies in the plane defined by the contact points and their surface normals. The following equations characterize this condition where i = 1, 2: = 0
Si(pi>
i
n&h
> x n2@2)
nib
> * (pz - PI) = 0
\(c
-
pi)
= 0
x (c -
pd
- ndpd
(13) = 0.
For contact at a regular surface point p1 and an edge point p2, the tangent at p2 must lie in the supporting plane, and the necessary conditions are given by
f Sl(Pl) = 0 fdp2) 1
n&d
* f2@2)
= 0
n&l)
* (p2 -
PI) = 0
PI> X Cc -
pd
- ndpl)
= 0.
Finally, the two supporting points may lie on edges. Since the line between pl and p2 as well as the curve tangents at these points must be coplanar, the normal to the supporting plane is given by n = tl(p,) x (~2 - pd. The conditions for two such contact points i = 1, 2 are then given by
I
gi(pi)
= =
n * f2@2) (C -
Three Contact Points
Finally, as for a tripod, an object can contact the supporting plane at three noncollinear points (p,, p2 and p3). There are four cases when the points lie on regular surfaces or edges which are characterized by systems of nine equations in nine unknowns. The pose will be stable if the projection of the center of mass onto the supporting plane is within the triangle defined by the three points. When the contacts are regular surface points, the surface normals are orthogonal to the plane defined by pl, p2, and ~3, and the following system where i = 1,2, 3 defines a necessary condition for stability.
i
= 0
jXPi)
3.3.
Sitpi>
=
n&i)
* (p2
-
PII
ni(P;>
* @3
-
PI> = 0
0 =
0
(16)
= 0
g2@2)
, (c -
If tl is aligned with p2 - pI, then n = 0 and the last equation in (15) will be satisfied even when c is not above the line defined by pi and p2. For this case, the necessary condition is satisfied when n = t2(p2) x (~2 - p,) in the last equation. Each of the systems (13), (14), and (15) is composed of six equations and six unknowns which can be solved to find the coordinates of the two contact points. For each contact type, the test in (10) can be used to determine if the projection of the center of mass lies between the contact points. In addition, it must be confirmed that u” > 0 for motion corresponding to a sideways roll while two contact points are maintained.
pr)
O
When two regular surface points and an edge point are involved, the conditions for the regular points are the same as in (16). Additionally, the tangent to the edge point must be orthogonal to the surface normals, or with i= 1,2 f Si(pi)
=
0
fxp3)
=
0
&?3(P3) = 0 t
(17) ni(pi)
* (p2
-
pi>
=
0
ni(pil
’ (p3
-
PI>
=
0
,nhd
* f3@3)
=
0.
O =
0
x Cc -
~2) * n = 0.
The following system describes tripod stability with one regular surface point and two edge points where i = 2. 3:
114
DAVID J. KRIEGMAN Sl(Pl)
=
curve on a surface or at a point on an edge. After the candidate pose transformation is applied to the model, intersection with model edge i can be expressed as
0
A(Pi) = O
gi(Pi> = O nl(pd
* (~2
-
pd
=
0
nl(pJ
. (~3
-
pJ
=
0
ndpl)
’ ti(pi> = 0.
.0x, Y, z) = 0 z = 0.
Finally, the three contact points can lie on edges, in which case the curve tangents must be in the plane defined by these points and whose normal is n = (~2 - pt) x (p3 - p,). For i = 1, 2, 3, the defining system is
.fxpJ = 0 g&i)
To determine if the supporting plane intersects a surface in a curve that does not intersect an edge, it is sufficient to find a single point on that curve. For a bounded object, these curves will have at least two extremal points in any direction, and so arbitrarily choosing a direction, such a point is found by solving
(19)
= 0
SAX, Y, Z) = 0
i n ' ti(pJ = 0. 4.
IMPLEMENTATION
cm
gi(X, YT Z> = 0
z=o
AND RESULTS
Very few assumptions about the modeled surfaces were made in the previous section except that they are C2, defined implicitly, and intersect transversally. The implementation is more restrictive in that surfaces are represented by implicit algebraic (polynomial) equations; however, this class of objects subsumes most representations used in current CAD systems. It should be noted that rational parametric patch representations of surfaces can be implicitized using resultants [SAG84]. Alternatively, the conditions for stability can be reformulated in terms of the patch domain coordinates. To model surfaces by implicit algebraic equations, we have been construtting a geometric modeling system that uses a new curve tracing algorithm to find surface intersections [PK90, KP911. All of the necessary conditions for a stable pose described in Sections 2 and 3 are now expressed as systems of polynomial equations, and a large collection of symbolic and numerical techniques are available for manipulating and solving such systems. In particular, the generic form of homotopy continuation described by Morgan is used to find numerically all real and complex roots [Mor87]. Each real root corresponds to one, two, or three contact points, and along with the surface normals and edge tangents, they define a plane. The candidate pose simply corresponds to the transformation which aligns this plane with a plane orthogonal to the direction of gravity. Once the pose is determined, the sufficient local conditions for stability must be checked. Finally, the necessary conditions only relate the local geometry of the contact points, and so some distal part of the object might intersect the supporting plane. If so, the intersection of the object and the plane will occur in a
(21)
-$ Si(X, y, z) = 0.
The presented approach has been successfully applied to a number object models; Fig. 4 shows the stable poses of a PVC pipe fitting. The object is modeled as the union of five bounded right circular cylinders and a sphere.
FE .=*
FLPE
Liiz&!z
b.
C.
d.
I
I
I
FIG. 4. The stable poses of a PVC pipe fitting: Poses (a), (b), and(c) satisfy the conditions of one point contact even though contact actually occurs over the entire planar patch. (d) shows a pose with contact at three edge points as does (e) and (f). Note that contact occurs along a ruling which terminates at two edge points. (g) shows contact at one surface point and two edges.
STABLE POSES OF OBJECTS
b. 1
C. I
d. I
I
I
I
115
tions satisfied the necessary conditions for stability, U’ = 0, but did not satisfy the sufficient conditions. As illustrated in Fig. 5, these balance points include contact at one surface point, at edge and surface points, and at two edge points. Finally it should be noted that the algorithm, implemented in common lisp, required about two days to execute on a Sun SPARCstation 1 for this model. Nearly all of the time was spent solving the systems of polynomial equations describing the necessary conditions. Figure 6 shows the stable poses of another pipe fitting, while Fig. 7 shows the balance points. When the algorithm was applied to this model for the first time, the pose in Fig. 6(e) was surprising. Placing the actual fitting on a table confirmed that this precarious looking pose is actually stable. 5. SUMMARY AND CONCLUSIONS
In this paper, necessary and sufficient conditions were presented for the pose of three-dimensional piecewise e. f. g. smooth objects to be stable. In turn, these conditions FIG. 5. The balance poses of a PVC pipe fitting: (a) shows contact have been used in an implementation for objects whose at two edge points which correspond to a ruling of the cylinder. (b) and bounding surfaces are given by implicit algebraic equa(c) show contact at an edge and surface point. (d) dramatically shows tions. In the special case where surfaces are cylindrical, balancing on one surface point, (e), (f), and (g) correspond to contact at conical, or surfaces of revolution, computing stable two edge points. poses can be simplified to determining stable poses of the corresponding two-dimensional generating curves. As noted previously, the running time for computing Because contact with any point on a ruled surface such as a cylinder implies contact along an entire ruling which the results in Fig. 4 is quite long, and so it seems relevant always intersects an edge for a bounded surface, it is to explore the complexity of the algorithm. Most of the sufficient to only consider contact with the corresponding time is spent solving systems of polynomials using a nuedges. Additionally, the center of mass of the real pipe merical method, and one measure of complexity is the fitting was found experimentally and used in the compu- number of roots given by the total degree of the system (the product of the degrees of the individual equations). tation. In the process of finding the stable poses, seven solu-
FIG. 6. The stable poses of another PVC pipe fitting: Poses (a), (b) and (c) satisfy the conditions of one point contact, pose (e) has two points of contact, and poses (d) and (f) satisfy three point contact conditions though contact occurs along rulings of the cylinder.
FIG. 7. The balance poses of the other PVC pipe fitting: All four poses satisfy the necessary conditions of two point contact. Note that the pose in (c) (similarly (d)) is a local maximum of potential energy lying between the two minima shown in Fig. 6(e) and (f) for a rolling motion about the two edges.
116
DAVID J. KRIEGMAN
For an object whose stable poses can be computed without any vertices as in the pipe fitting, the number of roots is O((s + e)3d9), where s is the number of surfaces in the model, e is the number of edges, and d is the degree of the defining equations. This arises from considering triplets of edges and surfaces, and noting that the characterization of stable poses with three contact points has total degree O(d9). For example, the total degree of the system (16) describing support of three spheres is 512. The number of edges requiring consideration is given by the Euler-Poincare formula, and for an object of genus 0 without any vertices, there are two more edges than faces. Also, many objects, such as the pipe fittings, have planar faces and a mixture of curved surfaces of varying degree; this leads to faster running times than indicated by the highest degree surfaces. A number of improvements to the basic algorithm are possible. First, by precomputing a boundary representation (BREP) of the model, one only needs to consider the true number of edges rather than every pairing of participating surfaces. A priori, some surfaces and edges cannot contain contact points due to their local geometry. For example, surface patches that are globally hyperbolic as well as ones that are elliptic but concave cannot be participants. When the surfaces defining an edge form a concavity along the entire length of the edge, that edge does not need to be considered. Additionally, it may be unnecessary to consider all pairs or triplets of edges and surfaces as was done here. Instead, it might be possible to order the participants in a manner similar to convex hull algorithms. Such algorithms are available for planar objects, but not to the best of my knowledge for 3D objects [SW87]. In some cases, the symmetry of objects such as the pipe fitting can be utilized to simplify computation. Alternatively, it might be possible to reduce the total degree of the defining systems. In practice, there are many roots at infinity, indicating that the systems might be reducible. Additionally, since the Bezout number of these systems is lower than the total degree because of the roots at infinity, other homotopies that exploit their multihomogeneous structure could be utilized [LSY89, MS87]. Finally, since a given set of surfaces has very few contact points that satisfy the necessary conditions, it might be possible to parameterize the surface types, precompute the candidate stable poses for a set of generic surfaces, and then use coefficient-parameter polynomial continuation for a particular instance of the problem. This could greatly improve performance at the cost of increased computation for the generic system; however, this only needs to be solved once [LSY89, MS891. Finally, since the homotopy continuation method allows the roots to be computed independently, they can be found in parallel; a preliminary implementation, which is distributed over a network of SPARC stations, looks promising.
To conclude, let us consider how knowledge of the stable poses can be exploited in “CAD-based’ vision algorithms. Typically, when the pose of an object is computed from an image, all six parameters of its location must be found. However, if it is known that the object will be in a specific stable pose, it is only necessary to compute the three parameters of rigid body motion in a plane. Even if the specific stable pose is unclear, knowing that three of the parameters only take on certain values can be used to advantage. For example, when the parameterized implicit equation of image contours is fitted as in [KP90b], a set of initial conditions is required, and these could be the parameters of the stable poses. Alternatively, if all six parameters are fit, the stable poses can be used for confirmation. Finally, some algorithms for recognition and determining pose use viewer-centered representations such as aspect graphs. Under orthographic projection, the viewer’s location is modeled as lying on a sphere. However, when the pose is known, only a circle on the view sphere needs consideration. This could greatly reduce the number of views in an aspect graph, and simplify both their construction and use in interpreting images. For example, in the aspect graph of the vase shown in Fig. 3, there are 13 stable views separated by visual events lying on lines of latitude of the view sphere [EB89, KP90al. Since this solid of revolution has three stable poses, the simplified aspect graph for an overhead view only contains three views. When the camera axis is 30” from the vertical, there are only six views. Figure 8 shows the partitioning of the view sphere into regions where the aspect remains constrained with respect to changes of viewpoint as well as those viewpoints where the objects pose is stable. Even if it cannot be guaranteed that objects will be lying in stable poses, hypothesized poses (and consequently aspects) can be ordered to consider initially the more likely ones during recognition.
FIG. 8. The partitioning of the view sphere for the aspect graph of the vase in Fig. 3 with the allowable views for a camera situated 30” from vertical and the vase in its three stable poses as depicted in bold.
117
STABLE POSES OF OBJECTS
APPENDIX
In this appendix, it is shown that for stable poses with two points of contact, the center of mass must lie in the plane that includes these points and the normal to the supporting plane. This formed part of the necessary conditions of Section 3.2. Consider the case of stable support by two edge points. Without loss of generality, the edges are parameterized by arc length and given by x~(sJ and x&). The tangents are then t;(si) = x,‘(si). Contact with a plane implies that the tangents and the line between the two contact points are coplanar, or (x1 - x2) * (t, x f2) = 0.
-
t, x n2 = 0,
x f2 + kZ 2
x2> [klv
1
1
U = (c - x1) . n,
(24)
where n = tl x f2/(fl x f2(. Note that if the two tangents are parallel, n is undefined; in this case, the following line of reasoning will apply with n = tl X (XI - xz)//tl X (x1 x2)1. Differentiating U with respect to $1 to find the necessary condition for stability, we have
U = (c - xI(u, u)) . n(u, u).
U’ = (c - xl) * (n,,u’ + n,u’).
ACKNOWLEDGMENTS I thank Jean Ponce, Alfred Rizzi, and B. Vijayakumor for many useful discussions, and Michael Tarr for providing the PVC pipe fittings. This work was supported in part by the National Science Foundation under Grant IRI-901574.
REFERENCES [BG83]
where d = (~/c%I) (tl x f2) = klnl x f2 + k2fl x n2(ds2/dsl). The fact that c lies in the plane through the supporting points and is orthogonal to gravity can be expressed as c - XI = (Y(XZ- x1) + j3tl x f2, where (Y,p E R. From (29, we have tl X t2 . h/as, = 0, and along with (22) and (23), it is simple to show that dUlds, = 0. Now, both cases of contact at two points, where -one lies on a regular surface, can be considered simultaneously. By the implicit function theorem, the surface can be locally parameterized in the form xI(u, u), and the relationship of the two contact points in (13) and (14) is (~164
4
-
~2)
* 44
u)
=
0,
(2’3
(29)
To show that the center of mass lies in the plane defined by the contact points and is orthogonal to the support plane, let c - x1 = a(x2 - x1) + /3n, where (Y,/3 E R. Note that n, and n, must lie in the tangent plane [dC76], and so n * (n,u’ + n,u’) = 0. Using (27), it is clear that U’ = 0. Therefore, the center of mass must lie in the plane defined by the contact points and the normal to the supporting plane.
(25)
djtl x t212- [d * tl x f2]fI x f2 3 16 x f213
(28)
Now, the necessary condition is that U be a local minimum for changes in s. Differentiating U with respect to s yields
= (c - Xl) ’ E = (c - x,) *
(27)
where n, means an/au. Note that this expression holds whether x2 is a surface or edge point. The potential energy is given by
(23)
where kl and k2 are the curvatures at the edge points. The object’s potential energy is
z
(xI(u, u) - x2) * (n,u’ + n”u’) = 0,
(22)
This relates s1 to s2, and the partial derivatives are used to find &sz/&, , namely
(XI
where n(u, u) is the surface normal. The object is free to rock while maintaining two points of contact, and this corresponds to a curve of contact points given implicitly by (26) and nl x n2 = 0 for a second surface point. This curve can be represented in the domain of the surface as [u, u] = [u(s), u(s)]. Differentiating (26) with respect to s gives
J. W. Bruce and P. J. Giblin, Generic geometry, Amer. Math. Monthly, October 1983, 529-545. [Bro81] R. A. Brooks, Symbolic reasoning among 3-D models and 2-D images, ArtiJicial Intelligence, 17, 1981, 285-348. [Cha82] Indranil Chakravarty, The Use of Characteristic Views as a Basis for Recognition of Three-Dimensional Objects. Image Processing Laboratory IPL-TR-034, Rensselaer Polytechnic Institute, October 1982. [CLC84] D. M. Chelberg, H. S. Lim, and C. K. Cowan, Acronym model based vision in the intelligent task automation project, Proceedings, Image Understanding Workshop, October 1984, pp. 308-315. [dC76] M. P. do Carmo, Differential Geometry of Curves and Surfaces, Prentice-Hall, Englewood Cliffs, NJ, 1976. [DS90] D. P. Dobkin and D. L. Souvaine, Computational geometry in a curved world, Algorithmica, 5(3), 1990, 421-457. [EB89] D. Eggert and K. Bowyer, Computing the orthographic projection aspect graph of solids of revolution, in IEEE Workshop on Interpretation of 30 Scenes, Austin, TX, November 1989, pp. 102-108.
118 [Gri89]
DAVID J. KRIEGMAN W. E. L. Grimson, On the recognition of curved objects, IEEE Trans. Pattern
[Hof89] [Hors61 [Koe90] [KP90a]
[KP90b]
[Kill
[KV79]
[LSY89j
Anal. Mach. Intelligence,
11(6), 1989,
632-643. Christoph M. Hoffman, Geometric and So/id Modeling, Morgan Kaufmann, San Mateo, CA, 1989. B. K. P. Horn, Computer Vision, MIT Press, Cambridge, MA, 1986. Jan J. Koenderink, Solid Shape, MIT Press, Cambridge, MA, 1990. D. J. Kriegman and J. Ponce, Computing exact aspect graphs of curved objects: Solids of revolution, Int. J. Comput. Vision 5(2), 1990, 119-135. D. J. Kriegman and J. Ponce, On recognizing and positioning curved 3D objects from image contours, IEEE Trans. Pattern Anal. Mach. Intelligence 12(12), 1990, 1127-1137. D. J. Kriegman and J. Ponce, A new curve tracing algorithm and some application, in Curves and Surfaces, (P. J. Laurent, A. Le Mehaute, and L. L. Schumaker, Eds.), Academic Press, San Diego, 1991, pp. 267-270. J. J. Koenderink and A. J. Van Doorn, The internal representation of solid shape with respect to vision, Biol. Cybernet. 32, 1979, 211-216. T. Y. Li, T. Sauer, and J. A. Yorke, The cheaters homotopy: An efficient procedure for solving systems of polynomial equations, SIAM J. Numer. Anal. 26(5), 1989, 1241-1251.
[Mor87] Alexander Morgan, Solving Polynomial tinuation for Engineering
Systems Using Conand ScientiJic Problems, Prentice-
Hall, Englewood Cliffs, NJ, 1987. [MS871 Alexander Morgan and Andrew Sommese, A homotopy for solving genera1 polynomial systems that respects m-homogeneous structures, Appl. Math. Comput. 24, 1987, 101-113. [MS891 Alexander Morgan and Andrew Sommese, Coefficientparameter polynomial continuation, Appl. Math. Comput., 29, 1989, 123-160. [PK90] J. Ponce and D.‘ J. Kriegman, Computing Exact Aspect Graphs of Curved Objects: Parametric Patches, Technical Report UIUCDCS-R-90-1579, University of Illinois at Urbana-champaign, April 1990. [PSSSI F. P. Preparata and M. I. Shamos, Computational Geometry: Springer-Verlag, New York, 1985. An Introduction, [SAG841 Thomas W. Sederberg, D. C. Anderson, and Ronald N. Goldman, Implicit representation of parametric curves and surfaces, Comput. Vision Graphics Image Process. 28, 1984,7284. [SS87] J. T. Schwartz and M. Sharir, Identification of partially obscured objects in two and three dimensions by matching noisy characteristic curves, Int. J. Robot. Res. 6(2), 1987, 29-44. [SW871 A. A. Schaffer and C. J. Van Wyk, Convex hulls of piecewise-smooth jordan curves, J. Algorithms 8, 1987, 66-94.
IMAGE
UNDERSTANDING
Vol. 55, No. 2, March, pp. 109-118, 1992
Computing
Stable Poses of Piecewise Smooth Objects DAVID J. KRIEGMAN
Center for Systems Science, Department
of Electrical
Engineering,
Yale University,
New Haven,
Connecticut
06.520-1968
Received August 19, 1991; accepted November 13, 1991
When a three dimensional object is known to be lying on a planar surface, its pose is restricted from six to three degreesof freedom. Computer vision algorithms can exploit the few stable posesof modeled objects to simplify sceneinterpretation and more accurately determine object location. This paper presents necessary and sufficient conditions for the pose of a piecewise smooth curved three-dimensional object to be stable. For objects whose surfaces are represented by implicit algebraic equations, these conditions can be expressed as systems of polynomial equations that are readily solved by homotopy continuation. Examples from the implemented algorithm are presented. o 1992 hdemic PW, IW.
1. INTRODUCTION
lndependent of the task, “CAD-Based” vision algorithms should exploit all available knowledge of object geometry, lighting conditions, viewpoint constraints, and material characteristics. Consider the prototypical task of determining the pose (position and orientation) and identity of a three-dimensional object traveling along a conveyor belt into a robotic workcell. Geometric models are readily available from computer aided design (CAD) systems, a supporting surface constrains object pose, and the location of the observer relative to the support is known. If the scene is static, the object must be lying in a stable position, a local minimum of its potential energy. In general, an object only assumes a finite number of stable poses, and for each one, the unknown spatial relationship of an observer and the object is reduced to three degrees of freedom; i.e., an object’s pose p is restricted to lw x S0(2).’ When a camera can be located overhead with viewing conditions approximating orthographic projection, measures that are invariant to planar rigid body transformations can be used for recognition and to determine p. Such measures include 2D moments, distances between pairs of feature points, angles between the legs of triangles formed by triplets of features, and contour curvature. There is a wide body of literature that addresses the ’ Surfaces of revolution are exceptions, where the object is free to rotate about its axis, and the potential energy will remain unchanged.
use of these measures (see for example [Gri89, Hor86, SS87]). When the camera axis is not orthogonal to the supporting plane, these measures are no longer invariant under planar motion, and alternatives are required. One particularly appealing approach to “CAD based” vision is the use of viewer centered representations of an object’s appearance such as aspect graphs [KV79]. Whereas the view space of a general orthographic projection aspect graph is two-dimensional, the viewing direction is confined to a circle on the view sphere for each stable pose. Similarly, the dimension of a perspective projection aspect graph is reduced from three to two. Not only is their construction simplified, but the number of stable views is correspondingly reduced, aiding scene interpretation. In his thesis, Chakravarty used the stable poses of objects to reduce the number of characteristic views that were found by tessellating the lower-dimensional view space [Cha82]. The preferred orientations of objects with respect to the world were also used in ACRONYM to simplify recognition. In particular, the initial experiments involved overhead views of airport scenes [Bro81] while in later extensions, explicit constraints for the stable poses of objects were introduced [CLC84]. In each case, the stable poses were specified by a person rather than being derived from the available CAD models. It is well known that the stable poses can be found from the convex hull of the modeled object, and this is feasible for the case of polyhedral objects since many convex hull algorithms are available [PS85]. However, such algorithms are unavailable when objects have curved surfaces, though there has been recent progress [SW87, DS90]. In this paper, we explore an implemented algorithm for determining the stable poses of objects bounded by piecewise smooth curved surfaces. Object models can be described either by Boolean combinations of volumetric primitives through constructive solid geometry (CSG) or by a boundary representation (BREP) of surface splines. Necessary and sufficient conditions for a stable pose are derived that relate the differential geometry of the object at the contact points and the center of mass. In the implementation, surfaces are represented as implicit algebraic equations, and the necessary conditions can be formu-
109 1049-9660/92 $3.00 Copyright 0 1992 by Academic Press, Inc. All rights of reproduction in any form reserved.
110
DAVID J. KRIEGMAN
lated as finding the roots of systems of polynomial equations. These roots are found reliably by the numerical method of homotopy continuation. An algorithm for determining the stable poses of an object is (a) Solve the systems of equations corresponding to the necessary conditions for stability yielding a set S of candidate contact points. (b) For each element of b, compute the object’s pose. (c) Remove from S those elements that do not satisfy the presented sufficient conditions. (d) Since the conditions are only local, remove from S those poses where the object intersects the hypothesized supporting plane at some distal point. An outline of the rest of the paper is as follows. In Section 2, the simplified two-dimensional case is introduced, and it is shown how these results apply to certain classes of three-dimensional objects, namely solids of revolution, cones, and cylinders. These results are extended in Section 3 to more general three dimensional objects. In Section 4, the implementation is represented, and the stable poses of complex objects are shown. Finally, the paper is concluded in Section 5 with a few brief remarks on future directions of this algorithm and its applicability to CAD based vision. 2. TWO-DIMENSIONAL
with the supporting line, otherwise the object would fall. Without friction, the supporting line must also be aligned with i, otherwise the object would slide freely along the supporting line. Without any further loss in generality, the origin o is considered to lie on the supporting line. The object is still free to move along the supporting line at a fixed orientation while maintaining contact, and U will remain unchanged. So, computing the stable poses is reduced to finding the object’s orientation 8 and distance yo from c to the supporting line; the transformation of coordinates in 6 to W is then given by
We now consider conditions for a pose to be stable. In particular, points of contact between the object and the supporting line are found by expressions that relate local curve geometry and the center of mass. Once the contact points are found for a specific object, the pose can be computed easily. In general, these conditions hold in the neighborhood of the contact points, and it must also be determined whether the object intersects the supporting line elsewhere.
OBJECTS
2.1. Before discussing the three-dimensional case, we consider the stable poses of two-dimensional objects bound by piecewise smooth curves in a planar world with gravity. Let [o, i, j] be a basis for a right-handed Euclidean coordinate system called the world frame “Mr,where j is aligned with the gravity vector g, and i is orthogonal to j. The bounding curve can be partitioned into C2 branches which join at vertices. Without loss of generality, each branch is parameterized by arc length and given by xi: I+ [w2with respect to some object frame 6, where Z is an open interval of [w. The coordinates of the center of mass c = [x,, y,lt in the object frame are given by
m
c,l- Ix=-x Iy=-” li
II X
Ph, Y)
dxdy,
(1)
If the only contact point is a vertex, the pose is clearly unstable since the object will simply topple over. Now, consider contact of a smooth curve branch and the supporting halfplane. If the object does not intersect the halfplane at the contact point, the tangent to the bounding curve i(s) = d%(s)lds must be aligned with the supporting line. Since j = g, we have i(s) * g = 0.
(4)
This defines the condition for contact of a smooth curve branch with the supporting line, and consequently, the pose can be parameterized by s.~ Additionally, the potential energy for contact at x(s) is given by
Y
where p is the density of the object at a point (x, y) and m is its mass. The object’s pose p E [w2x SO(2) represents the rigid body transformation between the 6 and W. We are interested in those poses where the object does not intersect the halfplane bounded by a supporting line. An object’s potential energy I/ is defined as U=j-(c-o)+
One Contact Point
Uo,
(2)
where U0 is some constant. A pose is considered stable if it is a local minimum of U with respect to changes in p. Clearly, a pose is stable only if the object is in contact
U(s) = (c - S(s)) * g = (c - k(s)) - ii(s)
(5)
since the curve normal must be aligned with g. A pose is stable if it is a local minimum of U, and a necessary condition is dUlds = 0. Thus, by differentiating (5) and using (4), a necessary condition for stable contact at one point is (c - x(s)) ’ t(s) = 0.
2 Technically, this is called two point contact IBG831.
(6)
111
STABLE POSES OF OBJECTS
.C lILLx-* (4
lc
PI
porting line must lie between the two contact points (p, and p2), or
lc Cd)
(4
0<
FIG. 1. Local conditions for a stable pose of a planar object: (a) one point of contact; (b) contact with two vertices; (c) contact with two smooth curve points; (d) contact with a vertex and a curve point.
For values of s satisfying (6), the pose is then found from (3) and (4). Note that (6) is only a necessary condition for U(S) to be a local minimum. In general, it is sufficient to show that d*CJlds* > 0 at these points. In the case of one contact point, it can be shown from the Frenet-Set-ret equations that this corresponds to [dC76] 1 Ic - x(s)1 < k(s)’
(7)
In other words, the center of mass must be closer to the contact point than the center of the osculating circle. Thus, (6) and (7) are necessary and sufficient local conditions for a stable pose, and an example of a contact point satisfying these conditions is illustrated in Fig. l(a). 2.2.
Two Contact Points
The pose of a two-dimensional object will also be stable if there are two points of contact with the supporting line, and the projection of the center of mass onto the supporting line lies between the contacts. The contact points may either be vertices or lie on smooth curve branches. When both points are vertices as in Fig. l(b), the supporting line passes through the two vertices, and the pose is determined easily. As shown in Fig. l(c), a necessary condition for stable support by two points that lie on smooth curve branches is
1 (XI(SI)
-
x2Cs2N
. nh)
=
0
MSI)
-
x2(~2))
* n2b2)
=
0,
- PI) ’ cc - PI> <
lp2
(10)
- PII*.
If the projection of c falls outside of this interval, the object will “roll” about one of the contact points, hence the pose would be unstable. Figure 2 shows stable and unstable poses of a two-dimensional object bounded by a star shaped curve given by [x(0>, y(0)] = [r(e) cos 8, r(d) sin 01, where r(e) = 4 cos* 0 + 3 cos 8 + 3. The poses in Fig. 2 satisfy the necessary conditions for stability, however, Figs. 2(c) and 2(d) do not satisfy the sufficient condition (7). While the stability of two-dimensional objects may seem irrelevant to three-dimensional computer vision, these results are useful for computing stable poses of three important classes of objects whose bounding surfaces are cylinders, cones, and surfaces of revolution. A cylinder is a surface formed by moving a line (ruling) along a planar generating curve while keeping it perpendicular to that plane; a cone is formed similarly except the rulings pass through a common point. For solids bounded by cylindrical or conical surfaces, contact with the supporting plane occurs along rulings rather than at isolated points. Each ruling corresponds to one point on the generating curve, so stable poses of cylinders and cones can be determined by finding the stable poses of the generating curve in a plane. A surface of revolution is formed by revolving a planar generating curve about an axis lying in the same plane. The solid bounded by the
I
I
(8) a.
where n(s) is the unnormalized curve normal at s and is given by n(s) = dt(s)lds. This system of two equations can be solved for s1 and s2 to obtain a set of pairs of contact points. When one point p is a vertex and the other point lies on a smooth curve branch, the necessary condition for a stable pose is (p - x(s)) . n(s) = 0
(p2
(9)
which can be solved to find points on the curve branch. A sufficient condition for such a pose to be stable is that the projection of the center of mass c onto the sup-
lia
b.
&hi
/
C. 1 FIG. 2. Poses (a) and (b) are stable and satisfy (8). While poses (c) and (d) satisfy (6), they are unstable since they do not satisfy (7). If the center of mass in (d) were shifted downward (perhaps by embedding a lead weight), one can imagine the pose becoming stable, much like an inflatable toy clown.
DAVID J. KRIEGMAN
112
FIG. 3. The stable pose of a vase shaped object formed by revolving the planar curve defined by r(z) = .124z3 - .872z + ,145, z E t-2.5, 2.51, about the z axis. Of course, there are two other stable poses where the vase rests on its top and bottom.
surface of revolution will be stable when its planar generating curve is stable. Note that the pose is also stable when contact occurs along the circle where the tangent to the generating curve is perpendicular to the axis; consider the top and bottom of a torus. Figure 3 shows a stable pose of a tipped-over vase. 3. THREE-DIMENSIONAL
OBJECTS
Conditions for stable poses of three-dimensional curved objects are similar to those for stable poses of two-dimensional objects. In this case, the supporting line becomes a supporting plane, and the pose is a parameterization of R3 x SO(3). For stable poses to exist, the supporting plane must be orthogonal to the gravity vector, and the object must be in contact with the plane. In a stable pose, the object still has three degrees of freedom (rigid motion of the contact points in the supporting plane) where the potential energy remains constant. This leaves three degrees of freedom to describe a stable pose. A constructive solids geometry (CSG) approach is used to model objects; however, it is trivial to extend these results to parametric surface patches [Hof89]. Object models are composed of Boolean combinations (union, intersection, difference) of “primitive” halfspaces. The smooth surfaces bounding the halfspaces are given by some implicit equation S&X,y, z) = 0, where i is the ith surface in the model. The direction of the surface normal at a point p = (x, y, z) is given by ni(p) = Vsi(p). An edge is the intersection between two surfaces and is given implicitly by fi(x, y, z) = 0 and gi(X, y, z) = 0 for edge i. We assume that surfaces intersect transversally so the tangent direction of the edge at a point p is ti = Vh(p) x Vgi(p). With some care in an implementation, the necessity for transversal intersections can be lifted easily Finally, the intersection of three surfaces at a point is named a vertex. As Koenderink recently noted, the convex hull of a generic smooth object corresponds to contact of a plane with the object at one, two, or three points [Koe90]. Sim-
ilarly, the stable poses correspond to contact of the convex hull with the supporting plane; however, the convex hull does not need to be explicitly computed. Below, systems of equations are presented that characterize necessary conditions on the local object geometry for a pose to be stable. The systems of equations can be solved, yielding a set of contact points. From these points, the pose can be computed, and it must be determined if some other part of the object intersects the supporting halfspace. For piecewise smooth objects, contact can occur at regular surface points, along edges, or at vertices. Because the combinatorics of these three contact types leads to the consideration of many cases, only contacts involving edges and regular surface points are presented. Conditions for contact of vertices are simple to derive based on what is presented below. 3.1. One Contact Point Since contact between the supporting plane and a single vertex or edge point is always unstable, only contact at regular surface points will be considered. If you are unconvinced that an object’s pose can be stable with only one contact point, consider an M&M candy resting on a table. Following a similar line of argument as in the 2D case, the necessary condition for a pose to be stable with one contact point p is that the line between p and the center of mass is orthogonal to the tangent plane at p. This can be expressed as3
s(p) = 0 i (p - c) x Vs(p) = 0.
(11)
Points satisfying (11) are critical points of the potential energy function U, and for the pose to be stable, U must be a local minimum. Also note that the contact point must not be hyperbolic since the object would necessarily intersect the supporting halfplane; i.e., the surface crosses the tangent plane at hyperbolic points. Now, any perturbation of the contact point from p amounts to moving in some direction on the surface. Since the normal curvature at p lies between the two principal curvatures kl and kZ, the pose will be stable when lc - PI
3.2.
(
1 1 k,,~.
1
(12)
Two Contact Points
Like the runners on a rocking horse, an object can contact the supporting plane at two points (p, and pZ) in a
3 Note that the cross-product gives three equations, but only two of these are independent.
113
STABLE POSES OF OBJECTS
stable pose. Excluding vertices, there are three possible pairings of contact points, namely surface-surface, surface-edge, and edge-edge. In these cases, the line between the contact points must lie in the tangent space at the contact points. As in the two-dimensional case, the projection of the center of mass in the direction of gravity onto the line defined by the contact points must lie between these points. Otherwise, the object would topple over one of the contact points. Additionally, there is a relationship between center of mass and the curvature at the contact points ensuring that the object is not top heavy which would cause it to topple sideways. The necessary conditions for stability with two points of contact are detailed in the appendix. First consider contact at two regular surface points where the surface normals are aligned, the line between p1 and p2 lies in the tangent planes, and the center of mass lies in the plane defined by the contact points and their surface normals. The following equations characterize this condition where i = 1, 2: = 0
Si(pi>
i
n&h
> x n2@2)
nib
> * (pz - PI) = 0
\(c
-
pi)
= 0
x (c -
pd
- ndpd
(13) = 0.
For contact at a regular surface point p1 and an edge point p2, the tangent at p2 must lie in the supporting plane, and the necessary conditions are given by
f Sl(Pl) = 0 fdp2) 1
n&d
* f2@2)
= 0
n&l)
* (p2 -
PI) = 0
PI> X Cc -
pd
- ndpl)
= 0.
Finally, the two supporting points may lie on edges. Since the line between pl and p2 as well as the curve tangents at these points must be coplanar, the normal to the supporting plane is given by n = tl(p,) x (~2 - pd. The conditions for two such contact points i = 1, 2 are then given by
I
gi(pi)
= =
n * f2@2) (C -
Three Contact Points
Finally, as for a tripod, an object can contact the supporting plane at three noncollinear points (p,, p2 and p3). There are four cases when the points lie on regular surfaces or edges which are characterized by systems of nine equations in nine unknowns. The pose will be stable if the projection of the center of mass onto the supporting plane is within the triangle defined by the three points. When the contacts are regular surface points, the surface normals are orthogonal to the plane defined by pl, p2, and ~3, and the following system where i = 1,2, 3 defines a necessary condition for stability.
i
= 0
jXPi)
3.3.
Sitpi>
=
n&i)
* (p2
-
PII
ni(P;>
* @3
-
PI> = 0
0 =
0
(16)
= 0
g2@2)
, (c -
If tl is aligned with p2 - pI, then n = 0 and the last equation in (15) will be satisfied even when c is not above the line defined by pi and p2. For this case, the necessary condition is satisfied when n = t2(p2) x (~2 - p,) in the last equation. Each of the systems (13), (14), and (15) is composed of six equations and six unknowns which can be solved to find the coordinates of the two contact points. For each contact type, the test in (10) can be used to determine if the projection of the center of mass lies between the contact points. In addition, it must be confirmed that u” > 0 for motion corresponding to a sideways roll while two contact points are maintained.
pr)
O
When two regular surface points and an edge point are involved, the conditions for the regular points are the same as in (16). Additionally, the tangent to the edge point must be orthogonal to the surface normals, or with i= 1,2 f Si(pi)
=
0
fxp3)
=
0
&?3(P3) = 0 t
(17) ni(pi)
* (p2
-
pi>
=
0
ni(pil
’ (p3
-
PI>
=
0
,nhd
* f3@3)
=
0.
O =
0
x Cc -
~2) * n = 0.
The following system describes tripod stability with one regular surface point and two edge points where i = 2. 3:
114
DAVID J. KRIEGMAN Sl(Pl)
=
curve on a surface or at a point on an edge. After the candidate pose transformation is applied to the model, intersection with model edge i can be expressed as
0
A(Pi) = O
gi(Pi> = O nl(pd
* (~2
-
pd
=
0
nl(pJ
. (~3
-
pJ
=
0
ndpl)
’ ti(pi> = 0.
.0x, Y, z) = 0 z = 0.
Finally, the three contact points can lie on edges, in which case the curve tangents must be in the plane defined by these points and whose normal is n = (~2 - pt) x (p3 - p,). For i = 1, 2, 3, the defining system is
.fxpJ = 0 g&i)
To determine if the supporting plane intersects a surface in a curve that does not intersect an edge, it is sufficient to find a single point on that curve. For a bounded object, these curves will have at least two extremal points in any direction, and so arbitrarily choosing a direction, such a point is found by solving
(19)
= 0
SAX, Y, Z) = 0
i n ' ti(pJ = 0. 4.
IMPLEMENTATION
cm
gi(X, YT Z> = 0
z=o
AND RESULTS
Very few assumptions about the modeled surfaces were made in the previous section except that they are C2, defined implicitly, and intersect transversally. The implementation is more restrictive in that surfaces are represented by implicit algebraic (polynomial) equations; however, this class of objects subsumes most representations used in current CAD systems. It should be noted that rational parametric patch representations of surfaces can be implicitized using resultants [SAG84]. Alternatively, the conditions for stability can be reformulated in terms of the patch domain coordinates. To model surfaces by implicit algebraic equations, we have been construtting a geometric modeling system that uses a new curve tracing algorithm to find surface intersections [PK90, KP911. All of the necessary conditions for a stable pose described in Sections 2 and 3 are now expressed as systems of polynomial equations, and a large collection of symbolic and numerical techniques are available for manipulating and solving such systems. In particular, the generic form of homotopy continuation described by Morgan is used to find numerically all real and complex roots [Mor87]. Each real root corresponds to one, two, or three contact points, and along with the surface normals and edge tangents, they define a plane. The candidate pose simply corresponds to the transformation which aligns this plane with a plane orthogonal to the direction of gravity. Once the pose is determined, the sufficient local conditions for stability must be checked. Finally, the necessary conditions only relate the local geometry of the contact points, and so some distal part of the object might intersect the supporting plane. If so, the intersection of the object and the plane will occur in a
(21)
-$ Si(X, y, z) = 0.
The presented approach has been successfully applied to a number object models; Fig. 4 shows the stable poses of a PVC pipe fitting. The object is modeled as the union of five bounded right circular cylinders and a sphere.
FE .=*
FLPE
Liiz&!z
b.
C.
d.
I
I
I
FIG. 4. The stable poses of a PVC pipe fitting: Poses (a), (b), and(c) satisfy the conditions of one point contact even though contact actually occurs over the entire planar patch. (d) shows a pose with contact at three edge points as does (e) and (f). Note that contact occurs along a ruling which terminates at two edge points. (g) shows contact at one surface point and two edges.
STABLE POSES OF OBJECTS
b. 1
C. I
d. I
I
I
I
115
tions satisfied the necessary conditions for stability, U’ = 0, but did not satisfy the sufficient conditions. As illustrated in Fig. 5, these balance points include contact at one surface point, at edge and surface points, and at two edge points. Finally it should be noted that the algorithm, implemented in common lisp, required about two days to execute on a Sun SPARCstation 1 for this model. Nearly all of the time was spent solving the systems of polynomial equations describing the necessary conditions. Figure 6 shows the stable poses of another pipe fitting, while Fig. 7 shows the balance points. When the algorithm was applied to this model for the first time, the pose in Fig. 6(e) was surprising. Placing the actual fitting on a table confirmed that this precarious looking pose is actually stable. 5. SUMMARY AND CONCLUSIONS
In this paper, necessary and sufficient conditions were presented for the pose of three-dimensional piecewise e. f. g. smooth objects to be stable. In turn, these conditions FIG. 5. The balance poses of a PVC pipe fitting: (a) shows contact have been used in an implementation for objects whose at two edge points which correspond to a ruling of the cylinder. (b) and bounding surfaces are given by implicit algebraic equa(c) show contact at an edge and surface point. (d) dramatically shows tions. In the special case where surfaces are cylindrical, balancing on one surface point, (e), (f), and (g) correspond to contact at conical, or surfaces of revolution, computing stable two edge points. poses can be simplified to determining stable poses of the corresponding two-dimensional generating curves. As noted previously, the running time for computing Because contact with any point on a ruled surface such as a cylinder implies contact along an entire ruling which the results in Fig. 4 is quite long, and so it seems relevant always intersects an edge for a bounded surface, it is to explore the complexity of the algorithm. Most of the sufficient to only consider contact with the corresponding time is spent solving systems of polynomials using a nuedges. Additionally, the center of mass of the real pipe merical method, and one measure of complexity is the fitting was found experimentally and used in the compu- number of roots given by the total degree of the system (the product of the degrees of the individual equations). tation. In the process of finding the stable poses, seven solu-
FIG. 6. The stable poses of another PVC pipe fitting: Poses (a), (b) and (c) satisfy the conditions of one point contact, pose (e) has two points of contact, and poses (d) and (f) satisfy three point contact conditions though contact occurs along rulings of the cylinder.
FIG. 7. The balance poses of the other PVC pipe fitting: All four poses satisfy the necessary conditions of two point contact. Note that the pose in (c) (similarly (d)) is a local maximum of potential energy lying between the two minima shown in Fig. 6(e) and (f) for a rolling motion about the two edges.
116
DAVID J. KRIEGMAN
For an object whose stable poses can be computed without any vertices as in the pipe fitting, the number of roots is O((s + e)3d9), where s is the number of surfaces in the model, e is the number of edges, and d is the degree of the defining equations. This arises from considering triplets of edges and surfaces, and noting that the characterization of stable poses with three contact points has total degree O(d9). For example, the total degree of the system (16) describing support of three spheres is 512. The number of edges requiring consideration is given by the Euler-Poincare formula, and for an object of genus 0 without any vertices, there are two more edges than faces. Also, many objects, such as the pipe fittings, have planar faces and a mixture of curved surfaces of varying degree; this leads to faster running times than indicated by the highest degree surfaces. A number of improvements to the basic algorithm are possible. First, by precomputing a boundary representation (BREP) of the model, one only needs to consider the true number of edges rather than every pairing of participating surfaces. A priori, some surfaces and edges cannot contain contact points due to their local geometry. For example, surface patches that are globally hyperbolic as well as ones that are elliptic but concave cannot be participants. When the surfaces defining an edge form a concavity along the entire length of the edge, that edge does not need to be considered. Additionally, it may be unnecessary to consider all pairs or triplets of edges and surfaces as was done here. Instead, it might be possible to order the participants in a manner similar to convex hull algorithms. Such algorithms are available for planar objects, but not to the best of my knowledge for 3D objects [SW87]. In some cases, the symmetry of objects such as the pipe fitting can be utilized to simplify computation. Alternatively, it might be possible to reduce the total degree of the defining systems. In practice, there are many roots at infinity, indicating that the systems might be reducible. Additionally, since the Bezout number of these systems is lower than the total degree because of the roots at infinity, other homotopies that exploit their multihomogeneous structure could be utilized [LSY89, MS87]. Finally, since a given set of surfaces has very few contact points that satisfy the necessary conditions, it might be possible to parameterize the surface types, precompute the candidate stable poses for a set of generic surfaces, and then use coefficient-parameter polynomial continuation for a particular instance of the problem. This could greatly improve performance at the cost of increased computation for the generic system; however, this only needs to be solved once [LSY89, MS891. Finally, since the homotopy continuation method allows the roots to be computed independently, they can be found in parallel; a preliminary implementation, which is distributed over a network of SPARC stations, looks promising.
To conclude, let us consider how knowledge of the stable poses can be exploited in “CAD-based’ vision algorithms. Typically, when the pose of an object is computed from an image, all six parameters of its location must be found. However, if it is known that the object will be in a specific stable pose, it is only necessary to compute the three parameters of rigid body motion in a plane. Even if the specific stable pose is unclear, knowing that three of the parameters only take on certain values can be used to advantage. For example, when the parameterized implicit equation of image contours is fitted as in [KP90b], a set of initial conditions is required, and these could be the parameters of the stable poses. Alternatively, if all six parameters are fit, the stable poses can be used for confirmation. Finally, some algorithms for recognition and determining pose use viewer-centered representations such as aspect graphs. Under orthographic projection, the viewer’s location is modeled as lying on a sphere. However, when the pose is known, only a circle on the view sphere needs consideration. This could greatly reduce the number of views in an aspect graph, and simplify both their construction and use in interpreting images. For example, in the aspect graph of the vase shown in Fig. 3, there are 13 stable views separated by visual events lying on lines of latitude of the view sphere [EB89, KP90al. Since this solid of revolution has three stable poses, the simplified aspect graph for an overhead view only contains three views. When the camera axis is 30” from the vertical, there are only six views. Figure 8 shows the partitioning of the view sphere into regions where the aspect remains constrained with respect to changes of viewpoint as well as those viewpoints where the objects pose is stable. Even if it cannot be guaranteed that objects will be lying in stable poses, hypothesized poses (and consequently aspects) can be ordered to consider initially the more likely ones during recognition.
FIG. 8. The partitioning of the view sphere for the aspect graph of the vase in Fig. 3 with the allowable views for a camera situated 30” from vertical and the vase in its three stable poses as depicted in bold.
117
STABLE POSES OF OBJECTS
APPENDIX
In this appendix, it is shown that for stable poses with two points of contact, the center of mass must lie in the plane that includes these points and the normal to the supporting plane. This formed part of the necessary conditions of Section 3.2. Consider the case of stable support by two edge points. Without loss of generality, the edges are parameterized by arc length and given by x~(sJ and x&). The tangents are then t;(si) = x,‘(si). Contact with a plane implies that the tangents and the line between the two contact points are coplanar, or (x1 - x2) * (t, x f2) = 0.
-
t, x n2 = 0,
x f2 + kZ 2
x2> [klv
1
1
U = (c - x1) . n,
(24)
where n = tl x f2/(fl x f2(. Note that if the two tangents are parallel, n is undefined; in this case, the following line of reasoning will apply with n = tl X (XI - xz)//tl X (x1 x2)1. Differentiating U with respect to $1 to find the necessary condition for stability, we have
U = (c - xI(u, u)) . n(u, u).
U’ = (c - xl) * (n,,u’ + n,u’).
ACKNOWLEDGMENTS I thank Jean Ponce, Alfred Rizzi, and B. Vijayakumor for many useful discussions, and Michael Tarr for providing the PVC pipe fittings. This work was supported in part by the National Science Foundation under Grant IRI-901574.
REFERENCES [BG83]
where d = (~/c%I) (tl x f2) = klnl x f2 + k2fl x n2(ds2/dsl). The fact that c lies in the plane through the supporting points and is orthogonal to gravity can be expressed as c - XI = (Y(XZ- x1) + j3tl x f2, where (Y,p E R. From (29, we have tl X t2 . h/as, = 0, and along with (22) and (23), it is simple to show that dUlds, = 0. Now, both cases of contact at two points, where -one lies on a regular surface, can be considered simultaneously. By the implicit function theorem, the surface can be locally parameterized in the form xI(u, u), and the relationship of the two contact points in (13) and (14) is (~164
4
-
~2)
* 44
u)
=
0,
(2’3
(29)
To show that the center of mass lies in the plane defined by the contact points and is orthogonal to the support plane, let c - x1 = a(x2 - x1) + /3n, where (Y,/3 E R. Note that n, and n, must lie in the tangent plane [dC76], and so n * (n,u’ + n,u’) = 0. Using (27), it is clear that U’ = 0. Therefore, the center of mass must lie in the plane defined by the contact points and the normal to the supporting plane.
(25)
djtl x t212- [d * tl x f2]fI x f2 3 16 x f213
(28)
Now, the necessary condition is that U be a local minimum for changes in s. Differentiating U with respect to s yields
= (c - Xl) ’ E = (c - x,) *
(27)
where n, means an/au. Note that this expression holds whether x2 is a surface or edge point. The potential energy is given by
(23)
where kl and k2 are the curvatures at the edge points. The object’s potential energy is
z
(xI(u, u) - x2) * (n,u’ + n”u’) = 0,
(22)
This relates s1 to s2, and the partial derivatives are used to find &sz/&, , namely
(XI
where n(u, u) is the surface normal. The object is free to rock while maintaining two points of contact, and this corresponds to a curve of contact points given implicitly by (26) and nl x n2 = 0 for a second surface point. This curve can be represented in the domain of the surface as [u, u] = [u(s), u(s)]. Differentiating (26) with respect to s gives
J. W. Bruce and P. J. Giblin, Generic geometry, Amer. Math. Monthly, October 1983, 529-545. [Bro81] R. A. Brooks, Symbolic reasoning among 3-D models and 2-D images, ArtiJicial Intelligence, 17, 1981, 285-348. [Cha82] Indranil Chakravarty, The Use of Characteristic Views as a Basis for Recognition of Three-Dimensional Objects. Image Processing Laboratory IPL-TR-034, Rensselaer Polytechnic Institute, October 1982. [CLC84] D. M. Chelberg, H. S. Lim, and C. K. Cowan, Acronym model based vision in the intelligent task automation project, Proceedings, Image Understanding Workshop, October 1984, pp. 308-315. [dC76] M. P. do Carmo, Differential Geometry of Curves and Surfaces, Prentice-Hall, Englewood Cliffs, NJ, 1976. [DS90] D. P. Dobkin and D. L. Souvaine, Computational geometry in a curved world, Algorithmica, 5(3), 1990, 421-457. [EB89] D. Eggert and K. Bowyer, Computing the orthographic projection aspect graph of solids of revolution, in IEEE Workshop on Interpretation of 30 Scenes, Austin, TX, November 1989, pp. 102-108.
118 [Gri89]
DAVID J. KRIEGMAN W. E. L. Grimson, On the recognition of curved objects, IEEE Trans. Pattern
[Hof89] [Hors61 [Koe90] [KP90a]
[KP90b]
[Kill
[KV79]
[LSY89j
Anal. Mach. Intelligence,
11(6), 1989,
632-643. Christoph M. Hoffman, Geometric and So/id Modeling, Morgan Kaufmann, San Mateo, CA, 1989. B. K. P. Horn, Computer Vision, MIT Press, Cambridge, MA, 1986. Jan J. Koenderink, Solid Shape, MIT Press, Cambridge, MA, 1990. D. J. Kriegman and J. Ponce, Computing exact aspect graphs of curved objects: Solids of revolution, Int. J. Comput. Vision 5(2), 1990, 119-135. D. J. Kriegman and J. Ponce, On recognizing and positioning curved 3D objects from image contours, IEEE Trans. Pattern Anal. Mach. Intelligence 12(12), 1990, 1127-1137. D. J. Kriegman and J. Ponce, A new curve tracing algorithm and some application, in Curves and Surfaces, (P. J. Laurent, A. Le Mehaute, and L. L. Schumaker, Eds.), Academic Press, San Diego, 1991, pp. 267-270. J. J. Koenderink and A. J. Van Doorn, The internal representation of solid shape with respect to vision, Biol. Cybernet. 32, 1979, 211-216. T. Y. Li, T. Sauer, and J. A. Yorke, The cheaters homotopy: An efficient procedure for solving systems of polynomial equations, SIAM J. Numer. Anal. 26(5), 1989, 1241-1251.
[Mor87] Alexander Morgan, Solving Polynomial tinuation for Engineering
Systems Using Conand ScientiJic Problems, Prentice-
Hall, Englewood Cliffs, NJ, 1987. [MS871 Alexander Morgan and Andrew Sommese, A homotopy for solving genera1 polynomial systems that respects m-homogeneous structures, Appl. Math. Comput. 24, 1987, 101-113. [MS891 Alexander Morgan and Andrew Sommese, Coefficientparameter polynomial continuation, Appl. Math. Comput., 29, 1989, 123-160. [PK90] J. Ponce and D.‘ J. Kriegman, Computing Exact Aspect Graphs of Curved Objects: Parametric Patches, Technical Report UIUCDCS-R-90-1579, University of Illinois at Urbana-champaign, April 1990. [PSSSI F. P. Preparata and M. I. Shamos, Computational Geometry: Springer-Verlag, New York, 1985. An Introduction, [SAG841 Thomas W. Sederberg, D. C. Anderson, and Ronald N. Goldman, Implicit representation of parametric curves and surfaces, Comput. Vision Graphics Image Process. 28, 1984,7284. [SS87] J. T. Schwartz and M. Sharir, Identification of partially obscured objects in two and three dimensions by matching noisy characteristic curves, Int. J. Robot. Res. 6(2), 1987, 29-44. [SW871 A. A. Schaffer and C. J. Van Wyk, Convex hulls of piecewise-smooth jordan curves, J. Algorithms 8, 1987, 66-94.