Calibration of camera parameters using vanishing points

Calibration of Camera Parameters Using Vanishing Points by GUNA SEETHARAM .AN, HONG RAO and GURUPRASAU SHIVARAM Center for Advanced Computer Studies University Lafayette, LA 70504-44330, U .S.A

of Southwestern

Louisiana,

A new technique for calibrating the intrinsic and extrinsic parameters of 'a video imaging system, hosed on Banishing points is presented. The intrinsic parameters of the camera refers to the (beat length of the lens, the physical dimensions of each pixel and the exact position of the optical center on the image grid, as well cis the radial distortion of the lens . The extrinsic parameters refer to the position and orientation of the camera described in a predefined frame of reference called the world coordinate .system . The proposed method requires only one view of a specially designed test object, or two distinct views of a solid cuhe . Other objects rich in parallel lines can he used as well . The two rieus must he generated by subjecting the object to an arbitrary composite three-dimensional (3D) displacement within the field of view of the camera . It is not required to know the exact 3D motion parameters ; however, it is necessary to know the initial position and orientation of the cube with respect to the world coordinate system . An intrinsic parameter that is often not addressed is the radial distortion factor. It remains fixed for each lens and thus has been considered a part of the lens specification . The camera model used in this paper incorporates the radial distortion factor ; however, the basic equations become highly nonlinear (eighth order) . Two cameras connected to the same digitizer hardware have been calibrated to experimentally verify and illustrate the feasibility of the proposed neiv technique . ABSTRACT :

L. Introduction

The principal objective of many three-dimensional (3D) computer vision tasks is to recognize and locate (1) 3D objects based on accurate measurements of graylevel images of these objects . The images are generally derived by perspectively projecting the objects onto a 2D image plane . The effective focal length, radial distortion (introduced by the lens) and the physical dimensions of the pixels (after sampling) influence the resulting 2D gray-level images recorded by the camera and digitizer . To precisely locate and manipulate the 3D objects kept in space it is necessary to resolve the camera-based 2D measurements (and the inferred 3D scene features) in a predefined 3D frame of reference known as world coordinate system . It is necessary to know the exact value of the focal length and the pixel dimensions, in order to analyse perspective images . It is required to first resolve the information observed from the digitized image into the actual image plane before one could apply various 3D vision algorithms . The inferred 3D scene features can then be expressed in the world coordinate system uniquely based on the extrinsic parameters . 001"033(95)01

Pergamon

7

555

G . Seetharaman et al .

Practical, off-the-shelf cameras record images by perspectively projecting the scene features onto the image plane . A specific and finite area of the image plane, called the active region, is responsible for converting the optical image into an electrical signal . The active region is made up of a discrete array of CCD or CID cells, or a continuous coating of photosensitive materials . In all cases, it is typically 8 .8mm x 6 .6mm . The removable and replaceable nature of the lens permits the images to vary from a strongly perspective to nearly orthographic nature . A lens whose focal length is much smaller than or equal to the effective diagonal of the image sensor gives a strongly perspective image . Such a lens is called a wide angle lens . In contrast, a lens whose focal length is more than five times the diagonal of the sensor tends to uniformly distribute the nonlinearities associated with perspective projection throughout the image sensor, thus giving an orthographic image model . Such a lens is called a telephoto lens. Even when the lens has a fixed focal length, known a priori, the effective focal length may differ from the actual focal length due to mechanical mounting factors . The point at which the optical axis of the lens intersects with the image sensor (a planar patch) is called the optical center_ The exact position of the optical center varies due to misalignments as the lens is changed . For example, the size of each pixel in a typical CCD imaging system is 11 pm, while the physical dimensions of the lens, and mounting hardware is of the order of millimeters . Variations of the lens-axis with respect to lens-mount is of the order of 100 µm, causing a variation of 10 20 pixels in the exact position of the optical center in the measured image . In the case of analog video cameras, various forms of timing inaccuracy may occur in the scanning process which in turn affect the digitization of the resulting continuous 1 D signal . The variations of the sampling instances with respect to the most recent HSvnc (horizontal sync) signal contributes to this problem . Many 3D vision algorithms, such as shape from shading (2) and geometrical reasoning (3) involve nonlinear terms and require the exact location of the optical center . The resolution of digitized images produced by generic digitizers is usually 512 x 480, 512 x 512 or 768 x 512 pixels . The effective area of the image sensors generally conforms to a fixed aspect ratio of 4 : 3 . Thus, given a digitized image, it is not necessary that its pixels are always a square in the physical sense. As a result, the geometric interpretation of the resulting image could significantly differ from that of the ideal (continuous . undigitized) image, if the camera is rotated along its optical axis . For example, a circular disk kept parallel to the image plane might get imaged as an elliptical disk if the .x and y sampling intervals are unequal . It is necessary to know the aspect ratio of each pixel to successfully apply many 3D vision algorithms [such as geometrical reasoning systems (3, 4) and 3D motion analysis of monocular image sequences (5, 6)J . 1 .1 . Problem statement

Effectively, the imaging of a real world scene instance .?,(X) into a 2D intensity image f(x) involves a four-phase geometric transformation of a point (X,,., Y,,., Z„) from the 3D world coordinate system to a 2D point (xG, ye) in the grid of discretely sampled image points . It is fully described by a nonlinear, non-invertiblc transformation M of the form 556

lmrnal of the Franklin Institute Fisa~fw Suenw Ltd



Calibration of Camera Parameters Using Vanishing Points M(T,R,s,,s,K1)

1(X)-'

.f(X)

where, T . R, s x, s,., r„ r„ fc , K are the parameters that instantiate the imaging system . These parameters are defined and explained in the next section . Given one or more scene instances .V;(X), i >, 1, each sufficiently defined in its geometric details, and their corresponding images f(x), the objective of the calibration system is to recover the parameters, T, R, s„ S, rx , r„fC, K . The purpose of such an effort is to, in some way, be able to predict an arbitrary scene instance, J'(X) based on its image f(x), using various computer vision techniques discussed in the literature . II. Physical and Geometrical Models of Video Imaging Systems 2 .1 . Projective imaging in pinhole and practical cameras Video cameras project a certain object point X e t8' located on opaque objects onto an image point x e S c IR'-, where S, called the sensor active region, is a small finite area in the image plane. Also, the image plane is uniquely determined by the focal length f > 0 of the camera such that Z = -f. . Figure 1 illustrates the perspective projection inherent in pinhole cameras . A column vector, X`e , would he used to describe the position of a certain point P expressed in a specific coordinate system C . The superscript C would be dropped, whenever the underlying coordinate system is apparent . The imaging process illustrated in Fig . I is expressed as follows

FIG . 1 . A pinhole camera passes light through its aperture A, and images P' onto P. The

non-specular light emanating from a small patch dS' at dS at P . Vo1 . i71B, Vn .S .pp .55-SRS,1994 Printed in Great Britain.

P' is channeled to an area-element

557



G . Seetharaman et al .

= [P] -f where

Z

P=

1f

and

Z > 0.

(1 b)

0 0 Equation (1) is non-invertible in that given X one can determine x but not the reverse . Also, given a point x on the intensity image . X is constrained to a line (of points) passing through the image point x` and the center of projection 0 _ (0, 0, 0) . The nature of projection is independent of the depth P_ of the object . Let D be the diameter of the pupil (pinhole) of the camera . Let dA be an areaclement located in that pinhole . The irradiance received by dA from a lambertian surface dS' located on the object, and then delivered to the image plane is LdA . Its impact on an area dS on the sensor is d 2 0 = L dA cos 0 dS) where dQ =

dScos0 f//cos 0) 2

Thus, d 2 O = LdA dScos' 0 . If the aperture A is of finite diameter D, the net irradiance at dS is obtained by integrating the above expression over the entire aperture (pupil) area . Thus, 1r D2 cos " B d t/5 = L - dS . -4 f7

(2)

Therefore, the net intensity (energy/area) at point P is E=

rzL

I -- cos 0 . 4 (1 ; D)'

(3)

The term (fD) is called the effective F number of the camera . The smaller the number, the larger the diameter D, and hence the higher the E . Thus, by adjusting the size of the pinhole and/or adjusting the focal length, one could control the net amplification of the observed image . From the linear-systems point of view the

558

Journal of the Franklin Institute E[.sevier science Lot



Calibration of Camera Parameters Using Vanishing Points pinhole camera offers weak signals since only a small fraction (say I/n) of the nonspecular radiation from dS' effectively reaches the destination dS . 2 .2 . Effect of the lens on the image properties An optical lens in a camera is used to increase the net intensity of the image manifold, by effectively increasing the irradiance of dS due to a single lambertian source dS' . It becomes possible since the cross section of a lens is larger than that of a pinhole. Also, the amplification is achieved by capturing more than one ray of light originating from the point P' and focusing them all at a single point P in the image plane . The principles and operation of lenses are found in any standard book on optics (7, 8, 9) . The basic equation of the imaging through a bifocal lens can be written as 1/v+I/u=(n-1)(1/R,-l/R,.)

(4)

where u and v represent the radial distance of the point P' and P respectively from the center of projection, also known as the optical center of the lens . This equation is known as the thin-lens equation, and can be rearranged to resemble that of the pinhole camera model . In perspective projection, the thin-lens equation indicates that the optical image of a 3D object is still in 3D . The image point P and object point P' satisfy the thinlens equation, such that I

1

l

VP,+P, +P

v/Py+P,'+P`

Consider a circle of radius r, on the image plane Z = v, whose center is located at the point where the image plane intersects with the optical axis of the lens . For each point on this circle, there exists one point on the object that satisfies equation (I), and the loci of these points constitutes a circle, whose orientation (major cross section) is parallel to the original circle . Its location is essentially characterized by only one value, Z = a where 1 Jv'""+r,'

1

1

, u'+r11

.f

Both the size r„ and the exact distance u of this (object) circle vary with the radius of the image circle on which P is located . Even though the image is recorded on a small planar area S . the set of object points that contribute to the net irradiance at S is in fact a volume . The consequence of this approximation is known as the depth-of-field effect . However, for v - f. and for a very small r,., the depth-of-field becomes very high, and thus, the basic model of pinhole camera is satisfied . To simplify the algebraic models, the images will be analyzed based on the assumption that they have been measured at the Z = + f,, plane, i .e . the image plane is in the object side of the lens . The basic imaging equation is suitably rewritten as Vo1 3319, No . 5, pp 535-585 . 1994 Printed in Great Britain .

559



G. Seetharaman

ct al . X

and

Z

Y

/ =

(5)

Z,

and it is clear that the projection is perspective . In linear projection, the basic equations are similar to that of the perspective imaging model . However, the difference can he emphasized if the object points are described with respect to an object centered coordinate system . Let X` = (X, Y, Z)°+ (0, 0, Z d )`, where Z~, is the distance of the object from the camera. As Zd and J 'simultaneously tend to infinity the projection reduces to an affine form : X" = (l + [1']

Z7

Y [0

Y` 01

(6)

0] Z"+Z„

It was assumed that the origin of the object centered coordinate system was located on the optical axis . The method may not work for larger values of X° and Y° . The rest of this section presents the definition of distortion parameters . Figures 1 and 2 illustrate the basic model of a video image sensor . 2 .3 . Radial distortion in the observed image The lens equation used to describe the imaging process is exact for a and v being measured along the optical axis . However, in imaging for a point of height h at a

MMM ----s. MMMM iii .' I

Optical axis

. Opticol Center

Lx

,, I Grid axis

I

Lx Xw World coordinate system / Zw

Fir . 2 . A simple perspective imaging system illustrating the intrinsic parameters . 560

Journalofthe Franklin Inalaua Elxrrer Science Ltd



Calibration

of Camera

Parameters Using Vanishing Points

distance 7, one must measure the lengths u and v along the line joining P and P' in Fig . 12 (see Appendix) . It is apparent that u' _ 11/h 2+Z2 > u and thus e' < v . Various rays that emanate from P' will converge prematurely before they strike the focal plane Z= -f. The error introduced by this will be proportional to the magnitude of h, and is circularly symmetric in the image plane . The observed location (x,,y,,, f) of a point and its ideal location (x,,,y u ,f) arc related (10, 11) by sd=x„(1-K,r;;+x,r,',+ . . .), 1'a=vJl-x,r +x, r4+ . . . ), ru = v/x"„+Y ;

(7)

where n, and K, are known as the radial distortion coefficients . A detailed derivation (8) is given in the Appendix . Given x,,,y, and the distortion parameters, x,, x„ Eq . (7) compactly describes However, most useful computer vision algorithms need the computation of the transformation in the opposite direction, i .e . computation of x,,, v o from the observation points x,, yd of the distorted image . The nonlinear nature of Eq . (7) does not readily lend itself to this requirement . Based on first principles, it can be shown (see Appendix) that

. . .) 1' . .=ye(l+~~,rn-r<'I + (8) 2 .3 .1 . Parameters introduced by digitization . The distorted image formed by the lens on the sensor-region S is a positive valued continuous function . The distorted image must he discretized and quantized to facilitate intensity based image processing . The sampled and quantized image is generally described over a 2D rectangular grid G, whose origin may not coincide with the true optical center, (0, 0, f,)` . The grid and image coordinate system are illustrated in Fig . 2 . Their relationship is expressed as xd = V, =

.X

G S,

+

rs ,

+

(9)

which can be rewritten in the form x r; = s . 'z,,+C„ (10) The parameters s„s„r, and r, are the intrinsic parameters of the video sensing system that arise due to digitization . The terms s,, .s,: represent the sampling intervals in x, y directions of the rectangular grid respectively . From Eq . (10), the location of the true image center (0, 0, f)` measured in the grid (pixel) coordinate system is given by Vol . 331R, No . 5, pp . 555 5851994 1994 Pdnt 1n Go-t Brink,

561



G . Seetharanian et al . (Cs . Cr•) _ -(s,. t,., sy ' T,.) .

A commonly repeated mistake is to assume that the uniform sampling intervals s, and s, are equal . This is not the case for many practical image acquisition systems . Also, L, and L,, represent the physical dimensions of the actual image sensor, If the grid consists of N,x N, pixels then L,-= N's, and L, . = N,S v respectively . The aspect ratio of the image sensor is LIL,, . It is important to extract s„ s,,, a, ., and s, in order to transform quantities from observation in the x cG grid into the x` plane . Then one can show that X Y

(s.,-xG+i.r) (s,.Y G +r,) ',

=~Tc)(Z(xe,Yn,K)]

Z

(LIa)

t where

2 (v,Ye, K) =

;,D(r,,, x)

0

0 0

0

AD(r,, K)

0 0

0

0

i.

0

0

0 1

r = x/(SzxG+rr) +(S,YG+i, .)2 .

0

(Ilb)

and D(r,, K)

(I +K,r3+KZr,,+ -) .

=

(11c)

2 .4. Extrinsic parameters The 3D transformation from 3D world coordinate to 3D camera coordinate follows . Given the absolute position X' of a point X, measured with respect to the world coordinate system . its position X` with respect to the camera coordinate system C is expressed in the form X w Y - [TC Z I

X~ ° Y (12a) Z

where

Y TC _

z

(12b)

1 562

Journai of the Franklin institute Flsevier Science Ltd

Calibration of Camera Parameters Using Vanishing Points The terms, a, /t and y represent the direction cosines of the X . Y and Z axes of the camera, and O.' the vector position of the origin of the camera coordinate system, all expressed in the world coordinate system . The matrix Tc is uniquely characterized by six parameters called extrinsic camera parameters . These parameters are uniquely determined by the position O c and the orientation parameters a, /i and y of the camera with respect to the world coordinate system . III. Previous Work There are many techniques for calibrating the extrinsic parameters . Tsai (10) presents a good survey of literature and a comparative assessment of these techniques . These classical techniques do not involve image refinement parameters such as the aspect ratio, the radial distortion factors K and the image center . The classical techniques arc broadly classified into three categories . The first group of methods (12,13) involve very elaborate modeling and a fuliscale nonlinear optimization . Faig's method (12) for example, solves for upto seventeen unknowns . A good initial estimate is necessary to initiate the iterative nonlinear search . The second class of methods compute the perspective transformation parameters first, based on few selected control points (or features) and then extract the extrinsic parameters . The direct linear transform technique due to Abdel-Aziz (14) and Ganapathy (15) falls in this category . In most cases, the solution is obtained by solving a number of linear equations . Also, the number of unknowns in these models is usually more than the total degree of freedom of perspective transformation . Nonlinear optimization is required in some instances . The third group is known as the geometric modeling group . Fischler et al. (16) derive a set of equations constraining the camera location and object orientation . The method does not require nonlinear search . The method is very robust and is expandable . We extend this approach for extracting the extrinsic (translation) parameters . The calibration of intrinsic parameters is relatively new (10, 17) . It has been a common practice to take the center of the digitized image (generally of the framebuffer) as the true image center ; and assume that the sampling interval is the same in both directions, i .e . s, = s,. = 1 . Recent advances in 3D interpretation of 2D images (3, 4, 6, 18 20) indicate that it is very important to know the true position of the optical center . Experimental results (17) indicate that the center may be off by as much as 20 pixels from the center of the frame buffer . Even when the perspectiveness is less pronounced- the 3D interpretation of orthographic images (21) produce the wrong interpretation if the actual shape of each pixel is not exactly square . In principle, this can be compensated only if we know the ratio of the sampling interval used in x and y directions . Although many 3D vision techniques model the imaging with unit focal length, it becomes necessary to know the focal length in terms of pixel dimensions (and/or the inverse) to perform model based interpretation of perspective images (3, 4) . Tsai (10) proposed a model which takes into account the scale factors (in x direction s,.) and radial distortions in addition to the six extrinsic parameters . Various techniques were proposed to compute these parameters, at varying levels of complexity . Lenz and Tsai (17) refined the original model (10) in order to Vol .331B . No . .pp . W-595, 19194 Prima G.wl Britain .

m

563



G . Seetharaman et al . account for the variation in the exact location of the optical center within the digitized image_ Specifically, they proposed three techniques to estimate the image center . First, the direct optical method involves aligning a laser beam (using four degrees of freedom) in front of the camera in a specific way . One must identify an optimum alignment (utilizing the optical properties of the lens surface) where the beam corresponds to the optical axis . Then the image of that beam will correspond to the true optical center . The method is precise, but may potentially damage the image orthicon type of cameras . In their second method . a planar grid of points is imaged by the camera . As the lens is zoomed (by varying its focal length), the image of the grid points follow a set of epipolar lines, and these lines are concurrent at the optical center . In particular, this method is not sensitive to the radial distortion . The third method utilizes a radial alignment constraint, introduced in (10), and employs model fitting strategies to conduct a nonlinear optimization for two variables . The three methods are accurate and reliably reproducible . Penna (22) presented a technique to quickly compute the scale factor s, first, and formulated a nonlinear equation that could be solved for radial distortion and the image center . In particular, the radial distortion is easily measured by using the occluding boundary of a sphere . The method is nonlinear and produces acceptable results .

IV. Calibration Using Vanishing Points The method proposed in this paper uses simple industrial objects, such as a solid cube, for calibrating both the extrinsic and intrinsic parameters . The method makes use of certain geometrical properties of the vanishing points produced by each group of parallel lines of the solid cube . A recent paper by Kanatani (23) explains the error analysis in computing the vanishing points and suggests the advantages of using vanishing points for calibration purpose . 4 .1 . Basic equations Let L be a line passing through two points, P and Q. Also, let p and q be the images of P and Q respectively . Unless it is stated explicitly by using a superscript, it would be assumed that X's describe vectors in the world coordinate system, and x's express vectors in the camera coordinate system (in particular, on the image plane) . Thus, L : X(t)=P+iU

(13)

where tel8 and U

(14) Q-PI

is the direction cosine of that line . Any arbitrary point on that line is uniquely characterized by the value of the parameter t . Then, its image in the image plane may be described as 564

Jmn~xi of,hc rrenkii" lnslillnc eisevler science Ltd



Calibration of Camera Parameters Using Vanishing Points Px +tU x P 1 +tU Y ^_ [R : [R]O, ] Pz +tUz

where

where the relation °= represents that each component of one vector is scaled by a fixed unknown scale-factor to form the respective component of the other vector . We arc particularly interested in the image x(l) of the point X(t) as t approaches oo . Let the limit converge to a point (t},., v„ z = ./') . Then it can he seen that r

x(t)~,

a,

Ux

=[R]

(T,,

(16)

-U,J

f-

The interpretation of Eq . (16) is as follows . Given two or more lines that are parallel in the world coordinate system, all with direction cosines U, their respective images would all be concurrent at point (r .r, v„ z = f) in the image plane . The point v is uniquely determined by the direction cosines of that line . Furthermore, v,-=f

2TU

and

v,

Y U

Consider the triplets U' =(a1(1,

$'U, Y TU)

=f fl T U

(17)

Y L

and v =(r„ r,., f ) . It can be shown that (18)

vX Ur =a .

Therefore, the vectors v and U' are parallel and would assume the same direction cosines . Thus, given a set of parallel lines. the direction cosines of these lines arc uniquely determined by their vanishing points . The converse is also true . Then, ii is the direction cosine of the vanishing point v, and is expressed as Ux = [R] Liz

At this point, Eq (19) is exact . 4 .2 . Calibration of extrinsic parameters

Now, let e,, e, and U, be the direction cosines of three distinctly different sets of parallel lines, whose vanishing points v„ v, and v 3 respectively are to be extracted from the image plane . Then Vol 331B, No . 5, pp 353553 . 1994 . Great Brim ; . Primed,

565



G . Seetharatnan et al .

UIA _ [R]

UIY U17

to 2r U2Y

U3 .r

(20)

U3Y

U ,

U37

in which the U's arc known and Fs are observables . It is therefore possible to compute

[R] -

U Ir

Ux

U;x -

U1

Y

UY

U3Y

U I7

U 27

U37

t('2x

(21)

Thus, in principle, one could estimate the camera orientation (extrinsic) parameters (a, Q, y) . The inversion is guaranteed if the matrix U I ,, = [UI , U2 , U,] is a wellconditioned matrix . In the case of a solid cube, the three sets of parallel lines are in mutually orthogonal directions . As a consequence, 11, is always a wellconditioned matrix . Let P and Q he two points located on the object whose coordinates arc P' and Q` respectively . Also, let p and q he the coordinates of the images P and Q respectively . Then 2,,p = P`

and

(22)

1.,q = Q' .

From the geometrical illustration in Fig . 3, it can he shown that

Pi

(4X ( p (Q`- P`) r ' X 4)) (q-p)

r'

(23)

(4 x (!3 x 4))

IPQIv7 •e p- q I ' e

FIG . 3 . A specially designed calibration test cube . The object contains nine distinct sets of

parallel lines visible from any view .

3o~~nai

566

onne Ftit,I h, Intel Cure Science Ltd



Calibration of Camera Parameters Using Vanishing Points where P,y is the normalized vanishing point of the set of parallel lines consisting of PQ . The vectors p and q are directly observed, while 8, q and e are easily extracted from the image . In addition, the physical length of the line PQ is assumed to be known as a ground truth . Once Po is given, then P_, . and P y, are computed by the expression

Pz

P,

= f p,

and

Pr

=

Pz

(

7p,

24

)

The units in which P' is expressed deserves further discussion . A careful examination of Eq . (23) indicates that I PQ is typically available in millimeters, while ~P - ql is available in pixel units . In order to relate these quantities, and express P` in conventional units, say millimeters, it is required that we know f and pixelunits in millimeters, also . Thus, the calibration of intrinsic parameters becomes inevitable . 4 .3 . Calibration of intrinsic parameters Case I : Negligible amount of radial distortion . The objective is to calibrate for the intrinsic parameters, C„ C, and f. The method described in this paper is applicable for lenses with very low radial distortion_ Equations (7) and (8) can be rewritten in the form S, X, +T , SvJ4+tc Xd=x„= /C

I

06, .

A

t,

fl,

xz

-a' 1 0,

f_

_#Toe

X„, Y„ .

- Y Oc

(25)

Z,,. I

The radial distortion factor remains fixed for each lens, and is thus considered a part of the lens specs . The method to be described could easily be extended to incorporate the radial distortion factor also ; however, the equations become highly nonlinear (eighth order) . Equation (19) is rewritten as I V, N/ /

(s,v9+t)'+(s, .ry .+T' .)-+f-

(Sr,,+T,)

Ux

(s,vgy+T,) f

_ ER) Ur

Uz

(26)

in which (v, r q,-) are the only observables . In particular, these are the vanishing points as observed from the sampled and digitized image grid . Let i, , and b, be the vanishing points of two sets of parallel lines whose direction cosines in the world coordinates are U, and U. respectively . Then Vn1 .331B . No .5,pp.555-585,1994 Printed in Grrr, Britain

567



G . Seetharaman

et al . vi8, = U, U s .

In particular, if

UfU,

(27)

= 0, from Eqs (26) and (27) it follows that

(+.,vi v.+T .x)(sxUZ V +i ..)+(srci ,+T,) (s+T .)+f =0,

(28)

in which (v" ' , v,,,,) and (r 2 ,,,, v 2y ,.) are the only observables . There are five unknowns in Eq . (28) . It appears that at least five observations may be necessary to solve for s„ s,., r r, r, and f uniquely . However . only four are sufficient since the set essentially forms a system of homogeneous nonlinear equations . From any given view of the solid cube, it is possible to extract three vanishing points, and form three instances of Eq . (28) as (s', r i gx +T

,) (s~, L 29, +TJ + (svv, ,, +T,) (s, . r2J, . +T v) +f 2 = 0:

.r3" . +T,) +J 2 - 0 . (s .v 1%211 +T .,-) (7,,13,, v + T x) + (s,.r 2 , +T, .) (s,

(s, v,:,,-+ t J (sx-v, v.,x

+Tx) + ( s, v3,, + T v ) (S r i i qv

+ T,)

+f 2 =

0.

By setting s, = 1, these equations are further simplified into two quadratic equations having three unknowns and of the form T,(n1,-v3j,,)+s,T,(v,g,-r,_)+s2r2 (r,, .-03p .)= T.r(T„„--vro .,)+s .T, .(v,,,,-11 .,)+s ;vs .„('v",-III")=1% ;,v(r'iq,-cm,) .

(29)

In particular, if s, = s, . = 1, then these equations can be solved uniquely fore,, and z, . which could then be substituted in Eq . (29) to solve for f. A geometric interpretation of the solution is readily available in that the optical center coincides with the orthocenter of the triangle formed by the three vanishing points . If s . # s, then the observed triangle is an affine transformed version of original shapes . Consequently, the geometric method of locating the optical center using the orthocenter is not applicable . That is, Eq . (29) must be solved algebraically for three variables . Each image of a simple cube permits only three equations in six unknowns . At least two image frames are necessary to form six equations in six unknowns . The inaccuracies in estimating the vanishing points may significantly affect the exact result obtained in this process . Therefore it is desirable to take more than two image frames (or six equations) for this purpose . When, s,. Eq . (29) can be reduced into a single quadratic equation in two unknowns, by eliminating the s,,z, terms . Consider a number of images of the solid cube recorded by subjecting the cube to arbitrary rotation, while keeping the camera stationary and the lens settings unchanged . Each image contributes one such quadratic equation . These equations are then solved in a least squared sense to result in the optimal solution . A simple solid cube has been designed to provide upto nine distinct vanishing points from a single view . In particular, they form 12 distinct pairs of orthogonally related vanishing points . It is possible to use this cube to calibrate the camera fully . Case 2 : Radial distortion is moderate, When the lens exhibits a moderate amount 568

Journal ui the Prunklin Institute Eisevmr Science Ltd

Calibration of Camera Parameters Using Vanishing Points of distortion, one could still use this method . However, the basic equations become a set of eighth order nonlinear equations . For example . (1+K(UI gr s,+r,)

+li(tig

5 ,, + Tr)~)((t'lyss+rs),(rlgvsl'+rr))

(V2pssx+Z.r)

(l+K(v2os,+r,) +K(v2.vs,+ ;)')+f2=0 . (vig pa,.+r n) Case 3 : Radial distortion is significant . When the radial distortion is s ignificant. i t is still possible to characterize a set of parallel lines by a point

I+Kf2(

I' 1-))

~t',.<,l-'

where the distorted images of these lines concur. These points cannot be extracted directly from the image . Thus, the vanishing point method becomes unsuitable for lenses with large radial distortions .

V. Experimental Methods for Validating the Results 5 .1 . Verification of pixel scalefactor using a telephoto lens It is known that the intrinsic parameters K and f depend on the lens only ; also, s, and s,. depend on the sampling process and hence depend only on the digitizer . However, both r, and r, depend upon the lens (due to mounting factors) and on the digitizer, due to jitters in the sampling clock timings_ A simple method is introduced to extract the scale factors first, to enhance the practicality of our vanishing point based approach . The following technique is built on the basic idea due to Pcnna (22) . The advantage of using a spherical object, as suggested in (22) could be fully exploited if the underlying projection is orthographic . In contrast with the perspective projection, an orthographic projection will map a sphere in 3D onto a circle in the 2D image plane even when the spherical object is not placed exactly on the optical axis . Two factors influence our ability to acquire orthographic images . First, the size of the object must be comparable to that of the image sensing area, and/or second, the focal length of the lens must be much larger than the diameter of the sensing zone . For example, in an 11 mm camera, (8 .8 x 6 .6 mm'), a zoom lens of 120 mm will act as an orthographic lens . Consider Fig . 4 . The image is first segmented, either by using a simple Solid operator or by thresholding the image . The thresholding method is preferred to the Sobel technique because the Sobel method, by design, assumes square pixels, which could he a potential source of error . The equation of the observed circle will be (u,,-g)z+(um-h)2 =c 1 , Vol .331D, No.3, pp. 555 05. 1994 Printed in Great BeStxm

569

G. Seetharannan et al .

HG . 4 . The image of a simple sphere . regardless of the viewing direction must characterize a circle . In this case it appears as an ellipse due to uneven sampling in x and p direction . Note that f is set to a very large value to ensure linear projection .

570

Journal or the Franklin lnssitule hisorler 4cienu' J Id



Calibration

of Camera

Parameters Using Vanishing Points

IV [`11 J-

where (x„ y), i = 1, 2, . . . , are the observables . A curve fitting approach suggested by Penna could be adopted . An alternate approach has been used to robustly compute the aspect ratio . First, the image of the sphere, henceforth referred to as an ellipse, is obtained by a simple thresholding operation, followed by a contour tracing operation . Let X = f(x;, v;)} be the list of points on that contour . The centroid (P„ hy ) of this ellipse is obtained by computing its first moments . Also . the scatter values r,-' and UJ2 are computed such that

F,K-102

E,(y,_nL)'

and

N

(30a)

These variances indicate the strength of the radius on the sampled grid . Also, in the unsampled image they represent the square of the radius of the underlying circle . Therefore, (30b) It is important to note that the contour tracing may suffer an inaccuracy of e = 0.5 pixels . Let 6„ > 6,r . Then R,-F y S„ y 6x+ a, +r,

S,

6 r -6

(30c) 30C )

It is important to image the sphere such that the image at hand is the largest possible, and is fully visible . 5 .2, Verification of overall geometric accuracy A simple test object has been designed to test the accuracy of the calibration process . The object is made of a bright square pattern on a dark background . Also, a dark circle has been drawn inside the square, such that the center of the square and that of the circle coincide . Given the image of this object, it is possible to extract the vanishing points corresponding to the major directions of its straight edges . Also, it is possible to locate the image of its true center by constructing its diagonals, and computing their intersection . Let v, and v, be their vanishing points . The first test is to verify that

(v,r+T,)(v2, +TJ+ (v„sr +T,.)(v,s,+TI)+f' =0 . The exact 3D locations of all the points are computed using the extrinsic calibration procedure . The center of these points, and hence the center of the circle, is thus determined . Let x H be the image of that center, and Xg be its position is the camera coordinate system . Vol . 131H, No . 3 pp . 355 585, 1994 Printed in Cheat Britain

571

G . Seetharainan et al .

. 5 . A planar patch of square shape and a circle drawn in it . It is designed to provide ; Fr( two sets of parallel lines, and a quadratic curve in the observed image .

It is possible to compute the orientation n . of the plane by computing e„ = v l x v, . A new coordinate system 0 is constructed as follows e, = 3, x "v, e l =cxe, e, = e, x e l . The three axes of the system 0 are illustrated in Fig . 5 . Now any point P on the circle can be resolved into the camera coordinate system as X,

Yo

0 1 Specifically, we are interested in the points on the circle,

[Y,,, - [rsin0] 572

Joumul

or the Franklin Institute Elsevier Sacnce I .td

Calibration

of Camera Parameters Using Vanishing Points

Since the radius of the circle is already known, each of the points on the circle is uniquely determined by 0, and can be transformed onto the image plane . Then, for various values of 0 we generate x(0), and find its nearest neighbor x,(0) on the observed image of the ellipse . Up to K points are generated, and tested . The inaccuracy in the calibration process can be measured and expressed as e 1 = Y, IX(O1)-x.v(dr)1 2 . K

Vt. Experiment Results

The proposed calibration process has been implemented in C using UNIX, A specially designed test object, shown in Fig . 3 was used for our experimental study . Both real world and simulated images were considered for the validation process . The radial distortion factor was kept very minimal by choice, through selecting a high-performance lens (a Nikon 6mm lens) . The simulated images were generated using a computer graphics program from 3D geometric description of the scene . The 3D geometric description required of the object was derived (exact to dimension) based on the physical dimensions of the object used in the real world test . The program was designed to output the image points at a meta resolution (i .e . pre-digitization) and at the pixel grid resolution (i .e . post-digitization) . This in turn has allowed us to study the effect of errors introduced in sampling (digitization) and segmentation procedures on the overall applicability of our newly proposed method . The intrinsic parameters used to generate the simulated images were deliberately kept in same order of magnitude as that of the expected values of their (real world) counterparts . The expected range of each physical parameter of the imaging system was estimated from the manufacturer's specification of the digitizer board, the camera, and the lens . These parameters are listed in Table I . The real and two-simulated (i .e . post-sampled and pre-sampled) images were processed in two stages . The images are illustrated in Figs 6 and 7 and Table II . The first stage was responsible for extracting the calibration parameters . The purpose of the second stage was to suitably validate the results by a back-projection and verification method . The processing steps in the first stage included (1) extraction of lines using an edge-detection procedure followed by a Hough-transform based line extraction procedure (applicable to the pre-sampled and real images only), (2) grouping of these lines into sets of parallel lines, followed by the computation of their vanishing points, and (3) computation of the calibration parameters . The processing steps in the second stage involved (1) generating the image using the geometric description of the test object, and the calibrated imaging parameters, and (2) computation of a measure of disparity (mean-squared error) between the original and the newly generated images . The disparity is computed based on the Euclidean distance between the original and projected position for corner points, and both the length and orientation (for lines), or the invariant features (quadratic shape parameters) (for ellipses) . The results are tabulated in Table III . Vo1.3310, No . S, pp . 555 5R5. 1994 Prlnta] 1n GtCal Britain

573



G . Seetharaman et al . TABLE I

Parameters used Joy generating the simulation images

Intrinsic parameters Parameter name Focal length / Sampling interval s, Sampling intervals, . Optical centcrt 7,, Optical center? 7 .,

Physical value

Logical value

6 .000000 mm

256 .00000

23 .437500 tim 23 .437500 pm

100000 . 1 .00000 0 .00000 0 .00000

0,000000 j m 0 .000000 pin

Extrinsic parameters (in pixel units) Direction n Direction /i Direction ; Position P

(-0 .707107, 0.707107, 0 .000000) (-0 .408248, -0 .408248, 0 .816497) (- 0 .577350, 0 .577350, 0 .577350) (3000 .00, 3000 .00, 3000 .00)

t The range of these parameters can't be estimated using simple means .

Fic . 6 . Computer generated image of the test objects . Note that it has been sampled and quantized .

574

lsrrnuI of Js Franklin Implore El"", sa;,,,ee IAn

Calibration of Camera Parameters Using Vanishing Points

FIG . 7 . The real world image of the physical test object . The image was captured using an

ITI-150 imaging system, with a Pull nix 560 camera .

Vll. Conclusion Vanishing points provide a quick and easy method for estimating the pixel aspect ratios (scale factors), true optical center and focal length of an imaging system . Two views of a solid cube are generally sufficient to facilitate the calibration of extrinsic and intrinsic camera parameters . In particular, the method is useful when the radial distortion of the lens is negligible. Both the simulated results and the real world experimental results indicate that the method performs very well . The accuracy of the calibrated parameters has been computed in terms of (1) the disparity from the actual value, and (2) the difference between the actual and back-projected images . In situations where the distortion is moderate, one can used a precalibrated lens, and estimate the scale factors the optical center first . We do so, to reduce the problem from an eighth order nonlinear search to a sixth order nonlinear search . Thus, the method is still nonlinear . Also, if the radial distortion is too high, the practical difficulties lie with the extraction of vanishing points . This problem must be addressed in future research .

Val 3318, No 5, pp-555-585, 1994 Printed in Grenl Rdmin

575

G. Seetharaman et al .

TABLE 11

The representation of lines in the mete-scale (pre-sampling) version of a the image generated by our,graphics program Line id

2 4 5 6 7 8 9 10 I1 12 13 14 15 16 17 18 19 20 21

576

End Point 1 (x,y)

End Point 2 (x, y)

(35 .574, -61 .616) (67 .328, - 19 .436) (60 .190, 11 .584) (21 .435, -37 .127) (0 .000 . - 108 .869) (94 284, 54.435) (-71 .885, -23 .057) (-26 .680, -77 .020) (-11 .504, -33 .209) (-70 .222, 28 .959) (-94 .284, 54 .435) (-72 .486, -41 .850) (0 .000, 0 .000) (-44.115, 61 .856) (-22.958, 32 .190) (49 .975, 46.028) (16 .987, 70.895) (0.000, 83 .700) (-94 .284. 54 .435) (0.000, 0 .000) (94 .284, 54 .435)

(21 .435, -37 .127) (35 .574, -61 .616) (67 .328, -19 .436) (60 .190, 11 .584) (72 .486, -41 .850) (72 .486 . -41 .850) (-70 .222, 28 .959) (-71 .885, -23 .057) (-26 .680, -77 .020) (- 11 .504, -33 .209) (-72 .486, -41 .850) (0 .000, - 108 .869) (0 .000, - 108 .869) (16 .987, 70 .895) (-44.115, 61 .856) (--22 .958, 32 .190) (49 .975, 46 .028) (-94 .284, 54 .435) (0.000, 0 .000) (94 .284, 54 .435) (0 .000, 83 .700)

Journal of the Franklin Institute FIsevier Science Ltd

Calibration of Camera Parameters Using Vanishing Points

FIG. 8 .

Segmented version of the real world scene illustrated in Fig . 7 .

Fri. 9 . The image of the (simulated) test scene reconstructed using the calibration parameters (the results listed in Table I11) . Vol . 331B, No.3, pp.555-515, 1994 Rimed in Great Britain

577

G . Seetharaman et al .

FIG . 10 . The image of the (simulated) test scene reconstructed using the calibration par-

ameters (the results listed in Table IV) .

FIG . 11 . The image of the real world test scene reconstructed using the calibration parameters

(the results listed in Table VI) .

578

Journal of me Fmnklen institme LseeierScience List



Calibration of Camera Parameters Using Vanishing Points TABLE III Simulation results : calibrated parameters for metascale simulated image The computed values : intrinsic parameters Calibrated parameters

Absolute error

255 .999569 1 .000000 0 .999997 -0 .000421 0 .001609

0 .000431 0 .000000 0 .000003 0 .000421 0 .001609

The computed values : extrinsic parameters u ( - 0 .707106, 0 .707107, 0 .000001) (0 .000001, 0 .000000, 0 .000001) (-0 .408251, - 0 .408250, 0 .816494) (0 .000003, 0 .000002, 0 .000003) (0 .577350, .577348, 0 0 .577353) (0 .000000 . 0 .000002, 0 .000003) (3000 .003, 3000 .005, 2999 .993) (0 .003 . 0,005, 0 .007)

TABLE IV Simulation results : calibrated parameters extracted from the sampled input image The computed values : intrinsic parameters Calibration parameters

Error

255 .126445 1 .000000 0 .996714 L554628 1 .912161

0 .873555 .000000 0 0 .003286 1 .554628 1 .912161

S,

Calibrated parameters : extrinsic parameters v [3

P

(-0 .709008, 0 .705188, . 0.004142) (-0 .411033 . -( .410120, 0.814158) (0 .573054, 0 .579400, 0.579572) (3006 .997, 3001 .121, 2984 .076)

Vol. 33113, No . 5, pp. 555 555 . 1994 Punted in Great Britain

(0 .001901, 0.001919, 0 .004142) (0 .002785, 0 .001872, 0 .002339) (0 .0(4296, 0.002050, 0 .002222) (6 .997, 1121, 15 .924)

579



G. Seetharaman et al . TABLE V

The disparity measure between the actual image and the hack-projected image, for the simulated test scene . Nonlinear errors in the estimated values of radial distortion would affect the ellipse significantly_ The disparity in invariant features of the ellipse is reported. The above values indicate the difference in Figs 6, 9 and 10

Unsampled image case

Parameters Center X Center Y Semi major axis Semi minor axis Orientation

Extracted value

Computed value

Error

Sampled image case Extracted value

Computed value

Error

-0 .138 54 .029 25 .240 9 .341 0.003

0 .255 1 .035 1 .284 1 .575 0 .002

0 .0000

0 .0001

0 .0001

53 .5122 26 .7644 10 .8450

53 .5124 26 .7648 10 .8454

0 .0002 0 .0004 0.0004

0 .117 52 .994 26.524 10 .916

-0 .0000

-0 .0000

0.0000

0 .001

Points error, mean-distance : rota! number of points :

N/A N/A

2 .228 115

TABLE VI

The calibration parameters, computed from the real world input image of the test object

Calibrated parameters Intrinsic parameters 528 .536215 1 .000000 0 .854468 -2 .252496 -22 .919249

Extrinsic parameters (-0 .672005, 0 .740434, -0 .012916) (-0 .478466, -0.437838, 0 .761163) (0 .568668, 0.512723, 0 .643320) (3119 .935, 2774 .938, 4241 .7372)

580

fournal of the Franklin Institute Elseeicr Science Ltd



Calibration of Camera Parameters Using Vanishing Points

TABLE VII

The disparity measure between the actual image and the ba(k-projected image, for the real world scene . Both the point disparity and line feature disparity measures are reported . The abate values reflect the difference in Figs I1 and 8

No .

Line Segments based analysis Extracted Computed value value Error

Feature Points based analysis Extracted Actual value value Error

I 2

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 20

26 .122 -42 .051 -143 .818, 100 .952 7 .028, 206 .890 167 .195, 115 .927 22 .363, -195 .913 -102 .945, -66 .968 129 .934 . -49 .230 -18 .623 . 41 .492 -62 .818, 129 .226 37 .991, 164.417 96 .024 . 87 .226 54 .462, -79 .793 113 .062, 20.938 122 .072, - 19 .754 73 .791, -105 .982 5 .463, -84 .314 20 .850, -142 .044 -102 .284, -37 .321 -98 .319, 41 .969

35.705 . -46 .440 10 .540% -142 .349 . 107 .562 6 .771% 10 .436, 212 .306 6 .398% 118 .419 11 .809% 178 .739 . 29 .646, -197 .936 7 .559% -108 .362, -51 .213 16 .660% 143 .096, -39 .722 16 .237% -10 .837, 42 .809 7 .897% -58 .798, 134 .781 6 .856% 44.546, 168 .448 7 .696% 106 .016, 89 .414 10 .229% 64.208, -78 .702 9 .807% 24.185 11.397% 123 .987, 134.893, -12 .289 14 .836% 84.330, -101.477 11 .462% 13 .364, -83 .399 7 .954% -18 .296, - 135 .775 6 .769% -104.622, -23 .847 13 .675% 49 .076 7 .994% 94 .659 .

Mean error : IOJ34%

Vol . BIB . No. 5, pp. 555-585 . 1994 Ynnted m (tied Britain

222 .103 184 .330 184.195 211 .799 179.802 181 .899 153 .907 172 .823 169 .308 98 .237 106 .774 96 .x72 123 .432 116 .536 41 .678 98 .825 32 .550 63 .443 132 .659 79 .389 163 .457

235 .414 18).242 192.719 218 .2x9 201 .429 194 .686 151 .617 162.372 162 .107 103 .726 108 .690 100 .124 125 .804 118 .992 38 .070 102 .524 30 .391 61 .202 141351 73 .600 170 .934

5.993 0 .495% 4 .627 3 .050% 12 .028% 7 .030% 1 .48P0 6 .047% 4 .253% 5 .587% 1 .794% 3 .678% 1 .922% 2 .108% 8.656% 3 .743% 6 .631 3534% 6 .552% 7 .292% 4 .574

Meanerror :4 .813%

581



G . Seetharaman ct al . TABLE VIII

Result of computed scale factor from the orthographic image of a .sphere, shown in Fig . 4 . The value is in agreement with the calibration results listed in Table VI Geometric features of the hall-image I lnage size Number of edge points p,. p, a,. a,

480 x 480 1187 28 .915754 13 .497894 94520702 112 .021130

Expected range of the pixel-aspect ratio

a, a,

0 .843776

a,-e a, +e

0 .835583

a,-Fe a, -C

0 .852042

Calibrated pixel-aspect ratio Entry from Table VI

0 .854468

Acknowledgements This work was supported by the National Science Foundation under the grant NSF9210926 ; also in part by a grant from the Louisiana Educational Quality Support Fund, LEQSF-91 RDA42 .

References (1) D . Mart, "Vision", W . H . Freeman Company, New York, 1982 . (2) B . K . P . Horn . "Robot Vision", The MIT Press, Cambridge, MA, 1987 . (3) W . J. Shomar, C . Seetharaman and T . Y . Young, "An expert system for recovering 3D shape and orientation from a single perspective view", in R . and L . Shapiro 582

fmnnal orlhe Franklin Inshore El ne•,er Scicncc Ltd

Calibration of Camera Parameters Using Vanishing Points (Eds .), "Computer Vision, Graphics and Image Processing . The last and special issue", Academic Press, San Diego, CA . 1992 . (4) S . T . Barnard, "Interpreting perspective images", Artificial Intelligence, Vol . AI-21, pp . 435--462, 1983 . (5) G . Seetharaman, "Estimation of 3D Motion and Orientation of Rigid Objects from an Image Sequence : A Region Correspondence Approach" . Ph .D . Thesis, University of Miami, Coral Gables, Miami, August 1988 . (6) R . Y . Tsai and T. S . Huang, `Uniqueness and estimation of three dimensional motion parameters of rigid objects with curved surfaces", IEEE Trans . Pattern Analysis & Mach . Intel! ., PAMI Vol . 6, pp . 545-554, 1984 . (7) M . Young, "Optics and Lasers : Including Fibers and Integrated Optics", Springer Verlag, N .Y, 1984 . (8) H . Ban, "Calibration of Stereo Imaging Systems Using Vanishing Points", Ph .D . Thesis, University of Southwestern Louisiana, The Center for Advanced Computer Studies Lafayette, LA 70504, May 1994 . (9) S . Cornbleet, "Geometrical optics reviewed : A new light on an old subject" . Proc . IEEE, Vol . 71, pp . 471-502, April 1983 . (10) R . Y . Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses", IEEE .1 . Robotics Autoni ., Vol . RA-3, pp. 323-344, August 1987 . (11) J . Weng . P . Cohen and M . Herniou, "Camera calibration with distortion models and accuracy evaluation", IEEE Trans . Pattern Analysis & Mach . Intel!., Vol . 14 . pp . 965-980, October 1992 . (12) W . Faig, "Calibration of close-range photogrammetry systems : Mathematical formulation", Photogrammetric Engng Remote Sensing, Vol . 41, pp . 1479 1486, 1975 . (13) .1 Sobel, "On calibrating computer controlled cameras for perceiving 3D scenes" . Artificial Intel! ., Vol . 5, pp . 185-198, 1974 . (14) Y . I . Abdel-Aziz and H . M . Karara, "Direct linear transformation into object space coordinates in close-range photogrammetry", in "Pros Symp . Close-range Photogrammetry", Univ . Illinois at Urbana-Champaign, pp . 1-I8, January 1971 . (15) S . Ganapathy, "Camera location and determination problem", Technical Report 11358-841102-20TM, AT&T Laboratories . Flolmdel, NJ, 1984 . (16) M. A . Fischler and O . Firschein (Eds .), "Readings in Computer Vision : Issues, Problems, and Paradigms", Morgan Kaufmann, San Mateo, CA, 1987 . (17) R . K . Lenz and R . Y . Tsai, "Techniques for calibration of the scale factor and image center for high accuracy 31) machine vision metrology", IEEE Trans_ Pattern ., Vol . 10, pp . 713-720, September 1988 . Analysts & Mach . Intel! (18) G . Adiv, "Inherent ambiguities in recovering 3D motion and structure from noisy flow field", IEEE Trans . on Pattern Analysis & Mach . Intel! ., Vol . 11, pp . 477, May 1989 . (19) C. H . Lee, "Time varying images : The effect of finite resolution on uniqueness" . CVGIP : Image Understanding . Vol . 54, pp . 325 332, 1992 . (20) J . Weng, T . S . Huang and N . Ahuja . "Motion and structure from two perspective views : Algorithm, error analysis and error estimation", IEEE Trans . Pattern Analysis & Mach . Intel! ., Vol . 11, pp. 451-476, May 1989 . (21) T . Kanadc, "Recovery of the 3D shape of an object front a single view"', Artificial Intel!., Vol . 17, pp . 409-460, 1981 . (22) M . A . Penna, "Camera calibration : A quick and easy way to determine the scale factor" . IEEE Trans . Pattern Analysis & Mach . Intel!., Vol . 13, pp . 1240 1245, December 1991 . (23) K . Kanatani, "Statistical analysis of focal-length calibration using vanishing points", IEEE Trans . Robotics and Autom ., Vol . 8 . pp . 767-775, December 1992 . Vol, 331B . No . 5, pp. 555-585 . 1994 Printed in Great Britain

583



G . Seetharaman et al .

'0AA

r i I l i I

O

W

~a

C

i -'

Lens plane

FIG. 12 . Illustration of the defocus effect lens distortion model .

Appendix The radial distortion model described by Eq . (7) has been used in (11, 22) . No derivation has been given, or referred to . Penna (22) has presented a series of approximations to facilitate the computability of Eq . (8), as an inversion of Eq . (7) . In this appendix we derive Eq . (7) from first principles, which clearly suggests the invertible (symmetric) nature of Eq . (7) by Eq . (8) . Let P be a point on the object surface . All the rays emanating from it converge at a point Q, governed by the thin lens equation l ; Il+ 1 ;1 v = I if where a and f are the distances of the points P and Q from the center of the lens respectively and f is the focal length of the lens . An increase in the value of h . in Fig. 12 . increases r,, . consequently increasing Av . This results in the premature convergence of the rays in front of the image plane . Effectively, the image plane would be cutting a cone of rays, resulting in the point P effecting an elliptical region on the image plane, thus blurring the final image and introducing radial distortion . The relationship between r,,, the perpendicular distance of the point Q from the optical axis, and r,, for distortion model, illustrated in Fig . 12, has been derived below . Some equalities and approximations used in the derivation of the relations are V

B

584

[

(a

~ tan B

-.

r) .

=-- -, I

fJournal of the RanAlin Inelilulc Llsevier Science Ltd



Calibration of Camera Parameters Using Vanishing Points 04

o°

eosO=l-

z ,+ 41

Derivation 1 : r„ -, r,, : We observe that the uncertainty in the measured value of r5 of the

true value of r,,, . is proportional to At . Thus rA= i., _ K A,.

= r„ +K tan ODr = r„- Kr„ (1 -cos N) = r,,( 1- K1 r,',

Ci

r - . . .+(-ll~Klf,+ . . .)

where K

(i = 1 .2. . . . ) .

(7_i)!

Derivation 2 : r,, -e r,, : e -K4r

= r,,-K tan OAr = r,,+Kr,,(1 -cos O)

=r5(1+t1 r3-K,r°, + . . .+(-1)°+1

K,rs+ + . . .)

In short, we have r,(1 -

,+K2r~4,

. . .) +(- 1)K,r„`+

r„o-K,ri+"-+"e,d,+ . . .)

where K (2d)i J'''

Since „

v„

s,

Ve

ru r,,

then x,,=x„(1-K1ru+K,_r,,- . . .+(-1)'K; x„=x,,(l+K,rr

. . .+ .(-1)r+

and F,, =y.,.(1 -

r„+K2r;- ._+(-I)'K,r;;'+ . . .)

. . . +(-I)o+I,K .r$ + . . .)_ Y,. =Y5(1 +K1d+

Vol . 331 H, No. 5, pp. 555 181 . 1994 Printed in Gmst Britain .

585