- Email: [email protected]
Flexible calibration of camera with large FOV based on planar homography
Optik 126 (2015) 5218–5223
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Flexible calibration of camera with large FOV based on planar homography Ju Huo a , Ning Yang b , Ming Yang b,∗ , Jiashan Cui b a b
Department of Electrical Engineering, Harbin Institute of Technology, Harbin 150001, China Control and Simulation Center, Harbin Institute of Technology, Harbin 150080, China
a r t i c l e
i n f o
Article history: Received 21 October 2014 Accepted 9 September 2015 Keywords: Vision measurement Camera calibration Large FOV Planar homography Flexible target
a b s t r a c t In order to achieve the high precision calibration of camera with large FOV (field-of-view), sufficient calibration points uniformly distributed in the FOV are needed. However, the arrangements of calibration points are normally complicated and time-consuming; meanwhile, the precision of the assigned calibration points is difficult to guarantee. To solve this problem, a flexible calibration method based on planar homography is presented. In this method, a large scale calibration target covering the whole FOV is built with several feature points by means of homography, in which camera distortion is fully considered so as to pledge the precision of the constructed calibration points. Simultaneously, classical Tsai method is improved in the process of calibration so as to adapt the constructed calibration target, which makes the whole process of calibration linear. The experiments results show that the precision of the flexible calibration based on planar homography is equivalent to that of calibration with real large scale target, whose reconstruction error is within 1.5 mm with the FOV of 3000 mm × 4000 mm. The method presented in this paper can achieve the high precision calibration of camera with large FOV. © 2015 Elsevier GmbH. All rights reserved.
1. Introduction In recent years, visual measurement system is widely used in the industrial production for its features of high precision, non-contact, real-time, and so on. In the visual measurement system, camera is the main measuring device, whose calibration precision will directly affect the precision of measurement. In order to achieve the high precision calibration of camera, the calibration targets are required to cover the whole FOV. However, for the camera with large FOV, the large scale calibration targets are difficult and expensive to manufacture [1–3]. Referring to the method of Zhang, Sun et al. [4] proposed a calibration method for camera with large FOV by splicing small targets into a large target based on planar homography, while Li et al. [5] spliced small targets into a large target through polynomial projective model with difference solution, in both of which the layouts of small targets are rigorously restricted. Yu et al. proposed a calibration method, which utilized the intersection points of lines to construct a large scale target that
∗ Corresponding author at: Control and Simulation Center, Harbin Institute of Technology, Harbin 150080, China. Tel.: +86 15045104681. E-mail address: [email protected] (M. Yang). http://dx.doi.org/10.1016/j.ijleo.2015.09.165 0030-4026/© 2015 Elsevier GmbH. All rights reserved.
cover the whole FOV of camera; however, the method ignores the camera distortion when calibration points are constructed, and the constructed calibration points are not uniformly distributed in the FOV, for which the precision of calibration is affected [6]. Self-calibration does not need a target in the whole process, but neither the stability nor the precision can satisfy the requirements of the high precision calibration for the camera with large FOV [7–9]. For the above problem, a flexible calibration method for camera with large FOV is presented. In this method, a large scale target covering the whole FOV is constructed with fewer feature points by planar homography, in which the number and distribution of calibration points can be independently determined as required. Meanwhile, camera distortion is fully considered in the process of calibration points construction, which will adequately guarantee the precision of calibration. Then, classical Tsai method is improved so as to adapt the constructed calibration target, which makes the whole process of calibration linear. The paper is organized as follows. Section 2 analyzes the algorithm theories. Section 3 describes in detail the construction of calibration target. Section 4 describes the improvement of Tsai two-step method. Section 5 provides experimental results. Finally, Section 6 concludes the paper.
J. Huo et al. / Optik 126 (2015) 5218–5223
P
Zw
o0
Ow
u
Yw
Zc
Xw
3′
2′
Poz
p
v
5219
1′
4′
o1
π
R, t
1
x
3
2
4
L2 L1
y
Oc
Yc
Xc
O Fig. 2. Distortion of spatial line imaging.
Fig. 1. The mathematic model of camera.
2. Basic theory respectively. From (3), a transformation matrix H between M and m can be acquired,
2.1. Mathematic model of camera In ideal situation, a camera is modeled by the usual pinhole, as shown in Fig. 1. The relationship between a 3D point P(xw , yw , zw ) and its image projection p(uu , vu ) is given by
⎡
⎤
⎡
⎡
⎤
xw
⎤
uu fx 0 cx 0 ⎢ ⎥ y ⎥ ⎢ ⎥ ⎢ ⎥ R t ⎢ ⎢ w ⎥, s ⎣ vu ⎦ = ⎣ 0 fy cy 0 ⎦ T ⎢ 1 ⎣ zw ⎥ 0 ⎦ 1
0
0
1
(1)
= H M, sm
(4)
where H = A[ r 1 r 2 t ] is a 3 × 3 matrix, while is a constant factor. Matrix H realizes the transformation from the planar calibration target to the image plane, which expresses the homography between two planes. H is also called the homography matrix [12–14].
0
1 where s is a nonzero scale factor [10,11]. In the camera model, there are four intrinsic parameters: fx and fy are the scale factors along the image axes u and v, and (Cx ,Cy ) is the principle point. R and t, called the extrinsic parameters, are the rotation matrix and the translation vector from world coordinate system to camera coordinate system, respectively. Note that the skew of two image axes is not considered in this paper. A camera usually exhibits lens distortion, especially radial distortion. Let pu (xu ,yu ) and pd (xd ,yd ) be the distortion-free and distorted image coordinates, respectively. We have
xu = xd (1 + k1 r 2 + k2 r 4 + k3 r 6 · · ·) yu = yd (1 + k1 r 2 + k2 r 4 + k3 r 6 · · ·)
,
(2)
where r = xd2 + yd2 ; k1 , k2 , k3 , . . . are the coefficients of radial distortion. In this paper, we consider only the first term of radial distortion. The task of camera calibration is to determine the intrinsic parameters, extrinsic parameters and the lens distortion coefficient. 2.2. Planar homography In the camera calibration, suppose the calibration target lies on the xy plane of the world coordinate system, and zw = 0. The matrix of the camera intrinsic parameters is A, and ri is the ith column of rotation matrix R. According to (1), we have
⎡ ⎢
uu
s ⎣ vu 1
⎤ ⎥ ⎦= A r 1 r 2 r 3
⎡
⎤
⎡ ⎤ xw ⎥ ⎢
⎢ yw ⎥
⎢ ⎥ t ⎢ ⎥= A r 1 r 2 t ⎣ yw ⎦. ⎣ 0 ⎦ xw
1
1
Depict the point on the calibration target as M = [ xw and its corresponding image point is m = [ uu ˜ = [ xw geneous coordinates are M
(3)
T
T
yw ] ,
vu ] , whose homoT ˜ = [ uu vu 1 ] , yw ] and m T
3. Construction of the large scale calibration target In this paper, flexible calibration method is employed to achieve the high precision calibration of camera with large FOV. Flexible calibration is a method that calibrates the camera by constructing a large number of calibration points, which are not marked on the calibration target, with the geometric constraints of original feature points in the FOV. Although the constructed calibration points are unmarked, they really exist and satisfy certain constraints, so they can be used for the calibration of camera. In this paper, a large scale target covering the whole FOV is constructed by means of the planar homography. It can be seen from Section 2.2 that in the planar homography, all the involved image coordinates are ideal. As a result, the acquired real image coordinates of the original feature points should be corrected firstly so as to guarantee the precision of the constructed calibration points. Then the homography matrix between image plane and planar calibration target is calculated, on the basis of which the large scale target is constructed with the constraints of the original feature points in the FOV. In a word, the construction of the calibration target can be divided into three steps: (1) calculation of the camera distortion coefficient; (2) calculation of homography matrix; (3) construction of the large scale target.
3.1. Calculation of the camera distortion coefficient There are a great number of methods to calculate the camera distortion coefficient [15,16], in which the one based on invariant cross ratio is easier and flexible [17]. As shown in Fig. 2, line L1 is the image of spatial line L in ideal camera; however, the actual image of L in real camera is the curve L2 with certain radian due to the distortion. P w1 (xw1 , yw1 , zw1 ), P w2 (xw2 , yw2 , zw2 ), Suppose P w3 (xw3 , yw3 , zw3 ), P w4 (xw4 , yw4 , zw4 ) are the points on the spatial line L, whose corresponding points on L1 and L2 are P1 (xu1 , yu1 ), P2 (xu2 , yu2 ), P3 (xu3 , yu3 ), P4 (xu4 , yu4 ) and P 1 (xd1 , yd1 ),
5220
J. Huo et al. / Optik 126 (2015) 5218–5223
8
Interval
9 8
6
5
4
7 Constructed calibration points
3
1
1
Fig. 4. Construction of the large scale target.
2
Fig. 3. Flexible planar target used to construct the large scale target.
mi . Then, the maximum-likelihood estimation of the homography matrix H is obtained by minimizing the following functional
P 2 (xd2 , yd2 ), P 3 (xd3 , yd3 ), P 4 (xd4 , yd4 ), respectively. The cross ratio of the four spatial points P w1 , P w2 , P w3 , P w4 is shown as follows:
⎧ (xw1 − xw3 )(xw2 − xw4 ) ⎪ ⎪ ⎪ (xw2 − xw3 )(xw1 − xw4 ) = CR ⎪ ⎪ ⎨
(yw1 − yw3 )(yw2 − yw4 ) = CR , (yw2 − yw3 )(yw1 − yw4 )
⎪ ⎪ ⎪ ⎪ ⎪ ⎩ (zw1 − zw3 )(zw2 − zw4 ) = CR
u3
u1
u4
⎪ ⎩ (yu1 − yu3 )(yu2 − yu4 ) = CR
(5)
.
(6)
From (2), we acquire xui = xdi (1 + k1 r 2 ) yui = ydi (1 + k1 r 2 )
,
−1
1
T h¯ 1 M i T h¯ M i
i = 1, . . ., 4 ,
(8)
(7)
with h¯ i , the ith row of H. T h¯ 3 M i 2 In practice, simply assume mi = 2 I for all i. This is reasonable if points are extracted independently with the same procedure. In this case, the above problem becomes a nonlinear least-squares
(yu2 − yu3 )(yu1 − yu4 )
T
ˆ i ) m (mi − m ˆ i ), (mi − m i
ˆi= where m
where CR indicates the cross ratio. According to the invariant of cross ratio, we have
⎧ (x − x )(x − x ) ⎪ ⎨ (xu1 − xu3 )(xu2 − xu4 ) = CR
i
(zw2 − zw3 )(zw1 − zw4 )
u2
2
2
ˆ i . The nonlinear minimization is conone, i.e., minH i mi − m ducted with the Levenberg–Marquardt algorithm. After the homography matrix H is acquired, calibration points in the world coordinate system are constructed. In the construction, take the original feature points of No. 1, No. 2 and No. 8 as the boundary constraints, and construct grid points, which will be selected as the calibration points, in the closed region with certain interval, as shown in Fig. 4. The interval will control the number and distribution of the constructed points. Then, the coordinates of the constructed points in the image coordinate system are acquired by means of homography between planar target and image plane. Thus far we have acquired the world coordinates and image coordinates of the constructed calibration points and the construction of the large scale target is completed.
2 + y2 ; (x , y ) are the real image coordinates. xdi where r = di di di Substitute (7) into (6), and then the camera distortion coefficient k1 will be gotten. Given the coordinates and cross ratios of multigroup collinear points, the least squares solution of k1 would be acquired. Consequently, the flexible planar target, shown in Fig. 3, is applied to construct the large scale calibration target. In Fig. 3, the original feature points are numbered 1–9, which lie on three lines intersecting with each other, and are full of the FOV.
3.2. Calculation of homography matrix and construction of the large scale target Once the camera distortion coefficient k1 is acquired, the original feature points 1–9 on the flexible planar target are corrected so as to get their ideal image coordinates. Then the homography matrix between planar target and image plane is calculated. There are many ways to estimate the homography matrix. Here, a technique based on a maximum-likelihood criterion is presented. Let Mi and mi (i = 1, . . ., 9) be the points of target and image, respectively. Ideally, they should satisfy (4). In practice, they do not because of noise in the extracted image points. Assume that mi is corrupted by Gaussian noise with mean 0 and covariance matrix
4. Improvement of Tsai two-step method The image coordinates of the constructed points, which are acquired by homography, are ideal image coordinates; however, in Tsai two-step method, camera is modeled with radial distortion, and the image coordinates participating in the calibration are the real image coordinates. It can be found from (2) that calculating the real image coordinates from their respective ideal image coordinates is a process of solving a binary quartic equation set, which contains more than one solution and requires other constraints. As a result, Tsai two-step method is improved so as to accommodate the constructed large scale target in this paper, in which the ideal camera imaging model is adopted. Without considering camera distortion, the camera model is shown in Fig. 1, in which Poz is the projection of P on the camera optical axis, and o1¯ p is the line between camera principal point ¯ P, i.e., and the ideal image point of P. In this situation, o1¯ p//P oz ¯ P = 0. o1¯ p × P oz
(9)
The expression of (9) in coordinates is (xu , yu ) × (xc , yc ) = 0,
(10)
J. Huo et al. / Optik 126 (2015) 5218–5223
5221
Table 1 Camera parameters in the simulation experiment. f (mm)
k1
Cx (pixel)
Cy (pixel)
Roll (◦ )
Yaw (◦ )
Pitch (◦ )
t (mm)
24
0.0003
640
512
10
10
10
[10,20,6000]T
where (xu , yu ) is the ideal image coordinates of P, and (xc , yc ) is the coordinates of P in camera coordinate system. Above equation can be expanded as xu yc − xc yu = 0.
(11)
According to the relationship between world coordinate system and camera coordinate system, we have
⎧ x =r x +r y +r z +t ⎪ ⎨ c 1 w 2 w 3 w x ⎪ ⎩
(12)
xu (r4 xw + r5 yw + r6 zw + ty ) − yu (r1 xw + r2 yw + r3 zw + tx ) = 0.
(13)
Rearrange (13), and (14) is acquired r1 r2 tx r4 r5 + yw yu + yu − xw xu − yw xu = xu . ty ty ty ty ty
(14)
Expressing above equation with vector, we have
⎡
yu xw
yu yw
yu
where the row is known,
−xu xw
ty−1 r1
⎤
⎢ −1 ⎥ ⎢ ty r2 ⎥ ⎥
⎢ ⎥ ⎢ −xu yw ⎢ ty−1 tx ⎥ = xu , ⎥ ⎢ ⎢ t −1 r ⎥ ⎣ y 4⎦
vector while
≈ ac − FD ad , ı
≈ ac − FD ac − FD (ad , ı), ı
,
(16)
≈ ac − FD ac − FD ac − FD ad , ı , ı , ı
where
ac = [ xu
T
yu ] , T
zc = r7 xw + r8 yw + r9 zw + tz
in which (xw , yw , zw ) is the world coordinates of spatial point P, and tx , ty , tz are the components of the translation vector t from world coordinate system to camera coordinate system. Substitute (12) into (11), and then we have
ad
≈ ···
yc = r4 xw + r5 yw + r6 zw + ty ,
xw yu
points are obtained through (1), and their respective real image coordinates are acquired with the following iterative model [18]
ty−1 r5 yu xw yu yw the
T
yu −xu xw column
(15)
−xu yw vector
contains the paramety−1 r1 ty−1 r2 ty−1 tx ty−1 r4 ty−1 r5 ters to be calculated. The relationship between world coordinates and ideal image coordinates is given in (15). Once the relationship in (15) is acquired, camera parameters are solved referring to the way in Tsai two-step method: (1) calculate rotation matrix R and translation components tx , ty ; (2) calculate focal length f and translation component tz . For the ideal camera imaging model is employed in the calibration, the calculation of f and tz is accurate and linear. It can be found that, in this paper, the whole calculation of camera parameters, including distortion coefficient, intrinsic parameters and extrinsic parameters, is linear. In the application, nonlinear optimization can be employed to further improve the results. 5. Experiments and results analyses
ad = [ xd
T
yd ] ,
FD (ad , ı) =
[ xd · k1 · r 2 yd · k1 · r 2 ] , and r = xd2 + yd2 . The simulation experiment is implemented to validate the stability of the method presented in this paper. In the process of calibration, the real image coordinates of the 9 original feature points are corrupted by Gaussian noise with mean 0 and standard deviation = 0.1–1 pixel. The deviation curves of camera parameters change with the varieties of noise are shown in Fig. 5, in which the abscissas express the noise level, while the ordinates indicate the relative error (the ratio of error and true value) of camera parameters. As can be seen in Fig. 5, the relative errors of focal length f, distortion coefficient k1 and translation components tx , ty increase with the noise, but the range is limited. Meanwhile, the relative errors of tz and three rotation angles basically maintain invariable, and keep in lower values. In order to explain the phenomenon, analyze the reasons causing the results. In the whole calibration, only 9 original feature points are employed, whose real image coordinates are corrupted with the noise. The number of the points corrupted with noise is small. Furthermore, the 9 original feature points do not participate in the calibration directly, so the calibration results are less affected by the noise. The experiment results show that the flexible calibration method presented in this paper is stability. 5.2. Real experiment In the real experiment, the camera is MC3010 at 1280 × 1024 pixel resolution. According to the arrangement of the flexible planar target in Fig. 3, original feature points are fixed on the wall in the FOV with the range of 3000 mm × 4000 mm, as shown in Fig. 6, whose world coordinates are given by total station. In order to verify the flexible calibration method presented in this paper, the following experiments are implemented: (1) Construct abundant and evenly distributed calibration points on the wall by total station, and calibrate camera with classical Tsai two-step method. (2) Construct calibration points with the 9 original feature points on the wall by lines intersecting, and calibrate camera with classical Tsai two-step method. (3) Calibrate camera with the flexible calibration method presented in this paper by means of the 9 original feature points on the wall.
5.1. Simulation experiment In the simulation experiment, first give the world coordinates of the 9 original feature points and the parameters of camera. The arrangement of the 9 original feature points is as shown in Fig. 3, while the intrinsic and extrinsic parameters of camera are shown in Table 1. The ideal image coordinates of the 9 original feature
In the experiment, calibration points constructed by intersections of lines and planar homography are shown in Fig. 7, in which the points constructed by intersections of lines are not uniformly distributed in the FOV, while the latter has the contrast situation. For above experiments, 95 precision verification points on the wall that do not participate in calibration are reconstructed with the
5222
J. Huo et al. / Optik 126 (2015) 5218–5223 -3
2.45
x 10
2.4 2.35
f relative error
2.3 2.25 2.2 2.15 2.1 2.05
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(a) Deviation curve of f with the change of noise Fig. 6. Flexible planar target in real experiment.
0.04 0.035 0
0.025
-500
0.02 -1000
0.015
Y/mm
k1 relative error
0.03
0.01
-1500
0.005 0
-2000
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(b) Deviation curve of k 1 with the change of noise
-2500
-500
-1000
-1500
-2000
-2500
-3000
-3500
-4000
(a) Constructed calibration points by intersections of lines
0.05
0
0.04 -500
0.03 tx ty tz
0.02
-1000
Y/mm
Position relative error
0
X/mm
0.06
-1500
0.01
0
-2000
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(c) Deviation curves of translation components with the change of noise
-2500
-500
-1000
-1500
-2000
-2500
-3000
-3500
-4000
(b) Constructed calibration points by planar homography
0.058
Attitude relative error
0
X/mm
0.06
Fig. 7. Constructed calibration points by intersections of lines and planar homography.
0.056
0.054 Roll Yaw Pitch
0.052
0.05
0.048
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(d) Deviation curves of rotation components with the change of noise Fig. 5. Deviation curves of camera parameters in different noise level.
respective calibration parameters. Comparing the reconstructed results with the true values, the curves of reconstructed errors are shown in Fig. 8. Fig. 8a shows the reconstructed results of classical Tsai twostep method, whose reconstructed errors are within 1.5 mm. The reconstructed results of calibration with flexible target based on intersections of lines are shown in Fig. 8b and c. Fig. 8b shows the reconstructed results in the area with sparse calibration points, while the reconstructed results in the area that the calibration points are dense are shown in Fig. 8c. In Fig. 8c, the reconstructed errors are within 2 mm, which are a little larger than that of the classical Tsai two-step method; however, in Fig. 8b the maximum reconstructed error is nearly close to 4 mm. The reconstructed results of the calibration method presented in this paper are shown
J. Huo et al. / Optik 126 (2015) 5218–5223
in Fig. 8d, whose reconstructed errors are within 1.5 mm, which is similar to that of the classical Tsai two-step method, and better than that of the flexible calibration method based on the intersections of lines. The experiment results show that the flexible calibration method presented in this paper can achieve the high precision calibration for camera with large FOV with fewer points.
Errors of reconstructed points/mm
1.5 X Y Z
1
5223
0.5
0
6. Conclusions -0.5
-1 0
10
20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(a) Calibration of classical Tsai two-step method Errors of reconstructed points/mm
4 3 2 1 0 -1 X Y Z
-2 -3
0
5
10
15
20
A flexible calibration method for camera with large FOV based on planar homography is presented. In this method, a large scale target covering the whole FOV is firstly constructed with fewer original feature points by planar homography. In the construction of the target, camera distortion is adequately considered so as to guarantee the precision of calibration points. In addition, classical Tsai twostep method is improved to accommodate the constructed target, which makes the whole calibration linear. The flexible calibration method presented in this paper can achieve the high precision calibration of the camera with large FOV, which accesses the precision similar to that of the classical Tsai two-step method. In the flexible calibration method in this paper, planar homography is the key, so how to further improve the precision of the acquired homography matrix is the work in the future. Acknowledgments
25
Serial number of precision verification points
(b) Calibration with flexible target based on intersections of lines (in the area with sparse calibration points)
This research is supported by the National Natural Science Foundation of China (Grant No. 61473100) and the Fundamental Research Funds for the Central Universities (Grant No. HIT.IBRSEM.201308).
Errors of reconstructed points/mm
2
References
1.5 1
0.5 0
-0.5 X Y Z
-1 -1.5 20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(c) Calibration with flexible target based on intersections of lines (in the area that the calibration points are dense)
Errors of reconstructed points/mm
1.5 X Y Z
1
0.5
0
-0.5
-1 0
10
20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(d) Flexible calibration based on planar homography Fig. 8. Curves of reconstructed errors with different calibration methods.
[1] R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE J. Robot. Autom. RA-3 (1987) 323–344. [2] Z.Y. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000) 1330–1334. [3] J.H. Sun, et al., A calibration method for stereo vision sensor with large FOV based on 1D targets, Opt. Laser Eng. 49 (2011) 1245–1250. [4] J.H. Sun, et al., Camera calibration based on flexible 3D target, Acta Opt. Sin. 29 (2009) 3433–3439. [5] W.M. Li, Q.Y. Yu, C. Liu, Calibration method with separation pattern of a singlecamera based on difference coordinates, Acta Opt. Sin. 26 (2006) 697–701. [6] M.L. Yu, Z.Z. Wei, J.H. Sun, On-spot camera calibration method based on flexible planar target, J. Univ. Aeronaut. Astronaut. B 35 (2009) 347–350. [7] R. Hartley, Camera calibration and search for infinity, Proc. IEEE Int. Conf. Comput. Vis. 1 (1999) 510–517. [8] A. Heyden, D.Q. Huynh, Auto-calibration via the absolute quadric and scene constraints, Proc. Int. Conf. Pattern Recognit. 16 (2002) 631–634. [9] J. Huo, W. Yang, M. Yang, A self-calibration technique based on the geometry property of the vanish point, Acta Opt. Sin. 30 (2010) 465–472. [10] G. Chen, Y.B. Guo, H.P. Wang, et al., Stereo vision sensor calibration based on random spatial points given by CMM, Optik 123 (2012) 731–734. [11] Q.Z. Liu, Calibration method for geometry relationships of nonoverlapping cameras using light planes, Opt. Eng. 52 (2013) 074108. [12] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, 2000. [13] F.C. Wu, Mathematical Methods in Computer Vision, Science Press, Beijing, 2008. [14] Y.H. Xu, Geometry Calibration of Perspective Cameras: A Review, Microsoft Research, Beijing, 2003. [15] R.V. Carlos, S.S. Antonio-Jose, Correcting non-linear lens distortion in cameras without using a model, Opt. Laser Technol. 42 (2010) 628–639. [16] J.H. Wang, A new calibration model of camera lens distortion, Pattern Recognit. 41 (2008) 607–615. [17] G.J. Zhang, J.J. He, X.M. Yang, Calibrating camera radial distortion with crossratio invariability, Opt. Laser Technol. 35 (2003) 457–461. [18] H. Janne, Geometric camera calibration using circular control points, IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000) 1066–1077.
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Flexible calibration of camera with large FOV based on planar homography Ju Huo a , Ning Yang b , Ming Yang b,∗ , Jiashan Cui b a b
Department of Electrical Engineering, Harbin Institute of Technology, Harbin 150001, China Control and Simulation Center, Harbin Institute of Technology, Harbin 150080, China
a r t i c l e
i n f o
Article history: Received 21 October 2014 Accepted 9 September 2015 Keywords: Vision measurement Camera calibration Large FOV Planar homography Flexible target
a b s t r a c t In order to achieve the high precision calibration of camera with large FOV (field-of-view), sufficient calibration points uniformly distributed in the FOV are needed. However, the arrangements of calibration points are normally complicated and time-consuming; meanwhile, the precision of the assigned calibration points is difficult to guarantee. To solve this problem, a flexible calibration method based on planar homography is presented. In this method, a large scale calibration target covering the whole FOV is built with several feature points by means of homography, in which camera distortion is fully considered so as to pledge the precision of the constructed calibration points. Simultaneously, classical Tsai method is improved in the process of calibration so as to adapt the constructed calibration target, which makes the whole process of calibration linear. The experiments results show that the precision of the flexible calibration based on planar homography is equivalent to that of calibration with real large scale target, whose reconstruction error is within 1.5 mm with the FOV of 3000 mm × 4000 mm. The method presented in this paper can achieve the high precision calibration of camera with large FOV. © 2015 Elsevier GmbH. All rights reserved.
1. Introduction In recent years, visual measurement system is widely used in the industrial production for its features of high precision, non-contact, real-time, and so on. In the visual measurement system, camera is the main measuring device, whose calibration precision will directly affect the precision of measurement. In order to achieve the high precision calibration of camera, the calibration targets are required to cover the whole FOV. However, for the camera with large FOV, the large scale calibration targets are difficult and expensive to manufacture [1–3]. Referring to the method of Zhang, Sun et al. [4] proposed a calibration method for camera with large FOV by splicing small targets into a large target based on planar homography, while Li et al. [5] spliced small targets into a large target through polynomial projective model with difference solution, in both of which the layouts of small targets are rigorously restricted. Yu et al. proposed a calibration method, which utilized the intersection points of lines to construct a large scale target that
∗ Corresponding author at: Control and Simulation Center, Harbin Institute of Technology, Harbin 150080, China. Tel.: +86 15045104681. E-mail address: [email protected] (M. Yang). http://dx.doi.org/10.1016/j.ijleo.2015.09.165 0030-4026/© 2015 Elsevier GmbH. All rights reserved.
cover the whole FOV of camera; however, the method ignores the camera distortion when calibration points are constructed, and the constructed calibration points are not uniformly distributed in the FOV, for which the precision of calibration is affected [6]. Self-calibration does not need a target in the whole process, but neither the stability nor the precision can satisfy the requirements of the high precision calibration for the camera with large FOV [7–9]. For the above problem, a flexible calibration method for camera with large FOV is presented. In this method, a large scale target covering the whole FOV is constructed with fewer feature points by planar homography, in which the number and distribution of calibration points can be independently determined as required. Meanwhile, camera distortion is fully considered in the process of calibration points construction, which will adequately guarantee the precision of calibration. Then, classical Tsai method is improved so as to adapt the constructed calibration target, which makes the whole process of calibration linear. The paper is organized as follows. Section 2 analyzes the algorithm theories. Section 3 describes in detail the construction of calibration target. Section 4 describes the improvement of Tsai two-step method. Section 5 provides experimental results. Finally, Section 6 concludes the paper.
J. Huo et al. / Optik 126 (2015) 5218–5223
P
Zw
o0
Ow
u
Yw
Zc
Xw
3′
2′
Poz
p
v
5219
1′
4′
o1
π
R, t
1
x
3
2
4
L2 L1
y
Oc
Yc
Xc
O Fig. 2. Distortion of spatial line imaging.
Fig. 1. The mathematic model of camera.
2. Basic theory respectively. From (3), a transformation matrix H between M and m can be acquired,
2.1. Mathematic model of camera In ideal situation, a camera is modeled by the usual pinhole, as shown in Fig. 1. The relationship between a 3D point P(xw , yw , zw ) and its image projection p(uu , vu ) is given by
⎡
⎤
⎡
⎡
⎤
xw
⎤
uu fx 0 cx 0 ⎢ ⎥ y ⎥ ⎢ ⎥ ⎢ ⎥ R t ⎢ ⎢ w ⎥, s ⎣ vu ⎦ = ⎣ 0 fy cy 0 ⎦ T ⎢ 1 ⎣ zw ⎥ 0 ⎦ 1
0
0
1
(1)
= H M, sm
(4)
where H = A[ r 1 r 2 t ] is a 3 × 3 matrix, while is a constant factor. Matrix H realizes the transformation from the planar calibration target to the image plane, which expresses the homography between two planes. H is also called the homography matrix [12–14].
0
1 where s is a nonzero scale factor [10,11]. In the camera model, there are four intrinsic parameters: fx and fy are the scale factors along the image axes u and v, and (Cx ,Cy ) is the principle point. R and t, called the extrinsic parameters, are the rotation matrix and the translation vector from world coordinate system to camera coordinate system, respectively. Note that the skew of two image axes is not considered in this paper. A camera usually exhibits lens distortion, especially radial distortion. Let pu (xu ,yu ) and pd (xd ,yd ) be the distortion-free and distorted image coordinates, respectively. We have
xu = xd (1 + k1 r 2 + k2 r 4 + k3 r 6 · · ·) yu = yd (1 + k1 r 2 + k2 r 4 + k3 r 6 · · ·)
,
(2)
where r = xd2 + yd2 ; k1 , k2 , k3 , . . . are the coefficients of radial distortion. In this paper, we consider only the first term of radial distortion. The task of camera calibration is to determine the intrinsic parameters, extrinsic parameters and the lens distortion coefficient. 2.2. Planar homography In the camera calibration, suppose the calibration target lies on the xy plane of the world coordinate system, and zw = 0. The matrix of the camera intrinsic parameters is A, and ri is the ith column of rotation matrix R. According to (1), we have
⎡ ⎢
uu
s ⎣ vu 1
⎤ ⎥ ⎦= A r 1 r 2 r 3
⎡
⎤
⎡ ⎤ xw ⎥ ⎢
⎢ yw ⎥
⎢ ⎥ t ⎢ ⎥= A r 1 r 2 t ⎣ yw ⎦. ⎣ 0 ⎦ xw
1
1
Depict the point on the calibration target as M = [ xw and its corresponding image point is m = [ uu ˜ = [ xw geneous coordinates are M
(3)
T
T
yw ] ,
vu ] , whose homoT ˜ = [ uu vu 1 ] , yw ] and m T
3. Construction of the large scale calibration target In this paper, flexible calibration method is employed to achieve the high precision calibration of camera with large FOV. Flexible calibration is a method that calibrates the camera by constructing a large number of calibration points, which are not marked on the calibration target, with the geometric constraints of original feature points in the FOV. Although the constructed calibration points are unmarked, they really exist and satisfy certain constraints, so they can be used for the calibration of camera. In this paper, a large scale target covering the whole FOV is constructed by means of the planar homography. It can be seen from Section 2.2 that in the planar homography, all the involved image coordinates are ideal. As a result, the acquired real image coordinates of the original feature points should be corrected firstly so as to guarantee the precision of the constructed calibration points. Then the homography matrix between image plane and planar calibration target is calculated, on the basis of which the large scale target is constructed with the constraints of the original feature points in the FOV. In a word, the construction of the calibration target can be divided into three steps: (1) calculation of the camera distortion coefficient; (2) calculation of homography matrix; (3) construction of the large scale target.
3.1. Calculation of the camera distortion coefficient There are a great number of methods to calculate the camera distortion coefficient [15,16], in which the one based on invariant cross ratio is easier and flexible [17]. As shown in Fig. 2, line L1 is the image of spatial line L in ideal camera; however, the actual image of L in real camera is the curve L2 with certain radian due to the distortion. P w1 (xw1 , yw1 , zw1 ), P w2 (xw2 , yw2 , zw2 ), Suppose P w3 (xw3 , yw3 , zw3 ), P w4 (xw4 , yw4 , zw4 ) are the points on the spatial line L, whose corresponding points on L1 and L2 are P1 (xu1 , yu1 ), P2 (xu2 , yu2 ), P3 (xu3 , yu3 ), P4 (xu4 , yu4 ) and P 1 (xd1 , yd1 ),
5220
J. Huo et al. / Optik 126 (2015) 5218–5223
8
Interval
9 8
6
5
4
7 Constructed calibration points
3
1
1
Fig. 4. Construction of the large scale target.
2
Fig. 3. Flexible planar target used to construct the large scale target.
mi . Then, the maximum-likelihood estimation of the homography matrix H is obtained by minimizing the following functional
P 2 (xd2 , yd2 ), P 3 (xd3 , yd3 ), P 4 (xd4 , yd4 ), respectively. The cross ratio of the four spatial points P w1 , P w2 , P w3 , P w4 is shown as follows:
⎧ (xw1 − xw3 )(xw2 − xw4 ) ⎪ ⎪ ⎪ (xw2 − xw3 )(xw1 − xw4 ) = CR ⎪ ⎪ ⎨
(yw1 − yw3 )(yw2 − yw4 ) = CR , (yw2 − yw3 )(yw1 − yw4 )
⎪ ⎪ ⎪ ⎪ ⎪ ⎩ (zw1 − zw3 )(zw2 − zw4 ) = CR
u3
u1
u4
⎪ ⎩ (yu1 − yu3 )(yu2 − yu4 ) = CR
(5)
.
(6)
From (2), we acquire xui = xdi (1 + k1 r 2 ) yui = ydi (1 + k1 r 2 )
,
−1
1
T h¯ 1 M i T h¯ M i
i = 1, . . ., 4 ,
(8)
(7)
with h¯ i , the ith row of H. T h¯ 3 M i 2 In practice, simply assume mi = 2 I for all i. This is reasonable if points are extracted independently with the same procedure. In this case, the above problem becomes a nonlinear least-squares
(yu2 − yu3 )(yu1 − yu4 )
T
ˆ i ) m (mi − m ˆ i ), (mi − m i
ˆi= where m
where CR indicates the cross ratio. According to the invariant of cross ratio, we have
⎧ (x − x )(x − x ) ⎪ ⎨ (xu1 − xu3 )(xu2 − xu4 ) = CR
i
(zw2 − zw3 )(zw1 − zw4 )
u2
2
2
ˆ i . The nonlinear minimization is conone, i.e., minH i mi − m ducted with the Levenberg–Marquardt algorithm. After the homography matrix H is acquired, calibration points in the world coordinate system are constructed. In the construction, take the original feature points of No. 1, No. 2 and No. 8 as the boundary constraints, and construct grid points, which will be selected as the calibration points, in the closed region with certain interval, as shown in Fig. 4. The interval will control the number and distribution of the constructed points. Then, the coordinates of the constructed points in the image coordinate system are acquired by means of homography between planar target and image plane. Thus far we have acquired the world coordinates and image coordinates of the constructed calibration points and the construction of the large scale target is completed.
2 + y2 ; (x , y ) are the real image coordinates. xdi where r = di di di Substitute (7) into (6), and then the camera distortion coefficient k1 will be gotten. Given the coordinates and cross ratios of multigroup collinear points, the least squares solution of k1 would be acquired. Consequently, the flexible planar target, shown in Fig. 3, is applied to construct the large scale calibration target. In Fig. 3, the original feature points are numbered 1–9, which lie on three lines intersecting with each other, and are full of the FOV.
3.2. Calculation of homography matrix and construction of the large scale target Once the camera distortion coefficient k1 is acquired, the original feature points 1–9 on the flexible planar target are corrected so as to get their ideal image coordinates. Then the homography matrix between planar target and image plane is calculated. There are many ways to estimate the homography matrix. Here, a technique based on a maximum-likelihood criterion is presented. Let Mi and mi (i = 1, . . ., 9) be the points of target and image, respectively. Ideally, they should satisfy (4). In practice, they do not because of noise in the extracted image points. Assume that mi is corrupted by Gaussian noise with mean 0 and covariance matrix
4. Improvement of Tsai two-step method The image coordinates of the constructed points, which are acquired by homography, are ideal image coordinates; however, in Tsai two-step method, camera is modeled with radial distortion, and the image coordinates participating in the calibration are the real image coordinates. It can be found from (2) that calculating the real image coordinates from their respective ideal image coordinates is a process of solving a binary quartic equation set, which contains more than one solution and requires other constraints. As a result, Tsai two-step method is improved so as to accommodate the constructed large scale target in this paper, in which the ideal camera imaging model is adopted. Without considering camera distortion, the camera model is shown in Fig. 1, in which Poz is the projection of P on the camera optical axis, and o1¯ p is the line between camera principal point ¯ P, i.e., and the ideal image point of P. In this situation, o1¯ p//P oz ¯ P = 0. o1¯ p × P oz
(9)
The expression of (9) in coordinates is (xu , yu ) × (xc , yc ) = 0,
(10)
J. Huo et al. / Optik 126 (2015) 5218–5223
5221
Table 1 Camera parameters in the simulation experiment. f (mm)
k1
Cx (pixel)
Cy (pixel)
Roll (◦ )
Yaw (◦ )
Pitch (◦ )
t (mm)
24
0.0003
640
512
10
10
10
[10,20,6000]T
where (xu , yu ) is the ideal image coordinates of P, and (xc , yc ) is the coordinates of P in camera coordinate system. Above equation can be expanded as xu yc − xc yu = 0.
(11)
According to the relationship between world coordinate system and camera coordinate system, we have
⎧ x =r x +r y +r z +t ⎪ ⎨ c 1 w 2 w 3 w x ⎪ ⎩
(12)
xu (r4 xw + r5 yw + r6 zw + ty ) − yu (r1 xw + r2 yw + r3 zw + tx ) = 0.
(13)
Rearrange (13), and (14) is acquired r1 r2 tx r4 r5 + yw yu + yu − xw xu − yw xu = xu . ty ty ty ty ty
(14)
Expressing above equation with vector, we have
⎡
yu xw
yu yw
yu
where the row is known,
−xu xw
ty−1 r1
⎤
⎢ −1 ⎥ ⎢ ty r2 ⎥ ⎥
⎢ ⎥ ⎢ −xu yw ⎢ ty−1 tx ⎥ = xu , ⎥ ⎢ ⎢ t −1 r ⎥ ⎣ y 4⎦
vector while
≈ ac − FD ad , ı
≈ ac − FD ac − FD (ad , ı), ı
,
(16)
≈ ac − FD ac − FD ac − FD ad , ı , ı , ı
where
ac = [ xu
T
yu ] , T
zc = r7 xw + r8 yw + r9 zw + tz
in which (xw , yw , zw ) is the world coordinates of spatial point P, and tx , ty , tz are the components of the translation vector t from world coordinate system to camera coordinate system. Substitute (12) into (11), and then we have
ad
≈ ···
yc = r4 xw + r5 yw + r6 zw + ty ,
xw yu
points are obtained through (1), and their respective real image coordinates are acquired with the following iterative model [18]
ty−1 r5 yu xw yu yw the
T
yu −xu xw column
(15)
−xu yw vector
contains the paramety−1 r1 ty−1 r2 ty−1 tx ty−1 r4 ty−1 r5 ters to be calculated. The relationship between world coordinates and ideal image coordinates is given in (15). Once the relationship in (15) is acquired, camera parameters are solved referring to the way in Tsai two-step method: (1) calculate rotation matrix R and translation components tx , ty ; (2) calculate focal length f and translation component tz . For the ideal camera imaging model is employed in the calibration, the calculation of f and tz is accurate and linear. It can be found that, in this paper, the whole calculation of camera parameters, including distortion coefficient, intrinsic parameters and extrinsic parameters, is linear. In the application, nonlinear optimization can be employed to further improve the results. 5. Experiments and results analyses
ad = [ xd
T
yd ] ,
FD (ad , ı) =
[ xd · k1 · r 2 yd · k1 · r 2 ] , and r = xd2 + yd2 . The simulation experiment is implemented to validate the stability of the method presented in this paper. In the process of calibration, the real image coordinates of the 9 original feature points are corrupted by Gaussian noise with mean 0 and standard deviation = 0.1–1 pixel. The deviation curves of camera parameters change with the varieties of noise are shown in Fig. 5, in which the abscissas express the noise level, while the ordinates indicate the relative error (the ratio of error and true value) of camera parameters. As can be seen in Fig. 5, the relative errors of focal length f, distortion coefficient k1 and translation components tx , ty increase with the noise, but the range is limited. Meanwhile, the relative errors of tz and three rotation angles basically maintain invariable, and keep in lower values. In order to explain the phenomenon, analyze the reasons causing the results. In the whole calibration, only 9 original feature points are employed, whose real image coordinates are corrupted with the noise. The number of the points corrupted with noise is small. Furthermore, the 9 original feature points do not participate in the calibration directly, so the calibration results are less affected by the noise. The experiment results show that the flexible calibration method presented in this paper is stability. 5.2. Real experiment In the real experiment, the camera is MC3010 at 1280 × 1024 pixel resolution. According to the arrangement of the flexible planar target in Fig. 3, original feature points are fixed on the wall in the FOV with the range of 3000 mm × 4000 mm, as shown in Fig. 6, whose world coordinates are given by total station. In order to verify the flexible calibration method presented in this paper, the following experiments are implemented: (1) Construct abundant and evenly distributed calibration points on the wall by total station, and calibrate camera with classical Tsai two-step method. (2) Construct calibration points with the 9 original feature points on the wall by lines intersecting, and calibrate camera with classical Tsai two-step method. (3) Calibrate camera with the flexible calibration method presented in this paper by means of the 9 original feature points on the wall.
5.1. Simulation experiment In the simulation experiment, first give the world coordinates of the 9 original feature points and the parameters of camera. The arrangement of the 9 original feature points is as shown in Fig. 3, while the intrinsic and extrinsic parameters of camera are shown in Table 1. The ideal image coordinates of the 9 original feature
In the experiment, calibration points constructed by intersections of lines and planar homography are shown in Fig. 7, in which the points constructed by intersections of lines are not uniformly distributed in the FOV, while the latter has the contrast situation. For above experiments, 95 precision verification points on the wall that do not participate in calibration are reconstructed with the
5222
J. Huo et al. / Optik 126 (2015) 5218–5223 -3
2.45
x 10
2.4 2.35
f relative error
2.3 2.25 2.2 2.15 2.1 2.05
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(a) Deviation curve of f with the change of noise Fig. 6. Flexible planar target in real experiment.
0.04 0.035 0
0.025
-500
0.02 -1000
0.015
Y/mm
k1 relative error
0.03
0.01
-1500
0.005 0
-2000
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(b) Deviation curve of k 1 with the change of noise
-2500
-500
-1000
-1500
-2000
-2500
-3000
-3500
-4000
(a) Constructed calibration points by intersections of lines
0.05
0
0.04 -500
0.03 tx ty tz
0.02
-1000
Y/mm
Position relative error
0
X/mm
0.06
-1500
0.01
0
-2000
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(c) Deviation curves of translation components with the change of noise
-2500
-500
-1000
-1500
-2000
-2500
-3000
-3500
-4000
(b) Constructed calibration points by planar homography
0.058
Attitude relative error
0
X/mm
0.06
Fig. 7. Constructed calibration points by intersections of lines and planar homography.
0.056
0.054 Roll Yaw Pitch
0.052
0.05
0.048
0
0.1
0.2
0.3
0.4 0.5 0.6 Level of noise/pixel
0.7
0.8
0.9
1
(d) Deviation curves of rotation components with the change of noise Fig. 5. Deviation curves of camera parameters in different noise level.
respective calibration parameters. Comparing the reconstructed results with the true values, the curves of reconstructed errors are shown in Fig. 8. Fig. 8a shows the reconstructed results of classical Tsai twostep method, whose reconstructed errors are within 1.5 mm. The reconstructed results of calibration with flexible target based on intersections of lines are shown in Fig. 8b and c. Fig. 8b shows the reconstructed results in the area with sparse calibration points, while the reconstructed results in the area that the calibration points are dense are shown in Fig. 8c. In Fig. 8c, the reconstructed errors are within 2 mm, which are a little larger than that of the classical Tsai two-step method; however, in Fig. 8b the maximum reconstructed error is nearly close to 4 mm. The reconstructed results of the calibration method presented in this paper are shown
J. Huo et al. / Optik 126 (2015) 5218–5223
in Fig. 8d, whose reconstructed errors are within 1.5 mm, which is similar to that of the classical Tsai two-step method, and better than that of the flexible calibration method based on the intersections of lines. The experiment results show that the flexible calibration method presented in this paper can achieve the high precision calibration for camera with large FOV with fewer points.
Errors of reconstructed points/mm
1.5 X Y Z
1
5223
0.5
0
6. Conclusions -0.5
-1 0
10
20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(a) Calibration of classical Tsai two-step method Errors of reconstructed points/mm
4 3 2 1 0 -1 X Y Z
-2 -3
0
5
10
15
20
A flexible calibration method for camera with large FOV based on planar homography is presented. In this method, a large scale target covering the whole FOV is firstly constructed with fewer original feature points by planar homography. In the construction of the target, camera distortion is adequately considered so as to guarantee the precision of calibration points. In addition, classical Tsai twostep method is improved to accommodate the constructed target, which makes the whole calibration linear. The flexible calibration method presented in this paper can achieve the high precision calibration of the camera with large FOV, which accesses the precision similar to that of the classical Tsai two-step method. In the flexible calibration method in this paper, planar homography is the key, so how to further improve the precision of the acquired homography matrix is the work in the future. Acknowledgments
25
Serial number of precision verification points
(b) Calibration with flexible target based on intersections of lines (in the area with sparse calibration points)
This research is supported by the National Natural Science Foundation of China (Grant No. 61473100) and the Fundamental Research Funds for the Central Universities (Grant No. HIT.IBRSEM.201308).
Errors of reconstructed points/mm
2
References
1.5 1
0.5 0
-0.5 X Y Z
-1 -1.5 20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(c) Calibration with flexible target based on intersections of lines (in the area that the calibration points are dense)
Errors of reconstructed points/mm
1.5 X Y Z
1
0.5
0
-0.5
-1 0
10
20
30
40
50
60
70
80
90
100
Serial number of precision verification points
(d) Flexible calibration based on planar homography Fig. 8. Curves of reconstructed errors with different calibration methods.
[1] R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE J. Robot. Autom. RA-3 (1987) 323–344. [2] Z.Y. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000) 1330–1334. [3] J.H. Sun, et al., A calibration method for stereo vision sensor with large FOV based on 1D targets, Opt. Laser Eng. 49 (2011) 1245–1250. [4] J.H. Sun, et al., Camera calibration based on flexible 3D target, Acta Opt. Sin. 29 (2009) 3433–3439. [5] W.M. Li, Q.Y. Yu, C. Liu, Calibration method with separation pattern of a singlecamera based on difference coordinates, Acta Opt. Sin. 26 (2006) 697–701. [6] M.L. Yu, Z.Z. Wei, J.H. Sun, On-spot camera calibration method based on flexible planar target, J. Univ. Aeronaut. Astronaut. B 35 (2009) 347–350. [7] R. Hartley, Camera calibration and search for infinity, Proc. IEEE Int. Conf. Comput. Vis. 1 (1999) 510–517. [8] A. Heyden, D.Q. Huynh, Auto-calibration via the absolute quadric and scene constraints, Proc. Int. Conf. Pattern Recognit. 16 (2002) 631–634. [9] J. Huo, W. Yang, M. Yang, A self-calibration technique based on the geometry property of the vanish point, Acta Opt. Sin. 30 (2010) 465–472. [10] G. Chen, Y.B. Guo, H.P. Wang, et al., Stereo vision sensor calibration based on random spatial points given by CMM, Optik 123 (2012) 731–734. [11] Q.Z. Liu, Calibration method for geometry relationships of nonoverlapping cameras using light planes, Opt. Eng. 52 (2013) 074108. [12] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, 2000. [13] F.C. Wu, Mathematical Methods in Computer Vision, Science Press, Beijing, 2008. [14] Y.H. Xu, Geometry Calibration of Perspective Cameras: A Review, Microsoft Research, Beijing, 2003. [15] R.V. Carlos, S.S. Antonio-Jose, Correcting non-linear lens distortion in cameras without using a model, Opt. Laser Technol. 42 (2010) 628–639. [16] J.H. Wang, A new calibration model of camera lens distortion, Pattern Recognit. 41 (2008) 607–615. [17] G.J. Zhang, J.J. He, X.M. Yang, Calibrating camera radial distortion with crossratio invariability, Opt. Laser Technol. 35 (2003) 457–461. [18] H. Janne, Geometric camera calibration using circular control points, IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000) 1066–1077.