- Email: [email protected]
Chapter 14 General theory of geometric estimation
Chapter 14 General Theory of Geometric Estimation This chapter gives a mathematical extension to the theory of geometric correction in Chapter 5 and the theory of parametric fitting in Chapter 7. Here, the error distribution is no longer assumed to be Gaussian: the role of the covariance matrix for a Gaussian distribution is played by the Fisher information matrix. We derive a theoretical lower bound, called the Cramer-Rao lower bound, on the covariance matrix of an unbiased estimator of the parameter to be determined and show that the maximum likelihood estimator attains this bound in the first order if the problem belongs to the exponential family. We then express the computation of maximum likelihood estimation in the form suitable for numerical computation in practical applications.
14.1
Statistical Estimation in Engineering
The subject of this chapter is essentially statistical estimation. However, the following treatment is very different from traditional statistics, whose main objective is to infer the structure of a random phenomenon by observing multiple data with a view to evaluating and comparing effects and procedures in domains that involve a large degree of uncertainty, such as medicine, biology, agriculture, manufacturing, sociology, economics, and politics. In such a domain, the problem is usually translated into the mathematical language as estimating parameters involved in the probability distribution from multiple independent samples from it (see Section 3.5 for the classical results). Although this framework covers almost all types of relevant applications in the above mentioned domains, geometric estimation problems in computer vision and robotics have many non-traditional elements. In traditional statistics, errors are regarded as uncontrollable; the accuracy of estimation is improved only by repeated measurements. However, repeating measurements is costly. Hence, if the accuracy is the same, those methods which require a smaller number of data are more desirable. In other words, methods whose accuracy improves rapidly as the number of data increases are more desirable than those with slow increase of accuracy. Thus, the study of asymptotic properties of estimation in the limit of a large number of data has been one of the central subjects in traditional statistics (see Section 3.6.2 for the classical results). In such engineering domains as computer vision and robotics, where electronic sensing devices are used, errors are usually small and called noise. 451
452
Chapter 14.
General Theory of Geometric Estimation
Moreover, they are controllable" the accuracy of sensing can be improved by using high-resolution devices and controlling the environment (lighting, dust, temperature, humidity, vibration, etc.). However, such control is costly. Hence, if the accuracy is the same, those methods which tolerate a higher level of noise are more desirable. In other words, methods whose accuracy improves rapidly as the noise level decreases are more desirable than those with slow increase of accuracy. Thus, the study of the accuracy of estimation in the l i m i t of s m a l l noise is very important. In this sense, our approach of assuming local distributions and applying linear analysis, which has been the basic principle in the preceding chapters, can be justified. In many engineering domains, repeating measurements under the same conditions (which is easy) often produces the same results because the sources of inaccuracy in the device and the environment are fixed (but unknown). Also, the number of independent data is usually fixed. Hence, the basic premise of traditional statistics that i n d e p e n d e n t s a m p l e s f r o m the s a m e dist r i b u t i o n can be observed as m a n y t i m e s as desired does not hold. This observation also underlies the statistical treatment in the preceding chapters.
14.2
General Definition
Geometric
Correction
of the problem
Let F ( k ) ( u ) , k = 1, ..., L, be continuously differentiable scalar functions of argument u E 7~". The domain of u is assumed to be an n~-dimensional manifold g / C ~ , which we call the data space. G e o m e t r i c correction is the problem of correcting a given datum u to such a value/t that satisfies the L equations F (k) (it) - O,
k - 1, ..., L.
(14.1)
In Chapter 5.1, we considered N variables u 1 , . . . , n N, but the problem becomes equivalent if we regard their direct sum u - u l | | U N, n -nl ~ - ' . " + rig, as a new variable. As in Chapter 5.1, we call the L equations (14.1) simply the constraint, as opposed to which we call the constraint u E U the i n h e r e n t constraint on u. Let fi E U be the true value of u, which we assume is unknown. We regard the datum u as a random variable with probability density p ( u ; ~t) parameterized by ~. We assume that the distribution of u in the data space b/is local (see Section 3.2.2) and define the "true value" ~ by the relationship given by eq. (3.29). However, such an explicit characterization of fi will not be used in the subsequent analysis, so it can be thought of simply as s o m e value in U. The geometric correction problem is formally stated as follows (see Section 5.1.1):
14.2.
General Geometric Correction
453
P r o b l e m 14.1 From a given datum u E ld, estimate the true value ~t E ld that satisfies the constraint F (k) (~) - 0,
k - 1, ..., L.
(14.2)
As in Sections 3.5 and 3.6, the probability density p(u; ~t) is assumed to satisfy the following regularity condition: A s s u m p t i o n 14.1 The probability density p(u; ~t) is continuously differentiable with respect to u and s an arbitrary number of times, and
p(u; ~) > 0
(14.3)
for all u E hi. Furthermore, the integration operation fu du for any expression of p(u; ~t) with respect to u is, if the integration exists, interchangeable with the differentiation operation V~t with respect to ~t. The probability density p(u; ~t) is not defined for ~t ~/d. For the convenience of analysis, however, we extend it to ~ ~ / 4 in such a way that +
= ;(u;
+
(14.4)
for all ~ E b / a n d A ~ E Tu (b/) • where T~t (/4) C ~'~ is the tangent space to the manifold/d at ~t (see Section 3.5.1). In precise terms, this assumption is stated as follows (see eq. (3.112)): Assumption
14.2
yap e
14.2.2
(14.5)
T h e r a n k o f the c o n s t r a i n t
The L equations (14.1) may be algebraically dependent. If only r of them are independent, they give only r effective constraints and define an (n ~ - r)dimensional manifold $, or a manifold $ of codimension r, in the data space b/except at possible singular points. We assume that ~t E $ is not a singular point and call r the rank of the constraint (14.1). A precise statement of this assumption, which also gives a precise definition of the rank r, is as follows (see Section 7.1.1): A s s u m p t i o n 14.3 The L equations of the constraint define a manifold $ of codimension r in the data space ld in the neighborhood of ~t. As in the preceding chapter, let us call the manifold $ the (geometric) model of Problem 14.1. The solution is given by the projection of the datum u E ?d onto the model 8 C L/(Fig. 14.1).
454
Chapter 14.
General Theory of Geometric Estimation
Rn
Fig. 14.19 Projecting u E U onto the model S C/4. If we write u = ~ + Au, we have F <1)(~ + Au) / 9
( __
) ~7/~(1),--',
~7/~(L)
T Au
(14.6)
F (L) (~ + Au)
to a first approximation in Au, where and hereafter V F (k) (~) is abbreviated as V/~(k). The inherent constraint u E U requires that A u E Tu(U) to a first approximation. If A u ranges over the entire tangent space Tu(/4), eq. (14.6) defines a linear mapping from Tu(b/) to a linear subspace
- {
/
9
zx.)/
IA. e Tu(U)}
c n
The dimension of this subspace equals the dimension of the subspace {P~V/~(1) , ..., P-UuV F (L ) } L C ,]p~,n,
(14.8)
where P ~ be the n-dimensional projection matrix onto the tangent space Tu(U). Consequently, the dimension of s is at most the rank r of the constraint (14.1), but we further assume the following1: A s s u m p t i o n 14.4 The dimension of the linear subspace s rank r of the constraint.
is equal to the
In mathematical terms, Assumption 14.4 states that each of the L equations (14.1) defines a manifold of codimension 1 in the data space/J such that they intersect each other transversally at ~ (see Section 3.2.1). We say that the 1As a result, the L equations (14.1) cannot be replaced by a single equation, say E L F(k)(u) 2 = O, which still has rank r but s = {0} k--1
14.2.
General Geometric Correction
455
intersection of the L manifolds is a singular value of the constraint (14.1) if the dimension of the subspace L:~t is less than r, and an ordinary value otherwise. Assumption 14.4 requires that the true value ~ should not be a singular value. We say that the constraint (14.1) is nonsingular if Assumption 14.4 is satisfied, and singular otherwise (see Section 5.1.1). The linear subspace L:~t defined by eq. (14.7) can alternatively be defined as the tangent space to the manifold
F 1)(u) / " lu~}eT~
7-{
L
(14.9)
F(L)(u) at a
E td.
However,
a
can be a singular point
of the mapping
(F(I)(u),...,F(L)(u)) T" 5t -~ 7~L. In other words, the linear subspace L:u defined by replacing ~ by u (~ ~) in eq. (14.7) may have a higher dimension than r. If that is the case, we say that the constraint is degenerate (see Section 5.1.1).
14.2.3
Cramer-Rao lower bound for geometric correction
Let l be the score of u with respect to ~ (see eq. (3.113))"
- Vu logp.
(14.10)
The Fisher information matrix J with respect to ~ is defined as follows (see eq. (3.118))" J - E[HT]. (14.11) The symbol E[.] denotes the expectation with respect to the probability density p(u; ~t). Assumption 14.2 implies that I E T~t(b/). We assume that the distribution p(u; ~t) is regular with respect to ~t in the sense that I takes all orientations in T~t (5/) as u ranges over the entire data space L/(see Section 3.5.1). In precise terms, this assumption is stated as follows: 14.5 The Fisher information matrix J is positive semi-definite and has range Tu (l~).
Assumption
Define an (LL)-matrix 12 - (12(kl)) by (12 (kt)) - ((V/~(a), J - V / ~ ( t ) ) ) ,
(14.12)
and an (LL)-matrix I?V - (I?V(kl)) by I?V - V - , or in the notation used in Chapter 5 (l~ (kl)) - ((V/~(k), J - V / ~ ( l ) ) ) - .
(14.13)
456
Chapter 14.
General Theory of Geometric Estimation
Define (nn)-matrices N and S by L
(14.14) k,l=l
S-J--N.
(14.15)
L e t / t - / t ( u ) be an unbiased estimator of the true value ~t determined as a function of the d a t u m u. Since r is constrained to be in the model $, it must s a t i s f y / t ( u ) E $ for any u E/4. Its unbiasedness is defined by
psE[it - ~ ] - 0, where T~t($) of/t is matrix
(14.16)
p S is the n-dimensional projection matrix onto the tangent space to the model $ at fi (see eq. (3.126)). Assuming that the distribution sufficiently localized around ~ in the model $, we define the covariance o f / t in the following form (see eq. (3.115))" V[/t] - psE[(it - g)(/t - ~t)T]pS.
(14.17)
We now prove the following theorem 2" Theorem
14.1
s.
(14.1s)
Since this is an analogue of the Cramer-Rao inequality (3.134) in Section 3.5.2, we call eq. (14.18) and the bound it gives the Cramer-Rao inequality and the Cramer-Rao lower bound, respectively, for geometric correction.
1~.2.~
Proof of the main theorem
L e m m a 14.1 The (LL)-matrix V is positive semi-definite and has range f-.~t.
Proof. Let rI - (r/k) be an arbitrary L-vector. We have L
L
(17, V~7) - (~-~, rlkVF(k),J - ~ qlV/~(l)) 9 k=l
(14.19)
/=1
Since J is positive semi-definite, its generalized inverse J - is also positive semi-definite. Hence, the right-hand side of eq. (14.19) is nonnegative for an arbitrary r/, meaning that V is positive semi-definite. Suppose rl belongs to the null space of V. Then, the left-hand side of eq. (14.19) is 0. It follows that ~-~L which is identical with the k--1 ~k~Tt (k) belongs to the null space of J 2Recall that the relation A _ B means that matrix A - B is positive semi-definite (see Section 3.5.2).
14.2.
General Geometric Correction
457
null space of J. Assumption 14.5 implies that the null space of J coincides with T~t(L/) • Hence, L
E
~k(V/~(k)' Au) - 0
(14.20)
k=l
for an arbitrary Au E T~t(L/). From the definition of the subspace s this means that r/E s Conversely, if r/E s we obtain (r/, V~7) - 0 by following eqs. (14.20) and (14.19) backward. Hence, the null space of the (LL)-matrix V coincides with_ the ( L - r)-dimensional subspace L:~t C 7r L. Consequently, the range of V coincides with s [::1 _
Consider the (LL)-matrix W defined by eq. (14.13). Let P ~ be the Ldimensional projection matrix onto/:~t. Lemma 14.1 implies the following: C o r o l l a r y 14.1 The ( L L ) - m a t r i x W is positive semi-definite and has range l?,u, and the following relationship holds: VW-
P~.
(14.21)
NJN-
N.
(14.22)
L e m m a 14.2
Proof. Since J - J J NJN
- J - , we see that
=
m,n=l L
-
z k,l,rn,n= l
k,n=l
l
--1
)
L
- N,
(14.23)
k,n=l
where the identity 1 7 V V W - IZVW W - I;V is used. L e m m a 14.3
SJS-
S.
(14.24)
458
Chapter 14.
General Theory of Geometric Estimation
Fig. 14.2. The tangent space Tu(S) to model S at fi and its orthogonal complement with respect to Tu(b/). Proof. Since J - J J SJS
- J - , we see that
- (J- - N)J(J= J- - P~N-
- N) - J-J J- - J-JN NP~
+ N - J-
- N J J- + NJN
(14.25)
- N - S,
where the relationship J - J - J J - - P ~ implied by Assumption 14.5 and the relationship P ~ N - N P ~ - N implied by eq. (14.14)are used. El The following lemma plays an essential role in proving Theorem 14.1. L e m m a 14.4 The ( n n ) - m a t r i x S is positive semi-definite and has range
Tu (S). Proof. By definition, S is a positive semi-definite matrix. From Lemma 14.3, we see that for an arbitrary n-vector ~ we see that (~,S~) - (~,SJS~)
(14.26)
- (S~,J(S~)).
Since J is positive semi-definite, the above expression is nonnegative for any E T~". Hence, S is positive semi-definite. By definition, the tangent space T~t(8) is the orthogonal complement of the linear subspace { P - ~ V F ( k ) } L C T~t(L/) (Fig. 14.2). From eqs. (14.14) and (14.15), we see that
= (P~VF(k),J-P~VF(k)) L
-
E
17v(Im)(P~VF(k)'J-VF(t))(J-VF(m)'P~
V~(k))
l,m=l
= (V/~(k), P ~ J - P ~ V F ( k ) ) L
-
E l,m--1
I~('m)(v/~(k)' P g J - V / ~ ( O ) ( V / ~ ( m ) ' J - P ~
V~(k))
14.2.
General Geometric Correction
459
L ---(V~"(k)'J-VP(k))- E (V~"(k)'J-VF(l))~'r(lm)(vF(m)'J-VF(k)) l,m=l L = ?(aa) _ E
?(at)Vv(lm)?(mk) -- O,
(14.27)
l,m=l where the identity V W V - V V V - ~r is used. Eq. (14.27) implies that the linear subspace {PUV/~(k)}L is included in the null space of S. This means that the range of S is included in T u ( $ ) . Hence, the rank of S is at most n ~ - r. We now prove that the range of S exactly coincides with T u ( S ) by showing that S has at least rank n ~ - r. Let J be the linear subspace of 7~n obtained by mapping the r-dimensional subspace { p u V F ( k ) } L C Tu(ld) by J - . Assumption 14.5 implies that J has range Tu(b/), so Y is an r-dimensional subspace of Tu(b/). Let {vl, ..., v n , - . } be a basis of the orthogonal complement of ,7 with respect to Tu (/1/). Since the vectors vi, i - 1 n ~ - r, are orthogonal to vectors J - P U V f i ' ( k ) k - 1, ..., L, we obtain L
-
(14.28)
k,l--1 By Assumption 14.5, matrix J - is positive semi-definite and has range Tu(/d). Since vectors {vi } E Tu (b/), i - 1, ..., n ' - r , are linearly independent, vectors { J - v i } - {Svi}, i - 1, ..., n ' - r, are also linearly independent. Thus, S has at least rank n ~ - r. [] C o r o l l a r y 14.2 The variation A ~ of r defined by A~-
SA~
(14.29)
satisfies Aft E T u ( $ ) for any ~ E T4n. L e m m a 14.5
E [ P ~ ( i ~ - ~ ) ( S l ) T] -- S.
(14.30)
Proof. Since the estimator/~ is unbiased, eq. (14.16) holds identically for any E $. It follows that the first variation of the left-hand side of eq. (14.16) should be 0 for any A~ C T u ( $ ) . Hence, it should be 0 for A~ -- SA~ for any A~ E 7~~. Taking the first variation of the left-hand side of eq. (14.16) with respect to ~, substituting eq. (14.29) into it, noting Assumption 14.1, and using the logarithmic differentiation formula (3.115), we obtain A ( P ~ E [ / t - ~]) - - P S E [ A ~ ) ]
= P~SAf 1
: - {S\
+ P ~ .fTj(/t - ~)(V~t logp, A~t)pdu
- P ~ E [ ( i ~ - ~)iT]A~ \
E [ P S ( i ~ - ~(SI~T]~ Af. /k/J]
(14.31)
460
Chapter 14.
General Theory of Geometric Estimation
Here, we have used the assumption that PS+~uE[it - ~] ~ PSE[i~- ~t] (see Section 3.5.2) and the relationship PfftS = S. Since the above expression should be 0 for any A~ E 7~~, we obtain eq. (14.30). [] The proof of Theorem 14.1 is given as follows. Lemma 14.3, we see that
E[(SI)(Sl) T] - S E [ ~ T ] s -
From eq. (14.11) and
SJS-
S.
(14.32)
From this and Lemmas 14.5, we obtain E[
{ p S (/t_- ~t)
n S ( / t - fi)
T
I/t] S
\
S
"
(14.33)
This matrix is symmetric and positive semi-definite. If matrix A is positive semi-definite and symmetric, so is B T A B for any matrix B as long as the multiplication can be defined (see Section 2.2.3). Hence, the following matrix is positive semi-definite and symmetric:
- PPdS ) ( vb] S
sS ) (
_psPS P~ ) _ ( V [ / , ] - S
S)"
(14.34) Here, we have used the fact that matrices V[/t] and S share the same range Tu(S) and hence P s S = SP~t = S and pSv[it]Pfft = V[/t]. Since S is positive semi-definite, matrix V[/t] - S is also positive semi-definite. [] 14.3
14.3.
Maximum
Likelihood
Correction
M a x i m u m likelihood estimator
Maximum likelihood estimation for this problem is to choose fi that maximize the likelihood, i.e., the probability density p(u; ft) viewed as a function of ~ for a given datum u. The resulting estimate/t is called the maximum likelihood estimator of ~t (see Section 3.6.1). In order to distinguish "variables" from their specific "values", we regard the symbol ~ as the true value and use x when it is viewed as a variable. Since maximizing the likelihood is equivalent to minimizing its logarithm with opposite sign, the problem reduces to the minimization J =-21ogp(u;x)-+ min (14.35) with respect to x E S, i.e., under the constraint
F(k)(x) -- O,
k - 1, ...,L,
x e hi.
(14.36)
Define an (nn)-matrix i, by L -- - V 2 logplx= ~.
(14.37)
14.3.
Maximum Likelihood Correction
461
With the expectation that the maximum likelihood estimator /t is located near ~, we write x - ~t + Ax. (14.38) Substituting this into the function J in eq. (14.35) and expanding it in the neighborhood of ~, we obtain J - c - 2(1, Ax) + (Ax, L A x ) + O(Ax) 3,
(14.39)
where ~ is the value of J evaluated at ~. Assumption 14.2 implies that the rank of L is no more than the dimension n' of the tangent space Tx(14). In order to guarantee unique existence of the value x E b / t h a t minimizes J, we assume the following: A s s u m p t i o n 14.6 For any u E Lt, the (nn)-matrix L is positive semidefinite and has range T~t(bt). Substituting eq. (14.38) into eq. (14.36) and taking a linear approximation, we obtain (V/~(k), Ax) -- 0. (14.40) The inherent constraint x E/d requires that Ax E Tu(b/) to a first approximation. Introducing Lagrange multipliers ~(k) to eq. (14.40), differentiating J with respect to Ax, and ignoring higher order terms, we obtain the following linear equation in Ax" L
L A x -- 1 + E
A(k)PUV/~(k)"
(14.41)
k--1
By Assumptions 14.2 and 14.6, eq. (14.41) has the following unique solution (see Section 2.3.3): L
Ax -- L - 1 + E
A(k)s
(14.42)
k=l
Substituting eq. (14.42)into eq. (14.40), we obtain -
(14.43)
where we put s - (~(k)) and define the (LL)-matrix ? ' - (iF '(k0) by (lY'(kl))
-
((V/~(k), l , - V / ~ ( t ) ) ) .
(14.44)
Since matrix I, has range Tx(g4), matrix V~ can be shown to be positive semi-definite and have range s by exactly the same argument as given in
462
Chapter 14.
General Theory of Geometric Estimation
the proof of Lemma 14.6. Consequently, if u is an ordinary value (see Section 14.2.2), the solution of eq. (14.43) is uniquely given by L
~(k) = _ E
l/V'(kl)(V/~(l)' A u ) '
(14.45)
/--1
where 17V'(kl) is the (kl) element of the (LL)-matrix I ~ ' - V ' - , i.e., (ITV l(kl)) -- ((V/~(k), L--V/7(/))) - 9
(14.46)
Substituting eq. (14.45) into eq. (14.42), we can express the maximum likelihood estimator/t in the form /t - ~ + S'l,
(14.47)
where we define the (nn)-matrix S / by L
S'-
L--
E 17v'(kl)(L-V/~(k))(L-V/~(0) T" k,l=l
(14.48)
By the argument similar to that in the proofs of Lemmas 14.2 and 14.3, we can show that S~LS ~ = S ~. Hence, the (nn)-matrix S' can be shown to be positive semi-definite and have range Tu(,5) by the same argument as given in the proof of Lemma 14.4. Consequently, the covariance matrix of the maximum likelihood estimator/t has the following form: V[/,] - E[S'H TS'].
1~.3.2
(14.49)
Geometric correction of the exponential family
If the distribution of u belongs to the exponential family (see Section 3.6.1), the probability density p(u; ~t) has the following expression (see eq. (3.146)):
p(u; ft) - C(~t)exp[(J'(u), ~) + g(u)].
(14.50)
Here, f ( u ) is an n-dimensional function of u e 7~n, and C(u) and g(u) are scalar functions of u G 7~n. We call Problem 14.1 geometric correction of the exponential family if the probability density p(u; ~t) has the above expression. For geometric correction of the exponential family, we see from eqs. (14.37) and (14.50) that the matrix L does not depend on u. Hence, the following equality holds (see eq. (3.122)): J-
E[s
= I,.
From this, we obtain the following proposition:
(14.51)
14.3.
M a x i m u m Likelihood Correction
463
14.1 The maximum likelihood estimator it for geometric correction of the exponential family is unbiased in the first order.
Proposition
Proof. If eq. (14.51) holds, the matrix S ' defined by eq. (14.48) is a constant matrix. Since El1] : 0 (see eq. (3.117)), we see from eq. (14.47) that E [
-
:
S E [
] :
o.
(14.52)
An estimator is said to be efficient if its covariance matrix attains the Cramer-Rao lower bound (see Section 3.5.2). Extending this concept, we say that an estimator for geometric correction is e]ficient if its covariance matrix attains the Cramer-Rao lower bound given by Theorem 14.1. 14.2 The maximum likelihood estimator it for geometric correction of the exponential family is efficient in the first order.
Proposition
Proof. Since L - J , we see that V ' - V (see eqs. (14.12) and (14.44)) and hence I26' - W (see eqs. (14.13) and (14.46)). It follows that S' - S (see eqs. (14.15) and (14.48)). From eq. (14.49), we see that V[it]- SE[a7-]S- SJS-
S,
where we have used L e m m a 14.3.
(14.53) [:3
The proofs been obtained polynomial in approximation
of Propositions 14.1 an 14.2 are based on eq. (14.47), which has by approximating the function J in eq. (14.35) by a quadratic the neighborhood of their true values and applying a linear to the constraint (14.36). In other words, eq. (14.47) is an expression in the limit of small noise. Accordingly, Propositions 14.1 and 14.2 hold in this sense, which is what the phrase "in the first order" means. This type of first order analysis is compatible with our approach of studying statistical behavior for small noise (see Section 14.1).
1~.3.3
Computation of maximum likelihood correction
We now derive an analytical expression for the m a x i m u m likelihood estimat o r / t . First, we introduce a new assumption about the probability density p(u; x). We assume that viewed as a function of x it takes its m a x i m u m at u. In other words, given a single d a t u m u, the d a t u m u itself is the most likely value of the true value ~ if no other information is given. To be specific, what we need is the following" Assumption
14.7
Vxp(u; X)lx=u = 0.
(14.54)
464
Chapter 14.
General Theory of Geometric Estimation
For example, this assumption is satisfied if the probability density is expressed by a smooth scalar function f ( . ) that takes its maximum at the origin 0 in the form p(u; ~t) = f (u - ~t). The Gaussian distribution is a typical example. Assuming that the solution x of the optimization (14.35) is close to the datum u, we write x - u + Ax. Substituting this into the function J, expanding it in the neighborhood of the datum u, and using Assumption 14.7, we obtain J = c - (Ax, L A x ) + O ( A x ) 3. (14.55) The constant c is the value of J evaluated at u, and L is an (nn)-matrix defined by L
-Vx2 logplx=u.
-
(14.56)
In order to guarantee unique existence of the value x E L / t h a t minimizes J, we assume that Assumption 14.6 also applies to L" A s s u m p t i o n 14.8 For any u E 14, the (nn)-matrix L is positive semidefinite and has range Tu(N). The linear approximation of the constraint (14.36) is ( V F (k) , Ax) - - F (k) ,
(14.57)
where F (k) and V F (k) are the abbreviations of F(k)(u) and VF(k)(u), respectively. The inherent constraint x E L/requires that Ax E Tu(Lt) to a first approximation. Introducing Lagrange multipliers )~(k) to eq. (14.57), differentiating J with respect to Ax, and ignoring higher order terms, we obtain the solution Ax uniquely in the following form: L
-
(14.5s) k=l
Substituting this into eq. (14.57) and putting )~ - (A(k)), we obtain V)~ - - ( F ( k ) ) ,
(14.59)
where the (LL)-matrix V -- (V (kl)) is defined by
L-vF(')))
(14.60)
Here, a computational problem arises. If the L equations (14.36) are not algebraically independent but give only r constraints, the matrix V~ computed from the true value ~ by eq. (14.44) has rank r. However, the rank of the matrix V computed from the datum u by eq. (14.60) may be larger than r. In other words, among the L linear equations in Ax given by eq. (14.57), more than r equations may be linearly independent if evaluated at u r ft. If this
14.3.
Maximum Likelihood Correction
465
Fig. 14.3. The corrected value fi is in Tu(b/) but may not be in L/. Hence, a higher order correction is necessary. occurs, the constraint (14.36) is said to be degenerate (see Section 14.2.2). If the constraint (14.36) is degenerate, some of the positive eigenvalues of matrix V converge to 0 in the limit u --+ ~. Hence, eq. (14.59) viewed as a linear equation in ,k is ill-conditioned when the datum u is close to the true value s (see Section 2.3.2). In order to avoid this ill-posedness, we project both sides of eq. (14.59) onto the eigenspace defined by the largest r eigenvalues of V. In other words, we effectively use only r constraints from among the L linear equations (14.57). The resulting solution may be slightly different from the exact solution of eq. (14.59) but does not effectively affect the subsequent analysis because eq. (14.59) is a first order approximation. The solution of eq. (14.59) given by this projection is L
)~(k) = _ Z
W(kl)F(l)'
(14.61)
/=1
where W (kl) is the (kl) element of the rank-constrained generalized inverse (V)~- (see eq. (2.132)), which we write (14.62) Substituting eq. (14.61) into eq. (14.58), we obtain the following conclusion: P r o p o s i t i o n 14.3 The maximum likelihood estimator it is computed to a first approximation in the form L
it - u -
L- Z
W(kt)F(k)VF(l)"
(14.63)
k,l=l
This gives only a first order solution under the linearized inherent constraint i t - u E Tu(lg) (Fig. 14.3). In actual computation, we need to projected
466
Chapter 14.
General Theory of Geometric Estimation
T.(U) / ~ . . ~ ~U
u"
Fig. 14.4. Locally Gaussian distribution. thus computed value/t onto the data space L/by adding a higher order correction, as discussed in Section 5.1.3. The resulting solution still may not be exactly in the model 8 C b/, so the correction is repeated until the constraint is sufficiently satisfied.
~4.3.4
Locally Gaussian model
Suppose the distribution of u is locally Gaussian (Fig. 14.4; see Section 3.3.1): the probability density has the form p(u; r
- Ce - ( u - v t ' v [ u ] - ( u - g ) ) / 2 ,
(14.64)
where C is a normalization constant. The covariance matrix V[u] is assumed to have range T~t (L/) and have the expression
v[~] -
P~E[(~- a ) ( ~ -
(14.65)
a)~]P~.
This distribution belongs to the exponential family, and eq. (14.64) implies that L - I, - J -- V[/t]-. Hence, the minimization (14.35)is equivalent to (x - u, V [ u ] - ( x - u)) --+ min.
(14.66)
In other words, the solution /t is the Mahalanobis projection of u onto the model $, i.e., the nearest point to u in S measured in the Mahalanobis disthrice with respect to V[u] (see Section 13.2.1). From Proposition 14.3, the solution is obtained in the form L
u - V[u] ~ W(kOF(k)VF(o,
i~ -
(14.67)
k,l=l
(W (kl)) -
(( V F (k), V[/t]VF (0) )
,
(14.68)
P
which coincides with the expression derived in Section 5.1.2 (see eq. (5.17)). As shown in Section 5.1.4, the a posteriori covariance matrix of this estimator has the following expression (see eqs. (5.31) and (5.32)):
.t l-
L
E k,/--1
T
9
(14.69)
14.4.
General Parametric Fitting
467
T~(S)
Fig. 14.5. The datum u is projected onto the tangent point fi of the equilikelihood surface of u to b/, and the standard confidence region is projected onto Tu (U). Here, V[u] is the matrix obtained by projecting V[u] onto T/t(L/) (see eq. (5.26)), and l ~ (kl) is the (kl) element of the ( L L ) - m a t r i x I ~ - (W (kt)) defined by (l~ (kl)) -
((V/~(k), I}'[u]VF (0) )- ,
(14.70)
]o
where V/~(k) is an abbreviation of VF(k)(/t) (see eq. (5.33)). As discussed in Section 5.1.4, eq. (14.69) can be interpreted as the projection of the standard confidence region of u onto the tangent space T/t (S) to the model S at /t (Fig. 14.5); it gives a first order approximation to the Cramer-Rao lower bound. 14.4
1~.4.1
General Parametric Fitting
Definition of the problem
Let F(k)(a,u), k = 1, ..., L, be continuously differentiable scalar functions of arguments a E T4m and u E 7~n. The domains of a and u are assumed to be an ml-dimensional manifold A C gUn and an hi-dimensional manifold /2 C T4n, respectively; we call A the data space a n d / 2 the parameter space. Parametric fitting is the problem of computing the parameter u in such a way that the L equations F (k) (a, u) - 0,
k = 1, ..., L,
(14.71)
"fit" multiple instances {a~}, c~ = 1, ..., N, of the variable a. We call the vectors {a~} and u the data and the fitting parameter, respectively. As in Chapter 7.1, let us call the above L equations the hypothesis. Let a~ E A be the true value of a~, which we assume is unknown. We regard each d a t u m a~ as a r a n d o m variable with probability density p~(a~; gt~) parameterized by a~. As in the case of geometric correction, we assume that the distribution of a~ in the data space A is local and define the "true value"
468
Chapter 14.
General Theory of Geometric Estimation
g~ by the relationship given by eq. (3.29). However, such an explicit characterization of ds will not be used in the subsequent analysis, so it can be thought of simply as some value in A. The parametric fitting problem is formally stated as follows (see Section 7.1.1): P r o b l e m 14.2 Estimate the true value ~t of the fitting parameter u E Lt from
the data {as}, ~ = 1, ..., N, in such a way that their (unknown) true values {a.s}, a = 1, ..., N, satisfy the hypothesis F ( k ) ( a s , ~) = O,
k = 1,..., L.
(14.72)
As in the case of geometric correction, the probability density p~(as; as) is assumed to satisfy the same regularity conditions (see Section 14.2.1): 14.9 The probability density ps(as; ds) is continuously differentiable with respect to as and ds an arbitrary number of times, and
Assumption
ps(as; ds) > 0
(14.73)
for all as E A. Furthermore, the integration operation f A das with respect to a~ for any expression of ps(as; ds) is, if the integration exists, interchangeable with the differentiation operation Va, with respect to as. The probability density p~(as; a~) is not defined for ds ~ .A. For the convenience of analysis, however, we extend it to ds ~ ,4 in the same way as in the case of geometric correction. Namely, we assume that
ps(as; ds + A d s ) = ps(as; ds) + O ( A a , ) 2
(14.74)
for all ds E A and A d s E Tao(A) • where T a , ( A ) C T~m is the tangent space to the manifold jt at 5,s. In precise terms, we assume the following: Assumption
14.10
Vamp, E Ta, (A). 1~.~.2
(14.75)
The rank o f the hypothesis
The L equations (14.71) may be algebraically dependent. If only r of them are independent, they generally give only r effective constraints and define an (m ~ - r)-dimensional manifold 8, or a manifold S of codimension r, in the data space jt except at singular points. We assume that all {as } are not singular points and call r the rank of the hypothesis (14.71). A precise statement of this assumption, which also gives a precise definition of the rank r, is as follows: 14.11 The L equations of the hypothesis define a manifold $ of codimension r in the data space A in the neighborhood of as, a = 1, ..., N, for u = ~t. Assumption
14.4.
General Parametric Fitting
469
As in the preceding chapter, let us call the manifold $ the (geometric) model of Problem 14.2. If we write a s = a s + A a ~ , we have
/
§
t
9
__
Va/~(1)...,
VaF(~L)
(
)
Aa~
(14.76)
to a first approximation in A a a , where and hereafter VaF(k)(5~, ~t) is abbreviated as Va/Nk). The inherent constraint a s E ,4 requires that A a s E Tas (,4) to a first approximation. If A a s ranges over the entire tangent space T a , (A), eq. (14.76) defines a linear mapping from Tas (,4) to a linear subspace
z:a
-
{
9
I/Xa
e Tao (A)}
c
n
(14.77)
The dimension of this linem" subspace equals the dimension of the subspace
{ PaA VaF(1) , ..., PaAs VaF(L) } L C Ti m,
(14.78)
where p aA s is the m-dimensional projection matrix onto T a s ( A ) . Consequently, the dimension of s is at most the rank r of the hypothesis (14.71), but we further assume the following3: 14.12 The dimension of the linear subspace s the rank r of the hypothesis for all c~.
Assumption
is equal to
In m a t h e m a t i c a l terms, Assumption 14.12 states that each of the L equations of the hypothesis defines a manifold of codimension 1 in the d a t a space A such that they intersect each other transversally at 5,s, c~ = 1, ..., N. We say that a s is a singular datum if the dimension s is less than r, and an ordinary datum otherwise. Assumption 14.12 requires that singular data should be excluded from the fitting process. We say that the hypothesis (14.71) is nonsingularif Assumption 14.12 is satisfied, and singular otherwise (see Section 7.1.1). The linear s u b s p a c e / : a s defined by eq. (14.77) can alternatively be defined as the tangent space to the manifold
jz _ {
9 F(L) (a, ~t)
e 7"iLia e A} e Ti n
(14.79)
3As a result, the L equations (14.71) cannot be replaced by a single equation, say = {0} for all a.
L Ek--1 F(k)(a' u)2 = 0, which still has rank r but s
470
Chapter 14.
General Theory of Geometric Estimation
at a,m E A. However, am can be a singular point of the mapping (F(1)(a, fi), .... F (L)(a, ~))T: ,4 --+ 7~L. In other words, the linear subspace s defined by replacing gm by am ( r gin) in eq. (14.77) has a higher dimension that r. If this is the case, we say that the hypothesis is degenerate (see Section 7.1.1). We further assume that no effective constraints exist on ~t other than the inherent constraint fi E b/. Putting it differently, we assume that for an arbitrary variation A u of ~ there exists a variation A~m of am such that
F(k)(am + A6,m, ~ + Au) = 0,
k = 1,..., L,
(14.80)
for c~ - 1, ..., N. The above equation is equivalent to k-
1,...,L,
(14.81)
to a first approximation, where and hereafter VuF(k)(~m, ~) is abbreviated as VuF (k). Let Tu(/d) be the tangent space to the manifold/d at ~. The inherent constraint u E ?d requires that A u E T~t (b/) to a first approximation. Similarly, the inherent constraint ~m E A requires that A~m E Ta~ (A) to a first approximation. From the definition of the linear subspace s the above assumption is formally stated as follows: A s s u m p t i o n 14.13 For an arbitrary Au E T~t(Lt),
E s
9
1~.~.3
(14.82)
Cramer-Rao lower bound for parametric fitting
Let Im be the score of am with respect to am: lm - Va~ logpm.
(14.83)
The Fisher information matrix Jm with respect to am is defined by w
-T
gm - E[lmlm ].
(14.84)
The symbol El. ] denotes the expectation with respect to the joint probability density 4 I-I N m=l pro(am; am). Assumption 14.10 implies that lm E Ta~(A). We_ assume that the distribution pro(am; ~tm) is regular with respect to am, i.e., lm takes all orientations in Tam (A) as am ranges over the entire data space A. In precise terms, this assumption is stated as follows: 4Since each aa is assumed to be independent, the expectation E[.] of a quantity that depends on aa alone equals the expectation with respect to pa(aa;~a) alone.
14.4.
General Parametric Fitting
471
A s s u m p t i o n 14.14 The Fisher information matrix J s is positive semi-
definite and has range Tvt. (.A). Define an (LL)-matrix V~ - ( ~ ( k 0 )
by
(g~k,,)_ ((Var(k),j-~Va~(O)), (LL)-m
trix
-
(-Ws(kl) ) -
by
-
Y2,
(14.85)
or
((Va/>(k) , J-~Va />(0 )) - .
(14.86)
Define an (nn)-matrix M by N
- E s=l
L
E
u
-(k)
17Vak')(P~VuF~ ) ( P U V u )
/>(t) T ,
(14.87)
k,l=l
where p U is the n-dimensional projection matrix onto the tangent space Tu (b/) to the manifold gg at R. We c a l l / ~ the moment matrix (see eq. (r.2a)). We assume that the number of the data {as } is sufficiently large and their true values {R~} distribute "generally" in the model $. In other words, we exclude the possibility that { a s } have a special configuration (being identical, collinear, coplanar, etc.) such that the fitting parameter u is indeterminate. In precise terms, this assumption is stated as follows: A s s u m p t i o n 14.15 The moment matrix M is positive semi-definite and has
range Tu (gt). L e t / t - / t ( a l , . . . , aN) be an unbiased estimator of u determined from the data {as}. Since u is constrained to be in the parameter space U, it must satisfy/t(al,..., aN) E ~ for any a~ E A, a - 1, ..., N. Its unbiasedness is defined by P ~ E[/t - R] - 0. (14.88) As in the case of geometric correction, we assume that the distribution of /t is sufficiently localized around ~ in the parameter space b / a n d define the covariance matrix of/t in the following form: V [ / t ] - P-UuE[(it - R ) ( / t - R)q-]Pg.
(14.89)
We now prove the following theorem: T h e o r e m 14.2 V[/t] __>/~-.
(14.90)
Since Theorem 14.2 is an analogue of the Cramer-Rao inequality (3.134) in Section 3.5.2, we call eq. (14.90) and the bound it gives the Cramer-Rao inequality and the Cramer-Rao lower bound, respectively, for parametric fitting.
472
Chapter 14.
14.4.4
General Theory of Geometric Estimation
Proof of the main theorem
L e m m a 14.6 The (LL)-matrix Va is positive semi-definite and has range
Proof. Let v / - (~](k)) be an arbitrary L-vector. We have L
L
(~7, V~7) - ( E q(k)VaF(k)'J-~ E r](l)~af(l))" k=l
(14.91)
/=1
Since J ~ is positive semi-definite, its generalized inverse J ~ is also positive semi-definite. It follows that the right-hand side of eq. (14.91) is nonnegative for an arbitrary ~7, meaning that V~ is positive semi-definite. Suppose v/ belongs to the null space of V~. Then, the left-hand side of eq. (14.91) is 0. Hence, ~kL=l rl(k)VaF(~k) belongs to the null space of J ~ , which is identical with the null space of J a . Assumption 14.14 implies that the null space of J ~ coincides with Tas (A) • Hence, L
A o)
-
o
(14.92)
k--1
for an arbitrary Aa~ E Ta s (A). From the definition of the subspace /:as, this means that v/ E /:_x as" Conversely, if v/ E /:--P as, we obtain (r/, V~v/) - 0 by following eqs. (14.92) and (14.91) backward. Hence, the null space of the (LL)-matrix V~ coincides with the ( L - r)-dimensional subspace s a s C 7~L Consequently, the range of V~ coincides w i t h / : a s . [] Consider the (LL)-matrix ISr~ defined by eq. (14.86) 9 Let p aL be the Ls dimensional projection matrix o n t o / : a s - Lemma 14.6 implies the following: C o r o l l a r y 14.3 The (LL)-matrix 17Va is positive semi-definite and has range f-.(zs, and the following relationship holds: V~I?r
- p_C as
(14.93)
9
The following lemma plays an essential role in proving Theorem 14.2. L e m m a 14.7 The variation Ads of ~
defined by
L
(14.94) k,l=l
satisfies Aa E Tcts(.A) for any A u E T~t(bl), and to a first approximation F(k)(d~ + Aa~, ~ + A u ) - 0,
k-
1,..., L.
(14.95)
14.4.
General Parametric Fitting
473
Proof. Assumption 14.10 implies le E T a : (A). Hence, we have p_u a~ J e - J e from the definition (14.84) of the Fisher information matrix J e . Consequently, p_U a , ~ j-~ _ j-~ ~ and eq. (14.94)implies PU a ~ Age = Age ~ i.e. ~ Age E Ta~ (A) 9 Eq. (14.95) is proved by showing (Vu/~(k) , Age) - -(Va/~(k) , Au).
(14.96)
From eqs. (14.93) and (14.94), we see that
(VaF(k)' A g e ) - - - E
(~TaF(k) , J-~Va~'(rn)) - (ml)W~ (Vu~(1) , Au) /=1
m=l
L
= - E ( P a ~ )kt(Vu/~(/) , Au),
(14.97)
/=1
where (paC)kt is the (kl) element of the projection matrix p_Cas. Assumption 14.13 implies that the L-vector ((Vu/~a(k), Au)) belongs to the s u b s p a c e / : a . Hence, the last term of eq. (14.97) is equal to -(Vaf'(k),Au).
[]
Define an n-vector random variable m e by L
me-
W~-
E
(14.98)
k,l--1 L e m m a 14.8
T
(14.99)
Proof. Since the estimator it is unbiased, eq. (14.88) holds identically for any ge E S, ct = 1, ..., N, and ft E b/that satisfy eq. (14.72). This means that the first variation of the left-hand side of eq. (14.88) should be 0 for any Au E Tu(L/) if the variation Age defined by eq. (14.94) is added to each ge. Taking the first variation of the left-hand side of eq. (14.88), substituting eq. (14.94) into it, noting Assumption 14.9, and using the logarithmic differentiation formula (3.115), we obtain
N
N
"k /.A P~(it - ft) H Pfl E (~,~ l~ /3=1
N
= -Au-
E[P~' ( i t - ft)
Z e=l
Aae)dal "" .daN
e=l
k,/=l
W;
(-le,J-~VaF(O)Vu
]Au.
(14. loo)
474
Chapter 14.
General Theory of Geometric Estimation
Here, we have used the assumption that Pu+muE[it u - ~] "~ P~ E[it - ~] (see Section 3.5.2). The above expression should be 0 for any Au E T~t(U). If we let A u - p U A~, we have A u E T~t (5/) for any A~ E 7~~. Hence,
E[P-~(it- ~t)
w(kl)(~~
E
]P~A~ - -P~A~.
c~--I k , / = l
(14.101) []
Since this is an identity for A~ E 7~~, we obtain eq. (14.99). L e m m a 14.9
(
c~=l
,. 3=1
Proof. Since the expectation of the score l~ is 0 (see eq. (3.117)), eq. (14.98) implies that E[m~] - O. Noting that each a~ is independent, we see that T
E[
ms
mz
ol=l
l-
~-1
E
E[m~mT] - E E[m~mT]" (14.103)
o~,~ 1
c~=l
From eq. (14.98), we obtain L -
W~
-
.. ~
,g-~E[lo~l,~]g-~VaF(m))
(Va
k,l,m,n----1
= E l,n--1
,j~Va~(m)),.oe
-(lk) k,m--1
L -
-
E
T
I ~ ( ' n ) ( P ~ V u / ~ ( t ) ) (PuVu/~('~))
--/~'
(14.104)
l,n=l
where we have used the defining equation (14.86) of the matrix W a and the identity W ~ V ~ W ~ - W ~ W ~ W ~ - I~r [] The proof of Theorem 14.2 is as follows. From eq. (14.89) and Lemmas 14.8 and 14.9, we see that T
,
-P~M )"
(14.105)
14.5.
Maximum Likelihood Fit
475
This matrix is symmetric and positive semi-definite. If matrix A is positive semi-definite and symmetric, so is B - t A B for any matrix B as long as the multiplication can be defined (see Section 2.2.3). Hence, the following matrix is positive semi-definite and symmetric:
_ ( VIii-M- M_).
(14.106)
Here, we have noted that the covariance matrix V[/t] and the moment matrix / ~ share the same range T~t(/A) and hence / ~ / ~ - - / ~ - / 1 ~ - p U . Since/f//- is positive semi-definite, matrix V[/t] - / ~ is also positive semidefinite. [] Suppose the probability densities ps (as ;a) of the data {as } are expressed in terms of a common function p ( . ; - ) in the form ps (as; a s ) = p(as; as). Then, the weight 17V~ (kl) defined by eq. (14.86) is also expressed in terms of a common function lrv(al)( 9) in the form 17V~ (al) - l~(kl)(ds). Suppose the true values {gs } are chosen from a bounded subset $~ of the model $ with density P(a) (the number of data is given by N = fs, P(a)da). Define the following matrix (cf. eq. (14.87)): L
k,l=l
(14.107) If N is sufficiently large, the Cramer-Rao lower bound is approximated by M ~ , which we call the asymptotic approximation of the Crarner-Rao lower bound.
14.5 1~.5.1
M a x i m u m Likelihood Fit M a x i m u m likelihood estimator
Maximum likelihood estimation for this problem is to choose {as} and the fitting parameter u that maximize the likelihood, i.e., the joint probability density 1-Is=l N ps (as; as) viewed as a function of { a s } for given data { a s }. The resulting estimates {ds } a n d / t are called the maximum likelihood estimators of {as} and u, respectively. In order to distinguish "variables" from their specific "values", we regard the symbol a s as the true value and use xs when it is viewed as a variable. Since maximizing the likelihood is equivalent to minimizing its logarithm with opposite sign, the problem reduces to the
476
Chapter 14.
General Theory of Geometric Estimation
minimization N
J - -2 E
logp~(a~; x~) --+ min
(14.108)
ot:l
with respect to xa E $ and u E L/, i.e., under the hypothesis F (k) (x~, u) - 0,
x~ E A,
k - 1, ..., L,
u e/2.
(14.109)
Define an (mm)-matrix L~ by
].~ - - V 2 1 o g p ~ l x ~ = a .
(14.110)
With the expectation that the maximum likelihood estimators {&~} and /~ are located near { ~ } and ~, respectively, we write x~ - a~ + A x e ,
u-~
+ Au.
(14.111)
Substituting the first equation into the function J in eq. (14.108) and expanding it in the neighborhood of a~, we obtain
j
-
e-
N
N
c~=l
c~--1
2
+
(14.112)
where a is the value of J evaluated at ~ , a - 1, ..., N. Assumption 14.10 implies that the rank of L~ is no more than the dimension m' of the tangent space Tx~ (A). In order to guarantee unique existence of the values xa E A, a = 1, ..., N, that minimize J, we assume the following: 14.16 For any aa E ,4, matrix La is positive semi-definite and has range Ta~ (A).
Assumption
Substituting eqs. (14.111) into the hypothesis (14.109) and taking a linear approximation, we obtain
(vap2
),
-
Au).
(14.113)
The inherent constraint x~ E A requires that Ax~ E Ta~ (A) to a first approximation. We seek the value Ax~ that minimizes J for a fixed value of A u under the linearized hypothesis (14.113). Introducing Lagrange multipliers ~ ) to eq. (14.113), differentiating J with respect to A x e , and ignoring higher order terms, we obtain the following linear equation in Axe" L
t ~ A x ~ -- l~ + E k=l
'X~)P.Aa,~Va/Nk) .
(14.114)
14.5.
Maximum Likelihood Fit
477
By Assumptions 14.10 and 14.16, eq. (14.114) has the following unique solution: L
k=l
Substituting eq. (14.115) into eq. (14.113) and putting A~ -- (A~)), we obtain I F : A ~ - -((Vu/~(k), A u ) ) -
((Va/~(k), L : l ~ ) ) ,
(14.116)
where the (LL)-matrix V~ - (~-(kl))is defined by (14.117) - /
By Assumption 14.16, matrix V~ can be shown to be positive semi-definite and have range s by exactly the same argument as given in the proof of Lemma 14.6. Consequently, if a~ is an ordinary datum (see Section 14.4.2), the solution of eq. (14.116) is uniquely given by L
~(k) = _ E l~(kt) ( (Vu/~(t)' Au) + (VaF~(t)' L : l a ) ) '
(14.118)
l=l
where W~ (kl)- is the (kl) element of the (LL)-matrix W~-' defined by I2r V~ , i.e., --
]
=
m
(w,
(14.119)
Substituting eq. (14.118) into eq. (14.115), substituting the resulting expression for Ax~ into eq. (14.112), and ignoring higher order terms, we obtain the following form: J - ~+ E a=l
-t(kt) (v,.p: w;
+ (v.p:
t2L)
k,l=l
((VuF(I),Au) + (Va/~(t),t;la)) -(l~,.t.,~-lo~)) . (14.120) We now seek the value Au that minimizes eq. (14.120). The inherent constraint u E H requires that Au E Tu(Lt) to a first approximation. Differentiating eq. (14.120) with respect to Au, we obtain the following linear equation in Au: N
M'Au-
L
- E E l~(kl)(Vaf'(~k)'t-s a=l
k,l= l
(14.121)
478
Chapter 14.
General Theory of Geometric Estimation
Here, the matrix M ' is defined as follows (see eq. (14.87)): N
L
T
m--1
k,l--1
(14.122) We assume that Assumption 14.15 also applies to M~: A s s u m p t i o n 14.17 The matrix M ~ is positive semi-definite and has range
Tu(U). Then, the solution of eq. (14.121) is uniquely given by N
A u -- - M ' -
E
L
E
I~(kt)(Va/~(k)' L~Im)P~Vu/~(0"
(14.123)
m--1 k,l--1
Since the maximum likelihood estimator of u is given b y / t - ~ + A t , the covariance matrix V[/t] (see eq. (14.89))is evaluated as follows: V[/~]- E [ M ' - M " M ' - ] .
(14.124)
Here, the matrix M " is defined by N
M"=
L
E
E
m,/3-- 1
I~(kt)iZV~('~n)(Vaf-'(k)'L:lm)(Vaf-'(m)'L~ ~)
k,l,m,n--1
(P~Vu/~(t)) ( ' ~ V u / ~ ( n ) ) T .
1~.5.2
(14.125)
Fitting of the exponential family
If the distribution of each am belongs to the exponential family, the probability density pro(am;am) has the following expression (see eq. (3.146))" a.)
-
a.) +
(14.126)
Here, f~(a)is an m-dimensional function of a e 7~TM, and Cm(a) and gin(a) are scalar functions of a E 7~m. We call Problem 14.2 parametric fitting of the exponential family if the probability densities p~(am; am) have the above expression. For parametric fitting of the exponential family, we see from eqs. (14.110) and (14.126) that the matrix Lm does not depend on am. Consequently, we have the following relationship:
Jm - E[Lm] - L m. From this, we obtain the following proposition:
(14.127)
14.5.
Maximum Likelihood Fit
479
P r o p o s i t i o n 14.4 The maximum likelihood estimator it for parametric fit-
ting of the exponential family is unbiased in the first order. Proof. If eq. (14.127) holds, the matrix M ' defined by eq. (14.107)is a constant matrix. Since E[l] - O, we see from eq. (14.123) that -
E[au]
-
(14.128)
o.
As in the case of geometric correction, we say that an estimator for parametric fitting is efficient if its covariance matrix attains the Cramer-Rao lower bound given by Theorem 14.2. P r o p o s i t i o n 14.5 The maximum likelihood estimator it for parametric fit-
ting of the exponential family is efficient in the first order. --
_
]
-
Proof. Since L~ - J ~ , we see that V~ - V~ (see eqs. (14.85) and (14.117)) and hence IV~ - I ~ (see eqs. (14.86) and (14.119)). It follows that M ' = M (see eqs. (14.87) and (14.122)). Taking the expectation of eq. (14.125) and noting the independence of each as, we see that N
E[M'] -
L
~
~
~/~kl) W~ - (rnn) (Va ~ ( k ) , J-~ E[l,l ~ T ]J~ Va ~ ( m ) )
o~,fl--1 k,l,m,n--1 (P~TuF(l)) N
(P-Uu~TuF(n))T
L
W~
W~
, (J-~J,J-~)Va ['(m))
(~--1 k,l,m,n--1
u
= l --1
N
-(0
k
)
)
f-,(n) T
)
--1
L
- E
E
ITV(l~' (P-~Vu~(t)) ( P ~ V u / ~ ( ~ ) ) 7 -
M,
(14.129)
1,n--1
~--1
where we have used the identities J - ~ J ~ J ~ - J-~ and I?V~V~l~r~ = W ~ W ~ W ~ - W ~ . Thus, eq. (14.124)is written as V [ i t ] - M M M -
-
--M
-
-
-
.
-
-
-
-
-
-
-
m
[]
480
Chapter 14.
General Theory of Geometric Estimation
Recall that an estimator is consistent if it converges to the true value as the number of data increases (see Section 3.6.2). P r o p o s i t i o n 14.6 The maximum likelihood estimator for parametric fitting of the exponential family is consistent in the first order if all gts, c~ = 1, ..., N , are in a bounded subset of the model $.
Proof. If all a s are bounded, we h a v e / ~ - O ( N ) from the definition of the moment m a t r i x / ~ (see eq. (14.87)). Hence, V[/t] - / ~ - O(1/N). n The proofs of Propositions 14.4, 14.5, and 14.6 are based on eq. (14.123), which is obtained by approximating the function J in eq. (14.108) by a quadratic polynomial in the neighborhood of their true values and applying a linear approximation to the hypothesis (14.109). In other words, eq. (14.123) is an expression in the limit of small noise. Accordingly, Propositions 14.4, 14.5, and 14.6 hold in this sense, which is what the phrase "in the first order" means. As in the case of geometric correction, this type of first order analysis is compatible with our approach of studying statistical behavior for small noise (see Section 14.1).
1~.5.3
Computation
of m a x i m u m
likelihood fit
We now derive a computational scheme for the maximum likelihood estimator /t by eliminating the estimators {~i~}, c~ -- 1, ..., N. To this end, we first compute the solution x~ of the optimization (14.108) under the hypothesis (14.109) for a fixed u and then substitute the resulting expression back into eq. (14.108), reducing it to a function of u alone. As in the case of geometric correction, we first introduce a new assumption about the probability density ps(as; x~). We assume that viewed as a function of x s it takes its maximum at as. In other words, given a single datum a s , the datum a s itself is the most likely value of the true value a s if no other information is given. To be specific, what we need is the following: A s s u m p t i o n 14.18
Vx~ps(as;x~)lx~=a ~ =0.
(14.130)
For example, this assumption is satisfied if the probability density is expressed by a smooth scalar function f ( . ) that takes its maximum at the origin 0 in the form p~(as; gts) = f ( a ~ - gts). The Gaussian distribution is a typical example. Assuming that the solution x s of the optimization (14.108) is close to the data value a s , we write x~ = a s + Axe. Substituting this into the function J, expanding it in the neighborhood of the datum as, and using Assumption
14.5.
Maximum Likelihood Fit
481
14.18, we obtain N
N
J - c + E(Ax~,L~Ax~) c~--1
+ E O(Ax~)3"
(14.131)
c~----1
The constant c is the value of J evaluated at a s , a - 1, ..., N, and L~ is an (mm)-matrix defined by L~ - - V ~ , logp~lxo,=a .
(14.132)
In order to guarantee unique existence of the values x~ E A, a - 1, ..., N, that minimize J, we assume that Assumption 14.16 also applies to L~" A s s u m p t i o n 14.19 For any a~ E A, matrix I,~ is positive semi-definite
and has range T(z. (.A). The linear approximation of the hypothesis (14.109) is ( V a F (k) , A x e ) - - F (k) ,
(14.133)
F (k) and VaF (k) are the abbreviations of F(a)(a~,u) and VaF(k)(a~, u), respectively. The inherent constraint x~ E .4 requires that Ax~ E T a . (,4) to a first approximation. Introducing Lagrange multipliers
where
$~) to the linearized hypothesis (14.133), differentiating J with respect to A x a , and ignoring higher order terms, we obtain the solution A x a uniquely in the following form: L
Ax~ - E )~) L-~VaF(~k)"
(14.134)
k---1
Substituting this into eq. (14.133) and putting A - (A~)), we obtain
V~A~ = -(F(k)),
(14.135)
where the (LL)-matrix V , - (V(kl)) is defined by (u(kl))
-
((VaF(k), L-~VaF(O))
(14.136)
As in the case of geometric correction, a computational problem arises. Namely, if the L equations (14.109) are not algebraically independent but give only r constraints, the matrix V'o~ computed from the true value d~ by eq. (14.117) has rank r. However, the rank of the matrix V~ computed from the datum a~ by eq. (14.136) may be larger than r. In other words, among the L linear equations in Ax~ given by eq. (14.133), more than r equations may be linearly independent if evaluated at the datum a~ # ~ . If this occurs,
482
Chapter 14.
General Theory of Geometric Estimation
the hypothesis (14.109) is said to be degenerate (see Section 14.4.2). If the hypothesis (14.109) is degenerate, some of the positive eigenvalues of matrix V~ converge to 0 in the limit a s ~ as. Hence, eq. (14.135) viewed as a linear equation in ,ks is ill-conditioned when the data {as} are close to the true values {as} (see Section 2.3.2). This ill-posedness can be avoided in the same way as in the case of geometric correction: we project both sides of eq. (14.135) onto the eigenspace defined by the largest r eigenvalues of V a. In other words, we effectively use only r constraints from among the L linear equations (14.133). The resulting solution may be slightly different from the exact solution of eq. (14.135) but does not effectively affect the subsequent analysis because eq. (14.135) is a first order approximation. The solution of eq. (14.135) given by this projection is L
~ ) - - ~ W(kOF(0,
(14.137)
/=1
where W~(kl) is the (kl) element of the rank-constrained generalized inverse (V~);-, which we write
(W(akl)) -((VaF(k),L-s
(14.138)
Substituting eq. (14.137) into eq. (14.134), substituting the resulting expression into eq. (14.131), and ignoring higher order terms, we obtain the following expression: N
J-c+
E c~=l
L
E
Ws
F~(k F(1).
(14.139)
k,l=l
Since the constant c is irrelevant to optimization, we obtain the following conclusion: P r o p o s i t i o n 14.7 The maximum likelihood estimator i~ of u is given as the
solution of the nonlinear optimization N
J[u] = E s=l
L
E
W~(k')F(ak)F(~') ~ min,
u e ?d.
(14.140)
k,l=l
This proposition gives a rigorous mathematical foundation to the formulation of parametric fitting as presented Chapter 7, upon which the subsequent applications are all based. Let /t be the computed maximum likelihood esdata timator of u. If the model 8 has codimension r in the m-dimensional ' space .4 parameterized by u E ~, it is a model of dimension d = m ' - r , codimension r, and n' degrees of freedom. The argument given in Sections
14.5.
M a x i m u m Likelihood Fit
483
13.2 and 13.3 can be generalized to d a t a with non-Gaussian distributions, and the geometric information criterion can be defined as follows:
A I C ( $ ) - J[/~]-t- 2(dN -t- n').
(14.141)
Bibliography [1] G. Adiv, Determining three-dimensional motion and structure from optical flow generated by several moving objects, IEEE Transactions on Pattern Analysis and Machine Intelligence, 7-4 (1985), 384-401. [2] H. Akaike, A new look at the statistical model identification, IEEE Transactions on Automatic Control, 19-6 (1974), 716-723. [3] J. Aisbett, An iterated estimation of the motion parameters of a rigid body from noisy displacement vectors, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-11 (1990), 1092-1098. [4] A. Albano, Representation of digitized contours in terms of conic arcs and straight-line segments, Computer Graphics and Image Processing, 3-1 (1974), 23-33.
[5]
Y. Aloimonos (ed.), Active Perception, Lawrence Erlbaum Associates, Hillsdale, N J, U.S.A., 1993.
[6]
J. (Y.) Aloimonos and D. Shulman, Integration of Visual Modules: An Extension of the Mart Paradigm, Academic Press, San Diego, CA, U.S.A., 1989.
[7]
S. Amari, Differential-Geometrical Methods in Statistics, Springer, Berlin, F.R.G., 1985.
[8]
S. Amari and M. Kumon, Estimation in the presence of infinitely many nuisance parameters--Geometry for estimation functions, Annals of Statistics, 16-3 (1988), 1044-1068.
[9]
E. B. Andersen, Asymptotic properties of conditional maximum likelihood estimators, Journal of the Royal Society of London, B32-2 (1970), 283-301.
[10]
B. D. O. Anderson and J. B. Moore, Optimal Filtering, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1979.
[11]
K. S. Arun, T. S. Huang and S. D. Blostein, Least-squares fitting of two 3-D point sets, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-5 (1987), 698-700.
[12]
H. Ballard and C. M. Brown, Cliffs, N J, U.S.A., 1982.
[13]
A. Bani-Hashemi, A Fourier approach to camera orientation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-11 (1993), 1197-1202.
[14]
O. E. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, John Wiley and Sons, New York, NY, U.S.A., 1978.
[15]
O. E. Barndorff-Nielsen, Parametric Statistical Models and Likelihood, Lecture Notes in Statistics 50, Springer, Berlin, F.R.G., 1988.
Computer
Vision, Prentice-Hall, Englewood
[16] J. L. Barron, D. J. Fleet and S. S. Beauchemin, Performance of optical flow techniques, International Journal of Computer Vision, 12-1 (1994), 43-77.
[17]
J. Bhanja and J. K. Ghosh, Efficient estimation with many nuisance parameters, Parts 1, 2, 3, Sankhy~, A54-1,2,3 (1992), 1-39, 135-156, 297-308. 485
486
Bibliography
[ls]
P. J. Bickel, C. A. J. Klaassen, Y. Ritov and J. A. Wellner, Efficient and Adaptive Estimation for Semiparametric Models, Johns Hopkins University Press, Baltimore, MD, U.S.A., 1993.
[19]
S. D. Blostein and T. S. Huang, Error analysis in stereo determination of 3-D point positions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-6 (1987), 752-765 (corrected in 10-5 (1988), 765).
[20]
R. M. Bolle and B. C. Verumi, On three-dimensional surface reconstruction methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-1 (1991), 1-13.
[21]
F. L. Bookstein, Fitting conic sections to scattered data, Computer Graphics and Image Processing, 9-1 (1979), 56-71.
[22]
K. L. Boyer, J. Mirza and G. Ganguly, The robust sequential estimator: A general approach and its application to surface organization in range data, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-10 (1994), 987-1001.
[23]
B. Brillault-O'Mahony, New method for vanishing point detection, CVGIP: Image Understanding, 54-2 (1991), 289-300.
[241 B. Brillault-O'Mahony, High level 3D structure from a single view, Image and Vision Computing, 10-7 (1992), 508-520.
[25]
T. J. Broida, S. Chandrashekhar and R. Chellappa, Recursive estimation of 3D motion from a monocular image sequence, IEEE Transactions on Aerospace and Electronic Systems, 26-4 (1990), 639-656.
[26]
T. J. Broida and R. Chellappa, Estimation of object motion parameters from noisy images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-1 (1986), 90-99.
[271 T. J. Broida and R. Chellappa, Performance bounds for estimating threedimensional motion parameters from a sequence of noisy images, Journal of the Optical Society of America, A6-6 (1989), 879-898.
[2s]
T. J. Broida and R. Chellappa, Estimating the kinematics and structure of a rigid object from a sequence of monocular images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-6 (1991), 497-513.
[29]
R. C. Brown, Introduction to Random Signal Analysis and Kalman Filtering, John Wiley and Sons, New York, NY, U.S.A., 1983.
[3o]
A. R. Bruss and B. K. P. Horn, Passive navigation, Computer Vision, Graphics, and Image Processing, 21-1 (1983), 3-20.
[311
M. Campani and A. Verri, Motion analysis from first-order properties of optical flow, CVGIP: Image Understanding, 56-1 (1992), 90-107.
[32]
B. Caprile and V. Torte, Using vanishing points for camera calibration, International Journal of Computer Vision, 4-2 (1990), 127-140.
[33]
C. K. Chui and G. Chen, Kalman Filtering with Real-Time Applications, Springer, Berlin, F.R.G., 1991.
Bibliography
487
[34]
D. R. Cooper and N. Yalabik, On the computational cost of approximating and recognizing noise-perturbed straight lines and quadratic arcs in the plane, IEEE Transactions on Computers, 25-10 (1976), 1020-1032.
[35]
I. J. Cox, A review of statistical data association techniques for motion correspondence, International Journal of Computer Vision, 10-1 (1993), 53-66.
[36]
I. J. Cox, J. M. Rehg and S. Hingorani, A Bayesian multiple-hypothesis approach to edge grouping and contour segmentation, International Journal of Computer Vision, 11-1 (1993), 5-24.
[37]
D. R. Cox and D. V. Hinkley, Theoretical Statistics, Chapman and Hall, London, U.K., 1974.
[3s]
H. Cram~r, The Elements of Probability Theory and Some of Its Applications, Almqvist and Wiksell/Gebers, Stockholm, Sweden, 1955.
[39]
J. L. Crowley, P. Bobet and C. Schmid, Auto-calibration by direct observation of objects, Image and Vision Computing, 11-2 (1993), 67-81.
[4o]
K. Daniilidis and H.-H. Nagel, Analytical results on error sensitivity of motion estimation from two views, Image and Vision Computing, 8-4 (1990), 287-303.
[41]
E. R. Davies, Finding ellipses using the generalised Hough transform, Pattern Recognition Letters, 9-1 (1989), 87-96.
[42]
A. J. Dobson, An Introduction to Generalized Linear Models, Chapman and Hall, London, U.K., 1990.
[43]
L. S. Dreschler and H.-H. Nagel, Volumetric model and 3-D trajectory of a moving car derived from monocular TV-frame sequences of a street scene, Computer Graphics and Image Processing, 20-3 (1982), 199-228.
[44]
T. Echigo, A camera calibration technique using three sets of parallel lines, Machine Vision and Applications, 3-3 (1990), 159-167.
[45]
T. Ellis, A. Abboot and B. Brillault, Ellipse detection and matching with uncertainty, Image and Vision Computing, 10-5 (1992), 271-276.
[46]
T. Endoh, T. Toriu and N. Tagawa, A superior estimator to the maximum likelihood estimator on 3-D motion estimation from noisy optical flow, IEICE Transactions on Information and Systems, E77-D-11 (1994), 1240-1246.
[47]
O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint, MIT Press, Cambridge, MA, U.S.A., 1993.
[4s]
O. D. Faugeras and S. Maybank, Motion from point matches: Multiplicity of solutions, International Journal of Computer Vision, 4-3 (1990), 225-246.
[49]
W. Feller, Probability Theory and Its Applications, Vols. 1, 2, John Wiley and Sons, New York, NY, U.S.A., 1950, 1966.
[50]
D. Forsyth, J. L. Mundy and A. Zisserman, Transformation invariance--A primer, Image and Vision Computing, 10-1 (1992), 39-45.
[51]
D. Forsyth, J. L. Mundy, A. Zisserman, C. Coelho, A. Heller and C. Rothwell, Invariant descriptors for 3-D object recognition and pose, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-10 (1991), 971-991.
488
Bibliography
[52]
J. E. Freund, Mathematical Statistics, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1962.
[53]
J. E. Freund, Modern Elementary Statistics, 3rd Ed., Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1967.
[54]
A. Gelb (ed.), Applied Optimal Estimation, MIT Press, Cambridge, MA, U.S.A., 1974.
[55]
R. Gnanadesikan, Methods for Statistical Data Analysis of Multivariate Observations, John Wiley and Sons, New York, NY, U.S.A., 1977.
[56]
W. I. Grosky and L. A. Tamburino, A unified approach to the linear camera calibration problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-7 (1990), 663-671.
[57]
R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, Vols. 1, 2, Addison-Wesley, Reading, MA, U.S.A., 1992.
[ss]
J. C. Hay, Optical motions and space perception--An extension of Gibson's analysis, Psychological Review, 73-6 (1966), 550-565.
[59]
G. Healey and R. Kondepudy, Radiometric CCD camera calibration and noise estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 267-275.
[6o]
D. J. Heeger and A. D. Jepson, Subspace methods for recovering rigid motion I: Algorithm and implementation, International Journal of Computer Vision, 7-2 (1992), 95-117.
[61]
Y. Hel-Or and M. Werman, Pose estimation by fusing noisy data of different dimensions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17- 2 (1995), 195-201.
[62]
P. G. Hoel, Elementary Statistics, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1966.
[63]
P. G. Hoel, Introduction to Mathematical Statistics, 4th Ed., John Wiley and Sons, New York, NY, U.S.A., 1971.
[64]
R. Horaud, R. Mohr and B. Lorecki, On single-scanline camera calibration, IEEE Transactions on Robotics and Automation, 9-1 (1993), 71-74.
[65] [66]
B. K. P. Horn, Robot Vision, MIT Press, Cambridge, MA, U.S.A., 1986.
[67]
B. K. P. Horn, Closed-form solution of absolute orientation using unit quaternions, Journal of the Optical Society of America, A4-4 (1987), 629-642.
[6s]
B. K. P. Horn, Relative orientation, International Journal of Computer Vision, 4-1 (1990), 59-78.
[69]
B. K. P. Horn, H. M. Hilden and S. Negahdaripour, Closed-form solution of absolute orientation using orthonormal matrices, Journal of the Optical Society of America, A5-5 (1988), 1127-1135.
[7o]
B. K. P. Horn and B. Schunck, Determining optical flow, Artificial Intelligence, 17-1/2/3 (1981), 185-203.
B. K. P. Horn, Motion fields are hardly ever ambiguous, International Journal of Computer Vision, 1-3 (1987), 259-274.
Bibliography
489
[71]
X. Hu and N. Ahuja, Motion and structure estimation using long sequence motion models, Image and Vision Computing, 11-9 (1993), 549-569.
[72]
X. Hu and N. Ahuja, Necessary and sufficient conditions for a unique solution of planar motion and structure, IEEE Transactions on Robotics and Automation, 11-2 (1995), 304-308.
[73]
T. S. Huang and O. D. Faugeras, Some properties of the E matrix in twoview motion estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-12 (1989), 1310-1312.
[74]
A. H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, New York, NY, U.S.A., 1970.
[75]
C. Jerian and R. Jain, Polynomial methods for structure from motion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-12 (1990), 1150-1165.
[76]
C. P. Jerian and R. Jain, Structure from Motion--A critical analysis of methods, IEEE Transactions on Systems, Man, and Cybernetics, 21-3 (1991), 572588.
[77]
J.-M. Jolion, P. Meer and S. Bataouche, Robust clustering with applications in computer vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-8 (1991), 791-802.
ITs]
R. E. Kalman, A new approach to linear filtering and prediction problems, Transactions of ASME, Journal of Basic Engineering, 82D-1 (1960), 35-45.
[79]
R. E. Kalman and R. S. Bucy, New results in linear filtering and prediction theory, Transactions of ASME, Journal of Basic Engineering, 83D-1 (1961), 95-108.
[so]
B. Kamgar-Parsi and R. D. Eastman, Calibration of a stereo system with small relative angles, Computer Vision, Graphics, and Image Processing, 511 (1990), 1-19.
[Sl]
B. Kamgar-Parsi, B. Kamgar-Parsi and N. Netanyahu, A nonparametric method for fitting a straight line to a noisy image, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-9 (1989), 998-1001.
[82]
K. Kanatani, Distribution of directional data and fabric tensors, International Journal of Engineering Science, 22-2 (1984), 149-164.
Is3]
K. Kanatani, Structure and motion from optical flow under orthographic projection, Computer Vision, Graphics, and Image Processing, 35-2 (1986), 181199.
Is4]
K. Kanatani, Structure and motion from optical flow under perspective projection, Computer Vision, Graphics, and Image Processing, 38-2 (1987), 122146.
[ss]
K. Kanatani, Group-Theoretical Methods in Image Understanding, Springer, Berlin, F.R.G., 1990.
Is6]
K. Kanatani, Computational projective geometry, CVGIP: Image Understanding, 54-3 (1991), 333-348.
490
Bibliography
[87]
K. Kanatani, Hypothesizing and testing geometric properties of image data, CVGIP: Image Understanding, 54-3 (1991), 349-357.
[ss]
K. Kanatani, Statistical analysis of focal length calibration using vanishing points, IEEE Transactions on Robotics and Automation, 8-6, (1992), 767775.
IS9]
K. Kanatani, Unbiased estimation and statistical analysis of 3-D rigid motion from two views, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-1 (1993), 37-50.
[90]
K. Kanatani, Geometric Computation for Machine Vision, Oxford University Press, Oxford, U.K., 1993.
[91]
K. Kanatani, Statistical analysis of geometric computation, CVGIP: Image Understanding, 59-3 (1994), 286-306.
[92]
K. Kanatani, Statistical bias of conic fitting and renormalization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 320-326.
[93]
K. Kanatani, Analysis of 3-D rotation fitting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-5 (1994), 543-549.
[941
K. Kanatani, Statistical foundation for hypothesis testing of image data, CVGIP: Image Understanding, 60-2 (1994), 382-391.
[95]
K. Kanatani, Computational cross ratio for computer vision, CVGIP: Image Understanding, 60-2 (1994), 371-381.
[96]
K. Kanatani, Renormalization for motion analysis: Statistically optimal algorithm, IEICE Transactions on Information and Systems, E77-D-11 (1994), 1233-1239.
[971 K. Kanatani and W. Liu, 3-D interpretation of conics and orthogonality, CVGIP: Image Understanding, 58-3 (1993), 286-301.
[9s]
K. Kanatani and Y. Onodera, Anatomy of camera calibration using vanishing points, IEICE Transactions on Information and Systems, E74-10 (1991), 3369-3378.
[99]
K. Kanatani and S. Takeda, 3-D motion analysis of a planar surface by renormalization, IEICE Transactions on Information and Systems, E78-D-8 (1995), 1074-1079.
[~00]
Y. Kanazawa and K. Kanatani, Direct reconstruction of planar surfaces by stereo vision, IEICE Transactions on Information and Systems, E78-D-7 (1995), 917-922.
[101] Y. Kanazawa and K. Kanatani, Reliability of 3-D reconstruction by stereo vision, IEICE Transactions on Information and Systems, E78-D-10 (1995), 1301-1306. [102] Y. Kanazawa and K. Kanatani, Reliability of fitting a plane to range data, IEICE Transactions on Information and Systems, E78-D-12 (1995), 16301635.
[103] A.
Kara, D. M. Wilkes and K. Kawamura, 3D structure reconstruction from point correspondences between two perspective projections, CVGIP: Image Understanding, 60-3 (1994), 392-387.
Bibliography
491
[104] M. G. Kendall and A. Stuart, The Advanced Theory of Statistics, Vols. 1 (3rd Ed.), 2 (3rd Ed.), 3 (2nd Ed.), Griffin, London, U.K., 1969, 1973, 1968. [105] Y. C. Kim and J. K. Aggarwal, Determining object motion in a sequence of stereo images, IEEE Journal of Robotics and Automation, 3-6 (1987), 599614. [106] N. Kiryati and A. M. Bruckstein, What's in a set of points?, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-4 (1992), 496-500. [107] E. Kreyszig, Introductory Mathematical Statistics: Principle and Methods, John Wiley and Sons, New York, NY. U.S.A., 1970. [108] R. Kumar, Robust methods for estimating pose and a sensitivity analysis, CVGIP: Image Understanding, 60-3 (1994), 313-432. [109] M. Kumon and S. Amari, Estimation of a structural parameter in the presence of a large number of nuisance parameters, Biometrika, 71-3 (1984), 445-459. [110] J. Z. C. Lai, On the sensitivity of camera calibration, Image and Vision Computing, 11-10 (1993), 656-664. [111] S. M. LaValle and S. A. Hutchinson, A Bayesian segmentation methodology for parametric image models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17-2 (1995), 211-217. [112] C.-H. Lee, Time-varying images: The effect of finite resolution on uniqueness, CVGIP: Image Understanding, 54-3 (1991), 325-332. [113] C.-H. Lee, Computing three-dimensional motion parameters: A hypothesis testing approach, Image and Vision Computing, 11-3 (1993), 145-154. [114] S. Lee and Y. Kay, A Kalman filter approach for accurate 3-D motion estimation from sequence of stereo images, CVGIP: Image Understanding, 54-2 (1991), 244-258. [115] E. L. Lehman, Testing Statistical Hypotheses, 2nd Ed., John Wiley and Sons, New York, NY. U.S.A., 1986. [116] R. K. Lenz and R. Y. Tsai, Techniques for calibration of the scale factor and image center for high-accuracy 3-D machine vision metrology, IEEE Transactions on Pattern Analysis and Machine Intelligence, 10-5 (1988), 713-720. [117] R. K. Lenz and R. Y. Tsai, Calibrating a Cartesian robot with eye-on-hand configuration independent of eye-to-hand relationship, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-9 (1989), 916-928. [118] L. Li and J. H. Duncan, 3-D translational motion and structure from binocular image flow, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-7 (1993), 657-667. [119] B. G. Lindsay, Conditional score functions: Biometrika, 69-3 (1982), 503-512.
Some optimality results,
[120] B. G. Lindsay, Using empirical partially Bayes inference for increased efficiency, Annals of Statistics, 13-3 (1985), 914-931. [121] W. Liu and K. Kanatani, Interpretation of conic motion and its applications, International Journal of Computer Vision, 10-1 (1993), 67-84.
492
Bibliography
[122] H. C. Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, 293-10 (1981), 133-135. [123] H. C. Longuet-Higgins, The reconstruction of a scene from two projections-Configurations that defeat the 8-point algorithm, Proceedings of the First IEEE Conference on Artificial Intelligence Applications, Denver, CO, U.S.A., December 1984, pp. 395-397. [124] H. C. Longuet-Higgins, The visual ambiguity of a moving plane, Proceedings of the Royal Society of London, B223 (1984), 165-175. [125] H. C. Longuet-Higgins, The reconstruction of a plane surface from two perspective projections, Proceedings of the Royal Society of London, B227 (1986), 399-410. [126] H. C. Longuet-Higgins, Multiple interpretations of a pair of images of a surface, Proceedings of the Royal Society of London, A418 (1988), 1-15. [127] H. C. Longuet-Higgins and K. Prazdny, The interpretation of a moving retinal image, Proceedings of the Royal Society of London, B208 (1980), 385-397. [128] S. D. Ma, Conics-based stereo, motion estimation, and pose determination, International Journal of Computer Vision, 10-1 (1993), 7-25. [129] K. V. Mardia, Statistics of Directional Data, Academic Press, London, U.K., 1972. [130] D. Marr, Vision: A Computational Investigation into the Human Representation and Processin 9 of Visual Information, W. H. Freeman, San Francisco, CA, U.S.A., 1982. [131] L. Matthies, T. Kanade and R. Szeliski, Kalman filter-based algorithms for estimating depth from image sequences, International Journal of Computer Vision, 3-3 (1989), 209-236. [132] S. Maybank, The angular velocity associated with the optical flowfield arising from motion through rigid environment, Proceedings of the Royal Society of London, A401 (1985), 317-326. [133] S. J. Maybank, The projective geometry of ambiguous surface, Philosophical Transactions of the Royal Society of London, A332 (1990), 1-47. [134] S. J. Maybank, Ambiguity in reconstruction from image correspondences, Image and Vision Computing, 9-2 (1991), 93-99. [135] S. Maybank, Theory of Reconstruction from Image Motion, Springer, Berlin, F.R.G., 1993. [136] S. J. Maybank and O. D. Faugeras, A theory of self-calibration of a moving camera, International Journal of Computer Vision, 8-2 (1992), 123-151. [137] P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd edition, Chapman and Hall, London, U.K., 1989. [138] J. S. Meditch, Stochastic Optimal Linear Estimation and Control, McGrawHill, New York, NY, U.S.A., 1969. [139] P. Meet, D. Mintz and A. Rosenfeld, Robust regression methods for computer vision: A review, International Journal of Computer Vision, 6-1 (1991), 5970.
Bibliography
493
[140] J. Mendel, Lessons in Digital Estimation Theory, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1987. [141] E. De Micheli, V. Torre and S. Uras, The accuracy of the computation of optical flow and the recovery of motion parameters, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-5 (1993), 434-447. [1421 K. M. Mutch, Determining object translation information using stereoscopic motion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-6 (1986), 750-755. [143] M. K. Murray and J. W. Rice, Differential Geometry and Statistics, Chapman and Hall, London, U.K., 1993. [144] H.-H. Nagel, Representation of moving rigid objects based on visual observations, Computer, 14-8 (1981), 29-39. [145] Y. Nakagawa and A. Rosenfeld, A note on polygonal and elliptical approximation of mechanical parts, Pattern Recognition, 11-2 (1979), 133-142. [146] S. Negahdaripour, Critical surface pairs and triplets, International Journal of Computer Vision, 3-4 (1989), 293-312. [147] S. Negahdaripour, Multiple interpretations of the shape and motion of objects from two perspective images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-11 (1990), 1025-1030. [148] S. Negahdaripour, Closed-form relationship between the two interpretations of a moving plane, Journal of the Optical Society of America, A7-2 (1990), 279-285. [149] S. Negahdaripour and S. Lee, Motion recovery form image sequences using only first order optical flow information, International Journal of Computer Vision, 9-3 (1992), 163-184.
[:50]
A. N. Netravali, T. S. Huang, A. S. Krishnakumar and R. J. Holt, Algebraic methods in 3-D motion estimation from two-view point correspondences, International Journal of Imaging Systems and Technology, 1-1 (1989), 78-99.
[1511 R. Nevatia, Machine Perception, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1982. [152] J. Neyman and E. L. Scott, Consistent estimates based on partially consistent observations, Econometrica, 16-1 (1948), 1-32.
[:sa]
Y. Nonmra, M. Sagara, H. Naruse and A. Ide, Simple calibration algorithm for high-distortion-lens camera, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-12 (1992), 1095-1099.
[154] N. Ohta and K. Kanatani, Optimal structure from motion algorithm for optical flow, IEICE Transactions on Information and Systems, E78-D-12 (1995), 1559-1566.
[:55]
K. A. Paton, Conic sections in chromosome analysis, Pattern Recognition, 2-1 (1970), 39-51.
[156] M. A. Penna, Camera calibration: A quick and easy way to determine the scale factor, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1312 (1991), 1240-1245.
494
Bibliography
[157] J. Philip, Estimation of three-dimensional motion of rigid objects from noisy observations, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-1 (1991), 61-66. [158] T. Poggio and C. Koch, Ill-posed problems in early vision: From computational theory to analogue networks, Proceedings of the Royal Society of London, B226 (1985), 303-323.
[159]
T. Poggio, V. Torte and C. Koch, Computational vision and regularization theory, Nature, 317 (1985), 314-319.
[160]
R. Poli, G. Coppini and G. Valli, Recovery of 3D closed surfaces from sparse data, CVGIP: Image Understanding, 60-1 (1994), 1-25.
[1611
S. B. Pollard, J. Porrill and J. E. Mayhew, Experiments in vehicle control using predictive feed-forward stereo, Image and Vision Computing, 8-1 (1990), 6370.
[162]
S. B. Pollard, T. P. Pridmore, J. PorriU, J. E. Mayhew and J. P. Frisby, Geometric modeling from multiple stereo views, International Journal of Robotics Research, 8-4 (1989), 3-32.
[163] J. Porrill, Optimal combination and constraints for geometrical sensor data, International Journal of Robotics Research, 7-6 (1988), 66-77. [164] J. Porrill, Fitting ellipses and predicting confidence envelopes using a bias corrected Kalman filter, Image and Vision Computing, 8-1 (1990), 37-41. [165] J. Porrill, S. B. Pollard and J. E. W. Mayhew, Optimal combination of multiple sensors including stereo vision, Image and Vision Computing, 5-2 (1987), 174180.
[166]
T. Poston and I. Stewart, Catastrophe Theory and Its Applications, Pitman, London, U.K., 1978.
[167] V. Pratt, Direct least-squares fitting of algebraic surfaces, Computer Graphics, 21-4 (1987), 145-152.
[1681
K. Prazdny, Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer, Computer Graphics and Image Processing, 17-3 (1981), 238-248.
[1691
L. Quan, P. Gros and R. Mohr, Invariants of a pair of conics revisited, Image and Vision Computing, 10-5 (1992), 319-323.
[1701 C. R. Rao, Linear Statistical Inference and Its Applications, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1973. [1711 C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Applications, John Wiley and Sons, New York, NY, U.S.A., 1971. [1721 J. H. Rieger and D. T. Lawton, Processing differential image motion, Journal of the Optical Society of America, A2-2 (1985), 354-360. [173] J. Rissanen, Modeling by shortest data description, Automatica, 14-5 (1978), 465-471.
[174]
J. Rissanen, A universal prior for integers and estimation by minimum description length, Annals of Statistics, 11-3 (1983), 416-431.
Bibliography
495
[175] J. Rissanen, Universal coding, information, prediction, and estimation, IEEE Transactions on Information Theory, 30-4 (1984), 629-636. [1761 J. Rissanen, Stochastic Complexity in Statistical Inquiry, World Scientific, Singapore, 1987. [177] J. W. Roach and J. K. Aggarwal, Determining the movement of objects from a sequence of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2-6 (1980), 544-562. [1781 C. A. Rothwell, A. Zisserman, C. I. Marinos, D. A. Forsyth and J. L. Mundy, Relative motion and pose from arbitrary plane curves, Image and Vision Computing, 10-4 (1992), 250-262. [1791 R. Safaee-Rad, I. Tchoukanov, B. Benhabib and K. C. Smith, Accurate parameter estimation of quadratic curves from grey-level images, CVGIP: Image Understanding, 54-2 (1991), 259-274. [lS0] R. Safaee-Rad, I. Tchoukanov, K. C. Smith and B. Benhabib, Constraints on quadratic-curved features under perspective projection, Image and Vision Computing, 19-8 (1992), 532-548. [1811 R. Safaee-Rad, I. Tchoukanov, K. C. Smith and B. Benhabib, Threedimensional location estimation of circular features for machine vision, IEEE Transactions on Robotics and Automation, 8-5 (1992), 624-640. [182] P. D. Sampson, Fitting conic sections to "very scattered" data: An iterative refinement of the Bookstein algorithm, Computer Graphics and Image Processing, 18-1 (1982), 97-108. [183] J. G. Semple and G. T. Kneebone, Algebraic Projective Geometry, Clarendon Press, Oxford, U.K., 1952 (reprinted 1979). [184] J. G. Scruple and L. Roth, Introduction to Algebraic Geometry, Clarendon Press, Oxford, U.K., 1949 (reprinted 1987). [1851 Y. Shirai, Three-Dimensional Computer Vision, Springer, Berlin, F.R.G., 1987. [186] S. Smith, Note on small angle approximations for stereo disparity, Image and Vision Computing, 11-6 (1993), 395-398. [187] M. A. Snyder, The precision of 3-D parameters in correspondence based techniques: The case of uniform translation motion in a rigid environment, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-5 (1989), 523528.
[1881
M. E. Spetsakis, A linear algorithm for point and line-based structure from motion, CVGIP: Image Understanding, 56-2 (1992), 230-241.
[189] M. Spetsakis and J. (Y.) Aloimonos, A multi-frame approach to visual motion perception, International Journal of Computer Vision, 6-3 (1991), 245-255. [190] M. E. Spetsakis and Y. Aloimonos, Optimal visual motion estimation: A note, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-9 (1992), 959-964. [191] M. Spetsakis, Models of statistical visual motion estimation, CVGIP: Image Understanding, 60-1 (1994), 300-312.
496
Bibliography
[192] M. Subbarao and A. M. Waxman, Closed form solution to image flow equations for a planar surface in motion. Computer Vision, Graphics, and Image Processing, 36-2/3 (1986), 208-228. [193] M. Subbarao, Interpretation of image flow: Rigid curved surfaces in motion, International Journal of Computer Vision, 2-1 (1988), 77-96. [194] M. Subbarao, Bounds on time-to-collision and rotational component from first-order derivatives of image flow, Computer Vision, Graphics, and Image Processing, 50-3 (1990), 329-341. [195] N. Tagawa, T. Toriu and T. Endoh, Un-biased linear algorithm for recovering three-dimensional motion from optical flow, IEICE Transactions on Information and Systems., E76-10 (1993), 1263-1275. [196] N. Tagawa, T. Toriu and T. Endoh, Estimation of 3-D motion from optical flow with unbiased objective function, IEICE Transactions on Information and Systems., E77-D-10 (1994), 1148-1161.
[1971
H. Takeda, C. Facchinetti and J.-C. Latombe, Planning the motions of a mobile robot in a sensory uncertainty field, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-19 (1994), 1002-1017.
[198] T. N. Tan, K. D. Baker and G. D. Sullivan, 3D structure and motion estimation from 2D image sequences, Image and Vision Computing, 11-4 (1993), 203-210.
[199] K. Tarabanis, R. Y. Tsai and D. S. Goodman, Calibration of a computer controlled robotic vision sensor with a zoom lens, CVGIP: Image Understanding, 59-2 (1994), 226-241.
[200]
G. Taubin, F. Cukierman, S. Sullivan, J. Ponce and D. J. Kriegman, Parameterized families for bounded algebraic curve and surface fitting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 287-303.
[201] R. Thorn, Stabilitd Structurelle et Morphogdn~se, Benjamin, New York, NY, 1972.
[2021
J. I. Thomas, Refining 3D reconstructions: A theoretical and experimental study of the effect of cross-correlations, CVGIP: Image Understanding, 60-3 (1994), 359-370.
[203]
C. Tomasi and T. Kanade, Shape and motion from image streams under orthography--A factorization method, International Journal of Computer Vision, 9-2 (1992), 137-154.
[204]
P. H. S. Torr and D. W. Murray, Statistical detection of independent movement from a moving camera, Image and Vision Computing, 11-4 (1993), 180187.
[205]
H. P. Trivedi, Estimation of stereo and motion parameters using a variational principle, Image and Vision Computing, 5-2 (1987), 181-183.
[206]
R. Y. Tsai and T. S. Huang, Estimating three-dimensional motion parameters of a rigid planar patch, IEEE Transactions on Acoustics, Speech, and Signal Processing, 29-6 (1981), 1147-1152.
Bibliography
497
[207]
R. Y. Tsai and T. S. Huang, Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces, I E E E Transactions on Pattern Analysis and Machine Intelligence, 6-1 (1984), 13-27.
[208]
T. Y. Tsai and T. S. Huang, Estimating 3-D motion parameters of a rigid planar patch III. Finite point correspondences and the three-view problem, I E E E Transactions on Acoustics, Speech, and Signal Processing, 32-2 (1984), 213-220.
[209] R. Y. Tsai, T. S. Huang and W.-L. Zhu, Estimating three-dimensional motion parameters of a rigid planar patch, II: Singular value decomposition, I E E E Transactions on Acoustics, Speech, and Signal Processing, 30-3 (1982), 525534. [210] R. Y. Tsai and R. K. Lenz, A new technique for fully autonomous and efficient 3D robotics hand/eye calibration, I E E E Transactions on Robotics and Automation, 5-3 (1989), 345-358. [211] S. Ullman, The Interpretation of Visual Motion, MIT Press, Cambridge, MA, U.S.A., 1979. [212] S. Umeyama, Least-squares estimation of transformation parameters between two point patterns, I E E E Transactions on Pattern Analysis and Machine Intelligence, 13-4 (1991), 376-380. [213] T. Vi~ville, Auto-calibration of visual sensor parameters on a robotic head, Image and Vision Computing, 12-4 (1994), 227-237. [214] C.-C. Wang, Extrinsic calibration of a vision sensor mounted on a robot, IEEE Transactions on Robotics and Automation, 8-2 (1992), 161-175. [215] L.-L. Wang and W.-H. Tsai, Camera calibration by vanishing lines for 3-D computer vision, I E E E Transactions on Pattern Analysis and Machine Intelligence, 13-4 (1991), 370-376.
[2~6]
A. M. Waxman and J. H. Duncan, Binocular image flows: Steps toward stereomotion fusion, I E E E Transactions on Pattern Analysis and Machine Intelligence, 8-6 (1986), 715-729.
[217] A. M. Waxman, B. Kamgar-Parsi and M. Subbarao, Closed-form solutions to image flow equations for 3D structure and motion, International Journal of Computer Vision, 1-3 (1987), 239-258. [218] G.-Q. Wei and S. D. Ma, Implicit and explicit camera calibration: Theory and experiments, I E E E Transactions on Pattern Analysis and Machine Intelligence, 16-5 (1994), 469-480. [2191 I. Weiss, Line fitting in a noisy image, I E E E Transactions on Pattern Analysis and Machine Intelligence, 11-3 (1989), 325-329. [220] R. Weiss, H. Nakatani and E. M. Riseman, An error analysis for surface orientation from vanishing points, I E E E Transactions on Pattern Analysis and Machine Intelligence, 12-12 (1990), 1179-1185.
[221]
J. Weng, N. Ahuja and T. S. Huang, Motion and structure from point correspondences with error estimation: Planar surfaces, I E E E Transactions on Signal Processing, 39-12 (1991), 2691-2717.
498
Bibliography
[2221 J. Weng, N. Ahuja and T. S. Huang, Optimal motion and structure estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-9 (1993), 864-884. [2231 J. Weng, P. Cohen and M. Herniou, Camera calibration with distortion models and accuracy evaluation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-10 (1992), 965-980. [2241 J. Weng, P. Cohen and N. Rebibo, Motion and structure estimation from stereo image sequences, IEEE Transactions on Robotics and Automation, 8-3 (1992), 362-382.
[22~]
J. Weng, T. S. Huang and N. Ahuja, 3-D motion estimation, understanding, and prediction from noisy image sequences, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-3 (1987), 370-389.
[2261 J. Weng, T. S. Huang and N. Ahuja, Motion and structure from two perspective views: Algorithms, error analysis, and error estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-5 (1989), 451-467. [2271 J. Weng, T. S. Huang and N. Ahuja, Motion and Structure from Image Sequences, Springer, Berlin, F.R.G., 1993.
[2281
M. Werman and Z. Greyzel, Fitting a second degree curve in the presence of error, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17-2 (1995), 207-211.
[229]
P. H. Winston (ed), Psychology of Computer Vision, McGraw-Hill, New York, NY, U.S.A., 1975.
[230] T. H. Wonnacott and R. J. Wonnacott, Introductory Statistics, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1972. [2311 J. J. Wu, R. E. Rink, T. M. Caelli and V. G. Gourishankar, Recovery of the 3-D location and motion of a rigid object through camera image (an extended Kalman filter approach), International Journal of Computer Vision, 2-4 (1989), 373-394. [2321 Y. Yasumoto and G. Medioni, Robust estimation of three-dimensional motion parameters from a sequence of image frames using regularization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-4 (1986), 464471. [233] G.-S. J. Young and R. Chellappa, Statistical analysis of inherent ambiguities in recovering 3-D motion from a noisy flow field, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-10 (1992), 995-1013. [234] Z. Zhang and O. D. Faugeras, Three-dimensional motion computation and object segmentation in a long sequence of stereo frames, International Journal of Computer Vision, 7-3 (1992), 211-241. [2351 Y. T. Zhou, V. Venkateswar and R. Chellappa, Edge detection and linear feature extraction using a 2-D random field model, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-1 (1989), 84-95. [236] X. Zhuang, A simplification to linear two-view motion algorithm, Computer Vision, Graphics, and Image Processing, 46-2 (1989), 175-178.
Bibliography
499
[237] X. Zhuang, T. S. Huang, N. Ahuja and R. M. Haralick, A simplified linear optical flow-motion algorithm, Computer Vision, Graphics, and Image Processing, 42-3 (1988), 334-344. [238] X. Zhuang, T. S. Huang and R. M. Haralick, Two-view motion analysis: A unified algorithm, Journal of the Optical Society of America, A3-9 (1986), 1492-1500.
14.1
Statistical Estimation in Engineering
The subject of this chapter is essentially statistical estimation. However, the following treatment is very different from traditional statistics, whose main objective is to infer the structure of a random phenomenon by observing multiple data with a view to evaluating and comparing effects and procedures in domains that involve a large degree of uncertainty, such as medicine, biology, agriculture, manufacturing, sociology, economics, and politics. In such a domain, the problem is usually translated into the mathematical language as estimating parameters involved in the probability distribution from multiple independent samples from it (see Section 3.5 for the classical results). Although this framework covers almost all types of relevant applications in the above mentioned domains, geometric estimation problems in computer vision and robotics have many non-traditional elements. In traditional statistics, errors are regarded as uncontrollable; the accuracy of estimation is improved only by repeated measurements. However, repeating measurements is costly. Hence, if the accuracy is the same, those methods which require a smaller number of data are more desirable. In other words, methods whose accuracy improves rapidly as the number of data increases are more desirable than those with slow increase of accuracy. Thus, the study of asymptotic properties of estimation in the limit of a large number of data has been one of the central subjects in traditional statistics (see Section 3.6.2 for the classical results). In such engineering domains as computer vision and robotics, where electronic sensing devices are used, errors are usually small and called noise. 451
452
Chapter 14.
General Theory of Geometric Estimation
Moreover, they are controllable" the accuracy of sensing can be improved by using high-resolution devices and controlling the environment (lighting, dust, temperature, humidity, vibration, etc.). However, such control is costly. Hence, if the accuracy is the same, those methods which tolerate a higher level of noise are more desirable. In other words, methods whose accuracy improves rapidly as the noise level decreases are more desirable than those with slow increase of accuracy. Thus, the study of the accuracy of estimation in the l i m i t of s m a l l noise is very important. In this sense, our approach of assuming local distributions and applying linear analysis, which has been the basic principle in the preceding chapters, can be justified. In many engineering domains, repeating measurements under the same conditions (which is easy) often produces the same results because the sources of inaccuracy in the device and the environment are fixed (but unknown). Also, the number of independent data is usually fixed. Hence, the basic premise of traditional statistics that i n d e p e n d e n t s a m p l e s f r o m the s a m e dist r i b u t i o n can be observed as m a n y t i m e s as desired does not hold. This observation also underlies the statistical treatment in the preceding chapters.
14.2
General Definition
Geometric
Correction
of the problem
Let F ( k ) ( u ) , k = 1, ..., L, be continuously differentiable scalar functions of argument u E 7~". The domain of u is assumed to be an n~-dimensional manifold g / C ~ , which we call the data space. G e o m e t r i c correction is the problem of correcting a given datum u to such a value/t that satisfies the L equations F (k) (it) - O,
k - 1, ..., L.
(14.1)
In Chapter 5.1, we considered N variables u 1 , . . . , n N, but the problem becomes equivalent if we regard their direct sum u - u l | | U N, n -nl ~ - ' . " + rig, as a new variable. As in Chapter 5.1, we call the L equations (14.1) simply the constraint, as opposed to which we call the constraint u E U the i n h e r e n t constraint on u. Let fi E U be the true value of u, which we assume is unknown. We regard the datum u as a random variable with probability density p ( u ; ~t) parameterized by ~. We assume that the distribution of u in the data space b/is local (see Section 3.2.2) and define the "true value" ~ by the relationship given by eq. (3.29). However, such an explicit characterization of fi will not be used in the subsequent analysis, so it can be thought of simply as s o m e value in U. The geometric correction problem is formally stated as follows (see Section 5.1.1):
14.2.
General Geometric Correction
453
P r o b l e m 14.1 From a given datum u E ld, estimate the true value ~t E ld that satisfies the constraint F (k) (~) - 0,
k - 1, ..., L.
(14.2)
As in Sections 3.5 and 3.6, the probability density p(u; ~t) is assumed to satisfy the following regularity condition: A s s u m p t i o n 14.1 The probability density p(u; ~t) is continuously differentiable with respect to u and s an arbitrary number of times, and
p(u; ~) > 0
(14.3)
for all u E hi. Furthermore, the integration operation fu du for any expression of p(u; ~t) with respect to u is, if the integration exists, interchangeable with the differentiation operation V~t with respect to ~t. The probability density p(u; ~t) is not defined for ~t ~/d. For the convenience of analysis, however, we extend it to ~ ~ / 4 in such a way that +
= ;(u;
+
(14.4)
for all ~ E b / a n d A ~ E Tu (b/) • where T~t (/4) C ~'~ is the tangent space to the manifold/d at ~t (see Section 3.5.1). In precise terms, this assumption is stated as follows (see eq. (3.112)): Assumption
14.2
yap e
14.2.2
(14.5)
T h e r a n k o f the c o n s t r a i n t
The L equations (14.1) may be algebraically dependent. If only r of them are independent, they give only r effective constraints and define an (n ~ - r)dimensional manifold $, or a manifold $ of codimension r, in the data space b/except at possible singular points. We assume that ~t E $ is not a singular point and call r the rank of the constraint (14.1). A precise statement of this assumption, which also gives a precise definition of the rank r, is as follows (see Section 7.1.1): A s s u m p t i o n 14.3 The L equations of the constraint define a manifold $ of codimension r in the data space ld in the neighborhood of ~t. As in the preceding chapter, let us call the manifold $ the (geometric) model of Problem 14.1. The solution is given by the projection of the datum u E ?d onto the model 8 C L/(Fig. 14.1).
454
Chapter 14.
General Theory of Geometric Estimation
Rn
Fig. 14.19 Projecting u E U onto the model S C/4. If we write u = ~ + Au, we have F <1)(~ + Au) / 9
( __
) ~7/~(1),--',
~7/~(L)
T Au
(14.6)
F (L) (~ + Au)
to a first approximation in Au, where and hereafter V F (k) (~) is abbreviated as V/~(k). The inherent constraint u E U requires that A u E Tu(U) to a first approximation. If A u ranges over the entire tangent space Tu(/4), eq. (14.6) defines a linear mapping from Tu(b/) to a linear subspace
- {
/
9
zx.)/
IA. e Tu(U)}
c n
The dimension of this subspace equals the dimension of the subspace {P~V/~(1) , ..., P-UuV F (L ) } L C ,]p~,n,
(14.8)
where P ~ be the n-dimensional projection matrix onto the tangent space Tu(U). Consequently, the dimension of s is at most the rank r of the constraint (14.1), but we further assume the following1: A s s u m p t i o n 14.4 The dimension of the linear subspace s rank r of the constraint.
is equal to the
In mathematical terms, Assumption 14.4 states that each of the L equations (14.1) defines a manifold of codimension 1 in the data space/J such that they intersect each other transversally at ~ (see Section 3.2.1). We say that the 1As a result, the L equations (14.1) cannot be replaced by a single equation, say E L F(k)(u) 2 = O, which still has rank r but s = {0} k--1
14.2.
General Geometric Correction
455
intersection of the L manifolds is a singular value of the constraint (14.1) if the dimension of the subspace L:~t is less than r, and an ordinary value otherwise. Assumption 14.4 requires that the true value ~ should not be a singular value. We say that the constraint (14.1) is nonsingular if Assumption 14.4 is satisfied, and singular otherwise (see Section 5.1.1). The linear subspace L:~t defined by eq. (14.7) can alternatively be defined as the tangent space to the manifold
F 1)(u) / " lu~}eT~
7-{
L
(14.9)
F(L)(u) at a
E td.
However,
a
can be a singular point
of the mapping
(F(I)(u),...,F(L)(u)) T" 5t -~ 7~L. In other words, the linear subspace L:u defined by replacing ~ by u (~ ~) in eq. (14.7) may have a higher dimension than r. If that is the case, we say that the constraint is degenerate (see Section 5.1.1).
14.2.3
Cramer-Rao lower bound for geometric correction
Let l be the score of u with respect to ~ (see eq. (3.113))"
- Vu logp.
(14.10)
The Fisher information matrix J with respect to ~ is defined as follows (see eq. (3.118))" J - E[HT]. (14.11) The symbol E[.] denotes the expectation with respect to the probability density p(u; ~t). Assumption 14.2 implies that I E T~t(b/). We assume that the distribution p(u; ~t) is regular with respect to ~t in the sense that I takes all orientations in T~t (5/) as u ranges over the entire data space L/(see Section 3.5.1). In precise terms, this assumption is stated as follows: 14.5 The Fisher information matrix J is positive semi-definite and has range Tu (l~).
Assumption
Define an (LL)-matrix 12 - (12(kl)) by (12 (kt)) - ((V/~(a), J - V / ~ ( t ) ) ) ,
(14.12)
and an (LL)-matrix I?V - (I?V(kl)) by I?V - V - , or in the notation used in Chapter 5 (l~ (kl)) - ((V/~(k), J - V / ~ ( l ) ) ) - .
(14.13)
456
Chapter 14.
General Theory of Geometric Estimation
Define (nn)-matrices N and S by L
(14.14) k,l=l
S-J--N.
(14.15)
L e t / t - / t ( u ) be an unbiased estimator of the true value ~t determined as a function of the d a t u m u. Since r is constrained to be in the model $, it must s a t i s f y / t ( u ) E $ for any u E/4. Its unbiasedness is defined by
psE[it - ~ ] - 0, where T~t($) of/t is matrix
(14.16)
p S is the n-dimensional projection matrix onto the tangent space to the model $ at fi (see eq. (3.126)). Assuming that the distribution sufficiently localized around ~ in the model $, we define the covariance o f / t in the following form (see eq. (3.115))" V[/t] - psE[(it - g)(/t - ~t)T]pS.
(14.17)
We now prove the following theorem 2" Theorem
14.1
s.
(14.1s)
Since this is an analogue of the Cramer-Rao inequality (3.134) in Section 3.5.2, we call eq. (14.18) and the bound it gives the Cramer-Rao inequality and the Cramer-Rao lower bound, respectively, for geometric correction.
1~.2.~
Proof of the main theorem
L e m m a 14.1 The (LL)-matrix V is positive semi-definite and has range f-.~t.
Proof. Let rI - (r/k) be an arbitrary L-vector. We have L
L
(17, V~7) - (~-~, rlkVF(k),J - ~ qlV/~(l)) 9 k=l
(14.19)
/=1
Since J is positive semi-definite, its generalized inverse J - is also positive semi-definite. Hence, the right-hand side of eq. (14.19) is nonnegative for an arbitrary r/, meaning that V is positive semi-definite. Suppose rl belongs to the null space of V. Then, the left-hand side of eq. (14.19) is 0. It follows that ~-~L which is identical with the k--1 ~k~Tt (k) belongs to the null space of J 2Recall that the relation A _ B means that matrix A - B is positive semi-definite (see Section 3.5.2).
14.2.
General Geometric Correction
457
null space of J. Assumption 14.5 implies that the null space of J coincides with T~t(L/) • Hence, L
E
~k(V/~(k)' Au) - 0
(14.20)
k=l
for an arbitrary Au E T~t(L/). From the definition of the subspace s this means that r/E s Conversely, if r/E s we obtain (r/, V~7) - 0 by following eqs. (14.20) and (14.19) backward. Hence, the null space of the (LL)-matrix V coincides with_ the ( L - r)-dimensional subspace L:~t C 7r L. Consequently, the range of V coincides with s [::1 _
Consider the (LL)-matrix W defined by eq. (14.13). Let P ~ be the Ldimensional projection matrix onto/:~t. Lemma 14.1 implies the following: C o r o l l a r y 14.1 The ( L L ) - m a t r i x W is positive semi-definite and has range l?,u, and the following relationship holds: VW-
P~.
(14.21)
NJN-
N.
(14.22)
L e m m a 14.2
Proof. Since J - J J NJN
- J - , we see that
=
m,n=l L
-
z k,l,rn,n= l
k,n=l
l
--1
)
L
- N,
(14.23)
k,n=l
where the identity 1 7 V V W - IZVW W - I;V is used. L e m m a 14.3
SJS-
S.
(14.24)
458
Chapter 14.
General Theory of Geometric Estimation
Fig. 14.2. The tangent space Tu(S) to model S at fi and its orthogonal complement with respect to Tu(b/). Proof. Since J - J J SJS
- J - , we see that
- (J- - N)J(J= J- - P~N-
- N) - J-J J- - J-JN NP~
+ N - J-
- N J J- + NJN
(14.25)
- N - S,
where the relationship J - J - J J - - P ~ implied by Assumption 14.5 and the relationship P ~ N - N P ~ - N implied by eq. (14.14)are used. El The following lemma plays an essential role in proving Theorem 14.1. L e m m a 14.4 The ( n n ) - m a t r i x S is positive semi-definite and has range
Tu (S). Proof. By definition, S is a positive semi-definite matrix. From Lemma 14.3, we see that for an arbitrary n-vector ~ we see that (~,S~) - (~,SJS~)
(14.26)
- (S~,J(S~)).
Since J is positive semi-definite, the above expression is nonnegative for any E T~". Hence, S is positive semi-definite. By definition, the tangent space T~t(8) is the orthogonal complement of the linear subspace { P - ~ V F ( k ) } L C T~t(L/) (Fig. 14.2). From eqs. (14.14) and (14.15), we see that
= (P~VF(k),J-P~VF(k)) L
-
E
17v(Im)(P~VF(k)'J-VF(t))(J-VF(m)'P~
V~(k))
l,m=l
= (V/~(k), P ~ J - P ~ V F ( k ) ) L
-
E l,m--1
I~('m)(v/~(k)' P g J - V / ~ ( O ) ( V / ~ ( m ) ' J - P ~
V~(k))
14.2.
General Geometric Correction
459
L ---(V~"(k)'J-VP(k))- E (V~"(k)'J-VF(l))~'r(lm)(vF(m)'J-VF(k)) l,m=l L = ?(aa) _ E
?(at)Vv(lm)?(mk) -- O,
(14.27)
l,m=l where the identity V W V - V V V - ~r is used. Eq. (14.27) implies that the linear subspace {PUV/~(k)}L is included in the null space of S. This means that the range of S is included in T u ( $ ) . Hence, the rank of S is at most n ~ - r. We now prove that the range of S exactly coincides with T u ( S ) by showing that S has at least rank n ~ - r. Let J be the linear subspace of 7~n obtained by mapping the r-dimensional subspace { p u V F ( k ) } L C Tu(ld) by J - . Assumption 14.5 implies that J has range Tu(b/), so Y is an r-dimensional subspace of Tu(b/). Let {vl, ..., v n , - . } be a basis of the orthogonal complement of ,7 with respect to Tu (/1/). Since the vectors vi, i - 1 n ~ - r, are orthogonal to vectors J - P U V f i ' ( k ) k - 1, ..., L, we obtain L
-
(14.28)
k,l--1 By Assumption 14.5, matrix J - is positive semi-definite and has range Tu(/d). Since vectors {vi } E Tu (b/), i - 1, ..., n ' - r , are linearly independent, vectors { J - v i } - {Svi}, i - 1, ..., n ' - r, are also linearly independent. Thus, S has at least rank n ~ - r. [] C o r o l l a r y 14.2 The variation A ~ of r defined by A~-
SA~
(14.29)
satisfies Aft E T u ( $ ) for any ~ E T4n. L e m m a 14.5
E [ P ~ ( i ~ - ~ ) ( S l ) T] -- S.
(14.30)
Proof. Since the estimator/~ is unbiased, eq. (14.16) holds identically for any E $. It follows that the first variation of the left-hand side of eq. (14.16) should be 0 for any A~ C T u ( $ ) . Hence, it should be 0 for A~ -- SA~ for any A~ E 7~~. Taking the first variation of the left-hand side of eq. (14.16) with respect to ~, substituting eq. (14.29) into it, noting Assumption 14.1, and using the logarithmic differentiation formula (3.115), we obtain A ( P ~ E [ / t - ~]) - - P S E [ A ~ ) ]
= P~SAf 1
: - {S\
+ P ~ .fTj(/t - ~)(V~t logp, A~t)pdu
- P ~ E [ ( i ~ - ~)iT]A~ \
E [ P S ( i ~ - ~(SI~T]~ Af. /k/J]
(14.31)
460
Chapter 14.
General Theory of Geometric Estimation
Here, we have used the assumption that PS+~uE[it - ~] ~ PSE[i~- ~t] (see Section 3.5.2) and the relationship PfftS = S. Since the above expression should be 0 for any A~ E 7~~, we obtain eq. (14.30). [] The proof of Theorem 14.1 is given as follows. Lemma 14.3, we see that
E[(SI)(Sl) T] - S E [ ~ T ] s -
From eq. (14.11) and
SJS-
S.
(14.32)
From this and Lemmas 14.5, we obtain E[
{ p S (/t_- ~t)
n S ( / t - fi)
T
I/t] S
\
S
"
(14.33)
This matrix is symmetric and positive semi-definite. If matrix A is positive semi-definite and symmetric, so is B T A B for any matrix B as long as the multiplication can be defined (see Section 2.2.3). Hence, the following matrix is positive semi-definite and symmetric:
- PPdS ) ( vb] S
sS ) (
_psPS P~ ) _ ( V [ / , ] - S
S)"
(14.34) Here, we have used the fact that matrices V[/t] and S share the same range Tu(S) and hence P s S = SP~t = S and pSv[it]Pfft = V[/t]. Since S is positive semi-definite, matrix V[/t] - S is also positive semi-definite. [] 14.3
14.3.
Maximum
Likelihood
Correction
M a x i m u m likelihood estimator
Maximum likelihood estimation for this problem is to choose fi that maximize the likelihood, i.e., the probability density p(u; ft) viewed as a function of ~ for a given datum u. The resulting estimate/t is called the maximum likelihood estimator of ~t (see Section 3.6.1). In order to distinguish "variables" from their specific "values", we regard the symbol ~ as the true value and use x when it is viewed as a variable. Since maximizing the likelihood is equivalent to minimizing its logarithm with opposite sign, the problem reduces to the minimization J =-21ogp(u;x)-+ min (14.35) with respect to x E S, i.e., under the constraint
F(k)(x) -- O,
k - 1, ...,L,
x e hi.
(14.36)
Define an (nn)-matrix i, by L -- - V 2 logplx= ~.
(14.37)
14.3.
Maximum Likelihood Correction
461
With the expectation that the maximum likelihood estimator /t is located near ~, we write x - ~t + Ax. (14.38) Substituting this into the function J in eq. (14.35) and expanding it in the neighborhood of ~, we obtain J - c - 2(1, Ax) + (Ax, L A x ) + O(Ax) 3,
(14.39)
where ~ is the value of J evaluated at ~. Assumption 14.2 implies that the rank of L is no more than the dimension n' of the tangent space Tx(14). In order to guarantee unique existence of the value x E b / t h a t minimizes J, we assume the following: A s s u m p t i o n 14.6 For any u E Lt, the (nn)-matrix L is positive semidefinite and has range T~t(bt). Substituting eq. (14.38) into eq. (14.36) and taking a linear approximation, we obtain (V/~(k), Ax) -- 0. (14.40) The inherent constraint x E/d requires that Ax E Tu(b/) to a first approximation. Introducing Lagrange multipliers ~(k) to eq. (14.40), differentiating J with respect to Ax, and ignoring higher order terms, we obtain the following linear equation in Ax" L
L A x -- 1 + E
A(k)PUV/~(k)"
(14.41)
k--1
By Assumptions 14.2 and 14.6, eq. (14.41) has the following unique solution (see Section 2.3.3): L
Ax -- L - 1 + E
A(k)s
(14.42)
k=l
Substituting eq. (14.42)into eq. (14.40), we obtain -
(14.43)
where we put s - (~(k)) and define the (LL)-matrix ? ' - (iF '(k0) by (lY'(kl))
-
((V/~(k), l , - V / ~ ( t ) ) ) .
(14.44)
Since matrix I, has range Tx(g4), matrix V~ can be shown to be positive semi-definite and have range s by exactly the same argument as given in
462
Chapter 14.
General Theory of Geometric Estimation
the proof of Lemma 14.6. Consequently, if u is an ordinary value (see Section 14.2.2), the solution of eq. (14.43) is uniquely given by L
~(k) = _ E
l/V'(kl)(V/~(l)' A u ) '
(14.45)
/--1
where 17V'(kl) is the (kl) element of the (LL)-matrix I ~ ' - V ' - , i.e., (ITV l(kl)) -- ((V/~(k), L--V/7(/))) - 9
(14.46)
Substituting eq. (14.45) into eq. (14.42), we can express the maximum likelihood estimator/t in the form /t - ~ + S'l,
(14.47)
where we define the (nn)-matrix S / by L
S'-
L--
E 17v'(kl)(L-V/~(k))(L-V/~(0) T" k,l=l
(14.48)
By the argument similar to that in the proofs of Lemmas 14.2 and 14.3, we can show that S~LS ~ = S ~. Hence, the (nn)-matrix S' can be shown to be positive semi-definite and have range Tu(,5) by the same argument as given in the proof of Lemma 14.4. Consequently, the covariance matrix of the maximum likelihood estimator/t has the following form: V[/,] - E[S'H TS'].
1~.3.2
(14.49)
Geometric correction of the exponential family
If the distribution of u belongs to the exponential family (see Section 3.6.1), the probability density p(u; ~t) has the following expression (see eq. (3.146)):
p(u; ft) - C(~t)exp[(J'(u), ~) + g(u)].
(14.50)
Here, f ( u ) is an n-dimensional function of u e 7~n, and C(u) and g(u) are scalar functions of u G 7~n. We call Problem 14.1 geometric correction of the exponential family if the probability density p(u; ~t) has the above expression. For geometric correction of the exponential family, we see from eqs. (14.37) and (14.50) that the matrix L does not depend on u. Hence, the following equality holds (see eq. (3.122)): J-
E[s
= I,.
From this, we obtain the following proposition:
(14.51)
14.3.
M a x i m u m Likelihood Correction
463
14.1 The maximum likelihood estimator it for geometric correction of the exponential family is unbiased in the first order.
Proposition
Proof. If eq. (14.51) holds, the matrix S ' defined by eq. (14.48) is a constant matrix. Since El1] : 0 (see eq. (3.117)), we see from eq. (14.47) that E [
-
:
S E [
] :
o.
(14.52)
An estimator is said to be efficient if its covariance matrix attains the Cramer-Rao lower bound (see Section 3.5.2). Extending this concept, we say that an estimator for geometric correction is e]ficient if its covariance matrix attains the Cramer-Rao lower bound given by Theorem 14.1. 14.2 The maximum likelihood estimator it for geometric correction of the exponential family is efficient in the first order.
Proposition
Proof. Since L - J , we see that V ' - V (see eqs. (14.12) and (14.44)) and hence I26' - W (see eqs. (14.13) and (14.46)). It follows that S' - S (see eqs. (14.15) and (14.48)). From eq. (14.49), we see that V[it]- SE[a7-]S- SJS-
S,
where we have used L e m m a 14.3.
(14.53) [:3
The proofs been obtained polynomial in approximation
of Propositions 14.1 an 14.2 are based on eq. (14.47), which has by approximating the function J in eq. (14.35) by a quadratic the neighborhood of their true values and applying a linear to the constraint (14.36). In other words, eq. (14.47) is an expression in the limit of small noise. Accordingly, Propositions 14.1 and 14.2 hold in this sense, which is what the phrase "in the first order" means. This type of first order analysis is compatible with our approach of studying statistical behavior for small noise (see Section 14.1).
1~.3.3
Computation of maximum likelihood correction
We now derive an analytical expression for the m a x i m u m likelihood estimat o r / t . First, we introduce a new assumption about the probability density p(u; x). We assume that viewed as a function of x it takes its m a x i m u m at u. In other words, given a single d a t u m u, the d a t u m u itself is the most likely value of the true value ~ if no other information is given. To be specific, what we need is the following" Assumption
14.7
Vxp(u; X)lx=u = 0.
(14.54)
464
Chapter 14.
General Theory of Geometric Estimation
For example, this assumption is satisfied if the probability density is expressed by a smooth scalar function f ( . ) that takes its maximum at the origin 0 in the form p(u; ~t) = f (u - ~t). The Gaussian distribution is a typical example. Assuming that the solution x of the optimization (14.35) is close to the datum u, we write x - u + Ax. Substituting this into the function J, expanding it in the neighborhood of the datum u, and using Assumption 14.7, we obtain J = c - (Ax, L A x ) + O ( A x ) 3. (14.55) The constant c is the value of J evaluated at u, and L is an (nn)-matrix defined by L
-Vx2 logplx=u.
-
(14.56)
In order to guarantee unique existence of the value x E L / t h a t minimizes J, we assume that Assumption 14.6 also applies to L" A s s u m p t i o n 14.8 For any u E 14, the (nn)-matrix L is positive semidefinite and has range Tu(N). The linear approximation of the constraint (14.36) is ( V F (k) , Ax) - - F (k) ,
(14.57)
where F (k) and V F (k) are the abbreviations of F(k)(u) and VF(k)(u), respectively. The inherent constraint x E L/requires that Ax E Tu(Lt) to a first approximation. Introducing Lagrange multipliers )~(k) to eq. (14.57), differentiating J with respect to Ax, and ignoring higher order terms, we obtain the solution Ax uniquely in the following form: L
-
(14.5s) k=l
Substituting this into eq. (14.57) and putting )~ - (A(k)), we obtain V)~ - - ( F ( k ) ) ,
(14.59)
where the (LL)-matrix V -- (V (kl)) is defined by
L-vF(')))
(14.60)
Here, a computational problem arises. If the L equations (14.36) are not algebraically independent but give only r constraints, the matrix V~ computed from the true value ~ by eq. (14.44) has rank r. However, the rank of the matrix V computed from the datum u by eq. (14.60) may be larger than r. In other words, among the L linear equations in Ax given by eq. (14.57), more than r equations may be linearly independent if evaluated at u r ft. If this
14.3.
Maximum Likelihood Correction
465
Fig. 14.3. The corrected value fi is in Tu(b/) but may not be in L/. Hence, a higher order correction is necessary. occurs, the constraint (14.36) is said to be degenerate (see Section 14.2.2). If the constraint (14.36) is degenerate, some of the positive eigenvalues of matrix V converge to 0 in the limit u --+ ~. Hence, eq. (14.59) viewed as a linear equation in ,k is ill-conditioned when the datum u is close to the true value s (see Section 2.3.2). In order to avoid this ill-posedness, we project both sides of eq. (14.59) onto the eigenspace defined by the largest r eigenvalues of V. In other words, we effectively use only r constraints from among the L linear equations (14.57). The resulting solution may be slightly different from the exact solution of eq. (14.59) but does not effectively affect the subsequent analysis because eq. (14.59) is a first order approximation. The solution of eq. (14.59) given by this projection is L
)~(k) = _ Z
W(kl)F(l)'
(14.61)
/=1
where W (kl) is the (kl) element of the rank-constrained generalized inverse (V)~- (see eq. (2.132)), which we write (14.62) Substituting eq. (14.61) into eq. (14.58), we obtain the following conclusion: P r o p o s i t i o n 14.3 The maximum likelihood estimator it is computed to a first approximation in the form L
it - u -
L- Z
W(kt)F(k)VF(l)"
(14.63)
k,l=l
This gives only a first order solution under the linearized inherent constraint i t - u E Tu(lg) (Fig. 14.3). In actual computation, we need to projected
466
Chapter 14.
General Theory of Geometric Estimation
T.(U) / ~ . . ~ ~U
u"
Fig. 14.4. Locally Gaussian distribution. thus computed value/t onto the data space L/by adding a higher order correction, as discussed in Section 5.1.3. The resulting solution still may not be exactly in the model 8 C b/, so the correction is repeated until the constraint is sufficiently satisfied.
~4.3.4
Locally Gaussian model
Suppose the distribution of u is locally Gaussian (Fig. 14.4; see Section 3.3.1): the probability density has the form p(u; r
- Ce - ( u - v t ' v [ u ] - ( u - g ) ) / 2 ,
(14.64)
where C is a normalization constant. The covariance matrix V[u] is assumed to have range T~t (L/) and have the expression
v[~] -
P~E[(~- a ) ( ~ -
(14.65)
a)~]P~.
This distribution belongs to the exponential family, and eq. (14.64) implies that L - I, - J -- V[/t]-. Hence, the minimization (14.35)is equivalent to (x - u, V [ u ] - ( x - u)) --+ min.
(14.66)
In other words, the solution /t is the Mahalanobis projection of u onto the model $, i.e., the nearest point to u in S measured in the Mahalanobis disthrice with respect to V[u] (see Section 13.2.1). From Proposition 14.3, the solution is obtained in the form L
u - V[u] ~ W(kOF(k)VF(o,
i~ -
(14.67)
k,l=l
(W (kl)) -
(( V F (k), V[/t]VF (0) )
,
(14.68)
P
which coincides with the expression derived in Section 5.1.2 (see eq. (5.17)). As shown in Section 5.1.4, the a posteriori covariance matrix of this estimator has the following expression (see eqs. (5.31) and (5.32)):
.t l-
L
E k,/--1
T
9
(14.69)
14.4.
General Parametric Fitting
467
T~(S)
Fig. 14.5. The datum u is projected onto the tangent point fi of the equilikelihood surface of u to b/, and the standard confidence region is projected onto Tu (U). Here, V[u] is the matrix obtained by projecting V[u] onto T/t(L/) (see eq. (5.26)), and l ~ (kl) is the (kl) element of the ( L L ) - m a t r i x I ~ - (W (kt)) defined by (l~ (kl)) -
((V/~(k), I}'[u]VF (0) )- ,
(14.70)
]o
where V/~(k) is an abbreviation of VF(k)(/t) (see eq. (5.33)). As discussed in Section 5.1.4, eq. (14.69) can be interpreted as the projection of the standard confidence region of u onto the tangent space T/t (S) to the model S at /t (Fig. 14.5); it gives a first order approximation to the Cramer-Rao lower bound. 14.4
1~.4.1
General Parametric Fitting
Definition of the problem
Let F(k)(a,u), k = 1, ..., L, be continuously differentiable scalar functions of arguments a E T4m and u E 7~n. The domains of a and u are assumed to be an ml-dimensional manifold A C gUn and an hi-dimensional manifold /2 C T4n, respectively; we call A the data space a n d / 2 the parameter space. Parametric fitting is the problem of computing the parameter u in such a way that the L equations F (k) (a, u) - 0,
k = 1, ..., L,
(14.71)
"fit" multiple instances {a~}, c~ = 1, ..., N, of the variable a. We call the vectors {a~} and u the data and the fitting parameter, respectively. As in Chapter 7.1, let us call the above L equations the hypothesis. Let a~ E A be the true value of a~, which we assume is unknown. We regard each d a t u m a~ as a r a n d o m variable with probability density p~(a~; gt~) parameterized by a~. As in the case of geometric correction, we assume that the distribution of a~ in the data space A is local and define the "true value"
468
Chapter 14.
General Theory of Geometric Estimation
g~ by the relationship given by eq. (3.29). However, such an explicit characterization of ds will not be used in the subsequent analysis, so it can be thought of simply as some value in A. The parametric fitting problem is formally stated as follows (see Section 7.1.1): P r o b l e m 14.2 Estimate the true value ~t of the fitting parameter u E Lt from
the data {as}, ~ = 1, ..., N, in such a way that their (unknown) true values {a.s}, a = 1, ..., N, satisfy the hypothesis F ( k ) ( a s , ~) = O,
k = 1,..., L.
(14.72)
As in the case of geometric correction, the probability density p~(as; as) is assumed to satisfy the same regularity conditions (see Section 14.2.1): 14.9 The probability density ps(as; ds) is continuously differentiable with respect to as and ds an arbitrary number of times, and
Assumption
ps(as; ds) > 0
(14.73)
for all as E A. Furthermore, the integration operation f A das with respect to a~ for any expression of ps(as; ds) is, if the integration exists, interchangeable with the differentiation operation Va, with respect to as. The probability density p~(as; a~) is not defined for ds ~ .A. For the convenience of analysis, however, we extend it to ds ~ ,4 in the same way as in the case of geometric correction. Namely, we assume that
ps(as; ds + A d s ) = ps(as; ds) + O ( A a , ) 2
(14.74)
for all ds E A and A d s E Tao(A) • where T a , ( A ) C T~m is the tangent space to the manifold jt at 5,s. In precise terms, we assume the following: Assumption
14.10
Vamp, E Ta, (A). 1~.~.2
(14.75)
The rank o f the hypothesis
The L equations (14.71) may be algebraically dependent. If only r of them are independent, they generally give only r effective constraints and define an (m ~ - r)-dimensional manifold 8, or a manifold S of codimension r, in the data space jt except at singular points. We assume that all {as } are not singular points and call r the rank of the hypothesis (14.71). A precise statement of this assumption, which also gives a precise definition of the rank r, is as follows: 14.11 The L equations of the hypothesis define a manifold $ of codimension r in the data space A in the neighborhood of as, a = 1, ..., N, for u = ~t. Assumption
14.4.
General Parametric Fitting
469
As in the preceding chapter, let us call the manifold $ the (geometric) model of Problem 14.2. If we write a s = a s + A a ~ , we have
/
§
t
9
__
Va/~(1)...,
VaF(~L)
(
)
Aa~
(14.76)
to a first approximation in A a a , where and hereafter VaF(k)(5~, ~t) is abbreviated as Va/Nk). The inherent constraint a s E ,4 requires that A a s E Tas (,4) to a first approximation. If A a s ranges over the entire tangent space T a , (A), eq. (14.76) defines a linear mapping from Tas (,4) to a linear subspace
z:a
-
{
9
I/Xa
e Tao (A)}
c
n
(14.77)
The dimension of this linem" subspace equals the dimension of the subspace
{ PaA VaF(1) , ..., PaAs VaF(L) } L C Ti m,
(14.78)
where p aA s is the m-dimensional projection matrix onto T a s ( A ) . Consequently, the dimension of s is at most the rank r of the hypothesis (14.71), but we further assume the following3: 14.12 The dimension of the linear subspace s the rank r of the hypothesis for all c~.
Assumption
is equal to
In m a t h e m a t i c a l terms, Assumption 14.12 states that each of the L equations of the hypothesis defines a manifold of codimension 1 in the d a t a space A such that they intersect each other transversally at 5,s, c~ = 1, ..., N. We say that a s is a singular datum if the dimension s is less than r, and an ordinary datum otherwise. Assumption 14.12 requires that singular data should be excluded from the fitting process. We say that the hypothesis (14.71) is nonsingularif Assumption 14.12 is satisfied, and singular otherwise (see Section 7.1.1). The linear s u b s p a c e / : a s defined by eq. (14.77) can alternatively be defined as the tangent space to the manifold
jz _ {
9 F(L) (a, ~t)
e 7"iLia e A} e Ti n
(14.79)
3As a result, the L equations (14.71) cannot be replaced by a single equation, say = {0} for all a.
L Ek--1 F(k)(a' u)2 = 0, which still has rank r but s
470
Chapter 14.
General Theory of Geometric Estimation
at a,m E A. However, am can be a singular point of the mapping (F(1)(a, fi), .... F (L)(a, ~))T: ,4 --+ 7~L. In other words, the linear subspace s defined by replacing gm by am ( r gin) in eq. (14.77) has a higher dimension that r. If this is the case, we say that the hypothesis is degenerate (see Section 7.1.1). We further assume that no effective constraints exist on ~t other than the inherent constraint fi E b/. Putting it differently, we assume that for an arbitrary variation A u of ~ there exists a variation A~m of am such that
F(k)(am + A6,m, ~ + Au) = 0,
k = 1,..., L,
(14.80)
for c~ - 1, ..., N. The above equation is equivalent to k-
1,...,L,
(14.81)
to a first approximation, where and hereafter VuF(k)(~m, ~) is abbreviated as VuF (k). Let Tu(/d) be the tangent space to the manifold/d at ~. The inherent constraint u E ?d requires that A u E T~t (b/) to a first approximation. Similarly, the inherent constraint ~m E A requires that A~m E Ta~ (A) to a first approximation. From the definition of the linear subspace s the above assumption is formally stated as follows: A s s u m p t i o n 14.13 For an arbitrary Au E T~t(Lt),
E s
9
1~.~.3
(14.82)
Cramer-Rao lower bound for parametric fitting
Let Im be the score of am with respect to am: lm - Va~ logpm.
(14.83)
The Fisher information matrix Jm with respect to am is defined by w
-T
gm - E[lmlm ].
(14.84)
The symbol El. ] denotes the expectation with respect to the joint probability density 4 I-I N m=l pro(am; am). Assumption 14.10 implies that lm E Ta~(A). We_ assume that the distribution pro(am; ~tm) is regular with respect to am, i.e., lm takes all orientations in Tam (A) as am ranges over the entire data space A. In precise terms, this assumption is stated as follows: 4Since each aa is assumed to be independent, the expectation E[.] of a quantity that depends on aa alone equals the expectation with respect to pa(aa;~a) alone.
14.4.
General Parametric Fitting
471
A s s u m p t i o n 14.14 The Fisher information matrix J s is positive semi-
definite and has range Tvt. (.A). Define an (LL)-matrix V~ - ( ~ ( k 0 )
by
(g~k,,)_ ((Var(k),j-~Va~(O)), (LL)-m
trix
-
(-Ws(kl) ) -
by
-
Y2,
(14.85)
or
((Va/>(k) , J-~Va />(0 )) - .
(14.86)
Define an (nn)-matrix M by N
- E s=l
L
E
u
-(k)
17Vak')(P~VuF~ ) ( P U V u )
/>(t) T ,
(14.87)
k,l=l
where p U is the n-dimensional projection matrix onto the tangent space Tu (b/) to the manifold gg at R. We c a l l / ~ the moment matrix (see eq. (r.2a)). We assume that the number of the data {as } is sufficiently large and their true values {R~} distribute "generally" in the model $. In other words, we exclude the possibility that { a s } have a special configuration (being identical, collinear, coplanar, etc.) such that the fitting parameter u is indeterminate. In precise terms, this assumption is stated as follows: A s s u m p t i o n 14.15 The moment matrix M is positive semi-definite and has
range Tu (gt). L e t / t - / t ( a l , . . . , aN) be an unbiased estimator of u determined from the data {as}. Since u is constrained to be in the parameter space U, it must satisfy/t(al,..., aN) E ~ for any a~ E A, a - 1, ..., N. Its unbiasedness is defined by P ~ E[/t - R] - 0. (14.88) As in the case of geometric correction, we assume that the distribution of /t is sufficiently localized around ~ in the parameter space b / a n d define the covariance matrix of/t in the following form: V [ / t ] - P-UuE[(it - R ) ( / t - R)q-]Pg.
(14.89)
We now prove the following theorem: T h e o r e m 14.2 V[/t] __>/~-.
(14.90)
Since Theorem 14.2 is an analogue of the Cramer-Rao inequality (3.134) in Section 3.5.2, we call eq. (14.90) and the bound it gives the Cramer-Rao inequality and the Cramer-Rao lower bound, respectively, for parametric fitting.
472
Chapter 14.
14.4.4
General Theory of Geometric Estimation
Proof of the main theorem
L e m m a 14.6 The (LL)-matrix Va is positive semi-definite and has range
Proof. Let v / - (~](k)) be an arbitrary L-vector. We have L
L
(~7, V~7) - ( E q(k)VaF(k)'J-~ E r](l)~af(l))" k=l
(14.91)
/=1
Since J ~ is positive semi-definite, its generalized inverse J ~ is also positive semi-definite. It follows that the right-hand side of eq. (14.91) is nonnegative for an arbitrary ~7, meaning that V~ is positive semi-definite. Suppose v/ belongs to the null space of V~. Then, the left-hand side of eq. (14.91) is 0. Hence, ~kL=l rl(k)VaF(~k) belongs to the null space of J ~ , which is identical with the null space of J a . Assumption 14.14 implies that the null space of J ~ coincides with Tas (A) • Hence, L
A o)
-
o
(14.92)
k--1
for an arbitrary Aa~ E Ta s (A). From the definition of the subspace /:as, this means that v/ E /:_x as" Conversely, if v/ E /:--P as, we obtain (r/, V~v/) - 0 by following eqs. (14.92) and (14.91) backward. Hence, the null space of the (LL)-matrix V~ coincides with the ( L - r)-dimensional subspace s a s C 7~L Consequently, the range of V~ coincides w i t h / : a s . [] Consider the (LL)-matrix ISr~ defined by eq. (14.86) 9 Let p aL be the Ls dimensional projection matrix o n t o / : a s - Lemma 14.6 implies the following: C o r o l l a r y 14.3 The (LL)-matrix 17Va is positive semi-definite and has range f-.(zs, and the following relationship holds: V~I?r
- p_C as
(14.93)
9
The following lemma plays an essential role in proving Theorem 14.2. L e m m a 14.7 The variation Ads of ~
defined by
L
(14.94) k,l=l
satisfies Aa E Tcts(.A) for any A u E T~t(bl), and to a first approximation F(k)(d~ + Aa~, ~ + A u ) - 0,
k-
1,..., L.
(14.95)
14.4.
General Parametric Fitting
473
Proof. Assumption 14.10 implies le E T a : (A). Hence, we have p_u a~ J e - J e from the definition (14.84) of the Fisher information matrix J e . Consequently, p_U a , ~ j-~ _ j-~ ~ and eq. (14.94)implies PU a ~ Age = Age ~ i.e. ~ Age E Ta~ (A) 9 Eq. (14.95) is proved by showing (Vu/~(k) , Age) - -(Va/~(k) , Au).
(14.96)
From eqs. (14.93) and (14.94), we see that
(VaF(k)' A g e ) - - - E
(~TaF(k) , J-~Va~'(rn)) - (ml)W~ (Vu~(1) , Au) /=1
m=l
L
= - E ( P a ~ )kt(Vu/~(/) , Au),
(14.97)
/=1
where (paC)kt is the (kl) element of the projection matrix p_Cas. Assumption 14.13 implies that the L-vector ((Vu/~a(k), Au)) belongs to the s u b s p a c e / : a . Hence, the last term of eq. (14.97) is equal to -(Vaf'(k),Au).
[]
Define an n-vector random variable m e by L
me-
W~-
E
(14.98)
k,l--1 L e m m a 14.8
T
(14.99)
Proof. Since the estimator it is unbiased, eq. (14.88) holds identically for any ge E S, ct = 1, ..., N, and ft E b/that satisfy eq. (14.72). This means that the first variation of the left-hand side of eq. (14.88) should be 0 for any Au E Tu(L/) if the variation Age defined by eq. (14.94) is added to each ge. Taking the first variation of the left-hand side of eq. (14.88), substituting eq. (14.94) into it, noting Assumption 14.9, and using the logarithmic differentiation formula (3.115), we obtain
N
N
"k /.A P~(it - ft) H Pfl E (~,~ l~ /3=1
N
= -Au-
E[P~' ( i t - ft)
Z e=l
Aae)dal "" .daN
e=l
k,/=l
W;
(-le,J-~VaF(O)Vu
]Au.
(14. loo)
474
Chapter 14.
General Theory of Geometric Estimation
Here, we have used the assumption that Pu+muE[it u - ~] "~ P~ E[it - ~] (see Section 3.5.2). The above expression should be 0 for any Au E T~t(U). If we let A u - p U A~, we have A u E T~t (5/) for any A~ E 7~~. Hence,
E[P-~(it- ~t)
w(kl)(~~
E
]P~A~ - -P~A~.
c~--I k , / = l
(14.101) []
Since this is an identity for A~ E 7~~, we obtain eq. (14.99). L e m m a 14.9
(
c~=l
,. 3=1
Proof. Since the expectation of the score l~ is 0 (see eq. (3.117)), eq. (14.98) implies that E[m~] - O. Noting that each a~ is independent, we see that T
E[
ms
mz
ol=l
l-
~-1
E
E[m~mT] - E E[m~mT]" (14.103)
o~,~ 1
c~=l
From eq. (14.98), we obtain L -
W~
-
.. ~
,g-~E[lo~l,~]g-~VaF(m))
(Va
k,l,m,n----1
= E l,n--1
,j~Va~(m)),.oe
-(lk) k,m--1
L -
-
E
T
I ~ ( ' n ) ( P ~ V u / ~ ( t ) ) (PuVu/~('~))
--/~'
(14.104)
l,n=l
where we have used the defining equation (14.86) of the matrix W a and the identity W ~ V ~ W ~ - W ~ W ~ W ~ - I~r [] The proof of Theorem 14.2 is as follows. From eq. (14.89) and Lemmas 14.8 and 14.9, we see that T
,
-P~M )"
(14.105)
14.5.
Maximum Likelihood Fit
475
This matrix is symmetric and positive semi-definite. If matrix A is positive semi-definite and symmetric, so is B - t A B for any matrix B as long as the multiplication can be defined (see Section 2.2.3). Hence, the following matrix is positive semi-definite and symmetric:
_ ( VIii-M- M_).
(14.106)
Here, we have noted that the covariance matrix V[/t] and the moment matrix / ~ share the same range T~t(/A) and hence / ~ / ~ - - / ~ - / 1 ~ - p U . Since/f//- is positive semi-definite, matrix V[/t] - / ~ is also positive semidefinite. [] Suppose the probability densities ps (as ;a) of the data {as } are expressed in terms of a common function p ( . ; - ) in the form ps (as; a s ) = p(as; as). Then, the weight 17V~ (kl) defined by eq. (14.86) is also expressed in terms of a common function lrv(al)( 9) in the form 17V~ (al) - l~(kl)(ds). Suppose the true values {gs } are chosen from a bounded subset $~ of the model $ with density P(a) (the number of data is given by N = fs, P(a)da). Define the following matrix (cf. eq. (14.87)): L
k,l=l
(14.107) If N is sufficiently large, the Cramer-Rao lower bound is approximated by M ~ , which we call the asymptotic approximation of the Crarner-Rao lower bound.
14.5 1~.5.1
M a x i m u m Likelihood Fit M a x i m u m likelihood estimator
Maximum likelihood estimation for this problem is to choose {as} and the fitting parameter u that maximize the likelihood, i.e., the joint probability density 1-Is=l N ps (as; as) viewed as a function of { a s } for given data { a s }. The resulting estimates {ds } a n d / t are called the maximum likelihood estimators of {as} and u, respectively. In order to distinguish "variables" from their specific "values", we regard the symbol a s as the true value and use xs when it is viewed as a variable. Since maximizing the likelihood is equivalent to minimizing its logarithm with opposite sign, the problem reduces to the
476
Chapter 14.
General Theory of Geometric Estimation
minimization N
J - -2 E
logp~(a~; x~) --+ min
(14.108)
ot:l
with respect to xa E $ and u E L/, i.e., under the hypothesis F (k) (x~, u) - 0,
x~ E A,
k - 1, ..., L,
u e/2.
(14.109)
Define an (mm)-matrix L~ by
].~ - - V 2 1 o g p ~ l x ~ = a .
(14.110)
With the expectation that the maximum likelihood estimators {&~} and /~ are located near { ~ } and ~, respectively, we write x~ - a~ + A x e ,
u-~
+ Au.
(14.111)
Substituting the first equation into the function J in eq. (14.108) and expanding it in the neighborhood of a~, we obtain
j
-
e-
N
N
c~=l
c~--1
2
+
(14.112)
where a is the value of J evaluated at ~ , a - 1, ..., N. Assumption 14.10 implies that the rank of L~ is no more than the dimension m' of the tangent space Tx~ (A). In order to guarantee unique existence of the values xa E A, a = 1, ..., N, that minimize J, we assume the following: 14.16 For any aa E ,4, matrix La is positive semi-definite and has range Ta~ (A).
Assumption
Substituting eqs. (14.111) into the hypothesis (14.109) and taking a linear approximation, we obtain
(vap2
),
-
Au).
(14.113)
The inherent constraint x~ E A requires that Ax~ E Ta~ (A) to a first approximation. We seek the value Ax~ that minimizes J for a fixed value of A u under the linearized hypothesis (14.113). Introducing Lagrange multipliers ~ ) to eq. (14.113), differentiating J with respect to A x e , and ignoring higher order terms, we obtain the following linear equation in Axe" L
t ~ A x ~ -- l~ + E k=l
'X~)P.Aa,~Va/Nk) .
(14.114)
14.5.
Maximum Likelihood Fit
477
By Assumptions 14.10 and 14.16, eq. (14.114) has the following unique solution: L
k=l
Substituting eq. (14.115) into eq. (14.113) and putting A~ -- (A~)), we obtain I F : A ~ - -((Vu/~(k), A u ) ) -
((Va/~(k), L : l ~ ) ) ,
(14.116)
where the (LL)-matrix V~ - (~-(kl))is defined by (14.117) - /
By Assumption 14.16, matrix V~ can be shown to be positive semi-definite and have range s by exactly the same argument as given in the proof of Lemma 14.6. Consequently, if a~ is an ordinary datum (see Section 14.4.2), the solution of eq. (14.116) is uniquely given by L
~(k) = _ E l~(kt) ( (Vu/~(t)' Au) + (VaF~(t)' L : l a ) ) '
(14.118)
l=l
where W~ (kl)- is the (kl) element of the (LL)-matrix W~-' defined by I2r V~ , i.e., --
]
=
m
(w,
(14.119)
Substituting eq. (14.118) into eq. (14.115), substituting the resulting expression for Ax~ into eq. (14.112), and ignoring higher order terms, we obtain the following form: J - ~+ E a=l
-t(kt) (v,.p: w;
+ (v.p:
t2L)
k,l=l
((VuF(I),Au) + (Va/~(t),t;la)) -(l~,.t.,~-lo~)) . (14.120) We now seek the value Au that minimizes eq. (14.120). The inherent constraint u E H requires that Au E Tu(Lt) to a first approximation. Differentiating eq. (14.120) with respect to Au, we obtain the following linear equation in Au: N
M'Au-
L
- E E l~(kl)(Vaf'(~k)'t-s a=l
k,l= l
(14.121)
478
Chapter 14.
General Theory of Geometric Estimation
Here, the matrix M ' is defined as follows (see eq. (14.87)): N
L
T
m--1
k,l--1
(14.122) We assume that Assumption 14.15 also applies to M~: A s s u m p t i o n 14.17 The matrix M ~ is positive semi-definite and has range
Tu(U). Then, the solution of eq. (14.121) is uniquely given by N
A u -- - M ' -
E
L
E
I~(kt)(Va/~(k)' L~Im)P~Vu/~(0"
(14.123)
m--1 k,l--1
Since the maximum likelihood estimator of u is given b y / t - ~ + A t , the covariance matrix V[/t] (see eq. (14.89))is evaluated as follows: V[/~]- E [ M ' - M " M ' - ] .
(14.124)
Here, the matrix M " is defined by N
M"=
L
E
E
m,/3-- 1
I~(kt)iZV~('~n)(Vaf-'(k)'L:lm)(Vaf-'(m)'L~ ~)
k,l,m,n--1
(P~Vu/~(t)) ( ' ~ V u / ~ ( n ) ) T .
1~.5.2
(14.125)
Fitting of the exponential family
If the distribution of each am belongs to the exponential family, the probability density pro(am;am) has the following expression (see eq. (3.146))" a.)
-
a.) +
(14.126)
Here, f~(a)is an m-dimensional function of a e 7~TM, and Cm(a) and gin(a) are scalar functions of a E 7~m. We call Problem 14.2 parametric fitting of the exponential family if the probability densities p~(am; am) have the above expression. For parametric fitting of the exponential family, we see from eqs. (14.110) and (14.126) that the matrix Lm does not depend on am. Consequently, we have the following relationship:
Jm - E[Lm] - L m. From this, we obtain the following proposition:
(14.127)
14.5.
Maximum Likelihood Fit
479
P r o p o s i t i o n 14.4 The maximum likelihood estimator it for parametric fit-
ting of the exponential family is unbiased in the first order. Proof. If eq. (14.127) holds, the matrix M ' defined by eq. (14.107)is a constant matrix. Since E[l] - O, we see from eq. (14.123) that -
E[au]
-
(14.128)
o.
As in the case of geometric correction, we say that an estimator for parametric fitting is efficient if its covariance matrix attains the Cramer-Rao lower bound given by Theorem 14.2. P r o p o s i t i o n 14.5 The maximum likelihood estimator it for parametric fit-
ting of the exponential family is efficient in the first order. --
_
]
-
Proof. Since L~ - J ~ , we see that V~ - V~ (see eqs. (14.85) and (14.117)) and hence IV~ - I ~ (see eqs. (14.86) and (14.119)). It follows that M ' = M (see eqs. (14.87) and (14.122)). Taking the expectation of eq. (14.125) and noting the independence of each as, we see that N
E[M'] -
L
~
~
~/~kl) W~ - (rnn) (Va ~ ( k ) , J-~ E[l,l ~ T ]J~ Va ~ ( m ) )
o~,fl--1 k,l,m,n--1 (P~TuF(l)) N
(P-Uu~TuF(n))T
L
W~
W~
, (J-~J,J-~)Va ['(m))
(~--1 k,l,m,n--1
u
= l --1
N
-(0
k
)
)
f-,(n) T
)
--1
L
- E
E
ITV(l~' (P-~Vu~(t)) ( P ~ V u / ~ ( ~ ) ) 7 -
M,
(14.129)
1,n--1
~--1
where we have used the identities J - ~ J ~ J ~ - J-~ and I?V~V~l~r~ = W ~ W ~ W ~ - W ~ . Thus, eq. (14.124)is written as V [ i t ] - M M M -
-
--M
-
-
-
.
-
-
-
-
-
-
-
m
[]
480
Chapter 14.
General Theory of Geometric Estimation
Recall that an estimator is consistent if it converges to the true value as the number of data increases (see Section 3.6.2). P r o p o s i t i o n 14.6 The maximum likelihood estimator for parametric fitting of the exponential family is consistent in the first order if all gts, c~ = 1, ..., N , are in a bounded subset of the model $.
Proof. If all a s are bounded, we h a v e / ~ - O ( N ) from the definition of the moment m a t r i x / ~ (see eq. (14.87)). Hence, V[/t] - / ~ - O(1/N). n The proofs of Propositions 14.4, 14.5, and 14.6 are based on eq. (14.123), which is obtained by approximating the function J in eq. (14.108) by a quadratic polynomial in the neighborhood of their true values and applying a linear approximation to the hypothesis (14.109). In other words, eq. (14.123) is an expression in the limit of small noise. Accordingly, Propositions 14.4, 14.5, and 14.6 hold in this sense, which is what the phrase "in the first order" means. As in the case of geometric correction, this type of first order analysis is compatible with our approach of studying statistical behavior for small noise (see Section 14.1).
1~.5.3
Computation
of m a x i m u m
likelihood fit
We now derive a computational scheme for the maximum likelihood estimator /t by eliminating the estimators {~i~}, c~ -- 1, ..., N. To this end, we first compute the solution x~ of the optimization (14.108) under the hypothesis (14.109) for a fixed u and then substitute the resulting expression back into eq. (14.108), reducing it to a function of u alone. As in the case of geometric correction, we first introduce a new assumption about the probability density ps(as; x~). We assume that viewed as a function of x s it takes its maximum at as. In other words, given a single datum a s , the datum a s itself is the most likely value of the true value a s if no other information is given. To be specific, what we need is the following: A s s u m p t i o n 14.18
Vx~ps(as;x~)lx~=a ~ =0.
(14.130)
For example, this assumption is satisfied if the probability density is expressed by a smooth scalar function f ( . ) that takes its maximum at the origin 0 in the form p~(as; gts) = f ( a ~ - gts). The Gaussian distribution is a typical example. Assuming that the solution x s of the optimization (14.108) is close to the data value a s , we write x~ = a s + Axe. Substituting this into the function J, expanding it in the neighborhood of the datum as, and using Assumption
14.5.
Maximum Likelihood Fit
481
14.18, we obtain N
N
J - c + E(Ax~,L~Ax~) c~--1
+ E O(Ax~)3"
(14.131)
c~----1
The constant c is the value of J evaluated at a s , a - 1, ..., N, and L~ is an (mm)-matrix defined by L~ - - V ~ , logp~lxo,=a .
(14.132)
In order to guarantee unique existence of the values x~ E A, a - 1, ..., N, that minimize J, we assume that Assumption 14.16 also applies to L~" A s s u m p t i o n 14.19 For any a~ E A, matrix I,~ is positive semi-definite
and has range T(z. (.A). The linear approximation of the hypothesis (14.109) is ( V a F (k) , A x e ) - - F (k) ,
(14.133)
F (k) and VaF (k) are the abbreviations of F(a)(a~,u) and VaF(k)(a~, u), respectively. The inherent constraint x~ E .4 requires that Ax~ E T a . (,4) to a first approximation. Introducing Lagrange multipliers
where
$~) to the linearized hypothesis (14.133), differentiating J with respect to A x a , and ignoring higher order terms, we obtain the solution A x a uniquely in the following form: L
Ax~ - E )~) L-~VaF(~k)"
(14.134)
k---1
Substituting this into eq. (14.133) and putting A - (A~)), we obtain
V~A~ = -(F(k)),
(14.135)
where the (LL)-matrix V , - (V(kl)) is defined by (u(kl))
-
((VaF(k), L-~VaF(O))
(14.136)
As in the case of geometric correction, a computational problem arises. Namely, if the L equations (14.109) are not algebraically independent but give only r constraints, the matrix V'o~ computed from the true value d~ by eq. (14.117) has rank r. However, the rank of the matrix V~ computed from the datum a~ by eq. (14.136) may be larger than r. In other words, among the L linear equations in Ax~ given by eq. (14.133), more than r equations may be linearly independent if evaluated at the datum a~ # ~ . If this occurs,
482
Chapter 14.
General Theory of Geometric Estimation
the hypothesis (14.109) is said to be degenerate (see Section 14.4.2). If the hypothesis (14.109) is degenerate, some of the positive eigenvalues of matrix V~ converge to 0 in the limit a s ~ as. Hence, eq. (14.135) viewed as a linear equation in ,ks is ill-conditioned when the data {as} are close to the true values {as} (see Section 2.3.2). This ill-posedness can be avoided in the same way as in the case of geometric correction: we project both sides of eq. (14.135) onto the eigenspace defined by the largest r eigenvalues of V a. In other words, we effectively use only r constraints from among the L linear equations (14.133). The resulting solution may be slightly different from the exact solution of eq. (14.135) but does not effectively affect the subsequent analysis because eq. (14.135) is a first order approximation. The solution of eq. (14.135) given by this projection is L
~ ) - - ~ W(kOF(0,
(14.137)
/=1
where W~(kl) is the (kl) element of the rank-constrained generalized inverse (V~);-, which we write
(W(akl)) -((VaF(k),L-s
(14.138)
Substituting eq. (14.137) into eq. (14.134), substituting the resulting expression into eq. (14.131), and ignoring higher order terms, we obtain the following expression: N
J-c+
E c~=l
L
E
Ws
F~(k F(1).
(14.139)
k,l=l
Since the constant c is irrelevant to optimization, we obtain the following conclusion: P r o p o s i t i o n 14.7 The maximum likelihood estimator i~ of u is given as the
solution of the nonlinear optimization N
J[u] = E s=l
L
E
W~(k')F(ak)F(~') ~ min,
u e ?d.
(14.140)
k,l=l
This proposition gives a rigorous mathematical foundation to the formulation of parametric fitting as presented Chapter 7, upon which the subsequent applications are all based. Let /t be the computed maximum likelihood esdata timator of u. If the model 8 has codimension r in the m-dimensional ' space .4 parameterized by u E ~, it is a model of dimension d = m ' - r , codimension r, and n' degrees of freedom. The argument given in Sections
14.5.
M a x i m u m Likelihood Fit
483
13.2 and 13.3 can be generalized to d a t a with non-Gaussian distributions, and the geometric information criterion can be defined as follows:
A I C ( $ ) - J[/~]-t- 2(dN -t- n').
(14.141)
Bibliography [1] G. Adiv, Determining three-dimensional motion and structure from optical flow generated by several moving objects, IEEE Transactions on Pattern Analysis and Machine Intelligence, 7-4 (1985), 384-401. [2] H. Akaike, A new look at the statistical model identification, IEEE Transactions on Automatic Control, 19-6 (1974), 716-723. [3] J. Aisbett, An iterated estimation of the motion parameters of a rigid body from noisy displacement vectors, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-11 (1990), 1092-1098. [4] A. Albano, Representation of digitized contours in terms of conic arcs and straight-line segments, Computer Graphics and Image Processing, 3-1 (1974), 23-33.
[5]
Y. Aloimonos (ed.), Active Perception, Lawrence Erlbaum Associates, Hillsdale, N J, U.S.A., 1993.
[6]
J. (Y.) Aloimonos and D. Shulman, Integration of Visual Modules: An Extension of the Mart Paradigm, Academic Press, San Diego, CA, U.S.A., 1989.
[7]
S. Amari, Differential-Geometrical Methods in Statistics, Springer, Berlin, F.R.G., 1985.
[8]
S. Amari and M. Kumon, Estimation in the presence of infinitely many nuisance parameters--Geometry for estimation functions, Annals of Statistics, 16-3 (1988), 1044-1068.
[9]
E. B. Andersen, Asymptotic properties of conditional maximum likelihood estimators, Journal of the Royal Society of London, B32-2 (1970), 283-301.
[10]
B. D. O. Anderson and J. B. Moore, Optimal Filtering, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1979.
[11]
K. S. Arun, T. S. Huang and S. D. Blostein, Least-squares fitting of two 3-D point sets, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-5 (1987), 698-700.
[12]
H. Ballard and C. M. Brown, Cliffs, N J, U.S.A., 1982.
[13]
A. Bani-Hashemi, A Fourier approach to camera orientation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-11 (1993), 1197-1202.
[14]
O. E. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, John Wiley and Sons, New York, NY, U.S.A., 1978.
[15]
O. E. Barndorff-Nielsen, Parametric Statistical Models and Likelihood, Lecture Notes in Statistics 50, Springer, Berlin, F.R.G., 1988.
Computer
Vision, Prentice-Hall, Englewood
[16] J. L. Barron, D. J. Fleet and S. S. Beauchemin, Performance of optical flow techniques, International Journal of Computer Vision, 12-1 (1994), 43-77.
[17]
J. Bhanja and J. K. Ghosh, Efficient estimation with many nuisance parameters, Parts 1, 2, 3, Sankhy~, A54-1,2,3 (1992), 1-39, 135-156, 297-308. 485
486
Bibliography
[ls]
P. J. Bickel, C. A. J. Klaassen, Y. Ritov and J. A. Wellner, Efficient and Adaptive Estimation for Semiparametric Models, Johns Hopkins University Press, Baltimore, MD, U.S.A., 1993.
[19]
S. D. Blostein and T. S. Huang, Error analysis in stereo determination of 3-D point positions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-6 (1987), 752-765 (corrected in 10-5 (1988), 765).
[20]
R. M. Bolle and B. C. Verumi, On three-dimensional surface reconstruction methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-1 (1991), 1-13.
[21]
F. L. Bookstein, Fitting conic sections to scattered data, Computer Graphics and Image Processing, 9-1 (1979), 56-71.
[22]
K. L. Boyer, J. Mirza and G. Ganguly, The robust sequential estimator: A general approach and its application to surface organization in range data, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-10 (1994), 987-1001.
[23]
B. Brillault-O'Mahony, New method for vanishing point detection, CVGIP: Image Understanding, 54-2 (1991), 289-300.
[241 B. Brillault-O'Mahony, High level 3D structure from a single view, Image and Vision Computing, 10-7 (1992), 508-520.
[25]
T. J. Broida, S. Chandrashekhar and R. Chellappa, Recursive estimation of 3D motion from a monocular image sequence, IEEE Transactions on Aerospace and Electronic Systems, 26-4 (1990), 639-656.
[26]
T. J. Broida and R. Chellappa, Estimation of object motion parameters from noisy images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-1 (1986), 90-99.
[271 T. J. Broida and R. Chellappa, Performance bounds for estimating threedimensional motion parameters from a sequence of noisy images, Journal of the Optical Society of America, A6-6 (1989), 879-898.
[2s]
T. J. Broida and R. Chellappa, Estimating the kinematics and structure of a rigid object from a sequence of monocular images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-6 (1991), 497-513.
[29]
R. C. Brown, Introduction to Random Signal Analysis and Kalman Filtering, John Wiley and Sons, New York, NY, U.S.A., 1983.
[3o]
A. R. Bruss and B. K. P. Horn, Passive navigation, Computer Vision, Graphics, and Image Processing, 21-1 (1983), 3-20.
[311
M. Campani and A. Verri, Motion analysis from first-order properties of optical flow, CVGIP: Image Understanding, 56-1 (1992), 90-107.
[32]
B. Caprile and V. Torte, Using vanishing points for camera calibration, International Journal of Computer Vision, 4-2 (1990), 127-140.
[33]
C. K. Chui and G. Chen, Kalman Filtering with Real-Time Applications, Springer, Berlin, F.R.G., 1991.
Bibliography
487
[34]
D. R. Cooper and N. Yalabik, On the computational cost of approximating and recognizing noise-perturbed straight lines and quadratic arcs in the plane, IEEE Transactions on Computers, 25-10 (1976), 1020-1032.
[35]
I. J. Cox, A review of statistical data association techniques for motion correspondence, International Journal of Computer Vision, 10-1 (1993), 53-66.
[36]
I. J. Cox, J. M. Rehg and S. Hingorani, A Bayesian multiple-hypothesis approach to edge grouping and contour segmentation, International Journal of Computer Vision, 11-1 (1993), 5-24.
[37]
D. R. Cox and D. V. Hinkley, Theoretical Statistics, Chapman and Hall, London, U.K., 1974.
[3s]
H. Cram~r, The Elements of Probability Theory and Some of Its Applications, Almqvist and Wiksell/Gebers, Stockholm, Sweden, 1955.
[39]
J. L. Crowley, P. Bobet and C. Schmid, Auto-calibration by direct observation of objects, Image and Vision Computing, 11-2 (1993), 67-81.
[4o]
K. Daniilidis and H.-H. Nagel, Analytical results on error sensitivity of motion estimation from two views, Image and Vision Computing, 8-4 (1990), 287-303.
[41]
E. R. Davies, Finding ellipses using the generalised Hough transform, Pattern Recognition Letters, 9-1 (1989), 87-96.
[42]
A. J. Dobson, An Introduction to Generalized Linear Models, Chapman and Hall, London, U.K., 1990.
[43]
L. S. Dreschler and H.-H. Nagel, Volumetric model and 3-D trajectory of a moving car derived from monocular TV-frame sequences of a street scene, Computer Graphics and Image Processing, 20-3 (1982), 199-228.
[44]
T. Echigo, A camera calibration technique using three sets of parallel lines, Machine Vision and Applications, 3-3 (1990), 159-167.
[45]
T. Ellis, A. Abboot and B. Brillault, Ellipse detection and matching with uncertainty, Image and Vision Computing, 10-5 (1992), 271-276.
[46]
T. Endoh, T. Toriu and N. Tagawa, A superior estimator to the maximum likelihood estimator on 3-D motion estimation from noisy optical flow, IEICE Transactions on Information and Systems, E77-D-11 (1994), 1240-1246.
[47]
O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint, MIT Press, Cambridge, MA, U.S.A., 1993.
[4s]
O. D. Faugeras and S. Maybank, Motion from point matches: Multiplicity of solutions, International Journal of Computer Vision, 4-3 (1990), 225-246.
[49]
W. Feller, Probability Theory and Its Applications, Vols. 1, 2, John Wiley and Sons, New York, NY, U.S.A., 1950, 1966.
[50]
D. Forsyth, J. L. Mundy and A. Zisserman, Transformation invariance--A primer, Image and Vision Computing, 10-1 (1992), 39-45.
[51]
D. Forsyth, J. L. Mundy, A. Zisserman, C. Coelho, A. Heller and C. Rothwell, Invariant descriptors for 3-D object recognition and pose, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-10 (1991), 971-991.
488
Bibliography
[52]
J. E. Freund, Mathematical Statistics, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1962.
[53]
J. E. Freund, Modern Elementary Statistics, 3rd Ed., Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1967.
[54]
A. Gelb (ed.), Applied Optimal Estimation, MIT Press, Cambridge, MA, U.S.A., 1974.
[55]
R. Gnanadesikan, Methods for Statistical Data Analysis of Multivariate Observations, John Wiley and Sons, New York, NY, U.S.A., 1977.
[56]
W. I. Grosky and L. A. Tamburino, A unified approach to the linear camera calibration problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-7 (1990), 663-671.
[57]
R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, Vols. 1, 2, Addison-Wesley, Reading, MA, U.S.A., 1992.
[ss]
J. C. Hay, Optical motions and space perception--An extension of Gibson's analysis, Psychological Review, 73-6 (1966), 550-565.
[59]
G. Healey and R. Kondepudy, Radiometric CCD camera calibration and noise estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 267-275.
[6o]
D. J. Heeger and A. D. Jepson, Subspace methods for recovering rigid motion I: Algorithm and implementation, International Journal of Computer Vision, 7-2 (1992), 95-117.
[61]
Y. Hel-Or and M. Werman, Pose estimation by fusing noisy data of different dimensions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17- 2 (1995), 195-201.
[62]
P. G. Hoel, Elementary Statistics, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1966.
[63]
P. G. Hoel, Introduction to Mathematical Statistics, 4th Ed., John Wiley and Sons, New York, NY, U.S.A., 1971.
[64]
R. Horaud, R. Mohr and B. Lorecki, On single-scanline camera calibration, IEEE Transactions on Robotics and Automation, 9-1 (1993), 71-74.
[65] [66]
B. K. P. Horn, Robot Vision, MIT Press, Cambridge, MA, U.S.A., 1986.
[67]
B. K. P. Horn, Closed-form solution of absolute orientation using unit quaternions, Journal of the Optical Society of America, A4-4 (1987), 629-642.
[6s]
B. K. P. Horn, Relative orientation, International Journal of Computer Vision, 4-1 (1990), 59-78.
[69]
B. K. P. Horn, H. M. Hilden and S. Negahdaripour, Closed-form solution of absolute orientation using orthonormal matrices, Journal of the Optical Society of America, A5-5 (1988), 1127-1135.
[7o]
B. K. P. Horn and B. Schunck, Determining optical flow, Artificial Intelligence, 17-1/2/3 (1981), 185-203.
B. K. P. Horn, Motion fields are hardly ever ambiguous, International Journal of Computer Vision, 1-3 (1987), 259-274.
Bibliography
489
[71]
X. Hu and N. Ahuja, Motion and structure estimation using long sequence motion models, Image and Vision Computing, 11-9 (1993), 549-569.
[72]
X. Hu and N. Ahuja, Necessary and sufficient conditions for a unique solution of planar motion and structure, IEEE Transactions on Robotics and Automation, 11-2 (1995), 304-308.
[73]
T. S. Huang and O. D. Faugeras, Some properties of the E matrix in twoview motion estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-12 (1989), 1310-1312.
[74]
A. H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, New York, NY, U.S.A., 1970.
[75]
C. Jerian and R. Jain, Polynomial methods for structure from motion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-12 (1990), 1150-1165.
[76]
C. P. Jerian and R. Jain, Structure from Motion--A critical analysis of methods, IEEE Transactions on Systems, Man, and Cybernetics, 21-3 (1991), 572588.
[77]
J.-M. Jolion, P. Meer and S. Bataouche, Robust clustering with applications in computer vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-8 (1991), 791-802.
ITs]
R. E. Kalman, A new approach to linear filtering and prediction problems, Transactions of ASME, Journal of Basic Engineering, 82D-1 (1960), 35-45.
[79]
R. E. Kalman and R. S. Bucy, New results in linear filtering and prediction theory, Transactions of ASME, Journal of Basic Engineering, 83D-1 (1961), 95-108.
[so]
B. Kamgar-Parsi and R. D. Eastman, Calibration of a stereo system with small relative angles, Computer Vision, Graphics, and Image Processing, 511 (1990), 1-19.
[Sl]
B. Kamgar-Parsi, B. Kamgar-Parsi and N. Netanyahu, A nonparametric method for fitting a straight line to a noisy image, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-9 (1989), 998-1001.
[82]
K. Kanatani, Distribution of directional data and fabric tensors, International Journal of Engineering Science, 22-2 (1984), 149-164.
Is3]
K. Kanatani, Structure and motion from optical flow under orthographic projection, Computer Vision, Graphics, and Image Processing, 35-2 (1986), 181199.
Is4]
K. Kanatani, Structure and motion from optical flow under perspective projection, Computer Vision, Graphics, and Image Processing, 38-2 (1987), 122146.
[ss]
K. Kanatani, Group-Theoretical Methods in Image Understanding, Springer, Berlin, F.R.G., 1990.
Is6]
K. Kanatani, Computational projective geometry, CVGIP: Image Understanding, 54-3 (1991), 333-348.
490
Bibliography
[87]
K. Kanatani, Hypothesizing and testing geometric properties of image data, CVGIP: Image Understanding, 54-3 (1991), 349-357.
[ss]
K. Kanatani, Statistical analysis of focal length calibration using vanishing points, IEEE Transactions on Robotics and Automation, 8-6, (1992), 767775.
IS9]
K. Kanatani, Unbiased estimation and statistical analysis of 3-D rigid motion from two views, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-1 (1993), 37-50.
[90]
K. Kanatani, Geometric Computation for Machine Vision, Oxford University Press, Oxford, U.K., 1993.
[91]
K. Kanatani, Statistical analysis of geometric computation, CVGIP: Image Understanding, 59-3 (1994), 286-306.
[92]
K. Kanatani, Statistical bias of conic fitting and renormalization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 320-326.
[93]
K. Kanatani, Analysis of 3-D rotation fitting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-5 (1994), 543-549.
[941
K. Kanatani, Statistical foundation for hypothesis testing of image data, CVGIP: Image Understanding, 60-2 (1994), 382-391.
[95]
K. Kanatani, Computational cross ratio for computer vision, CVGIP: Image Understanding, 60-2 (1994), 371-381.
[96]
K. Kanatani, Renormalization for motion analysis: Statistically optimal algorithm, IEICE Transactions on Information and Systems, E77-D-11 (1994), 1233-1239.
[971 K. Kanatani and W. Liu, 3-D interpretation of conics and orthogonality, CVGIP: Image Understanding, 58-3 (1993), 286-301.
[9s]
K. Kanatani and Y. Onodera, Anatomy of camera calibration using vanishing points, IEICE Transactions on Information and Systems, E74-10 (1991), 3369-3378.
[99]
K. Kanatani and S. Takeda, 3-D motion analysis of a planar surface by renormalization, IEICE Transactions on Information and Systems, E78-D-8 (1995), 1074-1079.
[~00]
Y. Kanazawa and K. Kanatani, Direct reconstruction of planar surfaces by stereo vision, IEICE Transactions on Information and Systems, E78-D-7 (1995), 917-922.
[101] Y. Kanazawa and K. Kanatani, Reliability of 3-D reconstruction by stereo vision, IEICE Transactions on Information and Systems, E78-D-10 (1995), 1301-1306. [102] Y. Kanazawa and K. Kanatani, Reliability of fitting a plane to range data, IEICE Transactions on Information and Systems, E78-D-12 (1995), 16301635.
[103] A.
Kara, D. M. Wilkes and K. Kawamura, 3D structure reconstruction from point correspondences between two perspective projections, CVGIP: Image Understanding, 60-3 (1994), 392-387.
Bibliography
491
[104] M. G. Kendall and A. Stuart, The Advanced Theory of Statistics, Vols. 1 (3rd Ed.), 2 (3rd Ed.), 3 (2nd Ed.), Griffin, London, U.K., 1969, 1973, 1968. [105] Y. C. Kim and J. K. Aggarwal, Determining object motion in a sequence of stereo images, IEEE Journal of Robotics and Automation, 3-6 (1987), 599614. [106] N. Kiryati and A. M. Bruckstein, What's in a set of points?, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-4 (1992), 496-500. [107] E. Kreyszig, Introductory Mathematical Statistics: Principle and Methods, John Wiley and Sons, New York, NY. U.S.A., 1970. [108] R. Kumar, Robust methods for estimating pose and a sensitivity analysis, CVGIP: Image Understanding, 60-3 (1994), 313-432. [109] M. Kumon and S. Amari, Estimation of a structural parameter in the presence of a large number of nuisance parameters, Biometrika, 71-3 (1984), 445-459. [110] J. Z. C. Lai, On the sensitivity of camera calibration, Image and Vision Computing, 11-10 (1993), 656-664. [111] S. M. LaValle and S. A. Hutchinson, A Bayesian segmentation methodology for parametric image models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17-2 (1995), 211-217. [112] C.-H. Lee, Time-varying images: The effect of finite resolution on uniqueness, CVGIP: Image Understanding, 54-3 (1991), 325-332. [113] C.-H. Lee, Computing three-dimensional motion parameters: A hypothesis testing approach, Image and Vision Computing, 11-3 (1993), 145-154. [114] S. Lee and Y. Kay, A Kalman filter approach for accurate 3-D motion estimation from sequence of stereo images, CVGIP: Image Understanding, 54-2 (1991), 244-258. [115] E. L. Lehman, Testing Statistical Hypotheses, 2nd Ed., John Wiley and Sons, New York, NY. U.S.A., 1986. [116] R. K. Lenz and R. Y. Tsai, Techniques for calibration of the scale factor and image center for high-accuracy 3-D machine vision metrology, IEEE Transactions on Pattern Analysis and Machine Intelligence, 10-5 (1988), 713-720. [117] R. K. Lenz and R. Y. Tsai, Calibrating a Cartesian robot with eye-on-hand configuration independent of eye-to-hand relationship, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-9 (1989), 916-928. [118] L. Li and J. H. Duncan, 3-D translational motion and structure from binocular image flow, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-7 (1993), 657-667. [119] B. G. Lindsay, Conditional score functions: Biometrika, 69-3 (1982), 503-512.
Some optimality results,
[120] B. G. Lindsay, Using empirical partially Bayes inference for increased efficiency, Annals of Statistics, 13-3 (1985), 914-931. [121] W. Liu and K. Kanatani, Interpretation of conic motion and its applications, International Journal of Computer Vision, 10-1 (1993), 67-84.
492
Bibliography
[122] H. C. Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, 293-10 (1981), 133-135. [123] H. C. Longuet-Higgins, The reconstruction of a scene from two projections-Configurations that defeat the 8-point algorithm, Proceedings of the First IEEE Conference on Artificial Intelligence Applications, Denver, CO, U.S.A., December 1984, pp. 395-397. [124] H. C. Longuet-Higgins, The visual ambiguity of a moving plane, Proceedings of the Royal Society of London, B223 (1984), 165-175. [125] H. C. Longuet-Higgins, The reconstruction of a plane surface from two perspective projections, Proceedings of the Royal Society of London, B227 (1986), 399-410. [126] H. C. Longuet-Higgins, Multiple interpretations of a pair of images of a surface, Proceedings of the Royal Society of London, A418 (1988), 1-15. [127] H. C. Longuet-Higgins and K. Prazdny, The interpretation of a moving retinal image, Proceedings of the Royal Society of London, B208 (1980), 385-397. [128] S. D. Ma, Conics-based stereo, motion estimation, and pose determination, International Journal of Computer Vision, 10-1 (1993), 7-25. [129] K. V. Mardia, Statistics of Directional Data, Academic Press, London, U.K., 1972. [130] D. Marr, Vision: A Computational Investigation into the Human Representation and Processin 9 of Visual Information, W. H. Freeman, San Francisco, CA, U.S.A., 1982. [131] L. Matthies, T. Kanade and R. Szeliski, Kalman filter-based algorithms for estimating depth from image sequences, International Journal of Computer Vision, 3-3 (1989), 209-236. [132] S. Maybank, The angular velocity associated with the optical flowfield arising from motion through rigid environment, Proceedings of the Royal Society of London, A401 (1985), 317-326. [133] S. J. Maybank, The projective geometry of ambiguous surface, Philosophical Transactions of the Royal Society of London, A332 (1990), 1-47. [134] S. J. Maybank, Ambiguity in reconstruction from image correspondences, Image and Vision Computing, 9-2 (1991), 93-99. [135] S. Maybank, Theory of Reconstruction from Image Motion, Springer, Berlin, F.R.G., 1993. [136] S. J. Maybank and O. D. Faugeras, A theory of self-calibration of a moving camera, International Journal of Computer Vision, 8-2 (1992), 123-151. [137] P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd edition, Chapman and Hall, London, U.K., 1989. [138] J. S. Meditch, Stochastic Optimal Linear Estimation and Control, McGrawHill, New York, NY, U.S.A., 1969. [139] P. Meet, D. Mintz and A. Rosenfeld, Robust regression methods for computer vision: A review, International Journal of Computer Vision, 6-1 (1991), 5970.
Bibliography
493
[140] J. Mendel, Lessons in Digital Estimation Theory, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1987. [141] E. De Micheli, V. Torre and S. Uras, The accuracy of the computation of optical flow and the recovery of motion parameters, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-5 (1993), 434-447. [1421 K. M. Mutch, Determining object translation information using stereoscopic motion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-6 (1986), 750-755. [143] M. K. Murray and J. W. Rice, Differential Geometry and Statistics, Chapman and Hall, London, U.K., 1993. [144] H.-H. Nagel, Representation of moving rigid objects based on visual observations, Computer, 14-8 (1981), 29-39. [145] Y. Nakagawa and A. Rosenfeld, A note on polygonal and elliptical approximation of mechanical parts, Pattern Recognition, 11-2 (1979), 133-142. [146] S. Negahdaripour, Critical surface pairs and triplets, International Journal of Computer Vision, 3-4 (1989), 293-312. [147] S. Negahdaripour, Multiple interpretations of the shape and motion of objects from two perspective images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12-11 (1990), 1025-1030. [148] S. Negahdaripour, Closed-form relationship between the two interpretations of a moving plane, Journal of the Optical Society of America, A7-2 (1990), 279-285. [149] S. Negahdaripour and S. Lee, Motion recovery form image sequences using only first order optical flow information, International Journal of Computer Vision, 9-3 (1992), 163-184.
[:50]
A. N. Netravali, T. S. Huang, A. S. Krishnakumar and R. J. Holt, Algebraic methods in 3-D motion estimation from two-view point correspondences, International Journal of Imaging Systems and Technology, 1-1 (1989), 78-99.
[1511 R. Nevatia, Machine Perception, Prentice-Hall, Englewood Cliffs, N J, U.S.A., 1982. [152] J. Neyman and E. L. Scott, Consistent estimates based on partially consistent observations, Econometrica, 16-1 (1948), 1-32.
[:sa]
Y. Nonmra, M. Sagara, H. Naruse and A. Ide, Simple calibration algorithm for high-distortion-lens camera, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-12 (1992), 1095-1099.
[154] N. Ohta and K. Kanatani, Optimal structure from motion algorithm for optical flow, IEICE Transactions on Information and Systems, E78-D-12 (1995), 1559-1566.
[:55]
K. A. Paton, Conic sections in chromosome analysis, Pattern Recognition, 2-1 (1970), 39-51.
[156] M. A. Penna, Camera calibration: A quick and easy way to determine the scale factor, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1312 (1991), 1240-1245.
494
Bibliography
[157] J. Philip, Estimation of three-dimensional motion of rigid objects from noisy observations, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13-1 (1991), 61-66. [158] T. Poggio and C. Koch, Ill-posed problems in early vision: From computational theory to analogue networks, Proceedings of the Royal Society of London, B226 (1985), 303-323.
[159]
T. Poggio, V. Torte and C. Koch, Computational vision and regularization theory, Nature, 317 (1985), 314-319.
[160]
R. Poli, G. Coppini and G. Valli, Recovery of 3D closed surfaces from sparse data, CVGIP: Image Understanding, 60-1 (1994), 1-25.
[1611
S. B. Pollard, J. Porrill and J. E. Mayhew, Experiments in vehicle control using predictive feed-forward stereo, Image and Vision Computing, 8-1 (1990), 6370.
[162]
S. B. Pollard, T. P. Pridmore, J. PorriU, J. E. Mayhew and J. P. Frisby, Geometric modeling from multiple stereo views, International Journal of Robotics Research, 8-4 (1989), 3-32.
[163] J. Porrill, Optimal combination and constraints for geometrical sensor data, International Journal of Robotics Research, 7-6 (1988), 66-77. [164] J. Porrill, Fitting ellipses and predicting confidence envelopes using a bias corrected Kalman filter, Image and Vision Computing, 8-1 (1990), 37-41. [165] J. Porrill, S. B. Pollard and J. E. W. Mayhew, Optimal combination of multiple sensors including stereo vision, Image and Vision Computing, 5-2 (1987), 174180.
[166]
T. Poston and I. Stewart, Catastrophe Theory and Its Applications, Pitman, London, U.K., 1978.
[167] V. Pratt, Direct least-squares fitting of algebraic surfaces, Computer Graphics, 21-4 (1987), 145-152.
[1681
K. Prazdny, Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer, Computer Graphics and Image Processing, 17-3 (1981), 238-248.
[1691
L. Quan, P. Gros and R. Mohr, Invariants of a pair of conics revisited, Image and Vision Computing, 10-5 (1992), 319-323.
[1701 C. R. Rao, Linear Statistical Inference and Its Applications, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1973. [1711 C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Applications, John Wiley and Sons, New York, NY, U.S.A., 1971. [1721 J. H. Rieger and D. T. Lawton, Processing differential image motion, Journal of the Optical Society of America, A2-2 (1985), 354-360. [173] J. Rissanen, Modeling by shortest data description, Automatica, 14-5 (1978), 465-471.
[174]
J. Rissanen, A universal prior for integers and estimation by minimum description length, Annals of Statistics, 11-3 (1983), 416-431.
Bibliography
495
[175] J. Rissanen, Universal coding, information, prediction, and estimation, IEEE Transactions on Information Theory, 30-4 (1984), 629-636. [1761 J. Rissanen, Stochastic Complexity in Statistical Inquiry, World Scientific, Singapore, 1987. [177] J. W. Roach and J. K. Aggarwal, Determining the movement of objects from a sequence of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2-6 (1980), 544-562. [1781 C. A. Rothwell, A. Zisserman, C. I. Marinos, D. A. Forsyth and J. L. Mundy, Relative motion and pose from arbitrary plane curves, Image and Vision Computing, 10-4 (1992), 250-262. [1791 R. Safaee-Rad, I. Tchoukanov, B. Benhabib and K. C. Smith, Accurate parameter estimation of quadratic curves from grey-level images, CVGIP: Image Understanding, 54-2 (1991), 259-274. [lS0] R. Safaee-Rad, I. Tchoukanov, K. C. Smith and B. Benhabib, Constraints on quadratic-curved features under perspective projection, Image and Vision Computing, 19-8 (1992), 532-548. [1811 R. Safaee-Rad, I. Tchoukanov, K. C. Smith and B. Benhabib, Threedimensional location estimation of circular features for machine vision, IEEE Transactions on Robotics and Automation, 8-5 (1992), 624-640. [182] P. D. Sampson, Fitting conic sections to "very scattered" data: An iterative refinement of the Bookstein algorithm, Computer Graphics and Image Processing, 18-1 (1982), 97-108. [183] J. G. Semple and G. T. Kneebone, Algebraic Projective Geometry, Clarendon Press, Oxford, U.K., 1952 (reprinted 1979). [184] J. G. Scruple and L. Roth, Introduction to Algebraic Geometry, Clarendon Press, Oxford, U.K., 1949 (reprinted 1987). [1851 Y. Shirai, Three-Dimensional Computer Vision, Springer, Berlin, F.R.G., 1987. [186] S. Smith, Note on small angle approximations for stereo disparity, Image and Vision Computing, 11-6 (1993), 395-398. [187] M. A. Snyder, The precision of 3-D parameters in correspondence based techniques: The case of uniform translation motion in a rigid environment, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-5 (1989), 523528.
[1881
M. E. Spetsakis, A linear algorithm for point and line-based structure from motion, CVGIP: Image Understanding, 56-2 (1992), 230-241.
[189] M. Spetsakis and J. (Y.) Aloimonos, A multi-frame approach to visual motion perception, International Journal of Computer Vision, 6-3 (1991), 245-255. [190] M. E. Spetsakis and Y. Aloimonos, Optimal visual motion estimation: A note, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-9 (1992), 959-964. [191] M. Spetsakis, Models of statistical visual motion estimation, CVGIP: Image Understanding, 60-1 (1994), 300-312.
496
Bibliography
[192] M. Subbarao and A. M. Waxman, Closed form solution to image flow equations for a planar surface in motion. Computer Vision, Graphics, and Image Processing, 36-2/3 (1986), 208-228. [193] M. Subbarao, Interpretation of image flow: Rigid curved surfaces in motion, International Journal of Computer Vision, 2-1 (1988), 77-96. [194] M. Subbarao, Bounds on time-to-collision and rotational component from first-order derivatives of image flow, Computer Vision, Graphics, and Image Processing, 50-3 (1990), 329-341. [195] N. Tagawa, T. Toriu and T. Endoh, Un-biased linear algorithm for recovering three-dimensional motion from optical flow, IEICE Transactions on Information and Systems., E76-10 (1993), 1263-1275. [196] N. Tagawa, T. Toriu and T. Endoh, Estimation of 3-D motion from optical flow with unbiased objective function, IEICE Transactions on Information and Systems., E77-D-10 (1994), 1148-1161.
[1971
H. Takeda, C. Facchinetti and J.-C. Latombe, Planning the motions of a mobile robot in a sensory uncertainty field, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-19 (1994), 1002-1017.
[198] T. N. Tan, K. D. Baker and G. D. Sullivan, 3D structure and motion estimation from 2D image sequences, Image and Vision Computing, 11-4 (1993), 203-210.
[199] K. Tarabanis, R. Y. Tsai and D. S. Goodman, Calibration of a computer controlled robotic vision sensor with a zoom lens, CVGIP: Image Understanding, 59-2 (1994), 226-241.
[200]
G. Taubin, F. Cukierman, S. Sullivan, J. Ponce and D. J. Kriegman, Parameterized families for bounded algebraic curve and surface fitting, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16-3 (1994), 287-303.
[201] R. Thorn, Stabilitd Structurelle et Morphogdn~se, Benjamin, New York, NY, 1972.
[2021
J. I. Thomas, Refining 3D reconstructions: A theoretical and experimental study of the effect of cross-correlations, CVGIP: Image Understanding, 60-3 (1994), 359-370.
[203]
C. Tomasi and T. Kanade, Shape and motion from image streams under orthography--A factorization method, International Journal of Computer Vision, 9-2 (1992), 137-154.
[204]
P. H. S. Torr and D. W. Murray, Statistical detection of independent movement from a moving camera, Image and Vision Computing, 11-4 (1993), 180187.
[205]
H. P. Trivedi, Estimation of stereo and motion parameters using a variational principle, Image and Vision Computing, 5-2 (1987), 181-183.
[206]
R. Y. Tsai and T. S. Huang, Estimating three-dimensional motion parameters of a rigid planar patch, IEEE Transactions on Acoustics, Speech, and Signal Processing, 29-6 (1981), 1147-1152.
Bibliography
497
[207]
R. Y. Tsai and T. S. Huang, Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces, I E E E Transactions on Pattern Analysis and Machine Intelligence, 6-1 (1984), 13-27.
[208]
T. Y. Tsai and T. S. Huang, Estimating 3-D motion parameters of a rigid planar patch III. Finite point correspondences and the three-view problem, I E E E Transactions on Acoustics, Speech, and Signal Processing, 32-2 (1984), 213-220.
[209] R. Y. Tsai, T. S. Huang and W.-L. Zhu, Estimating three-dimensional motion parameters of a rigid planar patch, II: Singular value decomposition, I E E E Transactions on Acoustics, Speech, and Signal Processing, 30-3 (1982), 525534. [210] R. Y. Tsai and R. K. Lenz, A new technique for fully autonomous and efficient 3D robotics hand/eye calibration, I E E E Transactions on Robotics and Automation, 5-3 (1989), 345-358. [211] S. Ullman, The Interpretation of Visual Motion, MIT Press, Cambridge, MA, U.S.A., 1979. [212] S. Umeyama, Least-squares estimation of transformation parameters between two point patterns, I E E E Transactions on Pattern Analysis and Machine Intelligence, 13-4 (1991), 376-380. [213] T. Vi~ville, Auto-calibration of visual sensor parameters on a robotic head, Image and Vision Computing, 12-4 (1994), 227-237. [214] C.-C. Wang, Extrinsic calibration of a vision sensor mounted on a robot, IEEE Transactions on Robotics and Automation, 8-2 (1992), 161-175. [215] L.-L. Wang and W.-H. Tsai, Camera calibration by vanishing lines for 3-D computer vision, I E E E Transactions on Pattern Analysis and Machine Intelligence, 13-4 (1991), 370-376.
[2~6]
A. M. Waxman and J. H. Duncan, Binocular image flows: Steps toward stereomotion fusion, I E E E Transactions on Pattern Analysis and Machine Intelligence, 8-6 (1986), 715-729.
[217] A. M. Waxman, B. Kamgar-Parsi and M. Subbarao, Closed-form solutions to image flow equations for 3D structure and motion, International Journal of Computer Vision, 1-3 (1987), 239-258. [218] G.-Q. Wei and S. D. Ma, Implicit and explicit camera calibration: Theory and experiments, I E E E Transactions on Pattern Analysis and Machine Intelligence, 16-5 (1994), 469-480. [2191 I. Weiss, Line fitting in a noisy image, I E E E Transactions on Pattern Analysis and Machine Intelligence, 11-3 (1989), 325-329. [220] R. Weiss, H. Nakatani and E. M. Riseman, An error analysis for surface orientation from vanishing points, I E E E Transactions on Pattern Analysis and Machine Intelligence, 12-12 (1990), 1179-1185.
[221]
J. Weng, N. Ahuja and T. S. Huang, Motion and structure from point correspondences with error estimation: Planar surfaces, I E E E Transactions on Signal Processing, 39-12 (1991), 2691-2717.
498
Bibliography
[2221 J. Weng, N. Ahuja and T. S. Huang, Optimal motion and structure estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15-9 (1993), 864-884. [2231 J. Weng, P. Cohen and M. Herniou, Camera calibration with distortion models and accuracy evaluation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-10 (1992), 965-980. [2241 J. Weng, P. Cohen and N. Rebibo, Motion and structure estimation from stereo image sequences, IEEE Transactions on Robotics and Automation, 8-3 (1992), 362-382.
[22~]
J. Weng, T. S. Huang and N. Ahuja, 3-D motion estimation, understanding, and prediction from noisy image sequences, IEEE Transactions on Pattern Analysis and Machine Intelligence, 9-3 (1987), 370-389.
[2261 J. Weng, T. S. Huang and N. Ahuja, Motion and structure from two perspective views: Algorithms, error analysis, and error estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-5 (1989), 451-467. [2271 J. Weng, T. S. Huang and N. Ahuja, Motion and Structure from Image Sequences, Springer, Berlin, F.R.G., 1993.
[2281
M. Werman and Z. Greyzel, Fitting a second degree curve in the presence of error, IEEE Transactions on Pattern Analysis and Machine Intelligence, 17-2 (1995), 207-211.
[229]
P. H. Winston (ed), Psychology of Computer Vision, McGraw-Hill, New York, NY, U.S.A., 1975.
[230] T. H. Wonnacott and R. J. Wonnacott, Introductory Statistics, 2nd Ed., John Wiley and Sons, New York, NY, U.S.A., 1972. [2311 J. J. Wu, R. E. Rink, T. M. Caelli and V. G. Gourishankar, Recovery of the 3-D location and motion of a rigid object through camera image (an extended Kalman filter approach), International Journal of Computer Vision, 2-4 (1989), 373-394. [2321 Y. Yasumoto and G. Medioni, Robust estimation of three-dimensional motion parameters from a sequence of image frames using regularization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8-4 (1986), 464471. [233] G.-S. J. Young and R. Chellappa, Statistical analysis of inherent ambiguities in recovering 3-D motion from a noisy flow field, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14-10 (1992), 995-1013. [234] Z. Zhang and O. D. Faugeras, Three-dimensional motion computation and object segmentation in a long sequence of stereo frames, International Journal of Computer Vision, 7-3 (1992), 211-241. [2351 Y. T. Zhou, V. Venkateswar and R. Chellappa, Edge detection and linear feature extraction using a 2-D random field model, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-1 (1989), 84-95. [236] X. Zhuang, A simplification to linear two-view motion algorithm, Computer Vision, Graphics, and Image Processing, 46-2 (1989), 175-178.
Bibliography
499
[237] X. Zhuang, T. S. Huang, N. Ahuja and R. M. Haralick, A simplified linear optical flow-motion algorithm, Computer Vision, Graphics, and Image Processing, 42-3 (1988), 334-344. [238] X. Zhuang, T. S. Huang and R. M. Haralick, Two-view motion analysis: A unified algorithm, Journal of the Optical Society of America, A3-9 (1986), 1492-1500.