What is wrong with mesh PCA in coordinate direction normalization

Pattern Recognition 39 (2006) 2244 – 2247 www.elsevier.com/locate/patcog

What is wrong with mesh PCA in coordinate direction normalization Heng Liua, b,∗ , Jingqi Yana , David Zhangc a Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200030, PR China b Southwest University of Science and Technology, Mianyang 621000, PR China c Department of Computing, The Hong Kong Polytechnic University, Hong Kong, PR China

Received 12 December 2005; accepted 10 May 2006

Abstract This work makes a detailed analysis on why using mesh PCA for coordinate direction normalization is always uncertainty in 3D surface registration. Our analysis takes the view of discrete signal statistical analyzing and is based on the specific process research of mesh PCA. Then we present a corrected method to improve mesh PCA effects. Such corrected method comes from the fact that the principal axes directions of 3D surface should be those in which the vertex distances are the longest among all 3D vertex distances. Corresponding experimental results on range scan data and synthetic models are provided. 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Mesh PCA; Surface registration; Coordinate direction normalization

1. Introduction Recently, there has been a constant increasing interest on surface registration for 3D pattern analysis. As in surface registration the same 3D model will have different representation if using different coordinate frames, coordinate frame normalization needs to be accomplished at first. Among all coordinate frame normalization processing, coordinate direction normalization is the most important process. For this task, mesh PCA is widely used in determining the direction transformation matrix in coordinate frame direction normalization (see Refs. [1,2] to cite a few). However, although mesh PCA method can always produce three principal axes for direction normalization, the results of coordinate transformation are uncertainty. Although work in [3] gives a simple discuss on the property, it is far from

∗ Corresponding author. Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Dong Chuan Road 800#, Min Hang District, Shanghai Jiao Tong, Shanghai 200240, China. Tel./fax: +86 2134202031. E-mail addresses: [email protected] (H. Liu), [email protected] (J. Yan), [email protected] (D. Zhang).

adequate explanation. In our work, we analyze such reasons exhaustively by discrete signal statistical view [4]. We present an improved mesh PCA method inspired by the fact that the principal axes directions of 3D surface should be those in which the vertex distances are the longest among all 3D vertex distances. Six coordinate transformation matrixes are used, respectively, in different cases for coordinate axes adjusting. Thus, mesh PCA uncertainty will be alleviated well. 2. Uncertainty of mesh PCA In this section, we first make a detailed analysis on mesh PCA uncertainty in the view of statistical signal analysis. And we conclude that three dominant factors affect the mesh PCA uncertainty. Then we present our method to improve mesh PCA results in 3D surface registration. 2.1. Mesh PCA uncertainty analysis Mesh PCA uses principal component analysis (PCA) principles to analyze surface vertices data and extract principal directions of three largest spreads of the vertice distribution.

0031-3203/$30.00 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.05.019

H. Liu et al. / Pattern Recognition 39 (2006) 2244 – 2247

Such mesh PCA transformation will change original coordinate axes to new ones while the geometry shape of 3D model will not alter. By Karhunen–Loeve (K–L) transformation and, sorted in descending order, the normalized eigenvectors of PCA caused covariance matrix are calculated as a coordinate transformation matrix. Thus, theoretically, the original coordinate axes will be transformed to three principal directions {[1, 0, 0], [0, 1, 0], [0, 0, 1]}. But in fact, the transformation result is always uncertainty. The following is a detailed analysis of mesh PCA in the view of statistical signal analysis. Let X be non-periodic random process with the autocorrelation matrix Rx . With unified sampling in [0, T ], X can be represented as X = [x(t1 ), x(t2 ), . . . , x(tD )]. Let
= { } be a set of deterministic functions, where they satisfy
Ti ∗ ∗ 0 n (t)m (t) = mn and m is the complex conjugate of m . Thus we have X =
W =

D 

w i i ,

(1)

i=1

where W is the coefficient matrix. In discrete cases, replacing
with a deterministic vector set U = {uj , j = 1, 2, . . . , ∞}, Eq. (1) can be rewritten as X=

∞ 

wi ui ,

(2)

i=1



uTi uj

=

1, i = j, 0, i  = j.

 If we use finite items to expand X as Xˆ = di=1 wi ui , M 1
d
D, truncation error is d = X − Xˆ = i=d+1 wi ui . Then mean square error (MSE) of signal X can be defined ˆ T (X − X)]. ˆ Combining with as  = E(Td d ) = E[(X − X) Eq. (2), it can be deduced as ∞   =E (3) wi2 . i=1

 T T Noticing wi = uTi X, thus  = E( ∞ i=d+1 ui XX ui ). For the deterministic vector ui , we have =

∞ 

uTi E(XX T )ui .

(4)

i=d+1

 T Obviously, Rx = E(XX T ). Therefore  = ∞ i=d+1 ui j R x ui . Lagrange factor method is used to compute numerical value of ui which will make MSE  to get the minimum, e.g., g(ui ) =

∞  i=d+1

uTi Rx ui −

∞ 

  i uTi ui − 1 .

i=d+1

Let jg(ui )/jui = 0, such formula can derive (Rx − i I )ui = 0, where j = d + 1, . . . , ∞. Thus we can get coordinate transformation matrix after normalization ui .

2245

From the above analysis of mesh PCA, we can find three dominant factors causing uncertainties: (i) In Eq. (3) MSE depends on the square of wi . Therefore MSE is not related to principal axes direction which will possibly cause the coordinate transformation result to bring an 180◦ overturn. (ii) Lagrange factor method only gets local extremum and not global minimum when computing ui . If data have much noise or are not unified sampled in different space directions, this method will possibly have quite a few minimum solution of vertices data. Corresponding to this, 3D model coordinate transformation results will be uncertainty. Furthermore, these principal coordinate axes {[1, 0, 0], [0, 1, 0], [0, 0, 1]} actually only represent three principal axes direction in the 3D surface itself and they will not correspond one by one to the real word right-hand coordinate frame known very well as X(1, 0, 0) − Y (0, 1, 0) − Z(0, 0, 1). A set of specific directions of coordinate axes should be confirmed to adjust such uncertainty axes states. (iii) Because PCA is a data statistical method, such principal axes determination depends on the all vertices space distribution law. If vertices do not comply with the same distribution principles under different sampling precision, or if noise among vertices is not introduced in the same form, mesh PCA results will be different certainly. Assuming 3D models keep the consistent sampling style, the covariance matrix N 1  C= wk ( vk − m)  · ( vk − m)  T wk k=1

can be used as PCA caused matrix, where vk is a vertex of a mesh, k is the vertex index, m  = (1/N ) N k k=1 wk v is the centroid of the mesh, wk = (N · Sk )/3S is the weight coefficient, Sk is the sum of surfaces of all triangles that have vk as a vertex, S is the surface area of the mesh (i.e. the sum of the areas of all triangles in the mesh), and N is the number of vertices. In most cases, C is invertible, and the matrix conditional number can be defined as Condp (C) = Cp C−1 p , where  · p is p-norm. If Cond p is abnormal great, matrix C will be morbid, and the greater the more. Matrix morbidity will lead to the solution of eigen vector ui to be unstable which makes coordinate transformation results uncertainty. Defining |i | ˜ i = D , I =1 |i |

i = 1, 2 . . . , D,

2246

H. Liu et al. / Pattern Recognition 39 (2006) 2244 – 2247

Fig. 1. Reconstructed ear models with different initial pose using improved mesh PCA direction normalization results. From left to right: original ear model; mesh PCA result; improved mesh PCA result.

Fig. 2. Range data models with different initial pose using improved mesh PCA direction normalization results. From left to right: original range model; mesh PCA result; improved mesh PCA result.

D ˜ i=1 i = 1. Thus mesh PCA uncertainty can be measured by entropy function:

HR = −

D 

˜ i log ˜ i .

(5)

i=1

Obviously, if all ˜ i are equal, then HR gets to its maximum. 2.2. Improvement of mesh PCA Assuming 3D models are sampled consistently equally, the basic idea to improve the uncertainty of mesh PCA is based on the observation that after coordinate transformation the longest vertex distance among a certain direction should correspond to the most principal axis and the two longer vertex distance directions correspond to the remaining two axes, respectively. Thus we can specify three fixed directions (for example {[0, 1, 0], [1, 0, 0], [0, 0, 1]}) for principal axes direction disorder adjustment.

Let matrix P be coordinate transformation matrix [u1 , u2 , u3 ], thus the improved coordinate transformation can be represented as follows:

0 1 0 MP T = 1 0 0 , (6) 0 0 1 where M is the original coordinate axes matrix and T is the corresponding rotation matrix for adjusting the coordinate axes directions. T contains six different forms according to length relationship among the longest three vertex distances along three principal axes directions:





−1 0 0 0 1 0 1 0 0 T = 0 1 0 −1 0 0 0 1 0 0 0 1 0 0 −1 0 0 1





1 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 −1 . 0 0 −1 0 −1 0 0 −1 0 Let Ni be normal vector of a triangle face, we can make the sum of all normal vector inner-products

H. Liu et al. / Pattern Recognition 39 (2006) 2244 – 2247

 S = i,j (i=j ) Ni , Nj  is greater than zero to guarantee the 3D surface visibility before coordinate transformation. Thus, the improved mesh PCA method is described in the following: (i) Re-sampling 3D object consistently evenly in the unified way and utilizing Gauss signal filter to wipe off vertex noise. (ii) Confirming surface visibility (if S < 0, make all faces normal vectors to be inverse, i.e., Ni = −Ni ). (iii) Treating matrix P to be the first coordinate transformation matrix to transform all vertices of surface. Confirming transformed surface visibility as the method stated in (ii). (iv) Using matrix T as the second coordinate transformation matrix to obtain the improved result. 3. Experimental results We first use synthetic 3D models, coming from multi-view image reconstruction (vertice numbers are about 100–300, mesh numbers are about 200–500), to show uncertainty of mesh PCA. Meshes will be subdivided or simplified in different sizes when needed. Fig. 1 shows such model direction normalization results. Experimental results on range data (vertice numbers are about 5000–8000, mesh numbers are about 10 000–20 000) are also proved in Fig. 2. Every model mesh PCA uncertainty guideline is presented in these two figures. In practical computing, we first calculate the covariance matrix conditional number Cond p (C) (taking ∞ norm). If it is too great, we need to resample the 3D model. The corresponding mesh PCA uncertainty value HR is computed also. The bigger the HR , the more the uncertainty using mesh PCA. From our experimental results, we can see the uncertainty of synthetic data is bigger than range data. The main reason is vertice distribution of synthetic data are not as unified as that of range data. Another reason is that synthetic data contain more noise than that in range data. Moreover,

2247

in many experiments, all model mesh PCA uncertainties are greater than 0.5. And, we find even improved mesh PCA method is not perfect for coordinate adjusting (accurate rate is about 92%). Actually, this is because mesh PCA is a linear analysis tool while coordinate direction normalization is a non-linear problem. 4. Conclusion We have made a detailed analysis on mesh PCA uncertainty in coordinate direction normalization for 3D surface registration. We present a fast method to improve mesh PCA result based on the vertex distance longest selection principles. By six rotation transformation, our method partially overcomes the problem on mesh PCA uncertainty. In fact, as PCA is a lineal technology essentially, mesh PCA method will face more challenging problems on coordinate direction normalization procedure. Thus we will make a further research on non-linear methods for coordinate direction normalization in 3D surface registration. Acknowledgments This work is supported by National Natural Science Foundation of China (No. 60402020). In addition, we would like to thank for all those people who have contributed to this work by providing their data and comments. References [1] D.V. Vranic, D. Saupe, J. Richter. Tools for 3D object retrieval: Karhunen–Loeve transform and spherical harmonics, in: Proceedings of the IEEE 2001 Workshop on Multimedia Signal Processing, 2001, pp. 293–298. [2] J.-J. Song, F. Golshani, Shape-based 3D model retrieval, Proceedings of the International Conference on Tools with Artificial Intelligence, 2003, pp. 636–640. [3] Zhi-yong Zhang. The Research on 3D Model Geometry Similarity Comparisons, Ph.D Thesis, Zhe Jiang University (China), 2004. [4] A.R. Webb, Statistics Pattern Recognition, second ed., Wiley, New York, 2002.