- Email: [email protected]
Surface mesh denoising via diffusing gradient field
Optics and Lasers in Engineering 49 (2011) 104–109
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Surface mesh denoising via diffusing gradient field Jianhuang Wu a,b,n, Jinting Xu c, Renbo Xia d a
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China The Chinese University of Hong Kong, Hong Kong, China School of Automotive Engineering, Dalian University of Technology, Dalian 116024, China d Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China b c
a r t i c l e in f o
a b s t r a c t
Article history: Received 27 May 2010 Received in revised form 26 July 2010 Accepted 18 August 2010 Available online 15 September 2010
Surface mesh denoising is an important preprocess for many digital geometrical applications. Unlike most previous smoothing algorithms, which are vertex- or normal-based, we present a smoothing approach based on gradient field. The proposed algorithm first applies an extended non-local means image filter to diffuse mesh gradient field and then reconstructs a smoothed surface from the smoothed field by solving the Poisson equations in the least-squares sense. Experimental results on various data demonstrate that the proposed approach can successfully remove noise from meshes while preserving geometrical detail features of the original object and avoiding significant volume shrinkage. & 2010 Elsevier Ltd. All rights reserved.
Keywords: Mesh processing Mesh denoising Gradient field Non-local means filtering
1. Introduction Advances in optics and three-dimensional (3D) laser scanner make 3D models widely available in many applications such as reverse engineering and entertainment industry. Also from medical image generation methods like CT and MRI devices, surfaces of interest are usually extracted for 3D visualization and quantitative analysis. However, due to the physical scanning devices, digitization process and modeling process, the reconstructed surface models, typically represented as triangle meshes, often contain undesirable noise. Such noise can degrade the visual quality and usability of 3D modes. Surface mesh denoising or smoothing, while preserving the geometrical details of original object as realistic as possible, therefore, is an important preprocess for many digital geometrical applications. Surface mesh denoising is one of the intensive research fields in computer graphics. Currently, many techniques are proposed and these techniques can be classified roughly into two categories: isotropic techniques and anisotropic techniques. Isotropic techniques: Laplacian smoothing is simple and efficient, but usually leads to volume shrinkage and oversmoothing. Therefore, different kinds of weighted Laplacian techniques are presented [1]. From the signal point of view, Taubin [2] presented an improved method based on the definition of Laplacian operator on surface meshes. Vollmer et al. [3] handled
n Corresponding author at: Chinese Academy of Sciences, Shenzhen Institutes of Advanced Technology, 1068 Xueyuan Boulvevard, Shenzhen 518055, China. E-mail addresses: [email protected], [email protected] (J. Wu).
0143-8166/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.optlaseng.2010.08.014
the shrinkage problem by moving each vertex back towards a weighted average of its original location and its previous location. Desbrun et al. [4] further developed Laplacian smoothing to irregular meshes using diffusion and mean curvature flow and scaled the mesh back to its original volume after each integration step in an attempt to achieve volume preservation. More recently, noise removal based on differential coordinates [5] is discussed in [6]. However, isotropic methods denoise the surface indiscriminately and hence prominent surface details are lost after smoothing. Anisotropic techniques: One class of anisotropic techniques is inspired by image processing filters. Fleishman et al. [7] proposed a feature-preserving mesh denoising method via shifting vertices along their normal directions in which speed is determined by bilateral filtering of the heights of neighboring vertices over the tangent plane. Independently, Jones et al. [8] presented a similar method based on statistics and local first-order predictors of the meshes. Ohtake et al. [9] first smoothed face normals with anisotropic Gaussian filter and then reconstructed a smoothed surface from the smoothed normal field. Similarly, Yagou et al. [10] applied mean and median filter to normals for anisotropic smoothing. Other image filters generalized to anisotropic denoising include Laplacian filter [11], fuzzy vector median filter [12], non-local means filter [13], random walk filter [14] and SUSAN filter [15]. Another type of anisotropic scheme is based on geometric diffusion. Desbrun et al. [16] introduced a geometric diffusion flow for irregular meshes via minimizing surface area flow. Hildebrandt and Polthier [17] improved this method based on defining discrete shape operator and prescribed mean curvature
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
flow. Clarenz et al. [18] formulated the anisotropic diffusion for smoothing as an anisotropic curvature evolution problem. Using a level set method, Tasdizen et al. [19] derived a variational partial differential equation (PDE) on the normal map that minimizes total curvature and then fit the vertex to the processed normals. Bajaj and Xu [20] suggested a procedure by combining an established anisotropic geometric diffusion PDE model with the limit function of the Loop subdivision scheme. These anisotropic smoothing techniques, however, are computationally expensive due to the PDEs or large linear systems. Very recently, supposing the underlying noisy mesh is piecewise smooth, Fan et al. [21] introduced an anisotropic technique by defining consistent subneighborhoods. The above-mentioned approaches for surface denoising can also be roughly classified into two categories: vertex-based and normal-based. In the first category, noise removal is achieved by iteratively moving vertices along their normals with a step or by constructing an optimization problem that minimizes certain functions, such as membrane energy and curvatures [1,7,4,18]. The main idea of the second category is first diffusing the vertex normal or face normal with filters and then manipulating the surface to fit the modified normals by solving the linear or nonlinear systems [9,19,22]. Compared differently with most previous approaches, we present in this paper a surface mesh denoising algorithm based on gradient field defined over meshes [22–24]. We first diffuse gradient field defined over meshes with an extended non-local means filter and then reconstruct a smooth surface from the smoothed vector field by solving the Poisson equations. The key observation behind our method is that gradient field can be viewed as signal defined over its domain, and thus signal and image processing filters can be applied to diffuse the vector field. By exploiting gradient field, local details of surface can be well preserved during the denoising procedure due to the gradient operator encapsulating differential properties of the local surface. At the same time, our method can achieve volume preservation and global optimization as the smoothed surface is reconstructed in the least-squares sense with barycenter constraints [25]. An example for the effectiveness of our method is illustrated in Fig. 1. In this paper, we use non-local means filter [26] to diffuse gradient field. To the best of our knowledge, there is no previous works on non-local means filtering of mesh gradient fields. With this filter, a smoother vector field can be robustly obtained since it takes into consideration the similarity between local neighborhood of the pixel being processed and the other pixels. However, one main difficulty of extending this filter to process mesh is how to define similarity neighborhood. We address this problem by
105
fitting a local bicubic B-spline surface. With our method, the similarity between neighborhoods can be easily measured when parameters are given.
2. Methods and materials 2.1. Gradient field Let M¼ (V, E, F) be a given triangular mesh with a set of vertices V¼{i91rirn} in Euclidean space R3. E is the set of edges and F denotes the set of faces. The vertex neighborhood (also referred to as 1-ring) of a vertex vi is the set of vertices connected by an edge to vi, represented by N(i)¼{j9(i,j)AE}. In this paper, surfaces are assumed to be manifolds. Manifold surface is a surface whose points have a neighborhood topologically equivalent to a disk (boundary points are topologically equivalent to a half-disk). Due to the mesh having vertices position in 3D space, the mesh itself can be viewed as three scalar fields defined over a certain domain, which is actually an abstract mesh called domain mesh [24]. Therefore, a mesh vector field over its domain is expressed as [22,24,27] X f ðvÞ ¼ ji ðvÞfi ð1Þ i
where ji( ) is a piecewise-linear basic function valued 1 at vertex vi and 0 at all other vertices; fi is the scalar value of vertex vi over the domain mesh. For convenience, discrete differential operators are often used to approximately describe the differential properties of continuous surface. For a mesh scalar function f, its discrete gradient operator over the domain mesh is defined as X rf ðvÞ ¼ rji ðvÞfi ð2Þ i
Given a piecewise constant vector field, where each triangle t A F over domain has a constant value w(t), the discrete divergence at vertex vi is expressed as X ðdiv wÞðvi Þ ¼ wðtÞSðtÞrji ðvÞ ð3Þ t A T ðvi Þ
where S(t) is the area of triangle t. The Laplace operator at each vertex vi is X 1 Df ðvi Þ ¼ ð4Þ 2ðcot aj þcot bj Þðfi fj Þ vj A Gðvi Þ
Fig. 1. Denoising a 3D model of Chinese ancient soldier with 149,996 vertices and 299,992 triangles. (a) Original noisy mesh. (b) Original mesh colored by mean curvature. (c) Mesh is smoothed by gradient field based method developed in this paper. (d) Smoothed mesh is colored by mean curvature. Note that details of surface and volume are preserved.
106
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
where aj and bj are the two angles opposite to the edge in two triangles having a common edge as illustrated in Fig. 2. Surface reconstruction: With the Poisson equation, the smoothed surface can be reconstructed from the above definition of vector field. The discretization of the Poisson equation with Direchlet boundary conditions is formulated as [22,24]
Df divðrf Þ ¼ divðwÞ,f 9@O ¼ f 9@O
ð5Þ
where f nprovides the desirable value on the boundary qO. For a mesh, Eq. (5) can be expressed as a sparse linear system: Ax ¼ b
ð6Þ
where A is a coefficient matrix defined by Eq. (4), the unknown vector x denotes the x coordinate vector to be reconstructed (the same goes for y and z coordinate vectors) and the vector b is the known vector field. To prevent volume shrinkage, we add the barycenter constraints [25] for each face to the linear system. Then we have the following linear system: A C
x¼
b
2.2. Non-local means filter We begin this section by simply introducing the non-local means filtering for images and then extending it to process gradient field of 3D mesh. Non-local image filter: The basic idea behind non-local means is that it assumes the image contains an extensive amount of selfsimilarity, and thus similarity is not measured conventionally based on single pixel but it is measured based on the geometrical configuration in the whole neighborhood. ~ For a given noisy image I¼{I(u)9uAO}, the estimated value IðuÞ at pixel u is obtained by a weighted average of all pixels in its neighborhood Y [26]: P v A Y cðu,vÞIðvÞ ~ IðuÞ ¼ P ð10Þ v A Y cðu,vÞ where weight c(u,v) is computed by the similarity of the Gaussian neighborhood between pixels u and v, which is defined as 0 1 2 99Yu Yv 992,a A ð11Þ cðu,vÞ ¼ exp@ h2
ð7Þ
c
where C is an m n matrix in which each row has only three non-zero elements whose value is 1/3 and the position corresponds to the index of vertices constituting the face, m is the number of faces of the mesh and c the vector of barycenter coordinate x of faces. The additional constraints make the linear system overdetermined and theoretically no accurate solution may exist. Fortunately, the system is full-rank and therefore there exists a unique solution in the least-squares sense: 2 x~ ¼ arg minð99Axb99 þ x
X
2
ðxÞ ðxÞ 9913ðvðxÞ i þvj þ vk Þc99 Þ
ð8Þ
where Yu is a square neighborhood centered at pixel u, 99 992,a the Gaussian weighted Euclidean distance function, a the standard deviation of the Gaussian kernel and h the delay parameter. Extension to 3D mesh: Gradient field can be considered as signal defined over its domain, and therefore we can apply non-local means filter to diffuse the vector field. However, when extending 2D non-local means scheme to process 3D mesh, one problem is the determination of similarity neighborhood, as images usually have a regular structure, which in most cases is not true for mesh. Yoshizawa et al. [13] constructed a local approximation based on radial basis functions to define the similarity kernel. In this work, we determinate the similarity neighborhood of a given vertex vi by fitting local bicubic B-spline surfaces [28]:
ði,j,kÞ A F
Q ðu,vÞ ¼
4 X 4 X
Bi,j Ni,k ðuÞMj,l ðvÞ
ð12Þ
i¼1j¼4
Rewriting the coefficient matrix of Eq. (7) as an (n + m)n matrix A~ and denoting the right-hand side vector of the linear system by ðxÞ T b~ ¼ ðb,c1ðxÞ ,c2ðxÞ , ,cm Þ , we have the analytical least-squares solution of Eq. (7): T ~ 1 A~ T b~ x~ ¼ ðA~ AÞ
ð9Þ
For implementation, direct solver and Cholesky factorization T ~ ~ x~ ¼ A~ T b. are applied to the associated normal equations, i.e. ðA~ AÞ T ~ ~ As matrix M ¼ ðA AÞis sparse and positive by definition, we can obtain a Cholesky factorization of M, e.g. M¼RTR, where R is an upper-triangular sparse matrix. Then the three vectors of coordinates can be solved by backsubstitution, for instance, ~ Rx ¼ x. RT x ¼ A~ b,
where Bi,j are control points and in their natural ordering; they are the vertices of the B-spline control net, as shown in Fig. 3, Ni,k(u) and Mj,l(v) are the B-spline basic functions in the biparametric u and v directions, respectively. The 2-ring vertices of vi are used as input data points and are parameterized by projecting them onto a plane (e.g. the tangent plane of vi) and scaling them to the range [0,1]. The advantage of using B-spline surfaces is that the underlying control net is topologically similar to the image grid structure, and thus the similarity between neighborhoods in 3D can be more easily measured by computing the corresponding 3D points on the surface when given uniformly parametric u and v in the u–v space of the B-spline surface.
B4,4
vj B4,1 vj+1
j
j
vj−1
vi
vi B1,1 Fig. 2. Vertex and its adjacent triangles.
B1,4
Fig. 3. Neighborhood for a vertex by fitting a local bicubic B-spline surface.
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
In the non-local image filter, the parameter h controlling spatial support of the filter is global and is set manually. In our work, h is determined adaptively based on the variance s of edge length distribution in the similarity neighborhood. In our experience, h¼ (10–15)s has a good performance.
107
3. Experimental results and discussion We demonstrate results of our denoising method in Figs. 1 and 4–6. We test our algorithm on various data. It is applied to the scanned data model (Figs. 1 and 5), to CAD models (Fig. 4) as well
Fig. 4. Comparison of feature-preservation properties of bilateral filtering, Yoshizawa et al.’s [13] algorithm and our approach on a CAD model. (a) The model is artificially corrupted by Gaussian noise (the variance is 1/5 of the mean edge length). (b) The model is smoothed by bilateral filtering. (c) The model is smoothed by Yoshizawa et al.’s [13] algorithm. (d) Model is smoothed by our approach. The mean curvature visualizations (bottom row) show that our approach preserves salient features more accurately than bilateral filtering and Yoshizawa et al. [13] algorithm, e.g. the rectangle region.
Fig. 5. Comparison of detail-preservation properties of the bilateral filtering and our approach on a Dragon model. (a) The model is artificially corrupted by Gaussian noise (the variance is 1/5 of the mean edge length). (b) The model is smoothed by bilateral filtering after two iterations. (c) The model is smoothed by our approach. The mean curvature visualizations show that our approach preserves details of surface more accurately than bilateral filtering.
108
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
Fig. 6. Comparison of smoothed results on medical data. (a) Vessel surface extracted by Marching Cubes. (b) Smoothed surface obtained by Yoshizawa et al.’s [13] algorithm. (c) Smoothed surface obtained by our approach. The zoomed region of vessel branch shows that our approach preserves finer vascular structures better than that of Yoshizawa et al.’s [13] algorithm, which leads to collapse of thin structures.
Table 1 Parameters setting and timings results. Here i stands for the number of iterations, s and N are the sizes of the neighborhoods used to measure the similarity, vol denotes the ratio of volume and ave denotes the average error of method noise. Mean of {sc,ss}and {Z1,Z2,Z3}are explained in Refs. [7] and [13], respectively. All results are tested on a Pentium D processor at 2.80 GHz with a 3.00 GB memory. Bilateral filtering Model
sc
ss
i
vol
ave
Time (s)
Fandisk Dragon Vessel
1.5 1.5 1.2
2.0 1.8 1.5
3 3 3
0.9605 0.9789 0.9758
0.006 0.006 0.007
0.34 10.04 1.51
Yoshizawa et al. [13] algorithm Model Z1
Z2
Z3
1 1 1
0.75 0.75 0.75
i
vol
ave
Time (s)
3 3 2
0.9801 0.9827 0.9795
0.004 0.005 0.005
9.83 176.5 28.75
Fandisk Dragon Vessel
0.4 0.35 0.45
Our approach Model
s
N
h
i
vol
ave
Time (s)
Fandisk Dragon Vessel
7 7 7
17 17 17
10 12 10
3 3 2
0.9989 0.9956 0.9990
0.002 0.004 0.003
7.75 149.1 22.42
as to the medical data model (Fig. 6). We compare our results with the bilateral filtering algorithm [7] and Yoshizawa et al.’s [13] algorithm. Table 1 shows timing results and parameter settings used for bilateral filtering and our approach. We use method noise [13,26] to measure the smoothed surfaces. The method noise in meshes is defined as the differences between the original and smoothed meshes, i.e. e ¼9voriginal vsmoothed9. In Fig. 1, we show the efficiency of our method on real-world scanned data. The images show that high-frequency noise on the Chinese ancient soldier is removed after only one iteration while lower-frequency details such as armor are well preserved. At the same time, the ratio of volume is 99.98%. Fig. 4 shows a comparison of the bilateral filter, Yoshizawa et al.’s [13] algorithm and our approach with regard to feature
preservation. It indicates that our approach produces a smooth surface by preserving sharp features more accurately than the bilateral filtering and Yoshizawa et al.’s [13] algorithm. In addition, the mean error of method noise produced by our method is 0.0023, which is smaller than that of both bilateral filter and Yoshizawa et al.’s [13] algorithm, i.e. 0.0067 and 0.0048, respectively. Fig. 5 illustrates a comparison of the bilateral filter and our approach concerning fine details preservation. It is observed from the mean curvature visualizations that our approach preserves local details such as scuba much better than that of bilateral filtering due to the gradient operator encapsulating local differential geometry and the diffusing filter averaging with similaritybased weights, which measure geometrical information in a local region. Furthermore, our method preserves high-curvature and
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
low-curvature regions accurately, for instance, the ridges and the ravines. In Fig. 6, we demonstrate a comparison of Yoshizawa et al.’s [13] algorithm and our approach on medical data. Due to the partial volume effect and the segmentation process, the vessel surface extracted from binary volume data in general contains artifacts. These artifacts will result in numerical problem if the surface is used for computational fluid dynamics simulation. On the other hand, a vessel surface after iterative smoothing will produce collapse of thin structures and may even be misinterpreted as false stenosis if the smoothing technique brings obvious volume shrinkage. It is shown in Fig. 6 that our approach preserves finer and elongated vessel structures better than Yoshizawa et al.’s [13] algorithm, which collapses thin structures and leads to further distortions and detached object parts.
4. Conclusion In this paper, we have presented a novel denoising method for surface meshes based on gradient field and non-local means filtering. To measure the similarity between neighborhoods and irregular meshes, we introduced a local approximation utilizing bicubic B-spline surfaces. The performances of various data show that our approach can preserve local details of surface and can avoid volume shrinkage during the denoising process. We also compared the proposed algorithm with the conventional mesh smoothing technique. The comparison demonstrated that the proposed algorithm outperformed the conventional one in terms of preserving fine details and preventing volume shrinkage. However, the speed of our method is too slow when compared to the efficient bilateral filtering technique [29], and because solving linear systems and computing similarities between neighborhoods of each vertex are both time-consuming procedures. One interesting direction of our future research is to develop a parallel smoothing strategy for our method.
Acknowledgments The Dragon model is courtesy of Stanford University. This work was supported by the National Natural Science Foundation of China under Grant No. 60803108. References [1] Kobbelt L, Campagna S, Vorsatz J, Seidel HP. Interactive multi-resolution modeling on arbitrary meshes. In: Proceedings of SIGGRAPH, 1998, p.105–14. [2] Taubin G. A signal processing approach to fair surface design. In: Proceedings of SIGGRAPH, 1995. p. 351–8. ¨ [3] Vollmer J, Mencl R, Muller H. Improved Laplacian smoothing of noisy surface meshes. Comput Graph Forum 2001;18(3):131–8.
109
¨ [4] Desbrun M, Meyer M, Schroder P, Barr AH. Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of SIGGRAPH, 1999. p. 317–324. [5] Alexa M. Differential coordinates for local mesh morphing and deformation. Vis Comput 2003;19:105–14. [6] Su Z, Wang H, Cao J. Mesh denoising based on differential coordinates. In: Proceedings of the IEEE international conference on shape modeling and applications, 2009. p. 1–6. [7] Fleishman S, Drori I, Cohen-Or D. Bilateral mesh denoising. ACM Trans Graph 2003;22(3):950–3. [8] Jones TR, Durand F, Desbrun M. Non-iterative feature-preserving mesh smoothing. ACM Trans Graph 2003;22(3):943–9. [9] Ohtake Y, Belyaev AG, Seidel HP. Mesh smoothing by adaptive and anisotropic Gaussian filter. In: Proceedings of the vision, modeling, and visualization conference, 2002. p. 203–10. [10] Yagou H, Ohtake Y, Belyaev AG. Mesh smoothing via mean and median filtering applied to face normals. In: Proceedings of the geometric modeling and processing conference, 2002. p. 124–31. [11] Taubin G. Linear anisotropic mesh filtering. IBM research report RC22213 (W0110-051), IBM, October 2001. [12] Shen Y, Barner KE. Fuzzy vector median-based surface smoothing. IEEE Trans Visual Comput Graphics 2004;10(3):252–65. [13] Yoshizawa S, Belyaev AG, Seidel HP. Smoothing by example: mesh denoising by averaging with similarity-based weights. In: Proceedings of the shape modeling international conference, 2006. p. 38–44. [14] Sun X, Rosin PL, Martin RR, Langbein FC. Random walks for featurepreserving mesh denoising. Comput Aided Geom Des 2008;25: 437–56. [15] Mao Z, Ma L, Zhao M, Xiao X. SUSAN structure preserving filtering for mesh denoising. Vis Comput 2006;22:276–84. ¨ [16] Desbrun M, Meyer M, Schroder P, Barr AH. Anisotropic feature-preserving denoising of height fields and bivariate data. In: Proceedings of the graphics interface conference, 2000. p. 145–52. [17] Hildebrandt K, Polthier K. Anisotropic filtering of non-linear surface features. Comput Graph Forum 2004;23(3):391–400. [18] Clarenz U, Diewald U, Rumpf M. Anisotropic geometric diffusion in surface processing. In: Proceedings of the IEEE visualization conference, 2000. p. 397–405. [19] Tasdizen T, Whitaker R, Burchard P, Osher S. Geometric surface smoothing via anisotropic diffusion of normals. In: Proceedings of the IEEE visualization; 2002. p. 125–32. [20] Bajaj CL, Xu G. Anisotropic diffusion of subdivision surfaces and functions on surfaces. ACM Trans Graph 2003;22(1):4–32. [21] Fan H, Yu Y, Peng Q. Robust feature-preserving mesh denoising based on consistent subneighborhoods. IEEE Trans Visual Comput Graph 2010;16(2):312–24. [22] Yu Y, Zhou K, Xu D, Shi X, Bao H, Guo B, Shum HY. Mesh editing with Poissonbased gradient field manipulation. ACM Trans Graph 2004;23(3): 644–51. [23] Huang J, Shi X, Liu XG, Zhou K, Wei LY, Teng SH, Bao HJ, Guo BN, Shum HY. Subspace gradient domain mesh deformation. ACM Trans Graph 2006;25(3):1126–34. [24] Xu D, Zhang H, Wang Q, Bao H. Poisson shape interpolation. In: Proceedings of the ACM solid and physical modeling conference, 2005. p. 267–74. [25] Liu XG, Bao HJ, Heng PA, Wong TT, Peng QS. Constrained fairing for meshes. Comput Graph Forum 2001;20(12):115–23. [26] Buades A, Coll B, Morel JM. A non-local algorithm for image denoising. In: Proceedings of the computer vision and pattern recognition conference, 2005, vol. 2. p. 60–5. [27] Tong Y, Lombeyda S, Hirani AN, Desbrun M. Discrete multiscale vector field decomposition. ACM Trans Graph 2003;22(3):445–52. [28] Rogers DF. An introduction to NURBS: with historical perspective.. Morgan Kaufmann; 2001. p. 179–81. [29] Sun XF, Rosin PL, Martin RR, Langbein FC. Fast and effective featurepreserving mesh denoising. IEEE Trans Visual Comput Graph 2007;13(5):925–38.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Surface mesh denoising via diffusing gradient field Jianhuang Wu a,b,n, Jinting Xu c, Renbo Xia d a
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China The Chinese University of Hong Kong, Hong Kong, China School of Automotive Engineering, Dalian University of Technology, Dalian 116024, China d Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China b c
a r t i c l e in f o
a b s t r a c t
Article history: Received 27 May 2010 Received in revised form 26 July 2010 Accepted 18 August 2010 Available online 15 September 2010
Surface mesh denoising is an important preprocess for many digital geometrical applications. Unlike most previous smoothing algorithms, which are vertex- or normal-based, we present a smoothing approach based on gradient field. The proposed algorithm first applies an extended non-local means image filter to diffuse mesh gradient field and then reconstructs a smoothed surface from the smoothed field by solving the Poisson equations in the least-squares sense. Experimental results on various data demonstrate that the proposed approach can successfully remove noise from meshes while preserving geometrical detail features of the original object and avoiding significant volume shrinkage. & 2010 Elsevier Ltd. All rights reserved.
Keywords: Mesh processing Mesh denoising Gradient field Non-local means filtering
1. Introduction Advances in optics and three-dimensional (3D) laser scanner make 3D models widely available in many applications such as reverse engineering and entertainment industry. Also from medical image generation methods like CT and MRI devices, surfaces of interest are usually extracted for 3D visualization and quantitative analysis. However, due to the physical scanning devices, digitization process and modeling process, the reconstructed surface models, typically represented as triangle meshes, often contain undesirable noise. Such noise can degrade the visual quality and usability of 3D modes. Surface mesh denoising or smoothing, while preserving the geometrical details of original object as realistic as possible, therefore, is an important preprocess for many digital geometrical applications. Surface mesh denoising is one of the intensive research fields in computer graphics. Currently, many techniques are proposed and these techniques can be classified roughly into two categories: isotropic techniques and anisotropic techniques. Isotropic techniques: Laplacian smoothing is simple and efficient, but usually leads to volume shrinkage and oversmoothing. Therefore, different kinds of weighted Laplacian techniques are presented [1]. From the signal point of view, Taubin [2] presented an improved method based on the definition of Laplacian operator on surface meshes. Vollmer et al. [3] handled
n Corresponding author at: Chinese Academy of Sciences, Shenzhen Institutes of Advanced Technology, 1068 Xueyuan Boulvevard, Shenzhen 518055, China. E-mail addresses: [email protected], [email protected] (J. Wu).
0143-8166/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.optlaseng.2010.08.014
the shrinkage problem by moving each vertex back towards a weighted average of its original location and its previous location. Desbrun et al. [4] further developed Laplacian smoothing to irregular meshes using diffusion and mean curvature flow and scaled the mesh back to its original volume after each integration step in an attempt to achieve volume preservation. More recently, noise removal based on differential coordinates [5] is discussed in [6]. However, isotropic methods denoise the surface indiscriminately and hence prominent surface details are lost after smoothing. Anisotropic techniques: One class of anisotropic techniques is inspired by image processing filters. Fleishman et al. [7] proposed a feature-preserving mesh denoising method via shifting vertices along their normal directions in which speed is determined by bilateral filtering of the heights of neighboring vertices over the tangent plane. Independently, Jones et al. [8] presented a similar method based on statistics and local first-order predictors of the meshes. Ohtake et al. [9] first smoothed face normals with anisotropic Gaussian filter and then reconstructed a smoothed surface from the smoothed normal field. Similarly, Yagou et al. [10] applied mean and median filter to normals for anisotropic smoothing. Other image filters generalized to anisotropic denoising include Laplacian filter [11], fuzzy vector median filter [12], non-local means filter [13], random walk filter [14] and SUSAN filter [15]. Another type of anisotropic scheme is based on geometric diffusion. Desbrun et al. [16] introduced a geometric diffusion flow for irregular meshes via minimizing surface area flow. Hildebrandt and Polthier [17] improved this method based on defining discrete shape operator and prescribed mean curvature
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
flow. Clarenz et al. [18] formulated the anisotropic diffusion for smoothing as an anisotropic curvature evolution problem. Using a level set method, Tasdizen et al. [19] derived a variational partial differential equation (PDE) on the normal map that minimizes total curvature and then fit the vertex to the processed normals. Bajaj and Xu [20] suggested a procedure by combining an established anisotropic geometric diffusion PDE model with the limit function of the Loop subdivision scheme. These anisotropic smoothing techniques, however, are computationally expensive due to the PDEs or large linear systems. Very recently, supposing the underlying noisy mesh is piecewise smooth, Fan et al. [21] introduced an anisotropic technique by defining consistent subneighborhoods. The above-mentioned approaches for surface denoising can also be roughly classified into two categories: vertex-based and normal-based. In the first category, noise removal is achieved by iteratively moving vertices along their normals with a step or by constructing an optimization problem that minimizes certain functions, such as membrane energy and curvatures [1,7,4,18]. The main idea of the second category is first diffusing the vertex normal or face normal with filters and then manipulating the surface to fit the modified normals by solving the linear or nonlinear systems [9,19,22]. Compared differently with most previous approaches, we present in this paper a surface mesh denoising algorithm based on gradient field defined over meshes [22–24]. We first diffuse gradient field defined over meshes with an extended non-local means filter and then reconstruct a smooth surface from the smoothed vector field by solving the Poisson equations. The key observation behind our method is that gradient field can be viewed as signal defined over its domain, and thus signal and image processing filters can be applied to diffuse the vector field. By exploiting gradient field, local details of surface can be well preserved during the denoising procedure due to the gradient operator encapsulating differential properties of the local surface. At the same time, our method can achieve volume preservation and global optimization as the smoothed surface is reconstructed in the least-squares sense with barycenter constraints [25]. An example for the effectiveness of our method is illustrated in Fig. 1. In this paper, we use non-local means filter [26] to diffuse gradient field. To the best of our knowledge, there is no previous works on non-local means filtering of mesh gradient fields. With this filter, a smoother vector field can be robustly obtained since it takes into consideration the similarity between local neighborhood of the pixel being processed and the other pixels. However, one main difficulty of extending this filter to process mesh is how to define similarity neighborhood. We address this problem by
105
fitting a local bicubic B-spline surface. With our method, the similarity between neighborhoods can be easily measured when parameters are given.
2. Methods and materials 2.1. Gradient field Let M¼ (V, E, F) be a given triangular mesh with a set of vertices V¼{i91rirn} in Euclidean space R3. E is the set of edges and F denotes the set of faces. The vertex neighborhood (also referred to as 1-ring) of a vertex vi is the set of vertices connected by an edge to vi, represented by N(i)¼{j9(i,j)AE}. In this paper, surfaces are assumed to be manifolds. Manifold surface is a surface whose points have a neighborhood topologically equivalent to a disk (boundary points are topologically equivalent to a half-disk). Due to the mesh having vertices position in 3D space, the mesh itself can be viewed as three scalar fields defined over a certain domain, which is actually an abstract mesh called domain mesh [24]. Therefore, a mesh vector field over its domain is expressed as [22,24,27] X f ðvÞ ¼ ji ðvÞfi ð1Þ i
where ji( ) is a piecewise-linear basic function valued 1 at vertex vi and 0 at all other vertices; fi is the scalar value of vertex vi over the domain mesh. For convenience, discrete differential operators are often used to approximately describe the differential properties of continuous surface. For a mesh scalar function f, its discrete gradient operator over the domain mesh is defined as X rf ðvÞ ¼ rji ðvÞfi ð2Þ i
Given a piecewise constant vector field, where each triangle t A F over domain has a constant value w(t), the discrete divergence at vertex vi is expressed as X ðdiv wÞðvi Þ ¼ wðtÞSðtÞrji ðvÞ ð3Þ t A T ðvi Þ
where S(t) is the area of triangle t. The Laplace operator at each vertex vi is X 1 Df ðvi Þ ¼ ð4Þ 2ðcot aj þcot bj Þðfi fj Þ vj A Gðvi Þ
Fig. 1. Denoising a 3D model of Chinese ancient soldier with 149,996 vertices and 299,992 triangles. (a) Original noisy mesh. (b) Original mesh colored by mean curvature. (c) Mesh is smoothed by gradient field based method developed in this paper. (d) Smoothed mesh is colored by mean curvature. Note that details of surface and volume are preserved.
106
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
where aj and bj are the two angles opposite to the edge in two triangles having a common edge as illustrated in Fig. 2. Surface reconstruction: With the Poisson equation, the smoothed surface can be reconstructed from the above definition of vector field. The discretization of the Poisson equation with Direchlet boundary conditions is formulated as [22,24]
Df divðrf Þ ¼ divðwÞ,f 9@O ¼ f 9@O
ð5Þ
where f nprovides the desirable value on the boundary qO. For a mesh, Eq. (5) can be expressed as a sparse linear system: Ax ¼ b
ð6Þ
where A is a coefficient matrix defined by Eq. (4), the unknown vector x denotes the x coordinate vector to be reconstructed (the same goes for y and z coordinate vectors) and the vector b is the known vector field. To prevent volume shrinkage, we add the barycenter constraints [25] for each face to the linear system. Then we have the following linear system: A C
x¼
b
2.2. Non-local means filter We begin this section by simply introducing the non-local means filtering for images and then extending it to process gradient field of 3D mesh. Non-local image filter: The basic idea behind non-local means is that it assumes the image contains an extensive amount of selfsimilarity, and thus similarity is not measured conventionally based on single pixel but it is measured based on the geometrical configuration in the whole neighborhood. ~ For a given noisy image I¼{I(u)9uAO}, the estimated value IðuÞ at pixel u is obtained by a weighted average of all pixels in its neighborhood Y [26]: P v A Y cðu,vÞIðvÞ ~ IðuÞ ¼ P ð10Þ v A Y cðu,vÞ where weight c(u,v) is computed by the similarity of the Gaussian neighborhood between pixels u and v, which is defined as 0 1 2 99Yu Yv 992,a A ð11Þ cðu,vÞ ¼ exp@ h2
ð7Þ
c
where C is an m n matrix in which each row has only three non-zero elements whose value is 1/3 and the position corresponds to the index of vertices constituting the face, m is the number of faces of the mesh and c the vector of barycenter coordinate x of faces. The additional constraints make the linear system overdetermined and theoretically no accurate solution may exist. Fortunately, the system is full-rank and therefore there exists a unique solution in the least-squares sense: 2 x~ ¼ arg minð99Axb99 þ x
X
2
ðxÞ ðxÞ 9913ðvðxÞ i þvj þ vk Þc99 Þ
ð8Þ
where Yu is a square neighborhood centered at pixel u, 99 992,a the Gaussian weighted Euclidean distance function, a the standard deviation of the Gaussian kernel and h the delay parameter. Extension to 3D mesh: Gradient field can be considered as signal defined over its domain, and therefore we can apply non-local means filter to diffuse the vector field. However, when extending 2D non-local means scheme to process 3D mesh, one problem is the determination of similarity neighborhood, as images usually have a regular structure, which in most cases is not true for mesh. Yoshizawa et al. [13] constructed a local approximation based on radial basis functions to define the similarity kernel. In this work, we determinate the similarity neighborhood of a given vertex vi by fitting local bicubic B-spline surfaces [28]:
ði,j,kÞ A F
Q ðu,vÞ ¼
4 X 4 X
Bi,j Ni,k ðuÞMj,l ðvÞ
ð12Þ
i¼1j¼4
Rewriting the coefficient matrix of Eq. (7) as an (n + m)n matrix A~ and denoting the right-hand side vector of the linear system by ðxÞ T b~ ¼ ðb,c1ðxÞ ,c2ðxÞ , ,cm Þ , we have the analytical least-squares solution of Eq. (7): T ~ 1 A~ T b~ x~ ¼ ðA~ AÞ
ð9Þ
For implementation, direct solver and Cholesky factorization T ~ ~ x~ ¼ A~ T b. are applied to the associated normal equations, i.e. ðA~ AÞ T ~ ~ As matrix M ¼ ðA AÞis sparse and positive by definition, we can obtain a Cholesky factorization of M, e.g. M¼RTR, where R is an upper-triangular sparse matrix. Then the three vectors of coordinates can be solved by backsubstitution, for instance, ~ Rx ¼ x. RT x ¼ A~ b,
where Bi,j are control points and in their natural ordering; they are the vertices of the B-spline control net, as shown in Fig. 3, Ni,k(u) and Mj,l(v) are the B-spline basic functions in the biparametric u and v directions, respectively. The 2-ring vertices of vi are used as input data points and are parameterized by projecting them onto a plane (e.g. the tangent plane of vi) and scaling them to the range [0,1]. The advantage of using B-spline surfaces is that the underlying control net is topologically similar to the image grid structure, and thus the similarity between neighborhoods in 3D can be more easily measured by computing the corresponding 3D points on the surface when given uniformly parametric u and v in the u–v space of the B-spline surface.
B4,4
vj B4,1 vj+1
j
j
vj−1
vi
vi B1,1 Fig. 2. Vertex and its adjacent triangles.
B1,4
Fig. 3. Neighborhood for a vertex by fitting a local bicubic B-spline surface.
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
In the non-local image filter, the parameter h controlling spatial support of the filter is global and is set manually. In our work, h is determined adaptively based on the variance s of edge length distribution in the similarity neighborhood. In our experience, h¼ (10–15)s has a good performance.
107
3. Experimental results and discussion We demonstrate results of our denoising method in Figs. 1 and 4–6. We test our algorithm on various data. It is applied to the scanned data model (Figs. 1 and 5), to CAD models (Fig. 4) as well
Fig. 4. Comparison of feature-preservation properties of bilateral filtering, Yoshizawa et al.’s [13] algorithm and our approach on a CAD model. (a) The model is artificially corrupted by Gaussian noise (the variance is 1/5 of the mean edge length). (b) The model is smoothed by bilateral filtering. (c) The model is smoothed by Yoshizawa et al.’s [13] algorithm. (d) Model is smoothed by our approach. The mean curvature visualizations (bottom row) show that our approach preserves salient features more accurately than bilateral filtering and Yoshizawa et al. [13] algorithm, e.g. the rectangle region.
Fig. 5. Comparison of detail-preservation properties of the bilateral filtering and our approach on a Dragon model. (a) The model is artificially corrupted by Gaussian noise (the variance is 1/5 of the mean edge length). (b) The model is smoothed by bilateral filtering after two iterations. (c) The model is smoothed by our approach. The mean curvature visualizations show that our approach preserves details of surface more accurately than bilateral filtering.
108
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
Fig. 6. Comparison of smoothed results on medical data. (a) Vessel surface extracted by Marching Cubes. (b) Smoothed surface obtained by Yoshizawa et al.’s [13] algorithm. (c) Smoothed surface obtained by our approach. The zoomed region of vessel branch shows that our approach preserves finer vascular structures better than that of Yoshizawa et al.’s [13] algorithm, which leads to collapse of thin structures.
Table 1 Parameters setting and timings results. Here i stands for the number of iterations, s and N are the sizes of the neighborhoods used to measure the similarity, vol denotes the ratio of volume and ave denotes the average error of method noise. Mean of {sc,ss}and {Z1,Z2,Z3}are explained in Refs. [7] and [13], respectively. All results are tested on a Pentium D processor at 2.80 GHz with a 3.00 GB memory. Bilateral filtering Model
sc
ss
i
vol
ave
Time (s)
Fandisk Dragon Vessel
1.5 1.5 1.2
2.0 1.8 1.5
3 3 3
0.9605 0.9789 0.9758
0.006 0.006 0.007
0.34 10.04 1.51
Yoshizawa et al. [13] algorithm Model Z1
Z2
Z3
1 1 1
0.75 0.75 0.75
i
vol
ave
Time (s)
3 3 2
0.9801 0.9827 0.9795
0.004 0.005 0.005
9.83 176.5 28.75
Fandisk Dragon Vessel
0.4 0.35 0.45
Our approach Model
s
N
h
i
vol
ave
Time (s)
Fandisk Dragon Vessel
7 7 7
17 17 17
10 12 10
3 3 2
0.9989 0.9956 0.9990
0.002 0.004 0.003
7.75 149.1 22.42
as to the medical data model (Fig. 6). We compare our results with the bilateral filtering algorithm [7] and Yoshizawa et al.’s [13] algorithm. Table 1 shows timing results and parameter settings used for bilateral filtering and our approach. We use method noise [13,26] to measure the smoothed surfaces. The method noise in meshes is defined as the differences between the original and smoothed meshes, i.e. e ¼9voriginal vsmoothed9. In Fig. 1, we show the efficiency of our method on real-world scanned data. The images show that high-frequency noise on the Chinese ancient soldier is removed after only one iteration while lower-frequency details such as armor are well preserved. At the same time, the ratio of volume is 99.98%. Fig. 4 shows a comparison of the bilateral filter, Yoshizawa et al.’s [13] algorithm and our approach with regard to feature
preservation. It indicates that our approach produces a smooth surface by preserving sharp features more accurately than the bilateral filtering and Yoshizawa et al.’s [13] algorithm. In addition, the mean error of method noise produced by our method is 0.0023, which is smaller than that of both bilateral filter and Yoshizawa et al.’s [13] algorithm, i.e. 0.0067 and 0.0048, respectively. Fig. 5 illustrates a comparison of the bilateral filter and our approach concerning fine details preservation. It is observed from the mean curvature visualizations that our approach preserves local details such as scuba much better than that of bilateral filtering due to the gradient operator encapsulating local differential geometry and the diffusing filter averaging with similaritybased weights, which measure geometrical information in a local region. Furthermore, our method preserves high-curvature and
J. Wu et al. / Optics and Lasers in Engineering 49 (2011) 104–109
low-curvature regions accurately, for instance, the ridges and the ravines. In Fig. 6, we demonstrate a comparison of Yoshizawa et al.’s [13] algorithm and our approach on medical data. Due to the partial volume effect and the segmentation process, the vessel surface extracted from binary volume data in general contains artifacts. These artifacts will result in numerical problem if the surface is used for computational fluid dynamics simulation. On the other hand, a vessel surface after iterative smoothing will produce collapse of thin structures and may even be misinterpreted as false stenosis if the smoothing technique brings obvious volume shrinkage. It is shown in Fig. 6 that our approach preserves finer and elongated vessel structures better than Yoshizawa et al.’s [13] algorithm, which collapses thin structures and leads to further distortions and detached object parts.
4. Conclusion In this paper, we have presented a novel denoising method for surface meshes based on gradient field and non-local means filtering. To measure the similarity between neighborhoods and irregular meshes, we introduced a local approximation utilizing bicubic B-spline surfaces. The performances of various data show that our approach can preserve local details of surface and can avoid volume shrinkage during the denoising process. We also compared the proposed algorithm with the conventional mesh smoothing technique. The comparison demonstrated that the proposed algorithm outperformed the conventional one in terms of preserving fine details and preventing volume shrinkage. However, the speed of our method is too slow when compared to the efficient bilateral filtering technique [29], and because solving linear systems and computing similarities between neighborhoods of each vertex are both time-consuming procedures. One interesting direction of our future research is to develop a parallel smoothing strategy for our method.
Acknowledgments The Dragon model is courtesy of Stanford University. This work was supported by the National Natural Science Foundation of China under Grant No. 60803108. References [1] Kobbelt L, Campagna S, Vorsatz J, Seidel HP. Interactive multi-resolution modeling on arbitrary meshes. In: Proceedings of SIGGRAPH, 1998, p.105–14. [2] Taubin G. A signal processing approach to fair surface design. In: Proceedings of SIGGRAPH, 1995. p. 351–8. ¨ [3] Vollmer J, Mencl R, Muller H. Improved Laplacian smoothing of noisy surface meshes. Comput Graph Forum 2001;18(3):131–8.
109
¨ [4] Desbrun M, Meyer M, Schroder P, Barr AH. Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of SIGGRAPH, 1999. p. 317–324. [5] Alexa M. Differential coordinates for local mesh morphing and deformation. Vis Comput 2003;19:105–14. [6] Su Z, Wang H, Cao J. Mesh denoising based on differential coordinates. In: Proceedings of the IEEE international conference on shape modeling and applications, 2009. p. 1–6. [7] Fleishman S, Drori I, Cohen-Or D. Bilateral mesh denoising. ACM Trans Graph 2003;22(3):950–3. [8] Jones TR, Durand F, Desbrun M. Non-iterative feature-preserving mesh smoothing. ACM Trans Graph 2003;22(3):943–9. [9] Ohtake Y, Belyaev AG, Seidel HP. Mesh smoothing by adaptive and anisotropic Gaussian filter. In: Proceedings of the vision, modeling, and visualization conference, 2002. p. 203–10. [10] Yagou H, Ohtake Y, Belyaev AG. Mesh smoothing via mean and median filtering applied to face normals. In: Proceedings of the geometric modeling and processing conference, 2002. p. 124–31. [11] Taubin G. Linear anisotropic mesh filtering. IBM research report RC22213 (W0110-051), IBM, October 2001. [12] Shen Y, Barner KE. Fuzzy vector median-based surface smoothing. IEEE Trans Visual Comput Graphics 2004;10(3):252–65. [13] Yoshizawa S, Belyaev AG, Seidel HP. Smoothing by example: mesh denoising by averaging with similarity-based weights. In: Proceedings of the shape modeling international conference, 2006. p. 38–44. [14] Sun X, Rosin PL, Martin RR, Langbein FC. Random walks for featurepreserving mesh denoising. Comput Aided Geom Des 2008;25: 437–56. [15] Mao Z, Ma L, Zhao M, Xiao X. SUSAN structure preserving filtering for mesh denoising. Vis Comput 2006;22:276–84. ¨ [16] Desbrun M, Meyer M, Schroder P, Barr AH. Anisotropic feature-preserving denoising of height fields and bivariate data. In: Proceedings of the graphics interface conference, 2000. p. 145–52. [17] Hildebrandt K, Polthier K. Anisotropic filtering of non-linear surface features. Comput Graph Forum 2004;23(3):391–400. [18] Clarenz U, Diewald U, Rumpf M. Anisotropic geometric diffusion in surface processing. In: Proceedings of the IEEE visualization conference, 2000. p. 397–405. [19] Tasdizen T, Whitaker R, Burchard P, Osher S. Geometric surface smoothing via anisotropic diffusion of normals. In: Proceedings of the IEEE visualization; 2002. p. 125–32. [20] Bajaj CL, Xu G. Anisotropic diffusion of subdivision surfaces and functions on surfaces. ACM Trans Graph 2003;22(1):4–32. [21] Fan H, Yu Y, Peng Q. Robust feature-preserving mesh denoising based on consistent subneighborhoods. IEEE Trans Visual Comput Graph 2010;16(2):312–24. [22] Yu Y, Zhou K, Xu D, Shi X, Bao H, Guo B, Shum HY. Mesh editing with Poissonbased gradient field manipulation. ACM Trans Graph 2004;23(3): 644–51. [23] Huang J, Shi X, Liu XG, Zhou K, Wei LY, Teng SH, Bao HJ, Guo BN, Shum HY. Subspace gradient domain mesh deformation. ACM Trans Graph 2006;25(3):1126–34. [24] Xu D, Zhang H, Wang Q, Bao H. Poisson shape interpolation. In: Proceedings of the ACM solid and physical modeling conference, 2005. p. 267–74. [25] Liu XG, Bao HJ, Heng PA, Wong TT, Peng QS. Constrained fairing for meshes. Comput Graph Forum 2001;20(12):115–23. [26] Buades A, Coll B, Morel JM. A non-local algorithm for image denoising. In: Proceedings of the computer vision and pattern recognition conference, 2005, vol. 2. p. 60–5. [27] Tong Y, Lombeyda S, Hirani AN, Desbrun M. Discrete multiscale vector field decomposition. ACM Trans Graph 2003;22(3):445–52. [28] Rogers DF. An introduction to NURBS: with historical perspective.. Morgan Kaufmann; 2001. p. 179–81. [29] Sun XF, Rosin PL, Martin RR, Langbein FC. Fast and effective featurepreserving mesh denoising. IEEE Trans Visual Comput Graph 2007;13(5):925–38.