A 3D shape descriptor based on spherical harmonics through evolutionary optimization

Author’s Accepted Manuscript A 3D shape descriptor based on spherical harmonics through evolutionary optimization Dingwen Wang, Shilei Sun, Xi Chen, Zhiwen Yu

www.elsevier.com/locate/neucom

PII: DOI: Reference:

S0925-2312(16)00243-5 http://dx.doi.org/10.1016/j.neucom.2016.01.081 NEUCOM16765

To appear in: Neurocomputing Received date: 9 November 2015 Revised date: 5 January 2016 Accepted date: 28 January 2016 Cite this article as: Dingwen Wang, Shilei Sun, Xi Chen and Zhiwen Yu, A 3D shape descriptor based on spherical harmonics through evolutionary optimization, Neurocomputing, http://dx.doi.org/10.1016/j.neucom.2016.01.081 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

A 3D shape descriptor based on spherical harmonics through evolutionary optimization Dingwen Wanga , Shilei Suna,∗, Xi Chena , Zhiwen Yub b School

a International School of Software, Wuhan University, Wuhan, China of Computer Science and Engineering, South China University of Technology, Guangzhou, China

Abstract This paper proposes an optimization approach based on a newly designed parameter set self-evolutionary process. This method improves the discriminability of the original shape descriptor of a 3D model based on spherical harmonics while retaining the efficiency and simplicity of the original shape descriptor. This method captures the critical characteristics, such as the distance, area, and normal distributions of a 3D model extracted along the latitude-longitude directions after the 3D model is normalized in the uniform coordinate frame, and obtains the 3D model’s features using a spherical harmonics transform. In order to determine the spherical harmonic basis function relationship, an additional weight (0,1) of each spherical harmonic coefficient as random variable is used to search for the optimal variables based on genetic optimization. The resulting transformed features for these random variables are then used as modified shape descriptors. Retrieval performance is examined using the public benchmarks: the Princeton Shape Benchmark, CCCC and NTU databases, and experiments have shown that the optimized additional weight for shape descriptors based on spherical harmonics results in a significant improvement in discriminability. Keywords: 3D model retrieval, spherical harmonics, optimization

∗ Corresponding

author Email address: [email protected] (Shilei Sun)

Preprint submitted to Journal of LATEX Templates

February 19, 2016

1. Introduction With the general availability of 3D digitizers, scanners, computed tomography (CT), magnetic resonance (MR), stereoscopic vision systems, graphics hardware, and software development, the number of available 3D models has been growing 5

explosively. It has become a great challenge to categorize and retrieve the continuously growing number of 3D models. The traditional retrieval method with keyword annotation cannot meet the retrieval requirements with the explosive growing number of unannotated 3D models. In this context, content-based 3D model retrieval has become an important issue[1], [2], [3], [4]. Many researchers

10

have investigated the performance of content-based 3D model retrieval by using shape properties instead of text [5], [6], [7], [8]. Several researchers have applied content-based 3D model retrieval in modern industry fields[9],[10], such as virtual reality[11], computer-aided design (CAD)[12], medicine[13], [14], molecular biology (3D protein models)[15], 3D head[16],[17], face recognition[18], [19], [20]

15

and entertainment[21]. Among the available shape descriptors, spherical harmonics shape descriptors achieve good retrieval performance [22], [1], [23], [24],[4], [25]. Moreover, spherical harmonics have also been applied in a few practical fields. Papadakis presented a 3D shape retrieval method with spherical harmonics[24]. Abdallah

20

analysed the 3D shape of the heart’s left ventricle with a unified parameter and spherical harmonics[26]. Chung presented a cortical asymmetry technique called the weighted spherical harmonics representation for 3D magnetic resonance images (MRI)[27]. [28] and [15] adopted spherical harmonics to compute the high resolution structures of the protein and molecule. [14] proposed 3D

25

particle size and shape distributions for X-ray computed tomography. [29] applied a spherical parameterization algorithm to retrieval biomedical imaging. [30] presented an approach with spherical harmonics to retrieval the 3D microscopy image when the topological structure of the model is equivalent to a sphere. However, different shape descriptor extraction approaches have different

30

focuses and limitations. For example, FDCS considers only the distance distri-

2

bution, while MSEGI pays more attention to the normal distribution. ADCS mainly considers the area distribution. Spherical harmonics, which are used to transform the functions of a sphere, were first introduced to 3D model retrieval by [31] and [32]. 3D model retrieval 35

based on spherical harmonics achieves good results. The theory of spherical harmonics holds that any spherical function can be represented as the sum of its harmonics, and each harmonic coefficient is represented as a contribution function on a spherical map. Based on the analysis of the harmonic coefficients of many 3D models, different spherical harmonic basis functions are supposed

40

to play different roles in representing 3D models. How to find the relationship of spherical harmonic basis functions? The proposed approach is to attach a weight to each spherical harmonic coefficient. In general, the first L-order coefficients is represented as 3D models according to the properties of spherical harmonics. Higher-order coefficients represent finer details of the models. In order to repre-

45

sent finer details of 3D models, their dimensions are adopted between 180 and 544. On the assumption that the weight of each spherical harmonic coefficient is between [0, 1] and that the precision is 0.1, the number of candidate resolutions is between 10180 and 10544 . It is very difficult to directly compute the optimal resolution[33]. This paper proposes the evolutionary algorithm for finding the

50

optimal solution to improve retrieval performance. This paper proposes a novel retrieval method to capturing 3D geometric relationships on a spherical map as much as possible by combining the parameter set self-evolutionary process with spherical harmonics so that a 3D object is described in as much detail as possible. The spatial features of 3D model in-

55

clude distance, normal and area of mesh. Analyzing previous works, this paper captures the farthest distance on each concentric sphere along longitude and latitude. Then, the harmonic coefficients is obtained by performing spherical harmonics on the distance, normal, and area distributions, which are regarded as composing a feature vector. Further, the properties and coefficients of spherical

60

harmonics are considered that the different harmonic coefficients play different roles in describing a 3D model. Adding different weight coefficient is an 3

effective way. In general, the feature dimensionality of a 3D model is on the order of several hundred. The problem of finding the optimal weight of each harmonic coefficient is known to be NP-hard in general. Here, the parameter 65

set self-evolutionary process is adopted to resolve this problem. In the next section, the spherical harmonics transform is adopted to extract shape descriptors from 3D models. In Section 3, the 3D model retrieval workflow is presented according to the furthest distance, normal, and area distributions on concentric spheres of extracted shape descriptors, and the parameter set self-

70

evolutionary process is proposed to optimize the descriptors based on spherical harmonics to search for the optimal spherical harmonic coefficients, which serve as more powerful shape descriptors. Section 4 presents the experimental results. The conclusions and future work are addressed in section 5. 2. Spherical Harmonics Transform

75

Spherical harmonics can effectively extract the features of 3D model in spherical mapping [34]. The theory of spherical harmonics states that any spherical function f (θ, φ) can be rewritten by the sum of its harmonics:

f (θ, φ) =

∞ m=l  

f˜l,m Ylm (θ, φ)

(1)

l=0 m=−l

where f˜l,m denotes the Fourier coefficient and Ylm (θ, φ) is the spherical harmonics base calculated by certain products of Legendre functions and complex 80

exponentials. Fig. 1 illustrates the spherical harmonics Ylm (θ, φ), which are complex functions defined on a sphere, up to degree 3. f (r, θ, φ) denotes the feature vectors which are obtained by spherical harmonic conversion of the mapping of furthest distance, normal, and area on each concentric sphere. Eq. 2 shows f (r, θ, φ) while the bandwidth is N/2 and l is

85

the degree of the spherical harmonics. According to the theory of spherical

4

Figure 1: Examples of spherical harmonics basis functions Ylm (θ, φ) up to degree 3.

harmonics, the larger l means the higher resolution.

f (r, θ, φ) =

N/2 m=l  

f˜l,m Ylm (r, θ, φ)

(2)

l=0 m=−l

While the value of l is given, the former l + 1 (l < N/2) rows of coefficients can be used as the feature vectors of 3D model. There are two properties of spherical harmonics on the use of coefficients. 90

2.1. Property 1 Let f ∈ L2 (S 2 ) be a real-valued function, i.e., f : S 2 → . Then, the following symmetry between coefficients exists: f˜l,m = (−1)m f˜l,−m ⇒ |f˜l,m | = |f˜l,−m |

(3)

where f˜l,m and (−1)m f˜l,−m are complex conjugates. 2.2. Property 2 95

A subspace of L2 (S 2 ) of dimension 2l + 1, spanned by the harmonics Ylm (−l ≤ m ≤ l) of degree l, is an invariant with respect to the sphere rotation S 2 . The feature vector F1 is denoted as the absolute values of f˜l,m . According to Property 1, the coefficients of f (θ, φ) is a symmetric relation while f (θ, φ)

5

is a real-valued function. Thus, the feature vector F1 is composed of l(l + 1)/2 100

elements of the former l + 1 (l < N/2) rows among the obtained coefficients. ˜ |, |f˜1,1 |, · · · , |f˜l,0 |, · · · , |f˜0,0 |) F1 = (|f˜0,0 |, |f1,0

(4)

According to Property 2, The feature vector F2 is rotation invariance regardless of the normalization of the orientation of a 3D model.   l  |f˜1,1 |2 F2 = (f0 , · · · , fdim−1 ), fi  =  where fi  =

 l

m=−l

(5)

m=−l

|f˜1,1 |2 and dim(F2 ) ≤ N/2. However, F2 drops

detailed information on degree l, thus, its resolution is reduced. In addition, 105

each subdivided spherical grids is not uniform as Fig.3, consequently the rotation of 3D model will change the sampling distribution function. Both reduce the resolution of the shape descriptors, therefore this paper prefers to use F1 rather than F2 . 3. Workflow of 3D Model Retrieval

110

The whole procedure includes four parts: normalization, sampling, number transformation, and the similar measure. 1. Normalization: The normalization of 3D models is done in the same canonical frame using translation (T ), rotation (R), and scale (S). P  = (P − T ) × R × S

115

2. Sampling: First, a 3D model is uniformly decomposed into multi-concentric shells; Then, the surface of these shells is divided into N × N cells. Function f (r, θ, ϕ) captures the furthest distance, normal, and area of the 3D object along longitude and latitude angle pairs in each concentric shell. 3. Transform: The spherical harmonics optimized by the parameter set self-

120

evolutionary process are used to transform the functions f (r, θ, ϕ) to obtain the feature vectors.

6

Figure 2: The workflow of the 3D model retrieval system.

4. Similarity measure: Considering the feedback retrieval performance of different features, an appropriate weight is used to obtain an overall similarity. 125

Here, for convenience in subsequent discussion, some definition of 3D models are given as follow. T = {T1 , T2 , · · · , TNt },

T i ⊂ 3

(6)

where Ti consists of a set of three vertices Ti = {PAi , PBi , PCi }

(7)

− ni are denoted as the center and normal of triangle Ti respectively. gi and → Si and S are denoted as the surface area of triangle Ti and the total area of a 130

3D model:

Si =

gi = (PAi + PBi + PCi )/3

(8)

− → ni = (PCi − PAi ) × (PBi − PAi )

(9)

1 |(PCi − PAi ) × (PBi − PAi )|, 2 7

S=

 i=1

mSi

(10)

3.1. Normalization The spatial geometric descriptor is relevant to the 3D model’s position, orientation, and scale. The rotation of 3D model will change these characteristics, thus the 3D model should be normalized in the uniform coordinate 135

frame. There have been several approaches to the normalization step. The most prominent one is principle component analysis (PCA), also known as the discrete Karhunen-Loeve transform, which produces an affine transformation of space [31]. The transform is defined by a set of vectors, e.g., the set of vertices of a 3D model. After a translation of the set moving its center of mass to the

140

origin of the coordinate system, a rotation is applied so that the largest spread of the transformed points is along the x-axis. Then a rotation around the x-axis is carried out so that the maximal spread in the yz-plane occurs along the yaxis. The ambiguity between positive and negative axes is resolved by choosing the direction of the axes so that the area of the model on the positive side of

145

the x-, y-, and z-axes is greater than the area on the negative side. 1. Translation: the model is translated to the center of mass. Nt 1 Si g i OI = (Ox , Oy , Oz ) = S i=1

(11)

2. Rotation: The eigenvectors and associated eigenvalues of the covariance matrix CI are computed by integrating the quadratic polynomials Pi · Pj at the centers on the surfaces of each polygon. The eigenvectors of CI represent the principal directions of the model’s surface area. A rotation matrix is made up of these three eigenvectors which are sorted by their associated eigenvalues in decreasing order, and can be used to rotate the model into a canonical position. CI =

M 1  (f (PAi ) + f (PB i ) + f (PC i ) + 9f (Pgi ))Si 12S i=1

fv = (Pi − OI ) · (Pi − OI )T

8

(12)

(13)

Figure 3: Decomposing a 3D space into many concentric shells.

Figure 4: Subdividing a surface into sub-surfaces.

3. Scale: The 3D object is normalized for size by isotropically rescaling it so that the average distance from points on its surface to the center of mass is 0.5. 3.2. Sampling 150

This section describes the procedure of how to capture the distance, normal, and area distributions of a 3D model along the latitude-longitude directions on each spherical grid by adopting a single-scan scheme on the mesh surface. First, a 3D model is decomposed into concentric shells with radii rc = 1, 2, · · · , Ns (c = 1, 2, · · · , Ns ) after normalization(Fig.3). Secondly, if the num-

155

ber of the triangle surfaces of the target 3d model is smaller than Ptotal , a predefined value, some bigger triangle surfaces will be divided into a finer granularity meshes until the total number reaches Ptotal (Fig.4). Then the center of each triangle surface is mapped to one particular cell, which is decided by its longitude, latitude, and distance from the origin. Then the fdistance (r, i, j) 9

160

records the furthest distance for each cell. Cell(r, i, j) = (rn , [θi , θi+1 ], [ϕj , ϕj+1 ]), i, j = 0, 1, 2, · · · , N − 1

(14)

θi = 2πi/N, ϕi = 2πi/N, i = 0, 1, 2, · · · , N

(15)

rn =

dk − min(dk ) × Ns , n = 1, 2, · · · , Nt max(dk ) − min(dk )

(16)

In general, the smaller the triangle surface and thus the higher the precision of sampling, but the more computing time is needed. Here, to balance retrieval performance and extraction time, the value of Ptotal is set to 40,000. Overall, the whole procedure of extracting the distance distribution is summarized in 165

the following steps: 1. Map a triangle surface to Cell(r, i, j) by its normal direction. fnormal (r, i, j) tracks the area of the triangle surface, mapped to Cell(r, i, j). 2. If Si < S/Ptotal , jump to the next step. Otherwise, further divide the triangle surface into Ptotal × (Si /S) sub-triangle surfaces.

170

3. fdistance (r, i, j) and farea (r, i, j) compute the furthest distance and area, respectively, of the triangle or sub-triangle surface which centers mapped to Cell(r, i, j). 4. Repeat the above procedures until every surface is scanned. 3.3. Optimizing spherical harmonics with the self-evolutionary process

175

The properties and coefficients of spherical harmonics are considered that the different spherical harmonics basis functions (as shown in Fig. 1) play different roles in describing the 3D model. Thus, each spherical harmonic basis function attaches an additional weight for different roles. However, the feature dimensionality of a 3D model is generally on the order of several hundred. It is

180

an NP-hard problem to find the optimal weight for each harmonic coefficient. Therefore, a parameter set self-evolutionary process is designed to deal with this problem. 10

The parameter set self-evolutionary process (PSSEP) is a newly proposed search technique in computing used to find exact or approximate solutions to op185

timization and search problems motivated by traditional evolution algorithms, such as genetic algorithms [35] and [36]. PSSEP is implemented as a computer simulation in which a population of abstract representations of candidate solutions to an optimization problem evolves toward better solutions. Specifically, each candidate solution is represented by a weight value set. The evolution usu-

190

ally starts from a population of randomly generated individuals and takes place over generations. In each generation, the fitness of every individual in the population is evaluated, and multiple individuals are stochastically selected from the current population based on their fitness and modified to form a new population. The new population is then used in the next iteration of the algorithm.

195

PSSEP is a global search heuristic algorithm, but it needs a large population of individuals and a number of generations when the candidate solution is for high-dimensional problem. Some evolutionary strategies are improved and preserve the diversity of individuals in a population. In the framework of PSSEP, the optimal spherical harmonic coefficient weights are obtained by solv-

200

ing the optimal retrieval performance, and the problem is transformed into an optimization problem. Here, the parameter set self-evolutionary process is given to optimize the spherical harmonics. Algorithm: procedure for the parameter set self-evolutionary process for

205

spherical harmonics Step 1 Initialization: Initialize a population of randomly generated individuals. Step 2 Parameter set cooperation: Randomly choose two individuals from a population to generate new individuals with a certain ratio. Step 3 Parameter set competition: Randomly choose an individual whose value

210

to change (i.e., mutate). Step 4 Keep better individuals and repeat the above steps until a predefined 11

number generations has been met or the best individual does not change over a certain number of generations. 3.3.1. Individual design 215

fv = (|f0,0 |, |f1,0 |, |f1,1 |, · · · , |fl,0 |, · · · , |fl,l |) is represented for the feature vector of 3D model. The parameter set ω = (ω0,0 , ω1,0 , ω1,1 , · · · , ωl,0 , · · · , ωl,l ) is a candidate solution, which represents the weight of the spherical harmonic coefficients. Then, the 3D model feature vector is represented as:

fv

ω

=

ω · fvT

=

(ω0,0 |f0,0 |, ω1,0 |f1,0 |, ω1,1 |f1,1 |, · · · , ωl,0 |fl,0 |, · · · , ωl,l |fl,l |)

(17)

=

ω0 , ω1 , ω2 , · · · , ωl(l+1)/2

(18)

In the initial procedure, a population of candidate solutions ω is randomly 220

generated, and each vector is equally distributed in the range [0, 1]. 3.3.2. Fitness function The fitness function is defined over the evolutionary representation and measures the quality of the represented solution. Some standard evaluation metrics includes the nearest neighbor(NN), first tier(FT), second tier(ST), e-

225

measure(EM), and discounted cumulative gain(DCG) which evaluate the performance of shape descriptors. 1. NN measure indicates the percentage of the closest matches belonging to the same query class. 2. FT and ST represent the percentage of top K matches, where K depends

230

on the size of the query’s class; For instance, given a |C| class, K = |C| − 1 for the first tier and K = 2(|C| − 1) for the second tier. 3. EM measures both the precision and recall for a fixed number of retrieved results.

12

Figure 5: Parameter set cooperation operators swapping part of the vectors of two parent individuals to produce two new offspring individuals.

4. DCG depicts the priority given to the correct results near the top of the 235

ranking list comparing to the ones at the bottom of the list, based on the assumption that a suer favors the elements near the top of the list. For all these methods, NN, FT, ST, EM, and DCG, higher values indicate better results. Here, we denote their sum (NN+FT+ST+EM+DCG) as the fitness function. Larger fitness represents better performance.

240

3.3.3. Evolutionary operators The parameter set cooperation operator combines the features of two parent individuals to form two similar offspring individuals by swapping corresponding segments of the parents, exchanging information between different potential solutions. Two parameter set cooperation operators are adopted in this paper.

245

One is the traditional approach where two of the solutions exchange the vectors of the individuals at a random position, as Fig. 5 shows. The other cooperation operator between two individuals is as follows. A random number ω is generated from an interval [0, 1]. Then, by making two linear combinations of vi and vj , Eq.19 obtains the new individuals:

vi = ωvi + (1 − ω)vj vj = (1 − ω)vi + ωvj

13

(19)

250

Both parameter set cooperation operators are used randomly in the evolutionary process. 3.3.4. Similarity measure The coefficient of each spherical harmony is a complex number. The L1 distance is computed to compare two shape descriptors.

255

Let f(l, m) = Re + jIm, f (l, m) = Re + jIm be two coefficients of two corresponding different shape descriptors. Let ω be the weight optimized by the parameter set self-evolutionary process. The L1 distance between them is |ω f(l, m) − ω f (l, m)| = ω



(Re − Re )2 + (Im − Im )2

(20)

Eq.20 is computed for all coefficients where l ≤ Lbands and m ≥ 0 owing to the property derived from Eq.4 that leads to the following:

(21)

=

|(−1)m f(l, −m) − (−1)m f (l, −m)| ω (Re − Re )2 + (Im − Im )2

=

ω f(l, −m) − ω f (l, −m)|

(23)

ω f(l, m) − ω f (l, m)| =

260

(22)

Thus, only the positive-order coefficients are necessary. The similarity measure of two different shape descriptors is as follows:

fv − fv

= =

   ω0,0 |f0,0 − f0,0 | + ω1,0 |f1,0 − f1,0 | + · · · + ωl,l |fl,l − fl,l | (24) i
ωi |fi − f i |

(25)

i=0

4. Experiments and Results To assess the impact on retrieval precision of 3D model descriptors based on spherical harmonics optimized by an evolutionary algorithm, a series of exper265

iments was performed; results are presented as follows. Our experiments were based on the following 3D model databases: classified models of the CCCC 14

database [32], classified models of the National Taiwan University database (NTU) [5], and the public Princeton Shape Benchmark [37]. The CCCC 3D Benchmark was introduced in [32] and consists of 547 3D mesh models spanning 270

47 categories serving as queries. The National Taiwan University database(NTU) was introduced in [5] and consists of 472 3D objects which are classified to 55 categories. In order to investigate the performance of the parameter set self-evolutionary process (PSSEP) on spherical harmonics, the 7 shape matching algorithms are

275

evaluated as follows: (1) furthest distance on concentric spheres optimized by the parameter set self-evolutionary process; (2) furthest distance on concentric spheres; (3) light field descriptor; (4) multi-shell extended Gaussian image optimized by the parameter set self-evolutionary process; (5) multi-shell extended Gaussian image; (6) area distribution on concentric spheres optimized by the

280

parameter set self-evolutionary process; and (7) area distribution on concentric spheres. 4.1. Experiments optimized by the parameter set self-evolutionary process In the first experiment, the parameter set self-evolutionary process optimizes shape descriptors based on spherical harmonics such as PSSEP-FDCS, PSSEP-

285

MSEGI, and PSSEP-ADCS. In the experiment, the population, parameter set cooperation operator, parameter set competition operator, and generation of the parameter set self-evolutionary process are respectively 150, 0.65, 0.05, and 200. Figs.6, 7, and 8 show the changes of NN, FT, ST, EM, and DCG in the optimization process of PSSEP-FDCS, PSSEP-MSEGI, and PSSEP-ADCS

290

executed on the CCCC and NTU databases. In the figures, the horizontal coordinate axis denotes the generation, and the vertical coordinate axis denotes the value of NN, FT, ST, EM, and DCG. 4.2. Experiments on datasets Secondly, precision vs. recall curves is used to demonstrate the effectiveness

295

of our algorithms (Figs. 9 and 10), and some standard evaluation metrics such as NN, FT, ST, EM, and DCG evaluate these shape descriptors(Tables 1 and 2). 15

NN FT ST EM DCG

0.7

0.6

0.5

0.4

0.3

0.2

0

20

40

60

80

100 generation

120

140

160

180

200

(a) CCCC database

0.7 NN FT ST EM DCG

0.65

0.6

0.55

0.5

0.45

0.4

0.35

0.3

0.25

0

20

40

60

80

100 generation

120

140

16

(b) NTU database Figure 6: FDCS optimized by PSSEP.

160

180

200

0.65 NN FT ST EM DCG

0.6

0.55

0.5

0.45

0.4

0.35

0.3

0.25

0.2

0

20

40

60

80

100 generation

120

140

160

180

200

(a) CCCC database

NN FT ST EM DCG

0.55

0.5

0.45

0.4

0.35

0.3

0.25

0.2

0

20

40

60

80

17

100 generation

120

140

(b) NTU database Figure 7: MSEGI optimized by PSSEP.

160

180

200

0.7 NN FT ST EM DCG

0.65

0.6

0.55

0.5

0.45

0.4

0.35

0.3

0.25

0.2

0

20

40

60

80

100 generation

120

140

160

180

200

(a) CCCC database

0.65 NN FT ST EM DCG

0.6

0.55

0.5

0.45

0.4

0.35

0.3

0.25

0.2

0

20

40

60

80

100 generation

18

120

140

(b) NTU database Figure 8: AD optimized by PSSEP.

160

180

200

NN

FT

ST

EM

DCG

retrieval time (s)

PSSEP-FDCS

72.3%

39.2%

49.1%

26.2%

65.2%

0.253

LFD

67.0%

36.5%

49.0%

27.7%

64.8%

21.332

FDCS

62.6%

34.5%

45.8%

24.8%

62.1%

0.251

PSSEP-MEGI

61.9%

31.2%

41.3%

22.8%

57.7%

0.132

MEGI

54.7%

27.5%

38.1%

21.5%

55.3%

0.131

PSSEP-AD

67.0%

32.4%

41.1%

21.8%

59.8%

0.238

AD

60.0%

29.3%

38.0%

20.9%

57.8%

0.236

Table 1: Retrieval measurements of 7 different shape feature vectors in the CCCC database.

As showing in the experimental results, the main advantages of the proposed algorithm are summarized as following: that shape descriptors optimized by the parameter set self-evolutionary process improve the distinguishability of the 300

original features. PSSEP is only used to search for optimal shape descriptors. It is a one-time effort that does not increase the computational cost of the retrieval process. Tables 1 and 2 show the retrieval time for all approaches. It is observed that PSSEP-FDCS, PSSEP-MSEGI, and PSSEP-ADCS do not increase computational costs when compared with FDCS, MSEGI, and ADCS.

305

This means that PSSEP is able to improve the effectiveness of retrieval without increasing computational cost.

4.3. 3D model retrieval system 310

A retrieval system of 3D model has been developed with FDCS, MSEGI, and ADCS and optimized them using the parameter set cooperation operator, which includes all 1841 3D models in the PSB dataset. After users update the target 3D model file to the retrieval system, the most similar models are generated. By choosing the above different shape descriptors, the retrieval system displays 19

NN

FT

ST

EM

DCG

retrieval time (s)

PSSEP-FDCS

69.1%

33.4%

43.4%

27.6%

62.9%

0.912

LFD

61.6%

34.3%

44.2%

28.3%

63.8%

36.323

FDCS

59.0%

29.4%

39.5%

25.2%

60.2%

0.91

PSSEP-MEGI

55.3%

26.1%

35.1%

22.4%

55.6%

0.803

MEGI

48.8%

23.1%

31.8%

20.1%

53.6%

0.801

PSSEP-AD

65.2%

27.4%

34.8%

23.1%

58.5%

0.891

AD

56.4%

24.8%

32.3%

21.4%

56.4%

0.889

Table 2: Retrieval measurements of 7 different shape feature vectors in the NTU database.

315

the most relevant models in the database. Fig. 11 shows the retrieval results based on PSSEP-FDCS and FDCS when the same car model was queried. 5. Conclusion In this paper, in order to investigate the relationship of spherical harmonic basis functions, the parameter set self-evolutionary process is proposed to op-

320

timize spherical harmonic shape descriptors. Experimental results on public benchmarks such as the CCCC, PSB, and NTU databases demonstrated that the parameter set self-evolutionary process significantly improves the performance of the original shape descriptors and retains the efficiency and simplicity of the original shape descriptor. For further comparison, a 3D model retrieval

325

system is developed with FDCS, MSEGI, and ADCS and optimized it using the parameter set self-evolutionary process, which directly displays retrieval results for different 3D model matching methods when a 3D model is given as the query. References [1] T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A. Halderman, D. Dobkin,

330

D. Jacobs, A search engine for 3d models, Acm Transactions on Graphics 22. 20

1 PSSEP−FDCS FDCS PSSEP−MSEGI MSEGI PSSEP−AD AD

0.9 0.8

Precision

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.2

0.4

0.6

0.8

1

Recall

Figure 9: Precision vs. recall curves for 11 approaches with the CCCC database.

[2] B. Leng, S. Guo, X. Zhang, Z. Xiong, 3d object retrieval with stacked local convolutional autoencoder, SIGNAL PROCESSING 112 (SI) (2015) 119–128. 335

[3] F. Wang, J. Peng, Y. Li, Hypergraph based feature fusion for 3-d object retrieval, NEUROCOMPUTING 151 (2) (2015) 612–619. [4] J. Yu, Y. Rui, D. Tao, Click Prediction for Web Image Reranking Using Multimodal Sparse Coding, IEEE TRANSACTIONS ON IMAGE PROCESSING 23 (5) (2014) 2019–2032. doi:{10.1109/TIP.2014.2311377}.

340

[5] D. Y. Chen, X. P. Tian, Y. T. Shen, O. Y. Ming, On visual similarity based 3d model retrieval, Computer Graphics Forum 22 (3) (2003) 223–232. [6] D. Smeets, J. Hermans, D. Vandermeulen, P. Suetens, Isometric defor-

21

0.9 PSSEP−FDCS FDCS PSSEP−MSEGI MSEGI PSSEP−AD AD

0.8 0.7

Precision

0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.2

0.4

0.6

0.8

1

Recall

Figure 10: Precision vs. recall curves for 11 approaches with the NTU database.

mation invariant 3d shape recognition, Pattern Recognition 45 (7) (2012) 2817–2831. 345

[7] A. Mian, M. Bennamoun, R. Owens, On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes, International Journal of Computer Vision 89. [8] C. T. Kuo, S. C. Cheng, 3d model retrieval using principal plane analysis and dynamic programming, Pattern Recognition 40.

350

[9] J. Yu, Y. Rui, B. Chen, Exploiting click constraints and multi-view features for image re-ranking, IEEE Transactions on Multimedia 16 (1) (2014) 159 – 168. [10] J. Yu, Y. Rui, Y. Y. Tang, D. Tao, High-order distance-based multiview

22

stochastic learning in image classification, IEEE Transactions on Cyber355

netics 44 (12) (2014) 2431 – 2442. [11] P. Spala, A. G. Malamos, A. Doulamis, G. Mamakis, Extending mpeg-7 for efficient annotation of complex web 3d scenes, Multimedia Tools and Applications 59 (2) (2012) 463–504. [12] S. Jayanti, Y. Kalyanaraman, N. Iyer, K. Ramani, Developing an engineer-

360

ing shape benchmark for cad models, Computer-Aided Design 38 (2006) 939–953. [13] J. D. Zhou, S. Cadavid, M. Abdel-Mottaleb, An efficient 3-d ear recognition system employing local and holistic features, Ieee Transactions on Information Forensics and Security 7 (2012) 978–991.

365

[14] E. J. Garboczi, Three dimensional shape analysis of jsc-1a simulated lunar regolith particles, Powder Technology 207 (1-3) (2011) 96–103. [15] L. Mavridis, B. D. Hudson, D. W. Ritchie, Toward high throughput 3d virtual screening using spherical harmonic surface representations, Journal of Chemical Information and Modeling 47 (5) (2007) 1787–1796.

370

[16] H.-S. Wong, B. Ma, Z. Yu, P. F. Yeung, H. H. S. Ip, 3d head model retrieval using a single face view query, IEEE Transactions on Multimedia 9 (5) (2007) 1026 – 1036. [17] Z. Yu, H.-S. Wong, H. Peng, Q. Ma, Asm: An adaptive simplification method for 3d point-based models, CAD Computer Aided Design 42 (7)

375

(2010) 598 – 612. [18] G. Passalis, I. A. Kakadiaris, T. Theoharis, Intraclass retrieval of nonrigid 3d objects: Application to face recognition, Ieee Transactions on Pattern Analysis and Machine Intelligence 29. [19] I. A. Kakadiaris, G. Passalis, G. Toderici, M. N. Murtuza, Y. L. Lu,

380

N. Karampatziakis, T. Theoharis, Three-dimensional face recognition in 23

the presence of facial expressions: An annotated deformable model approach, Ieee Transactions on Pattern Analysis and Machine Intelligence 29. [20] K. Lu, N. He, J. Xue, J. Dong, L. Shao, Learning view-model joint relevance 385

for 3d object retrieval, IEEE TRANSACTIONS ON IMAGE PROCESSING 24 (5) (2015) 1449–1459. [21] Y. Gao, J. H. Tang, R. C. Hong, S. C. Yan, Q. H. Dai, N. Y. Zhang, T. S. Chua, Camera constraint-free view-based 3-d object retrieval, Ieee Transactions on Image Processing 21 (4) (2012) 2269–2281.

390

[22] D. V. Vranic, An improvement of rotation invariant 3d-shape based on functions on concentric spheres, in: International Conference on Image Processing, Vol. 3, 2003, pp. 757–760. [23] D. Wang, J. Zhang, H.-S. Wong, Y. Li, Content-based 3d model retrieval based on the spatial geometric descriptor, Lecture Notes in Computer Sci-

395

ence (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 4810 LNCS (2007) 685–694. [24] P. Papadakis, I. Pratikakis, S. Perantonis, T. Theoharis, Efficient 3d shape matching and retrieval using a concrete radialized spherical projection representation, Pattern Recognition 40 (9) (2007) 2437–2452.

400

[25] J. Yu, D. Tao, M. Wang, Y. Rui, Learning to Rank Using User Clicks and Visual Features for Image Retrieval, IEEE TRANSACTIONS ON CYBERNETICS 45 (4) (2015) 767–779. doi:{10.1109/TCYB.2014.2336697}. [26] A. B. Abdallah, F. Ghorbel, K. Chatti, H. Essabbah, M. H. Bedoui, A new uniform parameterization and invariant 3d spherical harmonic shape

405

descriptors for shape analysis of the heart’s left ventricle - a pilot study, Pattern Recognition Letters 31 (13) (2010) 1981–1990. [27] M. K. Chung, R. Hartley, K. M. Dalton, R. J. Davidson, Encoding cortical surface by spherical harmonics, Statistica Sinica 18 (4) (2008) 1269–1291. 24

[28] J. Esquivel-Rodriguez, D. Kihara, Fitting multimeric protein complexes 410

into electron microscopy maps using 3d zernike descriptors, Journal of Physical Chemistry B 116 (23) (2012) 6854–6861. [29] L. Shen, F. Makedon, Spherical mapping for processing of 3d closed surfaces, Image and Vision Computing 24 (7) (2006) 743–761. [30] K. Khairy, J. Howard, Spherical harmonics-based parametric deconvolution

415

of 3d surface images using bending energy minimization, Medical Image Analysis 12 (2) (2008) 217–227. [31] M. Kazhdan, T. Funkhouser, S. Rusinkiewicz, Shape matching and anisotropy, Acm Transactions on Graphics 23 (3) (2004) 623–629. [32] D. V. Vranic, D. Saupe, Description of 3d-shape using a complex function

420

on the sphere, Ieee International Conference on Multimedia and Expo, Vol I and Ii, Proceedings (2002) 177–180. [33] S. Zhang, H.-S. Wong, Z. Yu, H. H. S. Ip, Hybrid associative retrieval of three-dimensional models, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 40 (6) (2010) 1582 – 1595.

425

[34] D. V. Vranic, D. Saupe, J. Richter, Tools for 3d-object retrieval: Karhunenloeve transform and spherical harmonics, 2001 Ieee Fourth Workshop on Multimedia Signal Processing (2001) 293–298. [35] J. Holland, Genetic algorithms, Scientific American (1992) 6672. [36] L. B. Booker, D. E. Goldberg, J. H. Holland, Classifier systems and genetic

430

algorithms, Artificial Intelligence 40 (1-3) (1989) 235–282. [37] P. Shilane, P. Min, M. Kazhdan, T. Funkhouser, The princeton shape benchmark, in: In Shape Modeling International, 2004, pp. 167–178.

25

(a)

(b) Figure 11: Retrieval results using a fighter aircraft and a car as queries.

26