- Email: [email protected]
Generic orthogonal moments: Jacobi–Fourier moments for invariant image description
Pattern Recognition 40 (2007) 1245 – 1254 www.elsevier.com/locate/pr
Generic orthogonal moments: Jacobi–Fourier moments for invariant image description Ziliang Ping a,∗ , Haiping Ren b , Jian Zou b , Yunlong Sheng c , Wurigen Bo d a Inner Mongolia Normal University, Huhhot, 010022 China b National Institute for Control of Pharmaceutical and Biological products, Beijing, 100050 China c Physical Department of Laval University, Que., Canada, G1K 7P4 d Mathematical Institute of Beijing University, Beijing, China
Received 8 January 2006; received in revised form 17 May 2006; accepted 25 July 2006
Abstract A multi-distorted invariant orthogonal moments, Jacobi–Fourier Moments (JFM), were proposed. The integral kernel of the moments was composed of radial Jacobi polynomial and angular Fourier complex componential factor. The variation of two parameters in Jacobi polynomial, and , can form various types of orthogonal moments: Legendre–Fourier Moments ( = 1, = 1); Chebyshev–Fourier Moments ( = 2, = 23 ); Orthogonal Fourier–Mellin Moments ( = 2, = 2); Zernike Moments and pseudo-Zernike Moments, and so on. Therefore, Jacobi–Fourier Moments are generic expressions of orthogonal moments formed by a radial orthogonal polynomial and angular Fourier complex component factor, providing a common mathematical tool for performance analysis of the orthogonal moments. In the paper, Jacobi–Fourier Moments were calculated for a deterministic image, and the original image was reconstructed with the moments. The relationship between Jacobi–Fourier Moments and other orthogonal moments was studied. Theoretical analysis and experimental investigation were conducted in terms of the description performance and noise sensibility of the JFM. 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Jacobi polynomial; Multi-distorted invariance; Jacobi–Fourier Moments; Image reconstruction error; Noise sensibility
1. Introduction There are a lot of literature on distorted-invariant pattern recognition including distorted-invariant descriptor [1–4] and synthetic filter method [15,16]. Those methods have solved the problem of some aspect of distorted invariant pattern recognition. It is important to acquire a set of orthogonal and multi-distorted invariant features of an image for image description and multi-distorted invariant pattern recognition. Hu [5] derived the moment invariant from geometrical moments in 1962, which are invariant for translation, rotation and scaling of the image. Hu’s moment invariants are not orthogonal themselves, so that
∗ Corresponding author. Tel.: +86 471 4392292; fax: +86 471 4392453.
E-mail address: [email protected] (Z. Ping).
reconstruction of the image with Hu’s moment invariant is impossible. The reconstruction of an image is an important question on how many moment invariants should be used and how well they describe the image. According to the orthogonal theory [6], an image function can be decomposed with orthogonal and completed function systems to form the independent orthogonal image moments, and original image can be reconstructed by the weighted superposition of the moments. Quality of the reconstructed image and quantity of the orthogonal moments needed for reconstructing an image can be evaluated by the reconstruction process. Teagure [7] first used the Zernike Moments [8] (ZM) for shape description. The kernel function of Zernike Moments is Zernike function system which is composed of a radial Zernike polynomial and an angular Fourier complex componential factor in polar coordinate system. Zernike
0031-3203/$30.00 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.07.016
1246
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Moments are orthogonal moments. Sheng et al. proposed the Orthogonal Fourier–Mellin Moments (OFMM) [9], which are constructed by the Gram–Schmidt orthogonization of a set of monomials of the lowest powers. Other orthogonal moments, such as Chebyshev–Fourier Moments (CHFM) [11], Radial-Harmonic-Fourier Moments (RHFM) [12] was proposed, as well. All the moments are constructed by the radial orthogonal polynomials and angular Fourier complex exponential factor to form the orthogonal kernel function to decompose the image function in polar coordinate system. Except this type of orthogonal moments, there are some other types of orthogonal moments, such as Legendre Moments (LM) [10], Discrete Chebyshev Moments (DCHM) [13] and Complex Moments (CM) [14] in a Cartesian coordinate system, but they are not rotation invariant. Teh et al. commented the description performance and noise sensibility of various image moments in 1988. They found that Zernike Moments have a superior performance over the others [10]. Sheng et al. [9] showed that the OFMM possesses a better performance over the Zernike Moments in 1994, especially for the description of small images. Ping et al. [11] showed that Chebishev–Fourier Moments possess the same performance as that of the OFMM. Ren et al. [12] showed that the Radial-Harmonic-Fourier Moments have the best performance in terms of image reconstruction and noise sensibility. Each of the above moments is independent and not associated with each other. We propose orthogonal moments, Jacobi–Fourier Moments (JFM), the kernel function of which consists of radial Jacobi polynomial and angular Fourier complex exponential factor. The JFM is a generic orthogonal moment. All orthogonal moments with the kernel function consisted of radial orthogonal polynomial and angular Fourier exponential factor are special cases of the JFM. A common formulation of orthogonal moments will benefit the performance analysis of the moments. In Section 2, the definition of the JFM is given and the behavior of Jacobi polynomials near the origin point of the coordinate system is investigated. In Section 3, the multidistortion invariance of JFM is discussed and the JFM is normalized for scale and intensity distorted invariance. In Section 4, JFM of English capital alphabet is calculated with various types of JFM and the original image is reconstructed with the moments, and the performance of JFM is analyzed in terms of image reconstruction error and noise sensibility. The last section is the conclusion. The relationship between JFM and other orthogonal moments is proved in the Appendix.
tial factor exp(jmϑ) to be an angular function: Pnm (r, ϑ) = Jn (, , r) exp(jmϑ),
(1)
where n and m are integers. The Jacobi–Fourier kernel function set is orthogonal in the interior of the unit circle, 0 r 1, 0 ϑ 2. 2 1 Pnm (r, ϑ)Pkl (r, ϑ)r dr dϑ = nk ml , (2) 0
0
where nk ml are Kronecker symbols and r = 1 is the maximum scale of the object in the concrete scene. The radial function Jn (r) and the Fourier angular kernel exp(jm) are separable. The exp(jm) is orthogonal and the radial function Jn (, , r) should be orthogonal in the interval 0 r 1 too: 1 Jn (r, , )Jk (r, , )r dr = nk (, ). (3) 0
In the orthogonal polynomial theory, the Jacobi polynomial Gn (, , r) is defined [6] as Gn (, , r) =
n n!( − 1)! (−1)s ( + n − 1)! s=0
( + n + s − 1)! rs. × (n − s)!s!( + s − 1)!
(4)
Jacobi polynomial Gn (, , r) is orthogonal in the interval 0 r 1: 1 Gn (, , r)Gm (, , r)w(, , r) dr 0
= bn (, )nm ,
(5)
where w(, , r) is the weight function and bn is the normalization constant: bn =
n![( − 1)!]2 ( − + n)! , ( + n − 1)!( + n − 1)!( + 2n)
(6)
which are a function of the parameters and , and the general weight function: w(, , r) = (1 − r)− r −1
− > − 1, > 0.
(7)
In the above formulas, and are real parameters, the value variation of which will form different Jacobi polynomials. Comparing formulas (3) and (5), we can get the radial function set: w(, , r) Jn (, , r) = Gn (, , r). (8) b(, )r
2. JFM
In the polar coordinate system, an image functionf (r, ϑ) can be decomposed into the superposition of weighted orthogonal components:
The JFM kernel function set Pnm (r, ϑ) consists of two separable function sets: the deformed Jacobi polynomial Jn (, , r) to be a radial function and the Fourier exponen-
f (r, ϑ) =
∞ ∞ n=0 m=−∞
nm Jn (r) exp(jmϑ),
(9)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
where nm are the coefficients of the decomposition and referred to as the JFM. 2 1 f (r, ϑ)Jn(,,r) exp(−jmϑ)r dr dϑ. (10) nm = 0
0
The explicit form of the radial function, a deformed Jacobi polynomial, Jn (, , r), depends on the values of the parameters, and , and are listed for some different integer numbers as follows: For = = 2: n
Jn (2, 2, r) = (−1)n
(−1)s
s=0
(n + s + 1)! rs. (n − s)!s!(s + 1)!
(11)
For = 3, = 2: Jn (3, 2, r) = (−1)n ×
n
(1 − r)(2n + 3) (n + 1)(n + 2)
(−1)s
s=0
(n + s + 2)! rs. (n − s)!s!(s + 1)!
(12)
For = = 3: n Jn (3, 3, r) = (−1)n (2n + 3)r (−1)s s=0
(n + s + 2)! rs. × (n − s)!s!(s + 2)!
(13)
For = 4, = 3: n
Jn (4, 3, r) = (−1) ×
n s=0
1247
tend to infinite when r tends to the origin point. Legendre polynomial and the radial function of CHFM do not satisfy the condition, and the radial function Jn (, , r) takes an infinite value at the origin point of the coordinate system. Fig. 1 shows the value distribution of the various radial functions Jn (, , r) in the interval 0 r 1. The order of the radial functions are n = 0, 1, 2, 9 and 10, respectively. All plot lines are vibrated and gradually declined with the increase the radial variable r. The distribution of zero points is almost uniform in the interval 0 r 1 for various types of radial function. The zero point numbers of radial functions are equal to the order number of the Jacobi polynomial. Although the profiles of the plot lines are approximately similar, the value distribution near the original point behaves differently for different radial functions. Some radial functions tend to infinite, others take the limited values. For the description of an image, uniform zero point distribution and value’s variation of radial function are better. The variation of a too large value of the radial function near the origin point will affect the reconstruction of the image. Comparing the value’s distribution diagram of different radial functions, we found that the radial function with parameter = 4, = 4 is better than others, which possesses a smaller value near the origin point than the other radial function, the reconstruction error of the image is a little smaller than the others (see Fig. 4), and the reconstruction situation is a little better than others (see Fig. 3). The prime JFM with the certain parameters, and , can be searched by mathematical analysis method on its general expression formula (10).
2r(1 − r)(n + 2) (n + 1)(n + 3)
(−1)s
(n + s + 3)! rs. (n − s)!s!(s + 2)!
(14)
Substituting, respectively, (11)–(14) into (10), we can get the different types of JFM with the different integral parameter. We show in the Appendix that the Jacobi Moments with = 1, = 1 are equal to the Legendre Moments in the Cartesian coordinate system. When = 2, = 23 , the JFM become the CHFM; the JFM with = 2, = 2 are equal to the OFMM. Because ZM and pseudo-Zernike polynomial are connected with the Jacobi polynomial by formula (10 ) and (11 ) in Appendix [9] , we can say that Zernike Moments and pseudo-Zernike Moments are special JFM, as well. All types of the JFM share the same angular function, but have different radial functions. In formula (8) the radial function consists of a Jacobi polynomial and a coefficient function including a radial variable r in the denominator. Comparing formulas (6)–(8), we can get that to ensure the limited value of radial function as r tend to original point of coordinate system, the conditions , 2 must be satisfied. Otherwise, radial function Jn (, , r) will
3. Normalization and invariance of the JFM The JFM are not multi-distorted invariants themselves, but we can get the multi-distorted invariance by normalization procedure for the translation, rotation, scale and intensity variation. First of all, the first order and zero order of the regular geometric moment, Gpq , was calculated, and the coordinate x = G10/G00, y = G01/G00 was taken as the original point of the coordinate system and center of the images. In this coordinate system, all the calculated moments are translation invariant. Secondly, because of the Fourier exponential factor e−jm in the integral kernel, any rotated angle of the image in the coordinate plane, , will cause a phase factor ejm for all moments, while the modular of the JFM, | nm |, reserves invariant. For the scale and intensity distortion of the image, the JFM can be normalized into an invariant according to the following steps: Suppose the distorted image is gf (r/k, ϑ), where g is the intensity variation factor, k is the scale variation factor. Using the low orders of Fourier–Mellin Moments [9] to calculate
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
10 8 6 4 2 0 -2 -4 -6 -8 -10
10 8 6 4 2 0 -2 -4 -6 -8 -10
n=10
Jacobi (3,2) radial function
Jacobi (3,2) radial function
1248
n=2 n=0
n=1
n=9
0
0.4
0.6 r (=2,=2)
0.2
0.8
n=2
n=1
n=10
0.2
0.4
0.6 r (=3,=2)
0.8
1
20
15 n=10
10 5
n=2
J (4,2,r)
Jacobi (3,3) radial function
n=0
0
1
20
n=0
0 n=1
-5 n=9
-10
15
n=10
10
n=2
5
n=0
0 -5
n=1
-10
-15
-15
-20
n=9
-20 0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 r (=4,=2)
r (=3,=3) 15
25 20
10
15
n=10
5
n=10
5
10 n=2
J (4,4,r)
Jacobi (4,3,r)
n=9
n=0
0 n=1
-5
n=2
n=0
0 n=1
-5 n=9
-10
-10
n=9
-15
-15
-20 0
0.2
r
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 r
(=4,=3)
(=4,=4)
0.4
0.6
0.8
1
Fig. 1. The value distribution of radial functions of different parameters in the interval 0 r 1.
the intensity factor and the scale factor: Msm =
2 0
ki =
r s f (r, ϑ) exp(−jmϑ)r dr dϑ,
(15)
0
i /M i ] [M10 00 , [M10 /M00 ]
gi =
1
(M10 /M00 ) i /M i ) (M10 00
(16) 2
i M00
M00
Taking the value of M10 /M00 to be a constant slightly i /M i of smaller than the minimum value in all the values M10 00 the training sample set, the scale factor, ki , and the intensity factor, gi , of each training image can be calculated according to formulas (16) and (17). JFM of each training image can be normalized as a distorted invariant by the following formula: inm =
.
(17)
2 0
ki
gi f (r/ki , )Jn (r/ki )e−jm r dr d,
(18)
0
inm = inm /gi ki2 .
(19)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Using the above steps, JFM of an image can be normalized as inm which is a multi-distorted invariance for translation, rotation, scale and intensity variation.
weighted superposition of limited lower orders of the orthogonal moments: f (r, ϑ) ≈
4. Image reconstruction and performance analysis of the JFM Image reconstruction can help to examine the image description performance of the moments and to determine how many moments are needed for an image reconstructed sufficiently well. By virtue of orthogonal function theory, the original image can be reconstructed approximately by
1249
M N
nm Jn (r) exp(jmϑ) = fˆ(r, ϑ),
(20)
n=0 m=−M
where N, M are the order numbers of JFM used in the reconstruction of the image, and fˆ(r, ϑ) is the partial sum of nm . Fig. 2 shows the 26 English characters reconstructed with JFM (=3, =2). Fig. 3 shows the detailed reconstruction process of character “E” of two different sizes, where ‘E3’ is a smaller one (64 × 64 pixels), and ‘E6’ is a larger one (128 × 128 pixels). In the experiment, six types of JFM were used for , = 2, 3, 4, respectively; the order of JFM is N = M = 2, 3, 5, 7, 10, 12, 15, 17 and 20, respectively. It can be seen that the reconstructed images are more approximate to the original image as more JFM orders are included. The vision effect is similar for various types of JFM, maybe the JFM with parameter = 4, = 4 is a little better. Figs. 4 and 5 show that the reconstruction error of image of JFM with parameter = 4, = 4 is a little smaller than the others. 4.1. The image description performance of JFM
Fig. 2. The reconstructed images of 26 English capitals with Jacobi– Fourier Moments = 3, = 2.
The normalization image reconstruction error (NIRE) [9] is used for the performance analysis of JFM, which is defined as the normalized mean square error for the reconstructed
Fig. 3. The reconstructed images of Jacobi–Fourier Moments of different parameters (moment’s order N = M = 2, 3, 5, 7, 10, 12, 15, 17, 20 for each piece of the image): (a) for small image ‘E3’; (b) for noise small image ‘E3’; (c) for large image ‘E6’; (d) for noise large image ‘E6’.
1250
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
NIRE curves of six types of JFM are almost overlapping with each other, showing similar image description performance. The image description ability of JFM for real image is also studied. A grayscale image with real noise can be reconstructed successfully with JFMs as shown in Fig. 6. There is no observed difference between the six types of JFM. 4.2. Noise sensibility of JFM Assume the image f (x, y) is a homogeneous random field with zero mean value and is corrupted by a zero-mean additive noise of spectral density 2 to form a noise image f (x, y) + n(x, y). The statistic signal–noise ratio is defined as [9]. Fig. 4. The NIRE of JFM with different parameters for deterministic image.
SNRnm =
var{(nm )f } 1 var{(nm )f }, = var{(nm )noise } 2
(23)
where 2 is the spectral density of the noise. The variance of the JFM for the random signal f (x, y) is defined as: 2k 2k var{(nm )f } =
Cff (x, y, u, v)Qn (r)Qn ( ) 0
0
0
0
× cos[m(ϑ − )]r dr dϑ d d, (24) Cff (x, y, u, v) = Cff (0, 0) × exp{−[(x − u)2 + (y − v)2]1/2 }, (25)
Fig. 5. The statistical NIRE of JFM with different parameters for a homogenous random image.
image fˆ(x, y): ∞ ∞ [f (x, y) − fˆ(x, y)]2 dx dy 2 .
= −∞ −∞ ∞ ∞ 2 −∞ −∞ f (x, y) dx dy
(21)
Fig. 4 shows the NIRE of six different types of JFM for the deterministic images “E3” and “E6”. Suppose the imagef (x, y) is a homogeneous random field with zero mean; then the statistical normalized image reconstruction error, 2 , can be defined as
∞ ∞ E −∞ −∞ [f (x, y) − fˆ(x, y)]2 dx dy ∞ ∞ , (22)
2 = E −∞ −∞ f 2 (x, y) dx dy where E{.} expresses the mathematical expectation. Fig. 5 shows the statistical NIRE of six types of JFM. It can be seen from Figs. 4 and 5 that the NIRE of a deterministic image and the statistic NIRE of a random image decrease with the increase in the number of JFM, and the
Cff (0, 0) = E{[f (x, y)]2 } 2 k 1 [f (r, ϑ)]2 dr dϑ, = 2 k 0 0
(26)
where k is the scale factor of the image determined by formula (16) and is a constant determined by the experiment. Fig. 7 shows the statistic signal–noise ratio (SSNR) of six JFM for a noise image, which decreases with the increase in the zero point’s numbers of radial function or moments’ orders of the JFM, showing the higher order of the moments is sensible to noise. For the reconstruction of a noise image the statistical normalization image reconstruction error (SNIRE) is defined as
1 1 E −1 −1 [f (x, y) − fˆ(x, y) − n(x, ˆ y)]2 dx dy
2n = 1 1 E −1 −1 [f (x, y)]2 dx dy
2 1 E [n(r, ˆ ϑ)]2 r dr dϑ
= 2 (N, M) + E
0 0
2 1
[f (r, ϑ)]2 r
dr dϑ
0 0
= 2 (N, M) +
Ntotal , SNRinput
(27)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
1251
Fig. 6. Reconstruction results for a grayscale image with real noise.
Fig. 7. The NIRE of different parameters for noise deterministic image (SNR = 100).
where 2n is the statistic NIRE of the noise image and 2 is the statistic of the JFM of the image without noise. n(x, y) is the partial sum of the ( nm )n , Ntotal is the total order numbers of the JFM used in reconstruction of the original image, and SNRinput = k 2 Cff (0, 0)/ 2 . Fig. 8 shows the NIRE of noise deterministic image (SNR = 100) reconstructed with JFM of different parameters. Fig. 9 shows the relationship of statistic NIRE 2n to the total numbers of the JFM used in reconstruction of noise image (k = 1, 0.5; SNRinput = 100). Fig. 3(b) and (d) show, respectively, the reconstruction process of the noise images “E3” and “E6” with JFM. Fig. 10 shows the reconstruction results using JFM for an image with the different SNR (SNR, respectively, equal to 100, 10, 1, 0.1). From Fig. 10, it can be seen that all noise images were reconstructed well; even though the signal–noise ratio is very low, the image was almost merged by noise, the image still can be reconstructed with the JFM, verifying that JFM possesses a strong power resistance to noise. The enhancement of a noise image is a very challenging issue in many research and application areas. Enhancement of an image includes both filtering various kinds of noise and preserving or possibly enhancing the image details and edges. The filtering image noise and preserving image de-
Fig. 8. The statistical NIRE of different parameters for noise image (SNR = 100).
Fig. 9. The statistic SNR of JFM of different parameters =2; m=0, 5, 10.
tails are intrinsically conflict operations. Based on fuzzy logic [17], the fuzzy image enhancement technology (FIE), which applies fuzzy inference to develop nonlinear filters [18–20], has been shown to be very effective in removing noise without destroying the useful information contained in the image data. The JFM can remove the image noise by decomposing and reconstructing the original image, which
1252
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Acknowledgments This work was supported by the National Natural Science Foundation of the Chinese Government (Grant No. 60562001) and by the Inner Mongolia National Science Foundation (Grant No. 2001301).
Appendix A. Relationship between the JFM and the other orthogonal moments The JFM are generic orthogonal moments. Other types of orthogonal moments can be derived from the JFM for different parameter values of the and in Jacobi Polynomial. A.1. Legendre moments
Fig. 10. The reconstruction of noise image with different SNR: from left to right SNR=∞, 100, 10, 1, 0.1; N =M =7: (a) original; (b) =2, =2; (c) = 3; = 2; (d) = 3, = 3.
According to the orthogonal polynomial theory [6], (,) there are two types of Jacobi polynomial: Pn (r) and Gn (, , r). The relationship between them is as follows: (,)
Pn
(r) =
is different from FIE in intrinsic. The JFM can provide nonoise image features and reconstruct the no-noise original image.
(2n + + + 1) n! (n + + + 1) r +1 × Gn + + 1, + 1, . 2
(1 )
The Jacobi polynomial family and Legendre polynomial family are related by 5. Conclusion We proposed the JFM, which are orthogonal and distorted invariants for translation, rotation, scale and intensity variation passing through the normalization processing. We have calculated six types of JFM with different parameters of the kernel function and reconstructed the original images. Because of the orthogonal property of the moments, the original image can be reconstructed well. The theory analysis and experiment investigation have verified that the JFM can describe the image perfectly, having a strong performance resist noise. The Jacobi polynomial has two variable parameters, and being the radial function of the orthogonal function system, the variation of parameter values will form a different kernel in the moments integral, and therefore form different types of JFM. In the Appendix we will demonstrate that the JFM is a generic orthogonal moment. Almost all orthogonal moments that consist of radial polynomial and angular Fourier exponential factor can be derived from the JFM in terms of different parameter values, and those orthogonal moments possess the same property of describing image and noise sensibility. The common formulation of the moments will be a benefit for performance analysis of the orthogonal moments and for searching a prime orthogonal moment.
Pn (r) = Pn(0,0) (r),
(2 ) (,)
where Pn (r) is Legendre polynomial of order n, Pn (r) is the Jacobi polynomial of order n with two real variable parameters , . Compare (1 ) and (2 ) and Let =0, =0; the relationship between Legendre polynomials and Jacobi polynomial can be obtained: r +1
(2n + 1) (0,0) Pn (r) = Pn (r) = Gn 1, 1, , (3 ) n! (n + 1) 2 where (.) is the Gamma function. The first few terms of the Legendre polynomial are Pn (r) =
[n/2] 1 2n − 2m n−2m n n r (−1) , m n 2n m=0
P1 (r) = r, P2 (r) = P3 (r) =
5 3 2r
−
P5 (r) =
63 5 8 r
3 2
−
3 2 2r
r,
70 3 8 r
− 21 ,
P4 (r) = +
15 8
35 4 8 r
r, . . . .
−
30 2 8 r
+ 38 ,
(4 )
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
The first few terms of the deformed Jacobi polynomial are
(2n + 1) r +1 1 Gn 1, 1, = n! (n + 1) 2 n! n
(2n + 1 − m) r + 1 n−m m n × (−1) , m (n + 1 − m) 2
r +1 1, 1, 2
G2
= r,
r +1 G3 1, 1, = 2 r +1 G4 1, 1, = 2 r +1 G5 1, 1, = 2
5 3 2r
r +1 1, 1, 2
(5 )
=
3 2 2r
−
The first few terms of deformed Jacobi polynomial are 22n Gn (2, 3/2, (r + 1)/2) = 22n
m=0
G1
1253
×
n
(−1)
m=0
G1 (r) = 2r,
1 2,
n
n m
(n + 3/2)
(2n + 2)
(2n + 2 − m) n−m r ,
(n + 3/2 − m)
(9 )
G2 (r) = 4r 2 − 1,
G3 (r) = 8r 3 − 4r,
G4 (r) = 16r 4 − 12r 2 + 1,
G5 (r) = 32r 5 − 32r 3 + 6r, . . . .
− 23 ,
35 4 8 r
−
30 2 8 r
+ 38 ,
63 5 8 r
−
70 3 8 r
+
15 8
The Chebyshev polynomials in formula (8 ) are identical to the Jacobi polynomial in formula (9 ). The Chebyshev polynomials are special cases of the Jacobi polynomial. They both are related by formula (7 ). r, . . . . A.3. Orthogonal Fourier–Mellin Moments
The expression of each order of the Legendre polynomial in formula (4 ) is equal to the expression of same order of the Jacobi polynomial in formula (5 ). The Legendre polynomial is a special situation of the Jacobi polynomial, and both are connected by formula (3 ).
It can be seen that when =2, =2, the radial function of the JFM shown in formula (11) of the normal text is identical to the radial polynomials of the Orthogonal Fourier–Mellin Moments [9]. The Orthogonal Fourier–Mellin Moments is the special situation of the JFM.
A.2. Chebyshev–Fourier Moments
A.4. Zernike moments and pseudo-Zernike moments
According to the orthogonal polynomial theory [6] the Jacobi polynomial family and the Chebyshev polynomial family are related by
In Appendix B of reference literature [9], the relationship of Jacobi polynomial with Zernike polynomial and pseudoZernike polynomial were given as follows: m+s m |m| Rm+2s (r) = (−1)m+2s r s
Un (r) =
√ (n + 1)! 2 (n + 23 )
(1/2,1/2)
Pn
(r),
(6 )
where Un (r) is the second kind Chebyshev polynomial of (1/2,1/2) order n, Pn (r) is the Jacobi polynomial of order n 1 with = 2 , = 21 . Substituting formula (1 ) in Eq. (6 ) and using the property of the function, we can get √ (n + 1)(2n + 1)! Gn (2, 23 , (r + 2)/2) Un (r) = 2(n + 1)! (n + 3/2) = 22n Gn [2, 23 , (r + 1)/2].
|m|
(7 )
The first few terms of the Chebyshev polynomial are Un (r) =
n/2
(−1)n
m=0
U1 (r) = 2r,
(n − m)! (2r)n−2m , m!(n − 2m)!
U2 (r) = 4r 2 − 1,
U3 (r) = 8r 3 − 4r,
U4 (r) = 16r 4 − 12r 2 + 1,
U5 (r) = 32r 5 − 32r 3 + 6r, . . . .
(8 )
(10 )
Gs (m + 1, m + 1, r 2 ), 2m + s + 1 m |m| Pm+s (r) = (−1)s r s Gs (2m + 2, 2m + 2, r),
(11 ) |m|
where Rm+2s (r) is the Zernike polynomial, Pm+s (r) is the pseudo-Zernike polynomial. From formulas (10 ) and (11 ), it can be seen that the Zernike polynomial and pseudoZernike polynomial are special cases of the Jacobi polynomials.
References [1] D. Casasent, D. Psaltis, Position rotation and scale invariant optical correlation, Appl. Opt. 15 (1976) 1795–1799. [2] H.H. Arsennault, Y. Sheng, Properties of the circular harmonic expansion for rotation-invariant pattern recognition, Appl. Opt. 25 (18) (1986) 3225–3229. [3] Z.L. Ping, D.X. Xu, Optical method for completing simultaneously polar coordinate transfer and radial logarithmic transfer, Acta Optica Sin. 4 (5) (1984) 425–429.
1254
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
[4] Z.L. Ping, Y.L. Sheng, Fourier–Mellin descriptor and interpolated feature space trajectories for three-dimensional object recognition, Opt. Eng. 39 (2000) 1260–1266. [5] M.K. Hu, Visual Pattern recognition by moment invariants, IRE. Trans. Inf. Theory IT-8 (1962) 179–187. [6] M. Abramowitz, I.A. Stegun, Handbook of Functions With Formulas, Graphs and Mathematical Tables, Dover, New York, 1964, pp. 733–792. [7] M.R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70 (1980) 920–930. [8] M. Born, E. Wolf, Principle of Optics, sixth ed., Pergamon Press, Oxford, 1980, pp. 767–772. [9] Y. Sheng, L. Shen, Orthogonal Fourier–Mellin moments for invariant pattern recognition, J. Opt. Soc. Am. A 11 (6) (1994) 1748–1757. [10] C.H. Teh, R.T. Chin, On image analysis by the methods of moments, IEEE Trans. Pattern Anal. Mach. Intell. 10 (1988) 496–513. [11] Z.L. Ping, R. Wu, Y.L. Sheng, Image description with Chebyshev– Fourier moments, J. Opt. Soc. Am. A 19 (9) (2002)1748–1754. [12] H.P. Ren, Z.L. Ping, Wurigen, Y.L. Sheng, Multi-distorted invariant Image recognition with Radial-Harmonic-Fourier Moments, J. Opt. Soc. Am. A 20 (4) (2003) 631–637.
[13] R. Mukundan, S.H. Ong, P.A. Lee, Image analysis by Tchebichef moments, IEEE Trans. Image Process. 10 (9) (2001). [14] Y.S. Abu-Mostafa, D. Psaltis, Recognition aspect of moment invariant, IEEE Trans. Pattern Anal. Mach. Intell. PAM-6 (1984) 698–706. [15] P. Ziliang, E. Dalsgaad, Synthetic circular harmonic phase-only filter for shifting, rotation and scaling invariant correlation, Appl. Opt. 34 (1995) 7527–7531. [16] P. Ziliang, S. Li, J. Wang, E. Dalsgaard, Circular harmonic minimum average energy filter for the multi-distorted invariant correlation, Chin. J. Laser B7 (4) (1998) 347–352. [17] L.A. Zadeh, Fuzzy set, Inf. Control 8 (1965) 338–353. [18] F. Russo, Fuzzy processing of image data using FIRE filter, J. Intell. Fuzzy Syst. 15 (4) (1997) 361–366. [19] A. Taguchi, H. Takashima, Y. Murata, Fuzzy filter for image smoothing, in: Proceedings of the SPIE Conference on Nonlinear Image Processing, V. San. Jose, CA, 7–9 February 1994, pp. 332–339. [20] Y. Choi, R. Krishnapuram, A robust approach to image enhancement based on fuzzy logic, IEEE Trans. Image Process. 6 (1997) 808–825.
About the Author—ZILIANG PING, received a BS degree in physics from Nankai University of China in 1968 and an MS degree in optics from The Post and Communication University of Beijing of China in 1983. He is now a professor at the Inner Mongolia Normal University of China. He has published about 50 journal papers and conference papers. He was awarded a special dedication allowance by the Chinese government in 1993. His research interests involve optical processing, digital image processing and pattern recognition. About the Author—HAIPING REN received her PhD degree in medical sciences in 2003 from the Peking Union Medical College, Beijing, China. Then she joined the Department of Medical Devices, National Institute for the Control of Pharmaceutical and Biological Products, State Food and Drug Administration, China, as a Researcher and, since 2006, she has been the Senior Engineer on Medical Devices. Her research interests are concerned with the application of computational sciences to medical fields, such as medical image processing, pattern recognition, quality control of medical devices. Her research has been supported by both National and Beijing Natural Science Foundation. More than 20 papers have been published in various journals and conference proceedings. About the Author—JIAN ZOU received his Bachelor’s degree in 1986 from Nanjin Pharmaceutical University. In 1987, he joined the National Institute for the Control of Pharmaceutical and Biological Products, State Food and Drug Administration, China, as a Pharmacist. Since 2006, he has been Chief Pharmacist. Since 2002, he has been the director of Department of Medical Devices. His research focuses are quality control and supervision of medical devices. More than 10 papers have been published in various journals and conference proceedings. About the Author—YUNLONG SHENG received his BS degree from the University of Sciences and Technology of china in 1964 and his MS, Dr, Doctor d’Etat degrees in physics from the University de Franche-Comte, Besancon, France, in 1980, 1982 and 1986, respectively. Since 1985 he has been with Centre d’Optique, Photonique et Laser, University Laval, and is now a professor. Dr. Sheng has authored and coauthored more than 70 refereed journal papers, book chapters and conference papers. His research interests involve optics, optical neural networks, and optical interconnects. Dr. Sheng is a fellow of SPIE and a member of OSA and the International Neural Network Society. He also serves as a consultant for industrial companies in the United States and Canada. About the Author—B. WURIGEN received his BS and MS degrees from Beijing University in 2001 and 2004, respectively. Currently he is a doctor degree student in New York University.
Generic orthogonal moments: Jacobi–Fourier moments for invariant image description Ziliang Ping a,∗ , Haiping Ren b , Jian Zou b , Yunlong Sheng c , Wurigen Bo d a Inner Mongolia Normal University, Huhhot, 010022 China b National Institute for Control of Pharmaceutical and Biological products, Beijing, 100050 China c Physical Department of Laval University, Que., Canada, G1K 7P4 d Mathematical Institute of Beijing University, Beijing, China
Received 8 January 2006; received in revised form 17 May 2006; accepted 25 July 2006
Abstract A multi-distorted invariant orthogonal moments, Jacobi–Fourier Moments (JFM), were proposed. The integral kernel of the moments was composed of radial Jacobi polynomial and angular Fourier complex componential factor. The variation of two parameters in Jacobi polynomial, and , can form various types of orthogonal moments: Legendre–Fourier Moments ( = 1, = 1); Chebyshev–Fourier Moments ( = 2, = 23 ); Orthogonal Fourier–Mellin Moments ( = 2, = 2); Zernike Moments and pseudo-Zernike Moments, and so on. Therefore, Jacobi–Fourier Moments are generic expressions of orthogonal moments formed by a radial orthogonal polynomial and angular Fourier complex component factor, providing a common mathematical tool for performance analysis of the orthogonal moments. In the paper, Jacobi–Fourier Moments were calculated for a deterministic image, and the original image was reconstructed with the moments. The relationship between Jacobi–Fourier Moments and other orthogonal moments was studied. Theoretical analysis and experimental investigation were conducted in terms of the description performance and noise sensibility of the JFM. 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Jacobi polynomial; Multi-distorted invariance; Jacobi–Fourier Moments; Image reconstruction error; Noise sensibility
1. Introduction There are a lot of literature on distorted-invariant pattern recognition including distorted-invariant descriptor [1–4] and synthetic filter method [15,16]. Those methods have solved the problem of some aspect of distorted invariant pattern recognition. It is important to acquire a set of orthogonal and multi-distorted invariant features of an image for image description and multi-distorted invariant pattern recognition. Hu [5] derived the moment invariant from geometrical moments in 1962, which are invariant for translation, rotation and scaling of the image. Hu’s moment invariants are not orthogonal themselves, so that
∗ Corresponding author. Tel.: +86 471 4392292; fax: +86 471 4392453.
E-mail address: [email protected] (Z. Ping).
reconstruction of the image with Hu’s moment invariant is impossible. The reconstruction of an image is an important question on how many moment invariants should be used and how well they describe the image. According to the orthogonal theory [6], an image function can be decomposed with orthogonal and completed function systems to form the independent orthogonal image moments, and original image can be reconstructed by the weighted superposition of the moments. Quality of the reconstructed image and quantity of the orthogonal moments needed for reconstructing an image can be evaluated by the reconstruction process. Teagure [7] first used the Zernike Moments [8] (ZM) for shape description. The kernel function of Zernike Moments is Zernike function system which is composed of a radial Zernike polynomial and an angular Fourier complex componential factor in polar coordinate system. Zernike
0031-3203/$30.00 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.07.016
1246
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Moments are orthogonal moments. Sheng et al. proposed the Orthogonal Fourier–Mellin Moments (OFMM) [9], which are constructed by the Gram–Schmidt orthogonization of a set of monomials of the lowest powers. Other orthogonal moments, such as Chebyshev–Fourier Moments (CHFM) [11], Radial-Harmonic-Fourier Moments (RHFM) [12] was proposed, as well. All the moments are constructed by the radial orthogonal polynomials and angular Fourier complex exponential factor to form the orthogonal kernel function to decompose the image function in polar coordinate system. Except this type of orthogonal moments, there are some other types of orthogonal moments, such as Legendre Moments (LM) [10], Discrete Chebyshev Moments (DCHM) [13] and Complex Moments (CM) [14] in a Cartesian coordinate system, but they are not rotation invariant. Teh et al. commented the description performance and noise sensibility of various image moments in 1988. They found that Zernike Moments have a superior performance over the others [10]. Sheng et al. [9] showed that the OFMM possesses a better performance over the Zernike Moments in 1994, especially for the description of small images. Ping et al. [11] showed that Chebishev–Fourier Moments possess the same performance as that of the OFMM. Ren et al. [12] showed that the Radial-Harmonic-Fourier Moments have the best performance in terms of image reconstruction and noise sensibility. Each of the above moments is independent and not associated with each other. We propose orthogonal moments, Jacobi–Fourier Moments (JFM), the kernel function of which consists of radial Jacobi polynomial and angular Fourier complex exponential factor. The JFM is a generic orthogonal moment. All orthogonal moments with the kernel function consisted of radial orthogonal polynomial and angular Fourier exponential factor are special cases of the JFM. A common formulation of orthogonal moments will benefit the performance analysis of the moments. In Section 2, the definition of the JFM is given and the behavior of Jacobi polynomials near the origin point of the coordinate system is investigated. In Section 3, the multidistortion invariance of JFM is discussed and the JFM is normalized for scale and intensity distorted invariance. In Section 4, JFM of English capital alphabet is calculated with various types of JFM and the original image is reconstructed with the moments, and the performance of JFM is analyzed in terms of image reconstruction error and noise sensibility. The last section is the conclusion. The relationship between JFM and other orthogonal moments is proved in the Appendix.
tial factor exp(jmϑ) to be an angular function: Pnm (r, ϑ) = Jn (, , r) exp(jmϑ),
(1)
where n and m are integers. The Jacobi–Fourier kernel function set is orthogonal in the interior of the unit circle, 0 r 1, 0 ϑ 2. 2 1 Pnm (r, ϑ)Pkl (r, ϑ)r dr dϑ = nk ml , (2) 0
0
where nk ml are Kronecker symbols and r = 1 is the maximum scale of the object in the concrete scene. The radial function Jn (r) and the Fourier angular kernel exp(jm) are separable. The exp(jm) is orthogonal and the radial function Jn (, , r) should be orthogonal in the interval 0 r 1 too: 1 Jn (r, , )Jk (r, , )r dr = nk (, ). (3) 0
In the orthogonal polynomial theory, the Jacobi polynomial Gn (, , r) is defined [6] as Gn (, , r) =
n n!( − 1)! (−1)s ( + n − 1)! s=0
( + n + s − 1)! rs. × (n − s)!s!( + s − 1)!
(4)
Jacobi polynomial Gn (, , r) is orthogonal in the interval 0 r 1: 1 Gn (, , r)Gm (, , r)w(, , r) dr 0
= bn (, )nm ,
(5)
where w(, , r) is the weight function and bn is the normalization constant: bn =
n![( − 1)!]2 ( − + n)! , ( + n − 1)!( + n − 1)!( + 2n)
(6)
which are a function of the parameters and , and the general weight function: w(, , r) = (1 − r)− r −1
− > − 1, > 0.
(7)
In the above formulas, and are real parameters, the value variation of which will form different Jacobi polynomials. Comparing formulas (3) and (5), we can get the radial function set: w(, , r) Jn (, , r) = Gn (, , r). (8) b(, )r
2. JFM
In the polar coordinate system, an image functionf (r, ϑ) can be decomposed into the superposition of weighted orthogonal components:
The JFM kernel function set Pnm (r, ϑ) consists of two separable function sets: the deformed Jacobi polynomial Jn (, , r) to be a radial function and the Fourier exponen-
f (r, ϑ) =
∞ ∞ n=0 m=−∞
nm Jn (r) exp(jmϑ),
(9)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
where nm are the coefficients of the decomposition and referred to as the JFM. 2 1 f (r, ϑ)Jn(,,r) exp(−jmϑ)r dr dϑ. (10) nm = 0
0
The explicit form of the radial function, a deformed Jacobi polynomial, Jn (, , r), depends on the values of the parameters, and , and are listed for some different integer numbers as follows: For = = 2: n
Jn (2, 2, r) = (−1)n
(−1)s
s=0
(n + s + 1)! rs. (n − s)!s!(s + 1)!
(11)
For = 3, = 2: Jn (3, 2, r) = (−1)n ×
n
(1 − r)(2n + 3) (n + 1)(n + 2)
(−1)s
s=0
(n + s + 2)! rs. (n − s)!s!(s + 1)!
(12)
For = = 3: n Jn (3, 3, r) = (−1)n (2n + 3)r (−1)s s=0
(n + s + 2)! rs. × (n − s)!s!(s + 2)!
(13)
For = 4, = 3: n
Jn (4, 3, r) = (−1) ×
n s=0
1247
tend to infinite when r tends to the origin point. Legendre polynomial and the radial function of CHFM do not satisfy the condition, and the radial function Jn (, , r) takes an infinite value at the origin point of the coordinate system. Fig. 1 shows the value distribution of the various radial functions Jn (, , r) in the interval 0 r 1. The order of the radial functions are n = 0, 1, 2, 9 and 10, respectively. All plot lines are vibrated and gradually declined with the increase the radial variable r. The distribution of zero points is almost uniform in the interval 0 r 1 for various types of radial function. The zero point numbers of radial functions are equal to the order number of the Jacobi polynomial. Although the profiles of the plot lines are approximately similar, the value distribution near the original point behaves differently for different radial functions. Some radial functions tend to infinite, others take the limited values. For the description of an image, uniform zero point distribution and value’s variation of radial function are better. The variation of a too large value of the radial function near the origin point will affect the reconstruction of the image. Comparing the value’s distribution diagram of different radial functions, we found that the radial function with parameter = 4, = 4 is better than others, which possesses a smaller value near the origin point than the other radial function, the reconstruction error of the image is a little smaller than the others (see Fig. 4), and the reconstruction situation is a little better than others (see Fig. 3). The prime JFM with the certain parameters, and , can be searched by mathematical analysis method on its general expression formula (10).
2r(1 − r)(n + 2) (n + 1)(n + 3)
(−1)s
(n + s + 3)! rs. (n − s)!s!(s + 2)!
(14)
Substituting, respectively, (11)–(14) into (10), we can get the different types of JFM with the different integral parameter. We show in the Appendix that the Jacobi Moments with = 1, = 1 are equal to the Legendre Moments in the Cartesian coordinate system. When = 2, = 23 , the JFM become the CHFM; the JFM with = 2, = 2 are equal to the OFMM. Because ZM and pseudo-Zernike polynomial are connected with the Jacobi polynomial by formula (10 ) and (11 ) in Appendix [9] , we can say that Zernike Moments and pseudo-Zernike Moments are special JFM, as well. All types of the JFM share the same angular function, but have different radial functions. In formula (8) the radial function consists of a Jacobi polynomial and a coefficient function including a radial variable r in the denominator. Comparing formulas (6)–(8), we can get that to ensure the limited value of radial function as r tend to original point of coordinate system, the conditions , 2 must be satisfied. Otherwise, radial function Jn (, , r) will
3. Normalization and invariance of the JFM The JFM are not multi-distorted invariants themselves, but we can get the multi-distorted invariance by normalization procedure for the translation, rotation, scale and intensity variation. First of all, the first order and zero order of the regular geometric moment, Gpq , was calculated, and the coordinate x = G10/G00, y = G01/G00 was taken as the original point of the coordinate system and center of the images. In this coordinate system, all the calculated moments are translation invariant. Secondly, because of the Fourier exponential factor e−jm in the integral kernel, any rotated angle of the image in the coordinate plane, , will cause a phase factor ejm for all moments, while the modular of the JFM, | nm |, reserves invariant. For the scale and intensity distortion of the image, the JFM can be normalized into an invariant according to the following steps: Suppose the distorted image is gf (r/k, ϑ), where g is the intensity variation factor, k is the scale variation factor. Using the low orders of Fourier–Mellin Moments [9] to calculate
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
10 8 6 4 2 0 -2 -4 -6 -8 -10
10 8 6 4 2 0 -2 -4 -6 -8 -10
n=10
Jacobi (3,2) radial function
Jacobi (3,2) radial function
1248
n=2 n=0
n=1
n=9
0
0.4
0.6 r (=2,=2)
0.2
0.8
n=2
n=1
n=10
0.2
0.4
0.6 r (=3,=2)
0.8
1
20
15 n=10
10 5
n=2
J (4,2,r)
Jacobi (3,3) radial function
n=0
0
1
20
n=0
0 n=1
-5 n=9
-10
15
n=10
10
n=2
5
n=0
0 -5
n=1
-10
-15
-15
-20
n=9
-20 0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 r (=4,=2)
r (=3,=3) 15
25 20
10
15
n=10
5
n=10
5
10 n=2
J (4,4,r)
Jacobi (4,3,r)
n=9
n=0
0 n=1
-5
n=2
n=0
0 n=1
-5 n=9
-10
-10
n=9
-15
-15
-20 0
0.2
r
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 r
(=4,=3)
(=4,=4)
0.4
0.6
0.8
1
Fig. 1. The value distribution of radial functions of different parameters in the interval 0 r 1.
the intensity factor and the scale factor: Msm =
2 0
ki =
r s f (r, ϑ) exp(−jmϑ)r dr dϑ,
(15)
0
i /M i ] [M10 00 , [M10 /M00 ]
gi =
1
(M10 /M00 ) i /M i ) (M10 00
(16) 2
i M00
M00
Taking the value of M10 /M00 to be a constant slightly i /M i of smaller than the minimum value in all the values M10 00 the training sample set, the scale factor, ki , and the intensity factor, gi , of each training image can be calculated according to formulas (16) and (17). JFM of each training image can be normalized as a distorted invariant by the following formula: inm =
.
(17)
2 0
ki
gi f (r/ki , )Jn (r/ki )e−jm r dr d,
(18)
0
inm = inm /gi ki2 .
(19)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Using the above steps, JFM of an image can be normalized as inm which is a multi-distorted invariance for translation, rotation, scale and intensity variation.
weighted superposition of limited lower orders of the orthogonal moments: f (r, ϑ) ≈
4. Image reconstruction and performance analysis of the JFM Image reconstruction can help to examine the image description performance of the moments and to determine how many moments are needed for an image reconstructed sufficiently well. By virtue of orthogonal function theory, the original image can be reconstructed approximately by
1249
M N
nm Jn (r) exp(jmϑ) = fˆ(r, ϑ),
(20)
n=0 m=−M
where N, M are the order numbers of JFM used in the reconstruction of the image, and fˆ(r, ϑ) is the partial sum of nm . Fig. 2 shows the 26 English characters reconstructed with JFM (=3, =2). Fig. 3 shows the detailed reconstruction process of character “E” of two different sizes, where ‘E3’ is a smaller one (64 × 64 pixels), and ‘E6’ is a larger one (128 × 128 pixels). In the experiment, six types of JFM were used for , = 2, 3, 4, respectively; the order of JFM is N = M = 2, 3, 5, 7, 10, 12, 15, 17 and 20, respectively. It can be seen that the reconstructed images are more approximate to the original image as more JFM orders are included. The vision effect is similar for various types of JFM, maybe the JFM with parameter = 4, = 4 is a little better. Figs. 4 and 5 show that the reconstruction error of image of JFM with parameter = 4, = 4 is a little smaller than the others. 4.1. The image description performance of JFM
Fig. 2. The reconstructed images of 26 English capitals with Jacobi– Fourier Moments = 3, = 2.
The normalization image reconstruction error (NIRE) [9] is used for the performance analysis of JFM, which is defined as the normalized mean square error for the reconstructed
Fig. 3. The reconstructed images of Jacobi–Fourier Moments of different parameters (moment’s order N = M = 2, 3, 5, 7, 10, 12, 15, 17, 20 for each piece of the image): (a) for small image ‘E3’; (b) for noise small image ‘E3’; (c) for large image ‘E6’; (d) for noise large image ‘E6’.
1250
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
NIRE curves of six types of JFM are almost overlapping with each other, showing similar image description performance. The image description ability of JFM for real image is also studied. A grayscale image with real noise can be reconstructed successfully with JFMs as shown in Fig. 6. There is no observed difference between the six types of JFM. 4.2. Noise sensibility of JFM Assume the image f (x, y) is a homogeneous random field with zero mean value and is corrupted by a zero-mean additive noise of spectral density 2 to form a noise image f (x, y) + n(x, y). The statistic signal–noise ratio is defined as [9]. Fig. 4. The NIRE of JFM with different parameters for deterministic image.
SNRnm =
var{(nm )f } 1 var{(nm )f }, = var{(nm )noise } 2
(23)
where 2 is the spectral density of the noise. The variance of the JFM for the random signal f (x, y) is defined as: 2k 2k var{(nm )f } =
Cff (x, y, u, v)Qn (r)Qn ( ) 0
0
0
0
× cos[m(ϑ − )]r dr dϑ d d, (24) Cff (x, y, u, v) = Cff (0, 0) × exp{−[(x − u)2 + (y − v)2]1/2 }, (25)
Fig. 5. The statistical NIRE of JFM with different parameters for a homogenous random image.
image fˆ(x, y): ∞ ∞ [f (x, y) − fˆ(x, y)]2 dx dy 2 .
= −∞ −∞ ∞ ∞ 2 −∞ −∞ f (x, y) dx dy
(21)
Fig. 4 shows the NIRE of six different types of JFM for the deterministic images “E3” and “E6”. Suppose the imagef (x, y) is a homogeneous random field with zero mean; then the statistical normalized image reconstruction error, 2 , can be defined as
∞ ∞ E −∞ −∞ [f (x, y) − fˆ(x, y)]2 dx dy ∞ ∞ , (22)
2 = E −∞ −∞ f 2 (x, y) dx dy where E{.} expresses the mathematical expectation. Fig. 5 shows the statistical NIRE of six types of JFM. It can be seen from Figs. 4 and 5 that the NIRE of a deterministic image and the statistic NIRE of a random image decrease with the increase in the number of JFM, and the
Cff (0, 0) = E{[f (x, y)]2 } 2 k 1 [f (r, ϑ)]2 dr dϑ, = 2 k 0 0
(26)
where k is the scale factor of the image determined by formula (16) and is a constant determined by the experiment. Fig. 7 shows the statistic signal–noise ratio (SSNR) of six JFM for a noise image, which decreases with the increase in the zero point’s numbers of radial function or moments’ orders of the JFM, showing the higher order of the moments is sensible to noise. For the reconstruction of a noise image the statistical normalization image reconstruction error (SNIRE) is defined as
1 1 E −1 −1 [f (x, y) − fˆ(x, y) − n(x, ˆ y)]2 dx dy
2n = 1 1 E −1 −1 [f (x, y)]2 dx dy
2 1 E [n(r, ˆ ϑ)]2 r dr dϑ
= 2 (N, M) + E
0 0
2 1
[f (r, ϑ)]2 r
dr dϑ
0 0
= 2 (N, M) +
Ntotal , SNRinput
(27)
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
1251
Fig. 6. Reconstruction results for a grayscale image with real noise.
Fig. 7. The NIRE of different parameters for noise deterministic image (SNR = 100).
where 2n is the statistic NIRE of the noise image and 2 is the statistic of the JFM of the image without noise. n(x, y) is the partial sum of the ( nm )n , Ntotal is the total order numbers of the JFM used in reconstruction of the original image, and SNRinput = k 2 Cff (0, 0)/ 2 . Fig. 8 shows the NIRE of noise deterministic image (SNR = 100) reconstructed with JFM of different parameters. Fig. 9 shows the relationship of statistic NIRE 2n to the total numbers of the JFM used in reconstruction of noise image (k = 1, 0.5; SNRinput = 100). Fig. 3(b) and (d) show, respectively, the reconstruction process of the noise images “E3” and “E6” with JFM. Fig. 10 shows the reconstruction results using JFM for an image with the different SNR (SNR, respectively, equal to 100, 10, 1, 0.1). From Fig. 10, it can be seen that all noise images were reconstructed well; even though the signal–noise ratio is very low, the image was almost merged by noise, the image still can be reconstructed with the JFM, verifying that JFM possesses a strong power resistance to noise. The enhancement of a noise image is a very challenging issue in many research and application areas. Enhancement of an image includes both filtering various kinds of noise and preserving or possibly enhancing the image details and edges. The filtering image noise and preserving image de-
Fig. 8. The statistical NIRE of different parameters for noise image (SNR = 100).
Fig. 9. The statistic SNR of JFM of different parameters =2; m=0, 5, 10.
tails are intrinsically conflict operations. Based on fuzzy logic [17], the fuzzy image enhancement technology (FIE), which applies fuzzy inference to develop nonlinear filters [18–20], has been shown to be very effective in removing noise without destroying the useful information contained in the image data. The JFM can remove the image noise by decomposing and reconstructing the original image, which
1252
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
Acknowledgments This work was supported by the National Natural Science Foundation of the Chinese Government (Grant No. 60562001) and by the Inner Mongolia National Science Foundation (Grant No. 2001301).
Appendix A. Relationship between the JFM and the other orthogonal moments The JFM are generic orthogonal moments. Other types of orthogonal moments can be derived from the JFM for different parameter values of the and in Jacobi Polynomial. A.1. Legendre moments
Fig. 10. The reconstruction of noise image with different SNR: from left to right SNR=∞, 100, 10, 1, 0.1; N =M =7: (a) original; (b) =2, =2; (c) = 3; = 2; (d) = 3, = 3.
According to the orthogonal polynomial theory [6], (,) there are two types of Jacobi polynomial: Pn (r) and Gn (, , r). The relationship between them is as follows: (,)
Pn
(r) =
is different from FIE in intrinsic. The JFM can provide nonoise image features and reconstruct the no-noise original image.
(2n + + + 1) n! (n + + + 1) r +1 × Gn + + 1, + 1, . 2
(1 )
The Jacobi polynomial family and Legendre polynomial family are related by 5. Conclusion We proposed the JFM, which are orthogonal and distorted invariants for translation, rotation, scale and intensity variation passing through the normalization processing. We have calculated six types of JFM with different parameters of the kernel function and reconstructed the original images. Because of the orthogonal property of the moments, the original image can be reconstructed well. The theory analysis and experiment investigation have verified that the JFM can describe the image perfectly, having a strong performance resist noise. The Jacobi polynomial has two variable parameters, and being the radial function of the orthogonal function system, the variation of parameter values will form a different kernel in the moments integral, and therefore form different types of JFM. In the Appendix we will demonstrate that the JFM is a generic orthogonal moment. Almost all orthogonal moments that consist of radial polynomial and angular Fourier exponential factor can be derived from the JFM in terms of different parameter values, and those orthogonal moments possess the same property of describing image and noise sensibility. The common formulation of the moments will be a benefit for performance analysis of the orthogonal moments and for searching a prime orthogonal moment.
Pn (r) = Pn(0,0) (r),
(2 ) (,)
where Pn (r) is Legendre polynomial of order n, Pn (r) is the Jacobi polynomial of order n with two real variable parameters , . Compare (1 ) and (2 ) and Let =0, =0; the relationship between Legendre polynomials and Jacobi polynomial can be obtained: r +1
(2n + 1) (0,0) Pn (r) = Pn (r) = Gn 1, 1, , (3 ) n! (n + 1) 2 where (.) is the Gamma function. The first few terms of the Legendre polynomial are Pn (r) =
[n/2] 1 2n − 2m n−2m n n r (−1) , m n 2n m=0
P1 (r) = r, P2 (r) = P3 (r) =
5 3 2r
−
P5 (r) =
63 5 8 r
3 2
−
3 2 2r
r,
70 3 8 r
− 21 ,
P4 (r) = +
15 8
35 4 8 r
r, . . . .
−
30 2 8 r
+ 38 ,
(4 )
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
The first few terms of the deformed Jacobi polynomial are
(2n + 1) r +1 1 Gn 1, 1, = n! (n + 1) 2 n! n
(2n + 1 − m) r + 1 n−m m n × (−1) , m (n + 1 − m) 2
r +1 1, 1, 2
G2
= r,
r +1 G3 1, 1, = 2 r +1 G4 1, 1, = 2 r +1 G5 1, 1, = 2
5 3 2r
r +1 1, 1, 2
(5 )
=
3 2 2r
−
The first few terms of deformed Jacobi polynomial are 22n Gn (2, 3/2, (r + 1)/2) = 22n
m=0
G1
1253
×
n
(−1)
m=0
G1 (r) = 2r,
1 2,
n
n m
(n + 3/2)
(2n + 2)
(2n + 2 − m) n−m r ,
(n + 3/2 − m)
(9 )
G2 (r) = 4r 2 − 1,
G3 (r) = 8r 3 − 4r,
G4 (r) = 16r 4 − 12r 2 + 1,
G5 (r) = 32r 5 − 32r 3 + 6r, . . . .
− 23 ,
35 4 8 r
−
30 2 8 r
+ 38 ,
63 5 8 r
−
70 3 8 r
+
15 8
The Chebyshev polynomials in formula (8 ) are identical to the Jacobi polynomial in formula (9 ). The Chebyshev polynomials are special cases of the Jacobi polynomial. They both are related by formula (7 ). r, . . . . A.3. Orthogonal Fourier–Mellin Moments
The expression of each order of the Legendre polynomial in formula (4 ) is equal to the expression of same order of the Jacobi polynomial in formula (5 ). The Legendre polynomial is a special situation of the Jacobi polynomial, and both are connected by formula (3 ).
It can be seen that when =2, =2, the radial function of the JFM shown in formula (11) of the normal text is identical to the radial polynomials of the Orthogonal Fourier–Mellin Moments [9]. The Orthogonal Fourier–Mellin Moments is the special situation of the JFM.
A.2. Chebyshev–Fourier Moments
A.4. Zernike moments and pseudo-Zernike moments
According to the orthogonal polynomial theory [6] the Jacobi polynomial family and the Chebyshev polynomial family are related by
In Appendix B of reference literature [9], the relationship of Jacobi polynomial with Zernike polynomial and pseudoZernike polynomial were given as follows: m+s m |m| Rm+2s (r) = (−1)m+2s r s
Un (r) =
√ (n + 1)! 2 (n + 23 )
(1/2,1/2)
Pn
(r),
(6 )
where Un (r) is the second kind Chebyshev polynomial of (1/2,1/2) order n, Pn (r) is the Jacobi polynomial of order n 1 with = 2 , = 21 . Substituting formula (1 ) in Eq. (6 ) and using the property of the function, we can get √ (n + 1)(2n + 1)! Gn (2, 23 , (r + 2)/2) Un (r) = 2(n + 1)! (n + 3/2) = 22n Gn [2, 23 , (r + 1)/2].
|m|
(7 )
The first few terms of the Chebyshev polynomial are Un (r) =
n/2
(−1)n
m=0
U1 (r) = 2r,
(n − m)! (2r)n−2m , m!(n − 2m)!
U2 (r) = 4r 2 − 1,
U3 (r) = 8r 3 − 4r,
U4 (r) = 16r 4 − 12r 2 + 1,
U5 (r) = 32r 5 − 32r 3 + 6r, . . . .
(8 )
(10 )
Gs (m + 1, m + 1, r 2 ), 2m + s + 1 m |m| Pm+s (r) = (−1)s r s Gs (2m + 2, 2m + 2, r),
(11 ) |m|
where Rm+2s (r) is the Zernike polynomial, Pm+s (r) is the pseudo-Zernike polynomial. From formulas (10 ) and (11 ), it can be seen that the Zernike polynomial and pseudoZernike polynomial are special cases of the Jacobi polynomials.
References [1] D. Casasent, D. Psaltis, Position rotation and scale invariant optical correlation, Appl. Opt. 15 (1976) 1795–1799. [2] H.H. Arsennault, Y. Sheng, Properties of the circular harmonic expansion for rotation-invariant pattern recognition, Appl. Opt. 25 (18) (1986) 3225–3229. [3] Z.L. Ping, D.X. Xu, Optical method for completing simultaneously polar coordinate transfer and radial logarithmic transfer, Acta Optica Sin. 4 (5) (1984) 425–429.
1254
Z. Ping et al. / Pattern Recognition 40 (2007) 1245 – 1254
[4] Z.L. Ping, Y.L. Sheng, Fourier–Mellin descriptor and interpolated feature space trajectories for three-dimensional object recognition, Opt. Eng. 39 (2000) 1260–1266. [5] M.K. Hu, Visual Pattern recognition by moment invariants, IRE. Trans. Inf. Theory IT-8 (1962) 179–187. [6] M. Abramowitz, I.A. Stegun, Handbook of Functions With Formulas, Graphs and Mathematical Tables, Dover, New York, 1964, pp. 733–792. [7] M.R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70 (1980) 920–930. [8] M. Born, E. Wolf, Principle of Optics, sixth ed., Pergamon Press, Oxford, 1980, pp. 767–772. [9] Y. Sheng, L. Shen, Orthogonal Fourier–Mellin moments for invariant pattern recognition, J. Opt. Soc. Am. A 11 (6) (1994) 1748–1757. [10] C.H. Teh, R.T. Chin, On image analysis by the methods of moments, IEEE Trans. Pattern Anal. Mach. Intell. 10 (1988) 496–513. [11] Z.L. Ping, R. Wu, Y.L. Sheng, Image description with Chebyshev– Fourier moments, J. Opt. Soc. Am. A 19 (9) (2002)1748–1754. [12] H.P. Ren, Z.L. Ping, Wurigen, Y.L. Sheng, Multi-distorted invariant Image recognition with Radial-Harmonic-Fourier Moments, J. Opt. Soc. Am. A 20 (4) (2003) 631–637.
[13] R. Mukundan, S.H. Ong, P.A. Lee, Image analysis by Tchebichef moments, IEEE Trans. Image Process. 10 (9) (2001). [14] Y.S. Abu-Mostafa, D. Psaltis, Recognition aspect of moment invariant, IEEE Trans. Pattern Anal. Mach. Intell. PAM-6 (1984) 698–706. [15] P. Ziliang, E. Dalsgaad, Synthetic circular harmonic phase-only filter for shifting, rotation and scaling invariant correlation, Appl. Opt. 34 (1995) 7527–7531. [16] P. Ziliang, S. Li, J. Wang, E. Dalsgaard, Circular harmonic minimum average energy filter for the multi-distorted invariant correlation, Chin. J. Laser B7 (4) (1998) 347–352. [17] L.A. Zadeh, Fuzzy set, Inf. Control 8 (1965) 338–353. [18] F. Russo, Fuzzy processing of image data using FIRE filter, J. Intell. Fuzzy Syst. 15 (4) (1997) 361–366. [19] A. Taguchi, H. Takashima, Y. Murata, Fuzzy filter for image smoothing, in: Proceedings of the SPIE Conference on Nonlinear Image Processing, V. San. Jose, CA, 7–9 February 1994, pp. 332–339. [20] Y. Choi, R. Krishnapuram, A robust approach to image enhancement based on fuzzy logic, IEEE Trans. Image Process. 6 (1997) 808–825.
About the Author—ZILIANG PING, received a BS degree in physics from Nankai University of China in 1968 and an MS degree in optics from The Post and Communication University of Beijing of China in 1983. He is now a professor at the Inner Mongolia Normal University of China. He has published about 50 journal papers and conference papers. He was awarded a special dedication allowance by the Chinese government in 1993. His research interests involve optical processing, digital image processing and pattern recognition. About the Author—HAIPING REN received her PhD degree in medical sciences in 2003 from the Peking Union Medical College, Beijing, China. Then she joined the Department of Medical Devices, National Institute for the Control of Pharmaceutical and Biological Products, State Food and Drug Administration, China, as a Researcher and, since 2006, she has been the Senior Engineer on Medical Devices. Her research interests are concerned with the application of computational sciences to medical fields, such as medical image processing, pattern recognition, quality control of medical devices. Her research has been supported by both National and Beijing Natural Science Foundation. More than 20 papers have been published in various journals and conference proceedings. About the Author—JIAN ZOU received his Bachelor’s degree in 1986 from Nanjin Pharmaceutical University. In 1987, he joined the National Institute for the Control of Pharmaceutical and Biological Products, State Food and Drug Administration, China, as a Pharmacist. Since 2006, he has been Chief Pharmacist. Since 2002, he has been the director of Department of Medical Devices. His research focuses are quality control and supervision of medical devices. More than 10 papers have been published in various journals and conference proceedings. About the Author—YUNLONG SHENG received his BS degree from the University of Sciences and Technology of china in 1964 and his MS, Dr, Doctor d’Etat degrees in physics from the University de Franche-Comte, Besancon, France, in 1980, 1982 and 1986, respectively. Since 1985 he has been with Centre d’Optique, Photonique et Laser, University Laval, and is now a professor. Dr. Sheng has authored and coauthored more than 70 refereed journal papers, book chapters and conference papers. His research interests involve optics, optical neural networks, and optical interconnects. Dr. Sheng is a fellow of SPIE and a member of OSA and the International Neural Network Society. He also serves as a consultant for industrial companies in the United States and Canada. About the Author—B. WURIGEN received his BS and MS degrees from Beijing University in 2001 and 2004, respectively. Currently he is a doctor degree student in New York University.