Optik 124 (2013) 5181–5183
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Shape-based foreign body recognition of train roof using invariant moments Kai Yang, Jinlong Li, Ping Lin ∗ , Yu Zhang, Xiaorong Gao Department of Photoelectric Engineering, Southwest Jiaotong University, Chengdu, Sichuan 610031, China
a r t i c l e
i n f o
Article history: Received 14 October 2012 Accepted 14 March 2013
Keywords: Multi-scale edge detection The cubic B-spline Invariant moment Foreign body recognition
a b s t r a c t Foreign body recognition of locomotive roof is an important security link. In this paper, a foreign body recognition method is presented based on shape feature matching. Roof images of the inherent devices (without foreign bodies) and those of the running train are processed respectively with the cubic B-spline wavelet edge detection to get multi-scale edge images and segment objects. And thus the invariant moments of each closed area in the edge images are extracted. Then, after calculating the similarity measure by Euclidean distance, we can judge whether the objects are foreign bodies. Test results show that this method can effectively recognize a foreign body in the size of 33 mm × 33 mm. © 2013 Elsevier GmbH. All rights reserved.
1. Introduction In recent years, in view of the mileage expansion of high-speed railway and the substantial increase in speed, the train device maintenance and preventive detection of varieties of potential faults and hazards are extremely important, which has aroused public concern widely. Researchers have done a series of studies on the factors, which caused the train accident and created significant economic and theoretical value [1–4]. Foreign bodies on the roof refer to a foreign substance or drop parts. Due to inertia, these foreign bodies may crash the pantograph or porcelain devices and thus cause damages. So we should recognition the foreign bodies on the roof correctly and take effective preventive measures to ensure trains’ normal running. Foreign body detection is widely used in the food test and medical diagnosis for larynx or digestive tract etc. [5–7], while such methods are almost impossible to apply directly to the train roof foreign body detection due to the complexity of the devices. This paper presents a foreign body recognition method based on shape feature matching, which can effectively classify objects with minute difference [8]. The method is often used in image retrieval and can do a good job even when the noise levels are very high [9]. The steps are: firstly, the roof images are processed with the cubic B-spline wavelet edge detection to get multi-scale edge images [10,11] and the existed objects in the images are segmented; secondly, with the invariance of shift, scale, and rotation of invariant moment, the shape features of the image is extracted [12]; finally, as the similarity measure, Euclidean distance is used to compare the objects to be identified in the roof image with those in the
stored reference library, so that the objects on the roof can be determined whether they are the foreign bodies, and at the same time, the foreign bodies are marked out. 2. Theory of wavelet multi-scale edge detection Edge detection is the first step to extract invariant moment features. The characteristic parameters of the wavelet multi-scale edge include direction, amplitude, location and scale, which are beneficial for effective denoising and removal of the pseudo-edges. The procession of the roof image f(u, v) with multi-scale edge detection is to look for the partial minimum of the wavelet transform. Let (u, v) be the two-dimensional smoothness function to meet
(u, v)ddv = 1,
0030-4026/$ – see front matter © 2013 Elsevier GmbH. All rights reserved. http://dx.doi.org/10.1016/j.ijleo.2013.03.100
(1)
R2
Let s (u, v) = (1/s2 )((u/s), (v/s)) and let’s defined two wavelets = ∂(u, v)/∂ and ϕ2 (u, v) = ∂(u, v)/∂v. So the two-dimensional wavelets transform of f(u, v) at scale s and position(, v) has two components:
ϕ1 (u, v)
W 1 f (s, u, v) =
R2 W 2 f (s, u, v) =
x−u 1 f (x, y) ϕ1 , s s
y−v dxdy = (f ∗ ϕs−1 )(u, v) s
x−u 1 , f (x, y) ϕ2 s s
y−v dxdy = (f ∗ ϕs−1 )(u, v) s
(2)
R2
Then we can get:
Wf (s, u, v) − s
∗ Corresponding author. Tel.: +86 15828572638. E-mail address:
[email protected] (P. Lin).
lim,v→∞ (u, v) = 0
(f
∗ϕ ¯ s1 )(u, v)
¯ s2 )(u, v) (f ∗ ϕ
⎛ ⎜
= s⎝
⎞
∂ (f ∗ ¯ s )(u, v) ∂u ⎟ ∂ (f ∗ ¯ s )(u, v) ∂v
¯ (u, v) = s∇ (f ∗ ) s
⎠ (3)
5182
K. Yang et al. / Optik 124 (2013) 5181–5183
¯ (u, v) is the gradient of the smoothness function, and where ∇ (f ∗ ) s ¯ (u, v)| is proportional to modulus of the wavelet transform |∇ (f ∗ ) s
of f(u, v), that is
Mf (s, u, v) =
|W 1 f (s, u, v)|2 + |W 2 f (s, u, v)|2
(4)
The angle between the gradient and the horizontal direction is
Af (s, u, v) = arctan
W 1 f (s, u, v) W 2 f (s, u, v)
(5)
So to calculate the modulus maximum of the smoothness func¯ (u, v) along the gradient direction is equivalent to tion (f ∗ ) s calculate the wavelet transform of the modulus maxima. Let Mf (s, u, v) change in the one-dimensional neighborhood (u, v) = (u1 , v1 ) + ∇ f (u1 , v1 ) along the gradient of the point (u1 , v1 ) to take the partial maximum when || is sufficiently small at the point and this ¯ (u, v). Generally, s = 2j , partial maximum is an edge point of (f ∗ ) s j ∈ Z, so we use dyadic wavelet transform for ensuring ϕ(ω) covering the entire frequency axis. In order to denoise and remove the pseudo-edges, we take the adaptive threshold to extract the edge of each scale. Then all the edge points under all the scales along the boundary direction are connected to detect the image edge. The smoothness function used in this paper is cubic B-spline, whose first order partial derivative is the wavelet function. The cubic B-spline can be described as Eq. (6):
⎧ (x + 2)s ⎪ ⎪ x ∈ [−2, −1) ⎪ ⎪ 6 ⎪ ⎪ ⎪ ⎪ xs 2 ⎪ ⎨ − = x2 + x ∈ [−1, 0) 2
ˇ3 (x) =
3
⎪ 2 ⎪ ⎪ − x2 + x ∈ [0, 1) ⎪ 2 3 ⎪ ⎪ ⎪ ⎪ s ⎪ ⎩ (2 − x) x ∈ [−1, 2) 6
6 exp (n + 1)
n
ˇ (x) ≈
6x2 − n+1
D(X, Y ) =
n i=1
(xi = yi )2
i = 1, 2, . . . n
(7)
3. Theory of the invariant moment feature extraction
4. Test results and analysis 4.1. Roof image acquisition In the testing, we used roof foreign body images and standard ones taken in workshop, among which the foreign bodies were artificially placed. The shooting device includes a linear camera and some fill flash installed on the beam in the center line of tracks. Keep the camera vertical or a certain depression angles (can be adjusted) and video the train roof to achieve roof panorama. Fig. 1 shows the sketch map, in which the camera is placed in the center of beam and flashes are on both sides. Besides, we should keep the camera field range covering the roof. Fig. 2 shows the real-time acquisition of the train roof images without foreign bodies.
As mentioned above, we use the cubic B-spline wavelet multi-edge detection method for detecting various objects and segmenting them from the background. Now let’s use a widely used shape feature depicting parameter – invariant moment to describe the regional features. To 2D function f(x,y), the p + q order moment is defined as
∞
xp uq f (x, y)dx dy
mpq =
p, q = 0, 1, 2 . . . . . .
(8)
−∞
The central moment is shown as
∞
∞
pq = −∞
¯ q f (x, y), dx dy (x − x¯ )p (y − y)
(9)
−∞
The normalized moment is defined as upq p+q pq = , = 2 u 00
(11)
This is called Euclidean distance between eigenvector X and Y. It matches successfully when Euclidean distance is the smallest, and if it failed, the object is seen as a foreign body.
As is well known, Gaussian function is the optimal basis function in both time and frequency domain because it can minimize the lower boundary of the uncertainty relation. Similarly, the Bspline function can minimize the loss of information, effectively smoothen the noise data, well balance the noise rejection and accurately orient edge. As Unser has proved that ˇn (x) and it’s Fourier Transform approximate to the Gaussian function if n → ∞ [13]. The relationship is given by
Hu proposed seven invariant moments group which could be derived from the second and third order moments, and the moments group had been proved that they were invariant with translation, rotation or size change [14]. Although describing complex shapes needs high order moment functions, only a few moments can accurately describe the target shape [15], so they can be used for describing the features of the region. After extraction of the shape features with invariant moments, the Euclidean distance is used for measuring the similarity between feature eigen vectors in feature matching. Setting two eigen vectors X = (x1 , x2 . . . xn ) and Y = (y1 , y2 . . .yn ), we get
(6)
xs
Fig. 1. Sketch map of the device.
(10) Fig. 2. Standard roof image.
K. Yang et al. / Optik 124 (2013) 5181–5183
5183
object images, in order to improve the luminance uniformity and strengthen the border to suppress false edges. We processed the partial foreign bodies of result images with color display, brightness display contrast adjustment, and zoom out, so that operators can judge the type of foreign bodies. Fig. 4 shows foreign body detection results with zoom out and brightness adjustment. 5. Conclusion
Fig. 3. Foreign bodies’ detection results: (a) and (b) live image with spanner on roof from industrial camera (c) and (d) spanner separated from background by invariant moment, Euclidean distance and background separation technology (e) and (f) mark out the spanner center with red rectangle.
This paper introduces a foreign body identification algorithm based on invariant moments of shape feature extraction. In this procession, the objects in the image are segmented with wavelet multi-edge detection, and then moment invariant features which are not sensitive to translation, scaling and rotation are extracted, and the similarity measure between the feature eigen vectors is determine by the Euclidean distance. And some experimental results are shown to prove that algorithm can effectively identify the foreign bodies of real-time image. However, this algorithm still needs to be improved: first, the accuracy of the algorithm to identify foreign bodies should be improved. Uneven illumination or high reflective images can be realized by improving the shooting conditions or pretreatment. Meanwhile, oil or rust stains may be mistakenly judged as foreign bodies on the roof, so algorithm should be improved in the ability of self-learning by classifying the foreign bodies. These problems would be resolved next, aiming at obtaining an autonomous non-destructive visual inspection tool for rail industry. References
Fig. 4. Foreign body display, (a) is the object image, and (b) is the display with zoom out and brightness adjustment.
4.2. Foreign body recognition In the experiment, the object images were processed with the above algorithm to separate the foreign bodies and marked them out with red square. Experimental results (as shown in Fig. 3) demonstrate that the results of the B-spline wavelet multi-edge detection are clear and complete, and the invariant moment-based matching results can orientate foreign bodies well. This method can reduce the impact of noise and large scale changes. Experimental results show that finer edges are detected together with some false edges when the scale is small, while some edges are missed when the scale is enlarged, but noise can be suppressed well. So for different type of train, different scale range should be set to extract edges accurately. Attention should be paid that additional light should be open, so that the method can correctly detect foreign bodies during night or under dark environment and thus the quality of the image can be improved. 4.3. Post-processing of display But in practical application, some images are not in uniform illumination. So we take image enhancement technology to process the
[1] M. Akkas, D. AY, N. Metin Aksu, M. Günalp, 10-year evaluation of train accidents, Ulus. Travma Acil Cerrahi Derg. 17 (5) (2011) 440–444. [2] M.T. Baysari, A.S. McIntosh, J.R. Wilson, Understanding the human factors contribution to railway accidents and incidents in Australia, Accid. Anal. Prev. 40 (5) (2008) 1750–1757. [3] G.J.M. Read, M.G. Lenne, S.A. Moss, Associations between task, training and social environmental factors and error types involved in rail incidents and accidents, Accid. Anal. Prev. 48 (2012) 416–422. [4] A.A. Holgersson, R.R. Forsberg, B.I. Saveman, Interior safety in trains neglected: a case study after the train crash in Kimstad, Lakartidningen 109 (1/2) (2012) 24–26. [5] G. Ginesu, D.D. Giusto, V. Margner, P. Meinlschmidt, Detection of foreign bodies in food by thermal image processing, IEEE Trans. Ind. Electron. 51 (2) (2004) 480–490. [6] L. Oesterhelweg, S.A. Bolliger, M.J. Thali, S. Ross, Virtopsy postmortem imaging of laryngeal foreign bodies, Arch. Pathol. Lab. Med. 133 (5) (2009) 806–810. [7] R. Shihada, M. Goldsher, S. Sbait, M. Luntz, Three-dimensional computed tomography for detection and management of ingested foreign bodies, ENT – Ear Nose Throat J. 88 (5) (2009) 910–911. [8] D. Shen, H.S.lp. Horace, Discriminative wavelet shape descriptors for recognition of 2-D patterns, Pattern Recognit. 32 (2) (1999) 151–165. [9] Arai Kohei, Visualization of 3D object shape complexity with wavelet descriptor and its application to image retrievals, J. Vis. 15 (2) (2012) 155–166. [10] Hong Jeong, C.I. Kim, Adaptive determination of filter scales for edge detection, IEEE Trans. PAMI 14 (1992) 579–585. [11] Lei Zhang, Paul Bao, A wavelet-based edge detection method by scale multiplication, IEEE Pattern Recognit. Lett. 3 (2002) 501–504. [12] Y. Li, Reforming the theory of invariant moments for pattern recognition, Pattern Recognit. 25 (7) (1992) 723–730. [13] M. Unser, I.A. Aldroub, M. Eden, On the asymptotic convergence of B-spline wavelets to Gabor function, IEEE Trans. Info. Theory 38 (2) (1992) 864–872. [14] M.K. Hu, Visual pattern recognition by moment invariants, IEEE Trans. Info. Theory 49 (2) (1962) 179–187. [15] M.R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70 (8) (1980) 920–930.