- Email: [email protected]
Research of object tracking based on soft feature
G Model IJLEO-52971; No. of Pages 4
ARTICLE IN PRESS Optik xxx (2013) xxx–xxx
Contents lists available at SciVerse ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Research of object tracking based on soft feature Wen-Tao Jiang ∗ , Wan-Jun Liu, Heng Yuan Centre for Image and Visual Information Calculating Research, Liaoning Technical University, Huludao 125105, China
a r t i c l e
i n f o
Article history: Received 11 August 2012 Accepted 2 January 2013 Keywords: Object tracking Soft feature Space spectrum Edge spectrum
a b s t r a c t To solve the problem that traditional object tracking method had disadvantage of too much time consumed by data process and object lost under complex background, a novel approach to object tracking based on the space spectrum and edge spectrum is proposed and verified experimentally. Analyzing precursor features and study the unity and complementary of space spectrum and edge spectrum, research the extraction technology of moving object based on space spectrum and edge spectrum in order to track and forecast information of the moving object in real time. Experiments show that the proposed approach overcomes the problems of data dependence and object occlusion and achieves good tracking results. © 2013 Elsevier GmbH. All rights reserved.
1. Introduction Moving object tracking is one of the hot topics in the field of computer vision. It is a technology used to object identification, object location information extraction and analysis, object motion tracking from the image signal in real time. It has a very wide range of applications in military, industrial, security, intelligent transportation, medical and other fields; the development prospects are bright. Object motion randomness and complexity exist in the real environment, so its theoretical and applied research has some challenging. At present, there are still many problems to be resolved [1,2]. Motion-based tracking method was brought forward earlier and used widely. Now it has become an important trend of computer vision field. For example, Huttenlocher used the definition and computation methods of Hausdorff distance to implement decorrelation of sequence images [3,4]. On the basis of Huttenlocher, a bimodal distribution of background model was established by Haritaoglu to realize object detection, and object appearance model was built and achieved better results [5]. In general, method based on spatial motion is relatively simple in principle and easy to implement, so it has been extensively applied. But with the same tracking effect, the tracking error of this method is lower than others. Another class of approaches is the density estimation of object domain based on model, it is an unsupervised clustering method which was addressed by Fukunaga and Hostetler in 1975. Each point can “drift” to the local maximum value point of density function in this method. Comaniciu is the first person who introduced
∗ Corresponding author. Tel.: +86 0429 531 0858; fax: +86 0429 531 0858. E-mail address: [email protected] (W.-T. Jiang).
the mean shift algorithm into object tracking. He put Bhattacharyya factor as a similarity measurement criterion of candidate objects in object model fields, and ideal tracking result was obtained [6,7]. The main drawback of Gaussian Model is that the covariance matrix and eigenvectors associated to original data must be calculated by the transform basis function, and these calculations are usually difficult to achieve. In object tracking, problems are found such as too much time consuming in data processing and track loss of complex background. To elevate these problems, the soft feature theory of moving object and the method of feature extraction were proposed in this work, concepts and definitions are given, and object tracking process based on soft feature is described. Finally, experiments are showed to validate the performance of the algorithm proposed in this paper, at the same time comparative analysis with the correlation algorithms is also given.
2. Soft feature description The first concern is tailing problem of object motion in space. Assuming that the moving object is expressed by rectangle template, denoted as M, the lengths of the template are 2a and 2b. In three-dimensional coordinate system (x,y,z), the tailing trajectory of rectangle template centroid is calibrated at time t the timedomain function of centroid trajectory ε is defined as ε = (x,y,z,t), and also at time t the time-domain function l = (x, y, z, t) represents the edge centroid trajectory l. The rectangle template is described by Mx,y,z,t , after a moving time of t, the template is Mx,y,z,t+t . Centroid during time t moves along the function ε from template Mx,y,z,t+t to Mx,y,z,t+t . After the moving, the left vertical lines of two templates are overlap. The relevant definitions and concepts are given as follows. The model diagram is shown in Fig. 1.
0030-4026/$ – see front matter © 2013 Elsevier GmbH. All rights reserved. http://dx.doi.org/10.1016/j.ijleo.2013.01.042
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model
ARTICLE IN PRESS
IJLEO-52971; No. of Pages 4
W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx
2
Properties: Soft feature in space spectrum is master frequency, and in edge spectrum it is auxiliary frequency. In the extraction process, master frequency will be given a higher weight, while auxiliary frequency a lower and it is used to revise and update master frequency. ៝ Definition 7. During the target moving, spectrum vector ¯ corresponding to centers of mass in the rectangle template produces precedent impact strength, the dynamically and flexibly changing component has a projection effect of the impact strength in spatial axis, and it is called dependence.
Fig. 1. Model chart.
Definition 1. In spatial domain, the vector between two centers ៝ the modulus of of mass is defined as spatial vector, denoted as I, ៝ spatial vector is |I|.
According to different dependences, the projection component which has a stronger effect is defined as full-dependent. On the contrary, the weaker is partial-dependent.
Definition 2. In frequency domain , the vector between two cen៝ ters of mass is defined as frequency vector, denoted as , ¯ and the ៝ modulus of it is ||. ¯
Definition 8. Full-dependent effect of the third-order differential of precedent impact strength plus partial-dependent effect of the third-order differential, the sum multiply spatial vector, we get constraint model of soft feature, and it is
៝ ៝ Definition 3. ¯ is a small vector taken from , ¯ the variation dur ៝ ing t o of ¯ is defined as precedent impact strength of moving
I៝ ∂3 ω ∂3 ω ∂3 ω ∂3 ω = + + 2 + 3 2 ៝ ∂x ∂x ∂y ∂x ∂y ∂y3 ¯
៝ target and it is assumed as ω, where ω = ¯ /t .
When x = ±a and y = ±b, the rectangle template presents as linear in video, precedent impact strength at this time is ω, the dependence effect of ω to axis is 0.
→0
Definition 4. Centroid motion of the center of gravity formed the periodic tailing trajectory spectrum, it is defined as space spectrum, denoted by . By Fourier transform time-domain orbit ε of centroid obtains frequency domain spectrum, it can be used to represent :
2a
2b
(u, v, r) = 0
where j = r ∈ (0, ||]. ¯
√
0
៝ | | ¯
៝
¯ dx dy dz ε(x, y, z)e−2 j(ux/2a+vy/2b+rz/||)
0
(1)
−1, and u,v,r are spectrum vectors, u ∈ (0,2a], v ∈ (0,2b],
Definition 5. Centroid motion of each edge formed the periodic (uniform or non-uniform) tailing trajectory spectrum, it is defined as edge spectrum, denoted by . By Fourier transform time-domain orbit l of edge centroid obtains frequency domain spectrum, it can be used to represent :
i
2a
2b
(u, v, r) = 0
៝ | | ¯
៝
¯ dx dy dz i (x, y, z)e−2 j(ux/2a+vy/2b+rz/||)
0
0
(2)
where √ i represents the number of edge center of mass, i ∈ [0,4ab], j = −1, and u,v,r are spectrum vectors, u ∈ (0,2a], v ∈ (0,2b], r ∈ 0, || ¯ .
(4)
3. Algorithm description (1) Extracting soft feature. The number of neighborhood pixels which have the same change with each edge soft feature pixel is N, the pixel group is labeled as soft feature Tit , where i is the serial number of soft feature, l ≤ i ≤ N, t is the moment. The simulation result of flexibly changing spectrum is shown in Fig. 2. (2) Object tracking. (a) Locates and tracks the moving object in videos according to soft feature, and moving status is predicted by precedent impact strength ω. (b) Pre-locates the moving object in neighborhood space before a range of time t. (3) Revising and updating the soft feature. According to the variation [ω ] of the precedent impact strength of moving object, if the precedent impact strength ω reduces to |ω|/2 and the deviation angle to /2, the direction angle of ω should be increased to so the deviation direction of ω is revised, and the soft feature is updated. (4) Predicting soft feature. Predicts the precedent soft feature of Tit (l ≤ i ≤ N) on the basis of definition 8. If t = / 0, go to step 2, else to step 5.
Definition 6. The frequency which changes continuously and flexibly in space spectrum and edge spectrum is defined as soft feature, denoted by T. The value range of T is the differentiable range of space spectrum (u,v,r) and edge spectrum i (u,v,r):
⎡ ⎢ ⎣
(u1,1 , v1,1 , r1,1 )
···
(um,1 , vm,1 , rm,1 )
.. .
..
.. .
(u1,n , v1,n , r1,n )
···
T =⎢
⎡
×∪
q ⎢ ⎢ ⎣ i=1
.
⎤ ⎥ ⎥ ⎦
(um,n , vm,n , rm,n )
i ) (ui1,1 , vi1,1 , r1,1
···
i ) (uiq,1 , viq,1 , rq,1
.. .
..
.. .
i ) (ui1,p , vi1,p , r1,p
···
.
⎤ ⎥ ⎥ , i ∈ [0, 4ab] ⎦
i ) (uiq,p , viq,p , rq,p
(3)
Fig. 2. Edge spectrum forecast (Edge spectrum, ES).
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model IJLEO-52971; No. of Pages 4
ARTICLE IN PRESS W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx Table 1 The performance indicators of monitoring and tracking algorithm.
Input: Video/image
Soft feature detection
Feature fusion figure 4
Object tracking
3
Forecast feature
Object precedent prediction
Feature revising and repairing
Output: Tracking box Fig. 3. Tracking process of soft feature.
Eliminated feature Soft feature Forecast feature Transitional fusion feature Eliminated fusion feature
Fig. 4. Fusion process of soft feature.
(5) Clear the cache, and loop to the second step of step 1. Tracking process of soft feature is shown in Fig. 3. Fusion process of soft feature is shown in Fig. 4. 4. Experiment and analysis In this paper, by C++ programming language, object tracking algorithm centering on the theory of soft feature is implemented on computer P43.0/1G. The test data in the experiment comes from IBM multi-object occlusion database [8] which have the same applied conditions with the algorithm proposed in this paper. At the same time, performance evaluation index of tracking and
Name of index
Calculation formula
TDR (Tracker detection rate) PP (Positive prediction) FNR (False negative rate) FAR (False alarm rate) ACC (Accuracy) SPE (Specificity) FPR (False positive rate) NP (Negative prediction)
TDR = TP/(TP + FN) PP = TP/(TP + FP) FNR = FN/(TP + FN) FAR = FP/(TP + FP) ACC = (TN + TP)/NTF SPE = TN/(TN + FP) FPR = FP/(TN + FP) NP = TN/(TN + FN)
surveillance performance assessment working group [9] is used to evaluate the performance of the algorithm. In order to evaluating the capability in dealing with precedent predication of moving object under occlusion quantitatively, a new evaluation index called Forecast Error Rate (FER) is introduced, it is used to describe the probability of error predication which is before and after the occlusion of object. Forecast Error Rate is ratio of the number of error predication before and after the occlusion to the number of splitting occlusion. A large number of tracing tests of sequence images is realized, and in these sequence images, the relative positions of multi-objective variety (including approaching, edge linking, occlusion, edge separating and departing) continuously. Each frame of the image is color bitmap. Fig. 5 shows the tracing effect of three moving persons. In this figure, there are some illustrations with representative results of continuous video tests. With different shooting angles, feature occlusion is produced when three objects are intermeshing, so there are more foreground pixels of three objects in the test (NO.187, NO.295). As the three moving objects merge a whole object visually, when edge separating of objects appears, High-dimensional Statistical Measure (HSM) and Probabilistic Fusion (PF) tracking algorithm of object occlusion proposed in [10,11] lose the objects. With the algorithm of precedent predication tracking based on soft feature put forward in this paper, discrete convolution effect of precedent impact strength is considered in the process of tracking, so locations of the moving objects are tracked and predicted accurately in the case of object occlusion (NO.179, NO.306). In the course of comprehensive assessment, 8 evaluation indexes of PETS and FER are adopted to evaluate the FOT tracking and predicting method, 8 evaluation indexes of PETS are shown in Table 1 (TP: True positive; TN: True negative; FP: False positive; FN: False negative). Fifteen video groups of precedent predication tracking method of IBM multi-object occlusion database are show in Fig. 6, it is the test result of 106 032 frames, and each experiment was repeated 20 times. Experimental data is shown in Table 2. To compare the tracking qualities of the performance contrast of
Fig. 5. Experiment results of three object tracking through similar background.
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model
ARTICLE IN PRESS
IJLEO-52971; No. of Pages 4
W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx
4
5. Conclusions
Fig. 6. Evaluation results of IBM database through disturbed background. Table 2 Experimental data of the proposed technique. First group
Second group
Third group
Test sample (video group) Object movement speed (pixel/ms) Former hasten impact size (pixel) Former hasten impact angle (◦ ) Former hasten impact range (pixel) Average sports curvature (1/tan) Error adjustment (◦ ) ( − ) Accuracy for the first time (%) Repeat prediction error rates (%) Average speed of features attenuation Average speed of feature updated
5 0.641 39 0.027 1.053 0.337 0.009 99.861 0.163 27 26
10 1.173 75 0.011 0.825 0.463 0.006 99.801 0.091 34 37
15 2.131 116 0.003 0.348 0.653 0.001 99.738 0.013 39 45
Absolute value of tracking error 100%
Parameter
0.5
HSM PF SF
0.4 0.3 0.2 0.1 0
3 6 9 12 Similar background interference transform video group
15
Real-time tracking error frame
Fig. 7. Performance contrast of similar background interference transform (absolute value of tracking error).
4 3 2
HSM PF SF
1 0 0
3 6 9 12 Similar background interference transform video group
15
Fig. 8. Performance contrast of similar background interference transform (realtime tracking error).
We have proposed and experimentally demonstrated a novel soft feature theory and a feature extraction method of object tracking. Effective and stable predication of moving objects neighborhood precedent was made by extracting precedent impact strength of moving object in real-time. Problems of object occlusion and loss of complicated background were solved at a certain range. In order to implement the real-time performance of this algorithm, the video processing of corresponding moving objects was omitted during the predication, and the dependence on high-frequency data was overcome. Extending the range of precedent predication is crucial to tracking performance, so further study should be highrange extending of forecast period. In future work, this issue will be researched and it will enhance the precedent predication performance of object tracking. Acknowledgments This work was supported by the National Natural Science Foundation of the Republic of China under Contract Nos. 61172144 and 40901222 and National Innovation Foundation of the Republic of China under Contract No. 20800118. References [1] M. Rezaeian, B.N. Vo, Error bounds for joint detection and estimation of a single object with random finite set observation, IEEE Trans. Signal Process. 58 (3) (2010) 1493–1506. [2] D. Clark, B. Ristic, B.N. Vo, B.T. Vo, Bayesian multi-object filtering with amplitude feature likelihood for unknown object SNR, IEEE Trans. Signal Process. 58 (1) (2010) 26–37. [3] D.P. Huttenlocher, G.A. Klanderman, W.J. Rucklidge, Comparing images using the Hausdorff distance, IEEE Trans. Pattern Anal. Mach. Intel. 15 (9) (1993) 850–863. [4] D.P. Huttenlocher, J.J. Noh, W.J. Rucklidge, Tracking nonrigid objects in complex scenes, in: The 4th International Conference on Computer Vision, Berlin, Germany, 1993. [5] I. Haritaoglu, D. Harwood, Real-time surveillance of people and their activities, IEEE Trans. Pattern Anal. Mach. Intel. 22 (8) (2008) 809–830. [6] D. Comaniciu, V. Ramesh, P. Meer, Real-time tracking of non-rigid objects using mean shift, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, South Carolina, USA, 2000, pp. 142–149. [7] D. Comaniciu, V. Ramesh, P. Meer, Kernel-based object tracking, IEEE Trans. Pattern Anal. Mach. Intel. 25 (5) (2007) 564–577. [8] L.M. Brown, A.W. Senior, Y.L. Tian, J. Connell, A. Hampapur, Performance evaluation of surveillance systems under varying conditions, available: http://www.research.ibm.com/peoplevision/performanceevaluation.html (Online) (20.04.11). [9] H. Grabner, P.M. Roth, H. Bischof, Is pedestrian detection really a hard task, in: Proceedings of the 10th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Rio de Janeiro, 2007, pp. 1–8. [10] S. Boltz, E. Debreuve, M. Barlaud, High-dimensional statistical measure for region-of-interest tracking, IEEE Trans. Image Process. 18 (6) (2009) 1266–1283. [11] B.H. Han, L.S. Davis, Probabilistic fusion-based parameter estimation for visual tracking, Comput. Vision Image Und. 113 (2009) 435–445.
similar background interference transform with soft feature (SF) are shown in Figs. 7 and 8. The results demonstrated that tracking performance in this method is higher.
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
ARTICLE IN PRESS Optik xxx (2013) xxx–xxx
Contents lists available at SciVerse ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Research of object tracking based on soft feature Wen-Tao Jiang ∗ , Wan-Jun Liu, Heng Yuan Centre for Image and Visual Information Calculating Research, Liaoning Technical University, Huludao 125105, China
a r t i c l e
i n f o
Article history: Received 11 August 2012 Accepted 2 January 2013 Keywords: Object tracking Soft feature Space spectrum Edge spectrum
a b s t r a c t To solve the problem that traditional object tracking method had disadvantage of too much time consumed by data process and object lost under complex background, a novel approach to object tracking based on the space spectrum and edge spectrum is proposed and verified experimentally. Analyzing precursor features and study the unity and complementary of space spectrum and edge spectrum, research the extraction technology of moving object based on space spectrum and edge spectrum in order to track and forecast information of the moving object in real time. Experiments show that the proposed approach overcomes the problems of data dependence and object occlusion and achieves good tracking results. © 2013 Elsevier GmbH. All rights reserved.
1. Introduction Moving object tracking is one of the hot topics in the field of computer vision. It is a technology used to object identification, object location information extraction and analysis, object motion tracking from the image signal in real time. It has a very wide range of applications in military, industrial, security, intelligent transportation, medical and other fields; the development prospects are bright. Object motion randomness and complexity exist in the real environment, so its theoretical and applied research has some challenging. At present, there are still many problems to be resolved [1,2]. Motion-based tracking method was brought forward earlier and used widely. Now it has become an important trend of computer vision field. For example, Huttenlocher used the definition and computation methods of Hausdorff distance to implement decorrelation of sequence images [3,4]. On the basis of Huttenlocher, a bimodal distribution of background model was established by Haritaoglu to realize object detection, and object appearance model was built and achieved better results [5]. In general, method based on spatial motion is relatively simple in principle and easy to implement, so it has been extensively applied. But with the same tracking effect, the tracking error of this method is lower than others. Another class of approaches is the density estimation of object domain based on model, it is an unsupervised clustering method which was addressed by Fukunaga and Hostetler in 1975. Each point can “drift” to the local maximum value point of density function in this method. Comaniciu is the first person who introduced
∗ Corresponding author. Tel.: +86 0429 531 0858; fax: +86 0429 531 0858. E-mail address: [email protected] (W.-T. Jiang).
the mean shift algorithm into object tracking. He put Bhattacharyya factor as a similarity measurement criterion of candidate objects in object model fields, and ideal tracking result was obtained [6,7]. The main drawback of Gaussian Model is that the covariance matrix and eigenvectors associated to original data must be calculated by the transform basis function, and these calculations are usually difficult to achieve. In object tracking, problems are found such as too much time consuming in data processing and track loss of complex background. To elevate these problems, the soft feature theory of moving object and the method of feature extraction were proposed in this work, concepts and definitions are given, and object tracking process based on soft feature is described. Finally, experiments are showed to validate the performance of the algorithm proposed in this paper, at the same time comparative analysis with the correlation algorithms is also given.
2. Soft feature description The first concern is tailing problem of object motion in space. Assuming that the moving object is expressed by rectangle template, denoted as M, the lengths of the template are 2a and 2b. In three-dimensional coordinate system (x,y,z), the tailing trajectory of rectangle template centroid is calibrated at time t the timedomain function of centroid trajectory ε is defined as ε = (x,y,z,t), and also at time t the time-domain function l = (x, y, z, t) represents the edge centroid trajectory l. The rectangle template is described by Mx,y,z,t , after a moving time of t, the template is Mx,y,z,t+t . Centroid during time t moves along the function ε from template Mx,y,z,t+t to Mx,y,z,t+t . After the moving, the left vertical lines of two templates are overlap. The relevant definitions and concepts are given as follows. The model diagram is shown in Fig. 1.
0030-4026/$ – see front matter © 2013 Elsevier GmbH. All rights reserved. http://dx.doi.org/10.1016/j.ijleo.2013.01.042
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model
ARTICLE IN PRESS
IJLEO-52971; No. of Pages 4
W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx
2
Properties: Soft feature in space spectrum is master frequency, and in edge spectrum it is auxiliary frequency. In the extraction process, master frequency will be given a higher weight, while auxiliary frequency a lower and it is used to revise and update master frequency. ៝ Definition 7. During the target moving, spectrum vector ¯ corresponding to centers of mass in the rectangle template produces precedent impact strength, the dynamically and flexibly changing component has a projection effect of the impact strength in spatial axis, and it is called dependence.
Fig. 1. Model chart.
Definition 1. In spatial domain, the vector between two centers ៝ the modulus of of mass is defined as spatial vector, denoted as I, ៝ spatial vector is |I|.
According to different dependences, the projection component which has a stronger effect is defined as full-dependent. On the contrary, the weaker is partial-dependent.
Definition 2. In frequency domain , the vector between two cen៝ ters of mass is defined as frequency vector, denoted as , ¯ and the ៝ modulus of it is ||. ¯
Definition 8. Full-dependent effect of the third-order differential of precedent impact strength plus partial-dependent effect of the third-order differential, the sum multiply spatial vector, we get constraint model of soft feature, and it is
៝ ៝ Definition 3. ¯ is a small vector taken from , ¯ the variation dur ៝ ing t o of ¯ is defined as precedent impact strength of moving
I៝ ∂3 ω ∂3 ω ∂3 ω ∂3 ω = + + 2 + 3 2 ៝ ∂x ∂x ∂y ∂x ∂y ∂y3 ¯
៝ target and it is assumed as ω, where ω = ¯ /t .
When x = ±a and y = ±b, the rectangle template presents as linear in video, precedent impact strength at this time is ω, the dependence effect of ω to axis is 0.
→0
Definition 4. Centroid motion of the center of gravity formed the periodic tailing trajectory spectrum, it is defined as space spectrum, denoted by . By Fourier transform time-domain orbit ε of centroid obtains frequency domain spectrum, it can be used to represent :
2a
2b
(u, v, r) = 0
where j = r ∈ (0, ||]. ¯
√
0
៝ | | ¯
៝
¯ dx dy dz ε(x, y, z)e−2 j(ux/2a+vy/2b+rz/||)
0
(1)
−1, and u,v,r are spectrum vectors, u ∈ (0,2a], v ∈ (0,2b],
Definition 5. Centroid motion of each edge formed the periodic (uniform or non-uniform) tailing trajectory spectrum, it is defined as edge spectrum, denoted by . By Fourier transform time-domain orbit l of edge centroid obtains frequency domain spectrum, it can be used to represent :
i
2a
2b
(u, v, r) = 0
៝ | | ¯
៝
¯ dx dy dz i (x, y, z)e−2 j(ux/2a+vy/2b+rz/||)
0
0
(2)
where √ i represents the number of edge center of mass, i ∈ [0,4ab], j = −1, and u,v,r are spectrum vectors, u ∈ (0,2a], v ∈ (0,2b], r ∈ 0, || ¯ .
(4)
3. Algorithm description (1) Extracting soft feature. The number of neighborhood pixels which have the same change with each edge soft feature pixel is N, the pixel group is labeled as soft feature Tit , where i is the serial number of soft feature, l ≤ i ≤ N, t is the moment. The simulation result of flexibly changing spectrum is shown in Fig. 2. (2) Object tracking. (a) Locates and tracks the moving object in videos according to soft feature, and moving status is predicted by precedent impact strength ω. (b) Pre-locates the moving object in neighborhood space before a range of time t. (3) Revising and updating the soft feature. According to the variation [ω ] of the precedent impact strength of moving object, if the precedent impact strength ω reduces to |ω|/2 and the deviation angle to /2, the direction angle of ω should be increased to so the deviation direction of ω is revised, and the soft feature is updated. (4) Predicting soft feature. Predicts the precedent soft feature of Tit (l ≤ i ≤ N) on the basis of definition 8. If t = / 0, go to step 2, else to step 5.
Definition 6. The frequency which changes continuously and flexibly in space spectrum and edge spectrum is defined as soft feature, denoted by T. The value range of T is the differentiable range of space spectrum (u,v,r) and edge spectrum i (u,v,r):
⎡ ⎢ ⎣
(u1,1 , v1,1 , r1,1 )
···
(um,1 , vm,1 , rm,1 )
.. .
..
.. .
(u1,n , v1,n , r1,n )
···
T =⎢
⎡
×∪
q ⎢ ⎢ ⎣ i=1
.
⎤ ⎥ ⎥ ⎦
(um,n , vm,n , rm,n )
i ) (ui1,1 , vi1,1 , r1,1
···
i ) (uiq,1 , viq,1 , rq,1
.. .
..
.. .
i ) (ui1,p , vi1,p , r1,p
···
.
⎤ ⎥ ⎥ , i ∈ [0, 4ab] ⎦
i ) (uiq,p , viq,p , rq,p
(3)
Fig. 2. Edge spectrum forecast (Edge spectrum, ES).
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model IJLEO-52971; No. of Pages 4
ARTICLE IN PRESS W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx Table 1 The performance indicators of monitoring and tracking algorithm.
Input: Video/image
Soft feature detection
Feature fusion figure 4
Object tracking
3
Forecast feature
Object precedent prediction
Feature revising and repairing
Output: Tracking box Fig. 3. Tracking process of soft feature.
Eliminated feature Soft feature Forecast feature Transitional fusion feature Eliminated fusion feature
Fig. 4. Fusion process of soft feature.
(5) Clear the cache, and loop to the second step of step 1. Tracking process of soft feature is shown in Fig. 3. Fusion process of soft feature is shown in Fig. 4. 4. Experiment and analysis In this paper, by C++ programming language, object tracking algorithm centering on the theory of soft feature is implemented on computer P43.0/1G. The test data in the experiment comes from IBM multi-object occlusion database [8] which have the same applied conditions with the algorithm proposed in this paper. At the same time, performance evaluation index of tracking and
Name of index
Calculation formula
TDR (Tracker detection rate) PP (Positive prediction) FNR (False negative rate) FAR (False alarm rate) ACC (Accuracy) SPE (Specificity) FPR (False positive rate) NP (Negative prediction)
TDR = TP/(TP + FN) PP = TP/(TP + FP) FNR = FN/(TP + FN) FAR = FP/(TP + FP) ACC = (TN + TP)/NTF SPE = TN/(TN + FP) FPR = FP/(TN + FP) NP = TN/(TN + FN)
surveillance performance assessment working group [9] is used to evaluate the performance of the algorithm. In order to evaluating the capability in dealing with precedent predication of moving object under occlusion quantitatively, a new evaluation index called Forecast Error Rate (FER) is introduced, it is used to describe the probability of error predication which is before and after the occlusion of object. Forecast Error Rate is ratio of the number of error predication before and after the occlusion to the number of splitting occlusion. A large number of tracing tests of sequence images is realized, and in these sequence images, the relative positions of multi-objective variety (including approaching, edge linking, occlusion, edge separating and departing) continuously. Each frame of the image is color bitmap. Fig. 5 shows the tracing effect of three moving persons. In this figure, there are some illustrations with representative results of continuous video tests. With different shooting angles, feature occlusion is produced when three objects are intermeshing, so there are more foreground pixels of three objects in the test (NO.187, NO.295). As the three moving objects merge a whole object visually, when edge separating of objects appears, High-dimensional Statistical Measure (HSM) and Probabilistic Fusion (PF) tracking algorithm of object occlusion proposed in [10,11] lose the objects. With the algorithm of precedent predication tracking based on soft feature put forward in this paper, discrete convolution effect of precedent impact strength is considered in the process of tracking, so locations of the moving objects are tracked and predicted accurately in the case of object occlusion (NO.179, NO.306). In the course of comprehensive assessment, 8 evaluation indexes of PETS and FER are adopted to evaluate the FOT tracking and predicting method, 8 evaluation indexes of PETS are shown in Table 1 (TP: True positive; TN: True negative; FP: False positive; FN: False negative). Fifteen video groups of precedent predication tracking method of IBM multi-object occlusion database are show in Fig. 6, it is the test result of 106 032 frames, and each experiment was repeated 20 times. Experimental data is shown in Table 2. To compare the tracking qualities of the performance contrast of
Fig. 5. Experiment results of three object tracking through similar background.
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042
G Model
ARTICLE IN PRESS
IJLEO-52971; No. of Pages 4
W.-T. Jiang et al. / Optik xxx (2013) xxx–xxx
4
5. Conclusions
Fig. 6. Evaluation results of IBM database through disturbed background. Table 2 Experimental data of the proposed technique. First group
Second group
Third group
Test sample (video group) Object movement speed (pixel/ms) Former hasten impact size (pixel) Former hasten impact angle (◦ ) Former hasten impact range (pixel) Average sports curvature (1/tan) Error adjustment (◦ ) ( − ) Accuracy for the first time (%) Repeat prediction error rates (%) Average speed of features attenuation Average speed of feature updated
5 0.641 39 0.027 1.053 0.337 0.009 99.861 0.163 27 26
10 1.173 75 0.011 0.825 0.463 0.006 99.801 0.091 34 37
15 2.131 116 0.003 0.348 0.653 0.001 99.738 0.013 39 45
Absolute value of tracking error 100%
Parameter
0.5
HSM PF SF
0.4 0.3 0.2 0.1 0
3 6 9 12 Similar background interference transform video group
15
Real-time tracking error frame
Fig. 7. Performance contrast of similar background interference transform (absolute value of tracking error).
4 3 2
HSM PF SF
1 0 0
3 6 9 12 Similar background interference transform video group
15
Fig. 8. Performance contrast of similar background interference transform (realtime tracking error).
We have proposed and experimentally demonstrated a novel soft feature theory and a feature extraction method of object tracking. Effective and stable predication of moving objects neighborhood precedent was made by extracting precedent impact strength of moving object in real-time. Problems of object occlusion and loss of complicated background were solved at a certain range. In order to implement the real-time performance of this algorithm, the video processing of corresponding moving objects was omitted during the predication, and the dependence on high-frequency data was overcome. Extending the range of precedent predication is crucial to tracking performance, so further study should be highrange extending of forecast period. In future work, this issue will be researched and it will enhance the precedent predication performance of object tracking. Acknowledgments This work was supported by the National Natural Science Foundation of the Republic of China under Contract Nos. 61172144 and 40901222 and National Innovation Foundation of the Republic of China under Contract No. 20800118. References [1] M. Rezaeian, B.N. Vo, Error bounds for joint detection and estimation of a single object with random finite set observation, IEEE Trans. Signal Process. 58 (3) (2010) 1493–1506. [2] D. Clark, B. Ristic, B.N. Vo, B.T. Vo, Bayesian multi-object filtering with amplitude feature likelihood for unknown object SNR, IEEE Trans. Signal Process. 58 (1) (2010) 26–37. [3] D.P. Huttenlocher, G.A. Klanderman, W.J. Rucklidge, Comparing images using the Hausdorff distance, IEEE Trans. Pattern Anal. Mach. Intel. 15 (9) (1993) 850–863. [4] D.P. Huttenlocher, J.J. Noh, W.J. Rucklidge, Tracking nonrigid objects in complex scenes, in: The 4th International Conference on Computer Vision, Berlin, Germany, 1993. [5] I. Haritaoglu, D. Harwood, Real-time surveillance of people and their activities, IEEE Trans. Pattern Anal. Mach. Intel. 22 (8) (2008) 809–830. [6] D. Comaniciu, V. Ramesh, P. Meer, Real-time tracking of non-rigid objects using mean shift, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, South Carolina, USA, 2000, pp. 142–149. [7] D. Comaniciu, V. Ramesh, P. Meer, Kernel-based object tracking, IEEE Trans. Pattern Anal. Mach. Intel. 25 (5) (2007) 564–577. [8] L.M. Brown, A.W. Senior, Y.L. Tian, J. Connell, A. Hampapur, Performance evaluation of surveillance systems under varying conditions, available: http://www.research.ibm.com/peoplevision/performanceevaluation.html (Online) (20.04.11). [9] H. Grabner, P.M. Roth, H. Bischof, Is pedestrian detection really a hard task, in: Proceedings of the 10th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Rio de Janeiro, 2007, pp. 1–8. [10] S. Boltz, E. Debreuve, M. Barlaud, High-dimensional statistical measure for region-of-interest tracking, IEEE Trans. Image Process. 18 (6) (2009) 1266–1283. [11] B.H. Han, L.S. Davis, Probabilistic fusion-based parameter estimation for visual tracking, Comput. Vision Image Und. 113 (2009) 435–445.
similar background interference transform with soft feature (SF) are shown in Figs. 7 and 8. The results demonstrated that tracking performance in this method is higher.
Please cite this article in press as: W.-T. Jiang, et al., Research of object tracking based on soft feature, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2013.01.042