- Email: [email protected]
Fast and Optimal Visual Tracking based on Spectral Method
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2017) 000–000 Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 116 (2017) 571–578
2nd International Conference on Computer Science and Computational Intelligence 2017, ICCSCI 2nd International Conference on Computer Science andBali, Computational 2017, 13-14 October 2017, Indonesia Intelligence 2017, ICCSCI 2017, 13-14 October 2017, Bali, Indonesia
Fast and Optimal Visual Tracking based on Spectral Method Fast and Optimal Visual Tracking based on Spectral Method
a a
b b Alexander A S Gunawana* a*, Wisnu Jatmikob, Aniati Murni Arymurthyb Alexander A S Gunawan , Wisnu Jatmiko , Aniati Murni Arymurthy
Computer Science Department, School of Computer Science, Bina Nusantara University, Jl.K.H. Syahdan No.9, Jakarta, 11480 Indonesia Computer Science Department, bSchool Nusantara University,Depok, Jl.K.H. Syahdan No.9, Jakarta, 11480 Indonesia FacultyofofComputer ComputerScience, Science,Bina University of Indonesia, Indonesia b Faculty of Computer Science, University of Indonesia, Depok, Indonesia
Abstract Abstract Visual object tracking is the process of continuously localizing visual object in a video sequence. We would like to investigate the Visual tracking model-free is the process of continuously visual object a video like to investigate the problemobject of short-term tracking which the localizing main purpose is to trackinany objectsequence. just basedWe on would an annotation box of object. problem of short-term tracking the main purpose is toVisual track any objectBenchmark, just based on an annotation of object. Many factors affect themodel-free performance of thewhich tracking algorithm. In the Tracker there are eleven box challenges in Many affect thehas performance of the tracker trackingthat algorithm. In the VisualallTracker are eleven challenges in object factors tracking. There not been a single successfully handles of theseBenchmark, scenarios. Inthere addition, the tracker must be object tracking. has a single tracker that successfully handles all of these scenarios. In addition, the tracker must be fast enough to beThere useful innot realbeen applications. We propose a new tracking algorithm within the Bayesian framework. The proposed fast enoughis to be useful in applications. propose a new tracking algorithm the methods. Bayesian Therefore, framework.the The proposed algorithm constructed byreal solving optimallyWe particle filters (OPF) efficiently usingwithin spectral constructed algorithm is constructed by solving particleST filters efficiently using spectral methods. Therefore, thethe constructed tracker is called as Spectral Trackeroptimally (ST). Although can (OPF) efficiently compute object position, it cannot estimate scale and tracker called asTo Spectral Tracker Although can efficiently position, it cannot estimate theand scale rotationisdirectly. overcome this (ST). weakness, it isST proposed to use compute multiple object observation points simultaneously to and use rotation directly. overcomepoint this movement weakness, to it estimate is proposed userotation. multiple points to use information on theTo observation scaletoand In observation the experiments, thesimultaneously performance ofand ST tracker information on with the observation movement estimate scale The and rotation. In theresults experiments, the performance of ST tracker was compared 9 relevant point trackers based onto100 data sets. experimental on on tracker performance show that was compared with 9 relevant trackers based on 100 and datasuccess sets. The increasing performance especially in tracker precision rate.experimental results on on tracker performance show that increasing performance especially in tracker precision and success rate. © 2017 The Authors. Published by Elsevier B.V. © 2017 2017 The The Authors. Published Elsevier B.V. © Authors. Published by by B.V. committee of the 2nd International Conference on Computer Science and Peer-review under responsibility of Elsevier the scientific Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. Computational Intelligence 2017. Computational Intelligence 2017. Keywords: short-term model-free tracking, optimal particle filter, spectral method, bayesian framework Keywords: short-term model-free tracking, optimal particle filter, spectral method, bayesian framework
1. Introduction 1. Introduction Visual object tracking has many practical applications in automated surveillance system, robot vision system, Visual object tracking has many practical applications in automated surveillance system, robot vision system,
* *
Corresponding author. Tel.: +62-21-534-5830 E-mail address:author. [email protected] Corresponding Tel.: +62-21-534-5830 E-mail address: [email protected]
1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review underThe responsibility of theby scientific of the 2nd International Conference on Computer Science and 1877-0509 © 2017 Authors. Published Elsevier committee B.V.
Peer-review under responsibility Computational Intelligence 2017.of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. 1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. 10.1016/j.procs.2017.10.069
572 2
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
intelligent traffic system and fault detection system. Visual tracking is the process of continuously localizing an object target in a video sequence. When the target is successfully tracked in each video frame gives more information about the identity and the activity of the object target. In general, tracking problem is easier than detection, tracking algorithms should be faster than running an object detector on every frame. Therefore, the key requirement for many tracking applications is ability to track an object target in video sequences in real-time. Currently, many existing tracking algorithms have been introduced for certain tasks. However, many of them have weakness because of their main approaches that is (1) they make basic assumptions about object appearances 1 and the environment 2 (2) most visual tracking algorithms are computationally expensive 3. In this paper, we would like to overcome these weakness by considering its fundamental approach. We focus to solve short-term model-free tracking problem 4. The short-term tracking in here means the tracker does not perform re-detection after the object target is lost. Out of the target is considered as a failure. While model-free property means that the only example for supervised learning is provided by the bounding box in the first frame. In addition, the tracker output is indicated by a rotated bounding box. First, we consider human visual perception in order to understand how visual information is represented and processed in the brain. It is clear that geometry of object and visual perception are closely related, but very little is known about their relation beyond the retinotopic association 5. In visual perception studies, retinotopic refers to the visual process where object geometry are projected in the retina by similar process as appearance models in a digital image. However, a recent study on human vision shows that the representation in higher visual areas of the visual cortex occurs in a nonretinotopic way 5. It means that visual perception creates dynamic layers for each moving object in the scene. This representation suggests that the object appearance and their position dynamics are marginal independent. Based on the inspiration from the above study, a tracking algorithm is proposed by imitating the nonretinotopic process in human visual processing. The suitable approach that can govern the relationships between the object appearance model and the object dynamic model is probabilistic Bayesian framework. The advantage of the Bayesian approach is able to maintain a history of probability estimation from the sequence of incoming visual perceptions. This means the Bayesian approach is able to integrate knowledge of the target object dynamics with visual observations of object appearance. Second, we need to choose a numerical approach for implementing Bayesian framework. Particle filter is a practical tool to implement dynamic state estimation in Bayesian framework 6. The basic idea is to approximate the posterior distribution of transition dynamics by using a finite set of weighted samples or particles, when given a sequence of observation measurements. Moreover, particle Þlter can overcome non-linear and non-Gaussian distribution in transition model and observation model. Particle filter is a sequential Monte-Carlo technique and can be proved to produce the correct posterior distribution from finite set of samples as the sample size toward infinity 7. In practice, it is necessary first to design its important sampling. There is, of course, trade off between accuracy and computational load in the chosen design. There are two main types of particle filters based on the design of its important sampling: bootstrap and optimal particle filter. The easiest particle filter to implement is a bootstrap particle filter (BPF). In BPF, the transition distribution is choosen as important density. BPF has the characteristic that the weights only depend on the likelihood observation distribution. For BPF, the sampling process is very easy by using the transition distribution to predict the new particle and then followed by the process of weight determination by using likelihood distribution. BPF has been used by many researchers in visual object tracking and often called as Condensation (Conditional Density Propagation) 8. Note that in BPF algorithm, its important density does not take into account the latest measurements. So even if BPF is easy to implement, BPF has some weaknesses associated with robustness in the face of unexpected noise. This is mainly due to the fact that variance of the weights can not go down 9. As a result, the degeneration phenomenon in BPF may appear. One way to overcome the degeneration phenomenon is by using resampling process. However, to really overcome the degeneration phenomenon, the important density should be chosen more carefully. The optimal choice for important density is the distribution, which is able to minimize the variance of the weights and it is called as optimal particle filter (OPF). In short, OPF can be achieved by incorporating the new observations when generating the new particles. However, the OPF has two major difficulties 6 that is: (1) we should be able to generate
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
573 3
samples from important density, when given current measurement. It means we must perform the measurement first before doing the sampling (2) we have to calculate the predictive likelihood. This means we should be able to predict the current measurements, based on the past state variables. In this paper, we propose a new tracking algorithm, which is constructed by solving optimally particle filters (OPF) efficiently using spectral methods. The remainder of this paper is composed as follows: first we discuss object tracking in Bayesian framework in section 2, and then is followed by the description of OPF in section 3. In section 4, we construct a new tracking algorithm called as spectral tracker (ST). Next, we report the experiment result in 100 video datasets in section 5. Finally, we summarize our work with notes on future research in section 6. 2. Object Tracking in Bayesian Framework The purpose of visual object tracking problem is to estimate a state variable xk based on a set of measurements
z1:k = z1 , z2 ,!, zk in discrete time k. To formulate this problem, starts by considering a set of state variables x1:k = x1 , x2 ,!, xk . Within Bayesian framework, the purpose of tracking problems can be achieved in two stages: first estimate the distribution of state variables x1:k that are the evolution of the object dynamics, given the set of measurements z1 : k -1 . This stage is called as the prediction stage and the result is the prior distribution. Prediction:
p( x1 : k | z1 : k -1 ) = p( xk | x1 : k -1 ) p( x1 : k -1 | z1 : k -1 )
In the second stage, the prior distribution is transformed into a posterior distribution using Bayes theorem incoporating the latest measurements. This stage is called the update stage which can be written as follows: Update: p( z | x , z ) p( x | z ) k
p ( x1 : k | z1 : k ) =
1: k
1 : k -1
1: k
10
(1) by
1 : k -1
p ( zk | z1 : k -1 )
(2) For practical reasons, the above equations can be simplified by using two basic assumptions: (1) the measurement at a given time is independent of the measurements made at other time, thus we get p( z k | x1 : k , z1 : k -1 ) = p( z k | xk ) and
p ( z k | z1 : k -1 ) = p ( z k ) (2) object dynamics are assumed to follow first-order Markov process. It means that the current state variable depends only on the last previous state variable and can be written as p( xk | x1 : k -1 ) = p( xk | xk -1 ) . With this simplification, the prediction and update stages can be written as follows: Prediction:
p( x1 : k | z1 : k -1 ) = p( xk | xk -1 ) p( x1 : k -1 | z1 : k -1 )
Update:
p( x1 : k | z1 : k ) =
(3)
p( zk | xk ) p( x1 : k | z1 : k -1 ) p( zk )
(4) At the beginning, the main purpose in object tracking can be stated to estimate the posterior distribution p ( xk | z1 :k ) . This distribution can be obtained from the posterior distribution in equation (4) by integrating it to all previous state variables. After the integration, the prediction and update stages in equations (3) and (4) can be written as: Prediction: p( xk | z1: k -1 ) = p( xk | xk -1 ) p( xk -1 | z1: k -1 ) dxk -1 (5) Update: p( zk | xk ) p( xk | z1 : k -1 ) p( xk | z1 : k ) = p ( zk ) (6)
ò
Solving recursive Bayesian equations in (5) and (6) is the essence of the object tracking problem. The main difficulty in here is how to calculate the integration in equation (5), which called as Chapman-Kolmogorov equation. There are two main approaches to solve the integral, namely: 1. All distributions involved in equations (5) and (6) are assumed to be Gaussian and system dynamics are assumed
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
574 4
2.
to be linear in order to derive the exact solution. This approach will result in a well known algorithm called Kalman filter 9. The integration in the Chapman-Kolmogorov equation is solved numerically using Monte Carlo method 6. This numerical approach is called particle filters, which give a discrete approximation of the posterior distribution of state variables. The particle filter does not produce an exact solution like the Kalman filter, but this approach can overcome the non-linear dynamics and the non-gaussian distribution in transition model and observation model. The following chapter will discuss about optimal design choice for particle filter, which called as optimal particle filters (OPF).
3. Particle Filters The particle filter is a numerical implementation of the recursive Bayesian model in equations (5) and (6) by approaching posterior distribution with a finite set of weighted samples, called as particles. The basic idea behind the particle filter is the Monte Carlo 7 simulation. Particle filters can be formulated as sequential important sampling (SIS) methods, which the formula derivation can be explained as follows: 1. Suppose the sample set is taken with i = 1,..., n. and each sample is associated with a normalized weight then a discrete version of the posterior distribution can be constructed as: n
p( xk | z1:k ) = å wki d ( xk - xki )
2.
i =1
The samples can not be taken from the posterior distribution directly, then in SIS method, the important density q( xk | z1 : k ) must be designed first. Important density can be considered as a simplification of posterior distribution with equivalent scale factors. By using important density, the approximation of posterior distribution is true if the normalized weight is defined as: p( x | z )
wk º
3.
k
1:k
q( xk | z1:k )
The next step is if n given particles x1 , ... , x n , and their weight w1 , ... , wn , then it can be derived recursive k -1 k -1 k -1 k -1 form of weight based on Bayesian model of object tracking as: p( xk(i ) | xk(i-)1 ) wk(i ) = wk(i-)1 p( zk | xk(i ) ) q( xk(i ) | xk(i-)1 , zk )
4.
i = 1...n
The final step in object tracking is to draw conclusion about the target object by using the maximum a posteriori (MAP) as follows: xk = arg max p( xk | z1:k ) xk
In the application of above particle filter procedure, it is necessary to design important sampling first. There are two main types of particle filters based on the design of its important sampling: bootstrap and optimal particle filter. In here, we just discuss the optimal choice for important density, which is able to minimize the weight variance, which is called as optimal particle filter (OPF) 3.1. Optimal Particle Filter To overcome the degeneration phenomenon 9, the important density should be chosen to minimize the variance of the weights wk . This requirement can be achieved by chosen the important density as 6: Next, the weight equation becomes:
q ( xk | xk(i-)1 , z k ) = p ( xk | xk(i-)1 , z k )
wk(i ) = wk(i-)1 p( zk | xk( i-)1 )
(7)
(8) However, OPF has two major difficulties in its implementation 6: 1. We should be able to draw samples from important density p ( xk | xk( i-)1 , z k ) , given current measurement. It means we must perform the measurement first before doing the sampling. 2. We have to calculate the predictive likelihood p( zk | xk( i-)1 ) . It means we should be able to predict the current measurements, based on the past state variables.
Author name / Procedia Computer Science 00 (2017) 000–000 Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578
5 575
The main contribution of this paper is to introduce a new tracking algorithm by solving the optimal particle filter (OPF) in the frequency domain using the sprektral method. By using our proposed approach, OPF can be solved directly with efficient computation. In this way the old bootstrap particle filter (BPF) scheme 8, which works on spatial domains and has been used by many researchers, becomes irrelevant. The proposed approach will be described in next chapter. 4. Spectral Tracker (ST) The proposed method is called as spectral tracker (ST) because its fundamental approach is spectral method. Spectral method is a numerical problem-solving technique using Fast Fourier Transform (FFT) which is widely used in applied mathematics. The basic idea is to decompose the solution of an equation as a certain number of periodic basic functions ie the cosine and sinus functions and then solve the corresponding coefficients in order to satisfy the given equation. The corresponding coefficient splitting is performed on the frequency domain. By using this approach, the tracking problem in the spatial domain is transformed into the problem in the frequency domain Spectral tracker (ST) is constructed by solving two major weaknesses of optimum particle filter (OPF) with spectral method, namely: 1. In OPF, the samples must be drawn from the important density in equation (7). This means that the current sample should be drawn based on the previous state variable xk -1 and also on the latest observation z k . The requirement to have the latest observation before the sampling process is impossible in the spatial domain. In our previous research 11, a gradual sampling process is performed in order to achive efficiency on particle filter schemes. This approach will just produce semi-optimal importance density. To solve equation (7), we propose to take all points on the domain as samples. This approach is called as dense sampling and the requirement to incorporate the latest observations in equation (7) will be met indirectly. Based on our previous research 12, domain of dense sampling does not need to be taken on all image pixels but simply taken locally around the target object in order to compute more efficient. In our previous research 12, this restricted domain is named as Search Window. 2. In OPF, we have to calculate the predictive likelihood, as consequence we must calculate the following integral:
p( zk | xki -1 ) = ò p( zk | xki ) p( xki | xki -1 ) dxki
(9) The above integral should be evaluated on all samples taken in the first step. Consequently, the integral must be calculated at all points on Search Window. The above integral is actually a convolution that can be calculated efficiently by using spectral method, as follows: p( zk | xki -1 ) = F -1 F p( zk | xki ) • F p( xki | xki -1 ) We use Fourier transform and its inverse in here. In addition, operation • is the hadamard product, ie multiplication between elements of the matrix. By using Fast Fourier Transform (FFT), equation (9) can be calculated efficiently with the computing order of O(n log n). However, the proposed solution has a major drawback in flexibility to choose representation of state variable. Note that in the frequency domain, the state variable must be position in x and y coordinate. If compared to solutions working on spatial domains where the representation of state variable has many alternatives. One of the most frequently used here is affine representation 13. In affine representation, the state variable is represented by 6 affine parameters, not only position of x and y, but also horizontal scaling, vertical scaling, rotation and shear. To overcome this weakness, we proposed to use multiple observation points simultaneously and to use information on the observation point movement to estimate scale and rotation.
[(
) (
)]
4.1. Spectral Tracker Algorithm In this subchapter, we would like to write down the steps of spectral tracker (ST) algorithm (see Table 1). Basically, ST algorithm implements optimal particle filter (OPF) through solving it in frequency domain by using spectral method. In short-term model-free tracking problem 4, the object target is determined by defining a bounding box on
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
576 6
the first frame manually. In ST algorithm, the representation of the object target is a center point of the bounding box. Therefore, the object target variable at time k which can be written as: xk = ( p x , p y ) With px and py is the position of x and y. Furthermore, it is necessary to calculate the new transition model in each frame and then integrate with previous transition model (see Table1). In summary, the proposed algorithm can be seen in the following table: Table 1. Spectral Tracker (ST) algorithm A. Initialization 1. Set time k=1 2. Set earch Window around the target x1 and the focus distribution 4. Compute initial attention distribution as transition model
éF ( p( x1 | z1 ))ù T1update = p( x2 | x1 ) = F -1 ê ú ëF ( p( z1 | x1 ))û
B. Optimal Particle Filter
Dense sampling in Search Window domain
3. Compute observation model of 1st frame
1. Set k=k+1 2. Generate update parameter ρ 3. Compute observation model of kth frame 4. Compute predictive likelihood
p( zk | xk -1 )
5. Compute maximum a posteriori (MAP) for kth frame 6. Construct new focus distribution based on the calculated MAP 7. Compute transition model and update using parameter ρ 8. Reset Search Window around the calculated MAP C. Go to the Optimal Particle Filter (B)
[(
) (
p( z k | xki -1 ) = F -1 F p( z k | xki ) • F Tkupdate
)]
Ns
xk = arg max å p( z k | xk(i-)1 ) d ( xk - xki ) xk
i =1
éF ( p( xk | zk ) )ù Tk = p( xk +1 | xk ) = F - 1 ê ú ëF ( p( zk | xk ) )û Tkupdate = (1 - r ) Tkupdate + r Tk -1
The algorithm in Table 1 above is the basic algorithm of the spectral tracker (ST). This algorithm is guaranteed theoretically optimal because it is an implementation of optimal particle filter (OPF). In addition, the computation is done quickly because the algorithm is based on Fast Fourier Transform (FFT). However, there is still major drawback that is spectral tracker (ST) can not calculate the change of scale and rotation directly. To solve this problem, multitarget solutions are proposed to modify the spectral tracker (ST) algorithm in order to calculate scale and rotation. Illustrations of the multi-target points can be seen in Fig 1 below:
Fig. 1. Multiple target points
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
577 7
5. Experiment Results To evaluate spectral tracker (ST) algorithm, a tracking experiment was conducted using open video datasets 3. The dataset and its ground truth can be downloaded at http://www.visual-tracking.net. The number of datasets is 100 benchmark datasets. The dataset consists of 25 grayscale image datasets and 75 RGB color image datasets. We used 9 renowned trackers as comparator, that is: DLT 14, IVT 13, CT 15, DFT 16, L1APG 17, ORIA 18, CSK 19, LOT 20, and GDPF 21. Comparator trackers were selected to represent the state-of-the-art in object tracking research that have comparable characteristics to the ST tracker. Evaluation of the trackers is done in speed, precision and success rate. The speed is calculated based on the processing speed in number of frames per second (fps). While the precision and success rate is calculated based on the percentage value (%) of the area under curve (AUC). In Table 2 below, summarized the results of speed, precision and success rate on 100 datasets. Table 2. Average of speed, precision and success rate on 100 datasets No 1 2 3 4 5 6 7 8 9 10
Tracker DLT IVT CT DFT L1APG ORIA CSK LOT GDPF ST
Speed (fps) 15.0470 * 35.6705 93.3131 14.6318 1.7610 13.0795 324.3594 0.6494 8.3294 31.8206
Note: * using GPU
Precision (%) 49.6 41.7 35.6 40.6 44.5 39.3 49.3 45.6 25.3 52.3
Success Rate (%) 37.8 32.2 27.0 32.8 35.6 31.3 38.2 34.5 18.8 38.8
From Table 2, it can be seen that the fastest tracker is Circulant Structure Kernels (CSK) which is about 324 fps, while spectral tracker (ST) is around 32 fps. CSK can achieve high speed from the exploitation of circulant structure in the image. On the other hand, ST uses multi observation points, which require relatively many computations. The speed of ST can be accelerated if the use of observation points can be made more efficiently. Furthermore, ST can achieve the highest precision 52.3% and the highest success rate 38.8% in the experiment results. From the results, it can be concluded that ST reaches the highest performance in the real time processing above the average video speed (30 fps). 6. Conclusion In this paper, we proposed a new framework for visual object tracking by solving the optimal particle filter (OPF) scheme in frequency domain using spectral method, called as spectral tracker (ST). By using Fast Fourier Transform (FFT), the OPF computation on the frequency domain can be performed efficiently. However, this spectral method approach has a drawback, that is scale and rotation parameter can not be calculated in frequency domain. To overcome this deficiency, we proposed multiple observation points to calculate scale and rotation based on movement of observation points. Nevertheless, the using of multiple observation points approach reduces ST tracker speed greatly. The speed of ST algorithm is influenced by the number of used observation points. ST can be accelerated if the observation points can be used more efficiently. In experiment results, ST can achieve the highest precision and success rate over 9 comparator trackers. References 1. Xi Li WH, Shen C, Zhang Z, Dick A, Hengel Avd. A Survey of Appearance Models in Visual Object Tracking. ACM Transactions on Intelligent Systems and Technology (TIST). 2013 october; 4(4): p. 48. 2. Yilmaz A, Javed O, Shah M. Object tracking: A survey. ACM Computing Surveys (CSUR). 2006 December; 38(4): p. 1-45.
578 8
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
3. Wu Y, Lim J, Yang MH. Object Tracking Benchmark. Transactions on Pattern Analysis and Machine Intelligence (TPAMI). 2015; 37(9): p. 1834 - 1848. 4. Kristan M, Pfugfelder R, Leonardis A, Matas J, Cehovin L, Nebehay G. The visual object tracking VOT2014 challenge results. Zurich:; 2014. 5. Gunawan AAS, Wasito I. Nonretinotopic Particle Filter for Visual Tracking. Journal of Theoretical and Applied Information Technology. 2014 May; 63(1): p. 104-111. 6. Haug AJ. Bayesian Estimation and Tracking: A Practical Guide: Wiley; 2012. 7. Doucet A, Godsill S, Andrieu C. On sequential Monte Carlo sampling methods for Bayesian filtering. Statistics and Computing. 2000 July; 10(3): p. 197-208. 8. Isard M, Blake A. CONDENSATION—Conditional Density Propagation for Visual Tracking. International Journal of Computer Vision. 1998 August ; 29(1): p. 5-28. 9. Ristic B, Arulampalam S, Gordon N. Beyond the Kalman Filter: Particle Filters for Tracking Applications Australia: Artech House; 2004. 10. Challa S, Morelande MR, Mušicki D, Evans RJ. Fundamentals of Object Tracking: Cambridge University Press; 2011. 11. Gunawan. AAS, Jatmiko W, Dewanto V, Rachmadi F, Jovan F. Tracking efficiency measurement of dynamic models on geometric particle filter using KLD-resampling. In International Conference on Advanced Computer Science and Information Systems (ICACSIS); 2014; Jakarta: IEEE. p. 385 - 388. 12. Gunawan AAS, Arymurthy AM. Visual Tracking for Abrupt Motions of Human Sperm Using Smoothing Stochastic Approximate Monte Carlo. In International Conference on Computer Science and Computational Intelligence (ICCSCI); 2015; Jakarta. p. 64–72. 13. Ross DA, Lim J, Lin RS, Yang MH. Incremental Learning for Robust Visual Tracking. International Journal of Computer Vision. 2008 May; 77(1): p. 125-141. 14. Wang N, Yeung DY. Learning a Deep Compact Image Representation for Visual Tracking. In Proceedings of Twenty-Seventh Annual Conference on Neural Information Processing Systems (NIPS) ; 2013; Lake Tahoe, Nevada. 15. Zhang K, Zhang L, Yang MH. Real-time Compressive Tracking. In Proceedings of the 12th European conference on Computer Vision (ECCV); 2012; Firenze, Italy: Springer-Verlag. p. 864-877. 16. Sevilla-Lara L. Distribution Fields for Tracking. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1910-1917. 17. Bao C, Wu Y, Ling H, Ji H. Real Time Robust L1 Tracker Using Accelerated Proximal Gradient Approach. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1830 - 1837. 18. Wu Y, Shen B, Ling H. Online Robust Image Alignment via Iterative Convex Optimization. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1808 - 1814. 19. Henriques JF, Caseiro R, Martins P, Batista J. Exploiting the Circulant Structure of Tracking-by-Detection with Kernels. In Proceedings of the 12th European conference on Computer Vision (ECCV); 2012; Firenze, Italy: Springer-Verlag. p. 702-715. 20. Oron S, Bar-Hillel A, Levi D, Avidan S. Locally Orderless Tracking. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1940 - 1947. 21. Gunawan AAS, Jatmiko W. Geometric Deep Particle Filter for Motorcycle Tracking: Development of Intelligent Traffic System in Jakarta. International Journal on Smart Sensing and Intelligent Systems. 2014; 8(1): p. 429 – 463.
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2017) 000–000 Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 116 (2017) 571–578
2nd International Conference on Computer Science and Computational Intelligence 2017, ICCSCI 2nd International Conference on Computer Science andBali, Computational 2017, 13-14 October 2017, Indonesia Intelligence 2017, ICCSCI 2017, 13-14 October 2017, Bali, Indonesia
Fast and Optimal Visual Tracking based on Spectral Method Fast and Optimal Visual Tracking based on Spectral Method
a a
b b Alexander A S Gunawana* a*, Wisnu Jatmikob, Aniati Murni Arymurthyb Alexander A S Gunawan , Wisnu Jatmiko , Aniati Murni Arymurthy
Computer Science Department, School of Computer Science, Bina Nusantara University, Jl.K.H. Syahdan No.9, Jakarta, 11480 Indonesia Computer Science Department, bSchool Nusantara University,Depok, Jl.K.H. Syahdan No.9, Jakarta, 11480 Indonesia FacultyofofComputer ComputerScience, Science,Bina University of Indonesia, Indonesia b Faculty of Computer Science, University of Indonesia, Depok, Indonesia
Abstract Abstract Visual object tracking is the process of continuously localizing visual object in a video sequence. We would like to investigate the Visual tracking model-free is the process of continuously visual object a video like to investigate the problemobject of short-term tracking which the localizing main purpose is to trackinany objectsequence. just basedWe on would an annotation box of object. problem of short-term tracking the main purpose is toVisual track any objectBenchmark, just based on an annotation of object. Many factors affect themodel-free performance of thewhich tracking algorithm. In the Tracker there are eleven box challenges in Many affect thehas performance of the tracker trackingthat algorithm. In the VisualallTracker are eleven challenges in object factors tracking. There not been a single successfully handles of theseBenchmark, scenarios. Inthere addition, the tracker must be object tracking. has a single tracker that successfully handles all of these scenarios. In addition, the tracker must be fast enough to beThere useful innot realbeen applications. We propose a new tracking algorithm within the Bayesian framework. The proposed fast enoughis to be useful in applications. propose a new tracking algorithm the methods. Bayesian Therefore, framework.the The proposed algorithm constructed byreal solving optimallyWe particle filters (OPF) efficiently usingwithin spectral constructed algorithm is constructed by solving particleST filters efficiently using spectral methods. Therefore, thethe constructed tracker is called as Spectral Trackeroptimally (ST). Although can (OPF) efficiently compute object position, it cannot estimate scale and tracker called asTo Spectral Tracker Although can efficiently position, it cannot estimate theand scale rotationisdirectly. overcome this (ST). weakness, it isST proposed to use compute multiple object observation points simultaneously to and use rotation directly. overcomepoint this movement weakness, to it estimate is proposed userotation. multiple points to use information on theTo observation scaletoand In observation the experiments, thesimultaneously performance ofand ST tracker information on with the observation movement estimate scale The and rotation. In theresults experiments, the performance of ST tracker was compared 9 relevant point trackers based onto100 data sets. experimental on on tracker performance show that was compared with 9 relevant trackers based on 100 and datasuccess sets. The increasing performance especially in tracker precision rate.experimental results on on tracker performance show that increasing performance especially in tracker precision and success rate. © 2017 The Authors. Published by Elsevier B.V. © 2017 2017 The The Authors. Published Elsevier B.V. © Authors. Published by by B.V. committee of the 2nd International Conference on Computer Science and Peer-review under responsibility of Elsevier the scientific Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. Computational Intelligence 2017. Computational Intelligence 2017. Keywords: short-term model-free tracking, optimal particle filter, spectral method, bayesian framework Keywords: short-term model-free tracking, optimal particle filter, spectral method, bayesian framework
1. Introduction 1. Introduction Visual object tracking has many practical applications in automated surveillance system, robot vision system, Visual object tracking has many practical applications in automated surveillance system, robot vision system,
* *
Corresponding author. Tel.: +62-21-534-5830 E-mail address:author. [email protected] Corresponding Tel.: +62-21-534-5830 E-mail address: [email protected]
1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review underThe responsibility of theby scientific of the 2nd International Conference on Computer Science and 1877-0509 © 2017 Authors. Published Elsevier committee B.V.
Peer-review under responsibility Computational Intelligence 2017.of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. 1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 2nd International Conference on Computer Science and Computational Intelligence 2017. 10.1016/j.procs.2017.10.069
572 2
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
intelligent traffic system and fault detection system. Visual tracking is the process of continuously localizing an object target in a video sequence. When the target is successfully tracked in each video frame gives more information about the identity and the activity of the object target. In general, tracking problem is easier than detection, tracking algorithms should be faster than running an object detector on every frame. Therefore, the key requirement for many tracking applications is ability to track an object target in video sequences in real-time. Currently, many existing tracking algorithms have been introduced for certain tasks. However, many of them have weakness because of their main approaches that is (1) they make basic assumptions about object appearances 1 and the environment 2 (2) most visual tracking algorithms are computationally expensive 3. In this paper, we would like to overcome these weakness by considering its fundamental approach. We focus to solve short-term model-free tracking problem 4. The short-term tracking in here means the tracker does not perform re-detection after the object target is lost. Out of the target is considered as a failure. While model-free property means that the only example for supervised learning is provided by the bounding box in the first frame. In addition, the tracker output is indicated by a rotated bounding box. First, we consider human visual perception in order to understand how visual information is represented and processed in the brain. It is clear that geometry of object and visual perception are closely related, but very little is known about their relation beyond the retinotopic association 5. In visual perception studies, retinotopic refers to the visual process where object geometry are projected in the retina by similar process as appearance models in a digital image. However, a recent study on human vision shows that the representation in higher visual areas of the visual cortex occurs in a nonretinotopic way 5. It means that visual perception creates dynamic layers for each moving object in the scene. This representation suggests that the object appearance and their position dynamics are marginal independent. Based on the inspiration from the above study, a tracking algorithm is proposed by imitating the nonretinotopic process in human visual processing. The suitable approach that can govern the relationships between the object appearance model and the object dynamic model is probabilistic Bayesian framework. The advantage of the Bayesian approach is able to maintain a history of probability estimation from the sequence of incoming visual perceptions. This means the Bayesian approach is able to integrate knowledge of the target object dynamics with visual observations of object appearance. Second, we need to choose a numerical approach for implementing Bayesian framework. Particle filter is a practical tool to implement dynamic state estimation in Bayesian framework 6. The basic idea is to approximate the posterior distribution of transition dynamics by using a finite set of weighted samples or particles, when given a sequence of observation measurements. Moreover, particle Þlter can overcome non-linear and non-Gaussian distribution in transition model and observation model. Particle filter is a sequential Monte-Carlo technique and can be proved to produce the correct posterior distribution from finite set of samples as the sample size toward infinity 7. In practice, it is necessary first to design its important sampling. There is, of course, trade off between accuracy and computational load in the chosen design. There are two main types of particle filters based on the design of its important sampling: bootstrap and optimal particle filter. The easiest particle filter to implement is a bootstrap particle filter (BPF). In BPF, the transition distribution is choosen as important density. BPF has the characteristic that the weights only depend on the likelihood observation distribution. For BPF, the sampling process is very easy by using the transition distribution to predict the new particle and then followed by the process of weight determination by using likelihood distribution. BPF has been used by many researchers in visual object tracking and often called as Condensation (Conditional Density Propagation) 8. Note that in BPF algorithm, its important density does not take into account the latest measurements. So even if BPF is easy to implement, BPF has some weaknesses associated with robustness in the face of unexpected noise. This is mainly due to the fact that variance of the weights can not go down 9. As a result, the degeneration phenomenon in BPF may appear. One way to overcome the degeneration phenomenon is by using resampling process. However, to really overcome the degeneration phenomenon, the important density should be chosen more carefully. The optimal choice for important density is the distribution, which is able to minimize the variance of the weights and it is called as optimal particle filter (OPF). In short, OPF can be achieved by incorporating the new observations when generating the new particles. However, the OPF has two major difficulties 6 that is: (1) we should be able to generate
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
573 3
samples from important density, when given current measurement. It means we must perform the measurement first before doing the sampling (2) we have to calculate the predictive likelihood. This means we should be able to predict the current measurements, based on the past state variables. In this paper, we propose a new tracking algorithm, which is constructed by solving optimally particle filters (OPF) efficiently using spectral methods. The remainder of this paper is composed as follows: first we discuss object tracking in Bayesian framework in section 2, and then is followed by the description of OPF in section 3. In section 4, we construct a new tracking algorithm called as spectral tracker (ST). Next, we report the experiment result in 100 video datasets in section 5. Finally, we summarize our work with notes on future research in section 6. 2. Object Tracking in Bayesian Framework The purpose of visual object tracking problem is to estimate a state variable xk based on a set of measurements
z1:k = z1 , z2 ,!, zk in discrete time k. To formulate this problem, starts by considering a set of state variables x1:k = x1 , x2 ,!, xk . Within Bayesian framework, the purpose of tracking problems can be achieved in two stages: first estimate the distribution of state variables x1:k that are the evolution of the object dynamics, given the set of measurements z1 : k -1 . This stage is called as the prediction stage and the result is the prior distribution. Prediction:
p( x1 : k | z1 : k -1 ) = p( xk | x1 : k -1 ) p( x1 : k -1 | z1 : k -1 )
In the second stage, the prior distribution is transformed into a posterior distribution using Bayes theorem incoporating the latest measurements. This stage is called the update stage which can be written as follows: Update: p( z | x , z ) p( x | z ) k
p ( x1 : k | z1 : k ) =
1: k
1 : k -1
1: k
10
(1) by
1 : k -1
p ( zk | z1 : k -1 )
(2) For practical reasons, the above equations can be simplified by using two basic assumptions: (1) the measurement at a given time is independent of the measurements made at other time, thus we get p( z k | x1 : k , z1 : k -1 ) = p( z k | xk ) and
p ( z k | z1 : k -1 ) = p ( z k ) (2) object dynamics are assumed to follow first-order Markov process. It means that the current state variable depends only on the last previous state variable and can be written as p( xk | x1 : k -1 ) = p( xk | xk -1 ) . With this simplification, the prediction and update stages can be written as follows: Prediction:
p( x1 : k | z1 : k -1 ) = p( xk | xk -1 ) p( x1 : k -1 | z1 : k -1 )
Update:
p( x1 : k | z1 : k ) =
(3)
p( zk | xk ) p( x1 : k | z1 : k -1 ) p( zk )
(4) At the beginning, the main purpose in object tracking can be stated to estimate the posterior distribution p ( xk | z1 :k ) . This distribution can be obtained from the posterior distribution in equation (4) by integrating it to all previous state variables. After the integration, the prediction and update stages in equations (3) and (4) can be written as: Prediction: p( xk | z1: k -1 ) = p( xk | xk -1 ) p( xk -1 | z1: k -1 ) dxk -1 (5) Update: p( zk | xk ) p( xk | z1 : k -1 ) p( xk | z1 : k ) = p ( zk ) (6)
ò
Solving recursive Bayesian equations in (5) and (6) is the essence of the object tracking problem. The main difficulty in here is how to calculate the integration in equation (5), which called as Chapman-Kolmogorov equation. There are two main approaches to solve the integral, namely: 1. All distributions involved in equations (5) and (6) are assumed to be Gaussian and system dynamics are assumed
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
574 4
2.
to be linear in order to derive the exact solution. This approach will result in a well known algorithm called Kalman filter 9. The integration in the Chapman-Kolmogorov equation is solved numerically using Monte Carlo method 6. This numerical approach is called particle filters, which give a discrete approximation of the posterior distribution of state variables. The particle filter does not produce an exact solution like the Kalman filter, but this approach can overcome the non-linear dynamics and the non-gaussian distribution in transition model and observation model. The following chapter will discuss about optimal design choice for particle filter, which called as optimal particle filters (OPF).
3. Particle Filters The particle filter is a numerical implementation of the recursive Bayesian model in equations (5) and (6) by approaching posterior distribution with a finite set of weighted samples, called as particles. The basic idea behind the particle filter is the Monte Carlo 7 simulation. Particle filters can be formulated as sequential important sampling (SIS) methods, which the formula derivation can be explained as follows: 1. Suppose the sample set is taken with i = 1,..., n. and each sample is associated with a normalized weight then a discrete version of the posterior distribution can be constructed as: n
p( xk | z1:k ) = å wki d ( xk - xki )
2.
i =1
The samples can not be taken from the posterior distribution directly, then in SIS method, the important density q( xk | z1 : k ) must be designed first. Important density can be considered as a simplification of posterior distribution with equivalent scale factors. By using important density, the approximation of posterior distribution is true if the normalized weight is defined as: p( x | z )
wk º
3.
k
1:k
q( xk | z1:k )
The next step is if n given particles x1 , ... , x n , and their weight w1 , ... , wn , then it can be derived recursive k -1 k -1 k -1 k -1 form of weight based on Bayesian model of object tracking as: p( xk(i ) | xk(i-)1 ) wk(i ) = wk(i-)1 p( zk | xk(i ) ) q( xk(i ) | xk(i-)1 , zk )
4.
i = 1...n
The final step in object tracking is to draw conclusion about the target object by using the maximum a posteriori (MAP) as follows: xk = arg max p( xk | z1:k ) xk
In the application of above particle filter procedure, it is necessary to design important sampling first. There are two main types of particle filters based on the design of its important sampling: bootstrap and optimal particle filter. In here, we just discuss the optimal choice for important density, which is able to minimize the weight variance, which is called as optimal particle filter (OPF) 3.1. Optimal Particle Filter To overcome the degeneration phenomenon 9, the important density should be chosen to minimize the variance of the weights wk . This requirement can be achieved by chosen the important density as 6: Next, the weight equation becomes:
q ( xk | xk(i-)1 , z k ) = p ( xk | xk(i-)1 , z k )
wk(i ) = wk(i-)1 p( zk | xk( i-)1 )
(7)
(8) However, OPF has two major difficulties in its implementation 6: 1. We should be able to draw samples from important density p ( xk | xk( i-)1 , z k ) , given current measurement. It means we must perform the measurement first before doing the sampling. 2. We have to calculate the predictive likelihood p( zk | xk( i-)1 ) . It means we should be able to predict the current measurements, based on the past state variables.
Author name / Procedia Computer Science 00 (2017) 000–000 Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578
5 575
The main contribution of this paper is to introduce a new tracking algorithm by solving the optimal particle filter (OPF) in the frequency domain using the sprektral method. By using our proposed approach, OPF can be solved directly with efficient computation. In this way the old bootstrap particle filter (BPF) scheme 8, which works on spatial domains and has been used by many researchers, becomes irrelevant. The proposed approach will be described in next chapter. 4. Spectral Tracker (ST) The proposed method is called as spectral tracker (ST) because its fundamental approach is spectral method. Spectral method is a numerical problem-solving technique using Fast Fourier Transform (FFT) which is widely used in applied mathematics. The basic idea is to decompose the solution of an equation as a certain number of periodic basic functions ie the cosine and sinus functions and then solve the corresponding coefficients in order to satisfy the given equation. The corresponding coefficient splitting is performed on the frequency domain. By using this approach, the tracking problem in the spatial domain is transformed into the problem in the frequency domain Spectral tracker (ST) is constructed by solving two major weaknesses of optimum particle filter (OPF) with spectral method, namely: 1. In OPF, the samples must be drawn from the important density in equation (7). This means that the current sample should be drawn based on the previous state variable xk -1 and also on the latest observation z k . The requirement to have the latest observation before the sampling process is impossible in the spatial domain. In our previous research 11, a gradual sampling process is performed in order to achive efficiency on particle filter schemes. This approach will just produce semi-optimal importance density. To solve equation (7), we propose to take all points on the domain as samples. This approach is called as dense sampling and the requirement to incorporate the latest observations in equation (7) will be met indirectly. Based on our previous research 12, domain of dense sampling does not need to be taken on all image pixels but simply taken locally around the target object in order to compute more efficient. In our previous research 12, this restricted domain is named as Search Window. 2. In OPF, we have to calculate the predictive likelihood, as consequence we must calculate the following integral:
p( zk | xki -1 ) = ò p( zk | xki ) p( xki | xki -1 ) dxki
(9) The above integral should be evaluated on all samples taken in the first step. Consequently, the integral must be calculated at all points on Search Window. The above integral is actually a convolution that can be calculated efficiently by using spectral method, as follows: p( zk | xki -1 ) = F -1 F p( zk | xki ) • F p( xki | xki -1 ) We use Fourier transform and its inverse in here. In addition, operation • is the hadamard product, ie multiplication between elements of the matrix. By using Fast Fourier Transform (FFT), equation (9) can be calculated efficiently with the computing order of O(n log n). However, the proposed solution has a major drawback in flexibility to choose representation of state variable. Note that in the frequency domain, the state variable must be position in x and y coordinate. If compared to solutions working on spatial domains where the representation of state variable has many alternatives. One of the most frequently used here is affine representation 13. In affine representation, the state variable is represented by 6 affine parameters, not only position of x and y, but also horizontal scaling, vertical scaling, rotation and shear. To overcome this weakness, we proposed to use multiple observation points simultaneously and to use information on the observation point movement to estimate scale and rotation.
[(
) (
)]
4.1. Spectral Tracker Algorithm In this subchapter, we would like to write down the steps of spectral tracker (ST) algorithm (see Table 1). Basically, ST algorithm implements optimal particle filter (OPF) through solving it in frequency domain by using spectral method. In short-term model-free tracking problem 4, the object target is determined by defining a bounding box on
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
576 6
the first frame manually. In ST algorithm, the representation of the object target is a center point of the bounding box. Therefore, the object target variable at time k which can be written as: xk = ( p x , p y ) With px and py is the position of x and y. Furthermore, it is necessary to calculate the new transition model in each frame and then integrate with previous transition model (see Table1). In summary, the proposed algorithm can be seen in the following table: Table 1. Spectral Tracker (ST) algorithm A. Initialization 1. Set time k=1 2. Set earch Window around the target x1 and the focus distribution 4. Compute initial attention distribution as transition model
éF ( p( x1 | z1 ))ù T1update = p( x2 | x1 ) = F -1 ê ú ëF ( p( z1 | x1 ))û
B. Optimal Particle Filter
Dense sampling in Search Window domain
3. Compute observation model of 1st frame
1. Set k=k+1 2. Generate update parameter ρ 3. Compute observation model of kth frame 4. Compute predictive likelihood
p( zk | xk -1 )
5. Compute maximum a posteriori (MAP) for kth frame 6. Construct new focus distribution based on the calculated MAP 7. Compute transition model and update using parameter ρ 8. Reset Search Window around the calculated MAP C. Go to the Optimal Particle Filter (B)
[(
) (
p( z k | xki -1 ) = F -1 F p( z k | xki ) • F Tkupdate
)]
Ns
xk = arg max å p( z k | xk(i-)1 ) d ( xk - xki ) xk
i =1
éF ( p( xk | zk ) )ù Tk = p( xk +1 | xk ) = F - 1 ê ú ëF ( p( zk | xk ) )û Tkupdate = (1 - r ) Tkupdate + r Tk -1
The algorithm in Table 1 above is the basic algorithm of the spectral tracker (ST). This algorithm is guaranteed theoretically optimal because it is an implementation of optimal particle filter (OPF). In addition, the computation is done quickly because the algorithm is based on Fast Fourier Transform (FFT). However, there is still major drawback that is spectral tracker (ST) can not calculate the change of scale and rotation directly. To solve this problem, multitarget solutions are proposed to modify the spectral tracker (ST) algorithm in order to calculate scale and rotation. Illustrations of the multi-target points can be seen in Fig 1 below:
Fig. 1. Multiple target points
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
577 7
5. Experiment Results To evaluate spectral tracker (ST) algorithm, a tracking experiment was conducted using open video datasets 3. The dataset and its ground truth can be downloaded at http://www.visual-tracking.net. The number of datasets is 100 benchmark datasets. The dataset consists of 25 grayscale image datasets and 75 RGB color image datasets. We used 9 renowned trackers as comparator, that is: DLT 14, IVT 13, CT 15, DFT 16, L1APG 17, ORIA 18, CSK 19, LOT 20, and GDPF 21. Comparator trackers were selected to represent the state-of-the-art in object tracking research that have comparable characteristics to the ST tracker. Evaluation of the trackers is done in speed, precision and success rate. The speed is calculated based on the processing speed in number of frames per second (fps). While the precision and success rate is calculated based on the percentage value (%) of the area under curve (AUC). In Table 2 below, summarized the results of speed, precision and success rate on 100 datasets. Table 2. Average of speed, precision and success rate on 100 datasets No 1 2 3 4 5 6 7 8 9 10
Tracker DLT IVT CT DFT L1APG ORIA CSK LOT GDPF ST
Speed (fps) 15.0470 * 35.6705 93.3131 14.6318 1.7610 13.0795 324.3594 0.6494 8.3294 31.8206
Note: * using GPU
Precision (%) 49.6 41.7 35.6 40.6 44.5 39.3 49.3 45.6 25.3 52.3
Success Rate (%) 37.8 32.2 27.0 32.8 35.6 31.3 38.2 34.5 18.8 38.8
From Table 2, it can be seen that the fastest tracker is Circulant Structure Kernels (CSK) which is about 324 fps, while spectral tracker (ST) is around 32 fps. CSK can achieve high speed from the exploitation of circulant structure in the image. On the other hand, ST uses multi observation points, which require relatively many computations. The speed of ST can be accelerated if the use of observation points can be made more efficiently. Furthermore, ST can achieve the highest precision 52.3% and the highest success rate 38.8% in the experiment results. From the results, it can be concluded that ST reaches the highest performance in the real time processing above the average video speed (30 fps). 6. Conclusion In this paper, we proposed a new framework for visual object tracking by solving the optimal particle filter (OPF) scheme in frequency domain using spectral method, called as spectral tracker (ST). By using Fast Fourier Transform (FFT), the OPF computation on the frequency domain can be performed efficiently. However, this spectral method approach has a drawback, that is scale and rotation parameter can not be calculated in frequency domain. To overcome this deficiency, we proposed multiple observation points to calculate scale and rotation based on movement of observation points. Nevertheless, the using of multiple observation points approach reduces ST tracker speed greatly. The speed of ST algorithm is influenced by the number of used observation points. ST can be accelerated if the observation points can be used more efficiently. In experiment results, ST can achieve the highest precision and success rate over 9 comparator trackers. References 1. Xi Li WH, Shen C, Zhang Z, Dick A, Hengel Avd. A Survey of Appearance Models in Visual Object Tracking. ACM Transactions on Intelligent Systems and Technology (TIST). 2013 october; 4(4): p. 48. 2. Yilmaz A, Javed O, Shah M. Object tracking: A survey. ACM Computing Surveys (CSUR). 2006 December; 38(4): p. 1-45.
578 8
Alexander A S Gunawan et al. / Procedia Computer Science 116 (2017) 571–578 Author name / Procedia Computer Science 00 (2017) 000–000
3. Wu Y, Lim J, Yang MH. Object Tracking Benchmark. Transactions on Pattern Analysis and Machine Intelligence (TPAMI). 2015; 37(9): p. 1834 - 1848. 4. Kristan M, Pfugfelder R, Leonardis A, Matas J, Cehovin L, Nebehay G. The visual object tracking VOT2014 challenge results. Zurich:; 2014. 5. Gunawan AAS, Wasito I. Nonretinotopic Particle Filter for Visual Tracking. Journal of Theoretical and Applied Information Technology. 2014 May; 63(1): p. 104-111. 6. Haug AJ. Bayesian Estimation and Tracking: A Practical Guide: Wiley; 2012. 7. Doucet A, Godsill S, Andrieu C. On sequential Monte Carlo sampling methods for Bayesian filtering. Statistics and Computing. 2000 July; 10(3): p. 197-208. 8. Isard M, Blake A. CONDENSATION—Conditional Density Propagation for Visual Tracking. International Journal of Computer Vision. 1998 August ; 29(1): p. 5-28. 9. Ristic B, Arulampalam S, Gordon N. Beyond the Kalman Filter: Particle Filters for Tracking Applications Australia: Artech House; 2004. 10. Challa S, Morelande MR, Mušicki D, Evans RJ. Fundamentals of Object Tracking: Cambridge University Press; 2011. 11. Gunawan. AAS, Jatmiko W, Dewanto V, Rachmadi F, Jovan F. Tracking efficiency measurement of dynamic models on geometric particle filter using KLD-resampling. In International Conference on Advanced Computer Science and Information Systems (ICACSIS); 2014; Jakarta: IEEE. p. 385 - 388. 12. Gunawan AAS, Arymurthy AM. Visual Tracking for Abrupt Motions of Human Sperm Using Smoothing Stochastic Approximate Monte Carlo. In International Conference on Computer Science and Computational Intelligence (ICCSCI); 2015; Jakarta. p. 64–72. 13. Ross DA, Lim J, Lin RS, Yang MH. Incremental Learning for Robust Visual Tracking. International Journal of Computer Vision. 2008 May; 77(1): p. 125-141. 14. Wang N, Yeung DY. Learning a Deep Compact Image Representation for Visual Tracking. In Proceedings of Twenty-Seventh Annual Conference on Neural Information Processing Systems (NIPS) ; 2013; Lake Tahoe, Nevada. 15. Zhang K, Zhang L, Yang MH. Real-time Compressive Tracking. In Proceedings of the 12th European conference on Computer Vision (ECCV); 2012; Firenze, Italy: Springer-Verlag. p. 864-877. 16. Sevilla-Lara L. Distribution Fields for Tracking. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1910-1917. 17. Bao C, Wu Y, Ling H, Ji H. Real Time Robust L1 Tracker Using Accelerated Proximal Gradient Approach. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1830 - 1837. 18. Wu Y, Shen B, Ling H. Online Robust Image Alignment via Iterative Convex Optimization. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1808 - 1814. 19. Henriques JF, Caseiro R, Martins P, Batista J. Exploiting the Circulant Structure of Tracking-by-Detection with Kernels. In Proceedings of the 12th European conference on Computer Vision (ECCV); 2012; Firenze, Italy: Springer-Verlag. p. 702-715. 20. Oron S, Bar-Hillel A, Levi D, Avidan S. Locally Orderless Tracking. In IEEE Conference 2012 on Computer Vision and Pattern Recognition (CVPR); 2012; Providence, Rhode Island: IEEE. p. 1940 - 1947. 21. Gunawan AAS, Jatmiko W. Geometric Deep Particle Filter for Motorcycle Tracking: Development of Intelligent Traffic System in Jakarta. International Journal on Smart Sensing and Intelligent Systems. 2014; 8(1): p. 429 – 463.