- Email: [email protected]
Tracking multiple workers on construction sites using video cameras
Advanced Engineering Informatics 24 (2010) 428–434
Contents lists available at ScienceDirect
Advanced Engineering Informatics journal homepage: www.elsevier.com/locate/aei
Tracking multiple workers on construction sites using video cameras J. Yang a,b, O. Arif b, P.A. Vela b, J. Teizer c,*, Zhongke Shi a a
College of Automation, Northwestern Polytechnical University, China School of Electrical and Computer Engineering, Georgia Tech, Atlanta, GA 30332, USA c School of Civil and Environmental Engineering, Georgia Tech, Atlanta, GA 30332, USA b
a r t i c l e
i n f o
Article history: Received 16 June 2010 Accepted 23 June 2010 Available online 31 July 2010 Keywords: Construction Worker Monitoring Tracking Video surveillance
a b s t r a c t This paper proposes a tracking scheme for tracking multiple workers on construction sites using video cameras. Prior work has compared several contemporary tracking algorithms on construction sites and identified several difficulties, one of which included the existence of interacting workforce. In order to address the challenge of multiple workers within the camera’s field of view, the authors have developed a tracking algorithm based upon machine learning methods. The algorithm requires several sample templates of the tracking target and learns a general model that can be applied to other targets with similar geometry. A parameterized feature bank is proposed to handle the case of variable appearance content. The tracking initialization has been discussed for different types of video cameras. A multiple tracking management module is applied to optimize the system. The principal objective of this paper is to test and demonstrate the feasibility of tracking multiple workers from statically placed and dynamically moving cameras. Published by Elsevier Ltd.
1. Introduction In recent years, arising from the affordable price of video cameras, the placement of cameras on construction sites for surveillance purposes is becoming more commonplace. If monitoring is performed at all, a worker or manager will be tasked to review or monitor the video, which is a time and money consuming task. Alternatively, the imagery will only be utilized to corroborate events after the fact should such a need arise. With the development of computer vision algorithms capable of processing the video stream, an intelligent construction site visual surveillance system would be feasible. Such a system could supply information for workforce productivity analysis, workforce training, and safety monitoring. However, little work has been done for these special applications mainly because of the complex and uncontrolled environment in any construction site. This paper presents a robust multiple workers tracking scheme for the construction site, which is one of the core components in a visual surveillance system. The tracking scheme extends previous work [1]. Considering the similarity among construction workers, a machine learning algorithm is used to train a general spatial model for tracking, which can be rapidly specialized to the appearance model of a given worker. To handle complex situations like
* Corresponding author. Tel.: +1 404 894 8269. E-mail addresses: [email protected] (J. Yang), [email protected] (O. Arif), [email protected] (P.A. Vela), [email protected] (J. Teizer), [email protected] (Z. Shi). 1474-0346/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.aei.2010.06.008
occlusions or interactions, a multiple tracking management module is applied. Initialization methods for static versus dynamic cameras are discussed. The remainder of the paper is organized as follows: a review of related work is given in Section 2, the description of the proposed tracking scheme follows in Section 3, with experimental results in Section 4, and a conclusion in Section 5. 2. Background and motivation Construction safety is a key motivation for this research since the United States has had more than 1100 construction related fatalities in each of the past 16 years. An additional reason for this research is to analyze the work that construction personnel perform on-site. Many management related duties, such as tracking the position, activity, and status of construction resources (personnel, equipment, material), are not in near- nor in real-time, thus limiting the awareness of project status in project management decision making at construction sites. Automated tracking of vital construction resources needs to be realized for timely feedback of construction progress [2]. While much research is devoted to analyzing the application of sensor technologies to the construction site [2], such as Global Positioning Systems or Radio Frequency ID technologies, this paper focuses on visual surveillance systems, which are becoming more common on the work-site. Vision-based monitoring research has principally investigated the use of vision for comparing as-planned versus as-built infrastructure, defect detection, and structural modelling in augmented
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
reality [3–5]. Trucco and Kaka [6] proposed an algorithm for automatically locating objects in unconstrained images of building sites and sketched a computational framework to capture and measure construction progress automatically from images taken on site. Lukins and Trucco [7] proposed a system capable of automatically detecting changes on a building site observed by a fixed camera, and to identify such changes as parts of the construction plan. Choi et al. [8] used a stereo vision system of 3D object recognition to extract specific features for comparison with pre-planned 3D CAD data for progress monitoring. Such works mostly seek to use vision for improved spatial awareness of construction work-site conditions or progress. Furthermore, these methods analyze individual images. Research dealing with video sequences are closer to the research described herein. Peddi et al. [9] proposed a system using blob tracking, pose extraction and pose classification to judge the productivity of the worker by classifying different poses into categories such as effective, ineffective, and contributory. Gong and Caldas [10] presented a study on developing a video interpretation model to interpret videos of construction operations automatically into productivity information. It focused on the detection and tracking of project resources as well as work state classifications and abnormal production scenario identifications. Most related to this paper is the work in Cordova and Brilakis [11], who detailed a static-camera, stereo-based tracking method for construction work-sites. However, the tracking of multiple, interacting personnel is not discussed. In prior work, we discussed the possibility and need for tracking workforce on construction jobsites using video cameras. A comparison among four tracking algorithms and evaluation of their associated results were presented [12]. One pressing need was the ability to track multiple workers with interacting trajectories and similar appearance models. This paper extends existing research efforts to the case of multiple, interacting, and occluding workforce.
[13]. For the PTZ camera, since a consistent background model cannot be generated, a people detector is needed. For people detection algorithm, please refer to [14]. Assuming the existence of a worker’s initial template from the detection module, to relate it with the spatial model, a segmentation method to distinguish the upper and lower body is needed. Inspired by Elgammal and Davis [15], the body is divided into three blobs – head, torso and legs. Each blob is modeled by its own color density and spatial density functions. Assuming every pixel inside every blob has the same color distribution and the vertical location of a blob inside the body is independent of its horizontal location, the joint distribution of pixel located at x with color c given blob A can be expressed as:
PA ðx; cÞ ¼ PA;spatial ðxÞPA;color ðcÞ where PA;spatial ðxÞ is the spatial distribution and P A;color ðcÞ is the color distribution. A pixel X ¼ ðx; cÞ can be classified to one of the three body blobs (head, torso, legs) by maximizing the joint distribution
X 2 Ak ;
such that k ¼ argk max PðXjAk Þ
where k ¼ 1; 2; 3. To learn the color and spatial models, samples pixels are extracted from the high-confidence regions associated to each blob. For a person in upright pose, the confident regions for the head, torso, and legs is chosen empirically. In our experiment, the confidence regions parameter matrix is [0.02, 0.08; 0.3, 0.4; 0.7, 0.8], which means vertically from 0.02 to 0.08 of the body is likely to be head, from 0.3 to 0.4 of the body is likely to be torso, and from 0.7 to 0.8 of the body is likely to be legs. For simplicity, we model the color intensity instead of the pixel RGB color using non-parametric kernel density estimation. Given a sample set S ¼ fci g where i ¼ 1 . . . N and ci is a one dimensional vector since we use color intensity, the probability that a test pixel t is from the same distribution as the sample set is
PðtÞ ¼
3. Visual tracking algorithm
429
N 1 X t ci K Nr i¼1 r
As shown in Fig. 1, depending on the type of camera used, there are two ways to initialize a tracker associated to a single worker. Both a spatial model and a color model are needed for a kernel covariance tracking. To handle occlusion or interaction, there needs to be a multiple tracking management module. The tracking system’s output – workers’ trajectories would then be the input for safety monitoring or progress analysis. The following will illustrate the system in detail.
where the Gaussian kernel is used. Fig. 2 shows the segmentation results of different people. Even though a person’s body is described by three regions, only the torso and legs will be used for tracking because a person’s head appearance changes with viewing angle. After the segmentation, the color distribution of a person’s torso and legs is learned with a certain distribution assumption. In our experiments, the multivariate Gaussian distribution is used for the RGB channels.
3.1. Online color model learning
3.2. Kernel covariance tracking
Both stationary and pan-tilt-zoom (PTZ) cameras are common for video surveillance. There are different ways to handle their output video. For the stationary camera, a background model is learned and workers are detected by background subtraction
Covariance tracking has been shown to provide good general purpose tracking capabilities [16]. This section describes a nonlinear extension to the covariance tracking, called kernel covariance tracking and detailed in [17].
Fig. 1. Flow chart depicting the detection and tracking algorithm.
430
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
bk ¼
N N X X aki pffiffiffiffi ffi kðu; ui Þ ¼ wki kðu; ui Þ: kk i¼1 i¼1
In practice, the KPCA procedure retains only nr < N of the eigenvectors which results in (i) a more compact representation, and (ii) noise rejection. Performing the projection for the nr eigenvectors kept, results in the transformation of the test point u 2 Rnf into the finite-dimensional, reduced Hilbert space b 2 Hnr . The procedure is depicted in Fig. 3, which shows the point u 2 Rnf being mapped to a 2 H, then projected to the reduced Hilbert space b 2 Hnr . When specifying that b arises from the nonlinear projection of u, we will write it as bðuÞ. The mapping from the input space to the KPCA space has one critical benefit. If the test point u agrees with the original set of training data it will align with one or more of the main nonlinear covariant directions, and will thus have maximal magnitude. If the test point does not match the training data, then it will map closer to the origin, and will have a small magnitude. As regards the KPCA space, a test point that agrees with the learned data will map onto the surface of a hyper ellipse in Hnr , whereas one that does not will map close to the origin. Fig. 2. Segmentation results: left image depicts intensity image and right image depicts the resulting segmentation.
3.2.1. Target representation Let f ðxÞ be the np -dimensional feature vector extracted from image I at the spatial location x,
f ðxÞ ¼ CðI; xÞ where C can be any image-to-feature mapping encoding such as color, image gradient, edge, texture, etc. The joint feature-spatial vector ui with dimension nf ¼ np þ 2 is constructed by concatenating the feature and the spatial data xi ; ui ¼ ½f ðxi Þ; xi T . Given N samP ples from a target region R with centered observations Ni¼1 ui ¼ 0, the covariance matrix C is
C¼
N X
Fig. 3. Depiction of KPCA projection.
ui uTi ;
i¼1
which measures the linear dependence between the feature and the spatial values of the samples. Linear dependence may not characterize all of the structure in a given data set. We propose to use the nonlinear covariance matrix obtained by first nonlinearly mapping the target samples into a higher-dimensional Hilbert space using a nonlinear mapping as
CH ¼
N 1X /ðui Þ/ðui ÞT : N i¼1
The nonlinear covariance matrix is used to define a reduced-dimension transformation via Kernel Principal Component Analysis (KPCA) [18]. To perform KPCA, the Gaussian Mercer kernel is employed. A Mercer kernel is a function kðui ; uj Þ allowing for the computation of the inner product in the Hilbert space without computing the mapping / explicitly. Mathematically speaking,
1 h/ðui Þ; /ðuj Þi ¼ kðui ; uj Þ ¼ exp uTi R1 uj ; 2 where R is a diagonal matrix of bandwidths for each feature-spatial entry, rf1 ; . . . ; rfp , rs1 , rs2 .The eigenvectors ai and eigenvalues ki , for i ¼ 1 . . . N, of the covariance matrix C H are computed and the eigenvectors are normalized. The projection of a test point u 2 Rnf onto the kth normalized eigenvector of the nonlinear covariance matrix is given by
Fig. 4. Identifying consistent appearance models by extracting the appearance and spatial models. (a) Person, (b) template models, (c) parameterized appearance model fit to RGB data, (d) spatial model.
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
3.2.2. Target localization From the discussed properties of the KPCA mapping, the squared distance from the origin indicates the amount a test point u aligns with the eigenvectors of the learned covariance matrix C H ,
JðuÞ ¼
nr X
bk ðuÞ
2
;
k¼1
where nr is the number of eigenvectors retained. Summing this calculation for all the test points falling within a region R results in a measure of how well, as a whole, the region R resembles the target model. Thus a measure of region similarity is
JðRÞ ¼
X
JðuÞ:
u2R
A local search for the optimal placement of the target region is accomplished through gradient descent. In our case, the mean shift procedure provides a good estimate of the best gradient step [19]. The update location is given by M P P
^x ¼
u2R^x k¼1
bk ðuÞ
N P i¼1
wki kðu; ui Þðx xi Þ
JðR^x Þ
:
3.2.3. Tracking refinement To ensure temporally consistent tracking of the personnel, a second-order constant velocity Kalman filter is instantiated for
431
each person being tracked. This has the effect of smoothing out the tracking signal and providing dynamically consistent predictions for the gradient descent template search procedure. Also, for the PTZ camera case, the movement between image frames can effect the stability of localization. A Kanade–Lucas– Tomasi (KLT) feature based registration method has been applied for the movement compensation from frame to frame [20]. 3.3. Offline spatial model training and parametrized feature function Since construction workers have very consistent geometry, it is possible to learn one target model if the varying appearance model can be characterized. One manner to achieve this is to have a parameterized feature function,
f ðxÞ ¼ CðTðxÞ; x; qÞ; where q is a parameter vector that alters the feature function and its selectivity. In our case, the feature function specifically targets the appearance model associated to the upper and lower body parts. Fig. 4 depicts such a procedure. Given a person, Fig. 4(a), and several templates of the person in different poses, Fig. 4(b), an appearance model can be generated. For example, using probabilistic PCA (PPCA) [21], an additive Gaussian mixture model can be found for the template appearance model, Fig. 4(c). The Gaussian mixture model parameters become the parameters, q of the feature function Cð:; :; ;qÞ. The feature function essentially generates a seg-
Fig. 5. Flow chart depicting the multiple tracking management module.
Fig. 6. Artificial scenario (the right most figure shows the comparison between tracking result and ground truth, the tracking signal is in black/red/blue line and the ground truth is in green dotted line. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
432
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
Fig. 7. Construction scenario 1.
mentation of the template images as in Fig. 4(d). The segmentations are then used as the model templates to learn. When given a new person to track, the algorithm automatically determines the Gaussian mixture model parameters to use in the feature function. 3.4. Multiple tracking management Considering the fact that most construction sites involve collaborative work, interactions between workers, workers and machines will happen frequently. The interactions will introduce static or dynamic occlusion, which causes difficulties for tracking. Occlusion handling is still an open problem in visual tracking. However, by tracking each worker, it is possible to use global information to tackle occlusion. Hence there is a management module in our scheme who can control every single tracking’s behavior
globally and make them collaborate with each other to reach the optimal performance. The management flow is illustrated in Fig. 5. Based on the assumption that every worker is on the same flat ground plane, we can get the depth ordering by comparing every worker’s location. The lower in the image the worker is located, the closer the worker is to the camera, and thus the more likely this worker will occlude other workers. The multiple track management module takes the output location of each single tracker and computes the presumed depth ordering. For the next frame of tracking, the occlusion evaluation module will evaluate the occlusion level between different workers according to the tracking result from the previous frame. No occlusion, partial occlusion or severe occlusion are the three possible consequences. For the no occlusion case, each worker will be tracked individually without any special procedure. For partial occlusion case, the occlusion seg-
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
mentation module will segment the workers under occlusion by their color models. The worker closer to the camera will be segmented and tracked first. The pixels belonging to this worker will be labeled and removed from consideration during the tracking procedure for the other workers. In this way, an occluded worker’s tracking module will not utilize the occluding worker’s information for localization. Such a measure efficiently prevents the loss of track or identity switches. For more details on occlusion segmentation, see [22]. For the severe occlusion case, such as when a worker is completely occluded by another worker or an object, the management module will not rely on the localization step and will instead rely on the Kalman filter prediction. 4. Experimental results The proposed multiple workers tracking scheme has been applied to several scenarios. The first is an artificial scenario where two of the tracked people pass by each other. The second is a construction scenario with three sequences which monitor the exterior area around a building under construction. The third is a construction scenario with two sequences which are during the initial clearing operations associated to an open work-site. Only the first one was taken by a stationary camera. The rest were from a PTZ camera. A visual comparison shows that the real construction video is noisier and more complicated. Except the artificial scenario, all the construction sequences use the same spatial model learned from a worker in the second scenario. Five templates with differing walking poses were chosen to train the spatial model offline. The covariance matrix formed by the extracted feature vectors was learned using KPCA and the top five eigenvectors were retained. Figs. 6–8 contain snapshots of individual frames associated to the tracking task. Within each
433
frame is depicted the trajectories of the tracked individuals for some time period prior to the depicted frame. As can be seen, the general spatial model works robustly for every worker to track. The artificial sequence contains both trajectory crossing and severe occlusion. In Fig. 6, the first frame depicts the situation where one person is occluded by another person due to a trajectory crossing. The second frame shows the trajectories after the both persons cross paths. In the third frame, the person with the red shirt walks under the tree. The majority of his body is occluded by the leaves, however, the tracking algorithm can still follow the person. The second scenario includes cases where the trajectories of personnel were overlapping and their appearances were similar. In Fig. 7(a), for example, the two workers cross each others’ paths. Both persons wear reflective (green polychloroprene) vests that were both severely occluded in the work process. Fig. 7(b) depicts a situation where one worker crosses the paths of two workers cooperating on a task. One of them walks away from the camera and two of them walk towards the camera resulting in a scale change. The proposed tracking algorithm can handle the scale change well. As seen in Fig. 7(c), both workers that wear the same type of safety vest walk close by each other and cross paths. From their trajectories, we can see that the algorithm did not lose track nor switch tracks in spite of the similar appearance (safety vest). In the last scenario, Fig. 8(a) depicts the tracking results of two workers joining up then walking together. Later, it can be seen that several trajectories overlap at some time during the observation of the construction workers. Fig. 8(b) contains a later sequence where three people meet up and walk off together. From all the shown results, our multiple workers tracking scheme works robustly for the complex situations, like worker interaction, occlusions, and scale change.
Fig. 8. Construction scenario 2.
434
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
Table 1 Table of tracking error metrics and target size (height width). All units are in pixels. Lower performance values are better. The star above the target ID means this target experienced occlusion. Scenario
Target ID
Target size
L2 error
L1 error
Artificial scenario
1 2 3
76 56 54 38 66 32
7.54 3.40 8.45
18.18 7.63 16.82
Scenario 1(a)
1 2
46 18 44 22
5.00 2.61
17.92 9.20
Scenario 1(b)
1 2 3
48 22 48 22 46 22
2.68 2.41 2.91
4.72 6.46 7.31
Scenario 1(c)
1 2
28 18 28 18
3.15 3.12
6.77 9.37
Scenario 2(a)
1 2
40 20 40 20
2.78 6.07
5.73 24.01
Scenario 2(b)
1 2 3
28 18 28 18 28 18
5.09 5.13 2.50
8.74 21.64 8.91
4.1. Performance evaluation Table 1 contains the performance results of all the scenarios shown above. The ground truth was generated manually. The L2 and L1 errors were computed between the given tracking signal and the ground truth, which is denoted by x ðÞ; y ðÞ. The L2 error is
jjejj2 ¼
N 1X ððxðkÞ x ðkÞÞ2 þ ðyðkÞ y ðkÞÞ2 Þ N k¼1
!12
whereas the L1 error is 1
jjejj1 ¼ maxððxðkÞ x ðkÞÞ2 þ ðyðkÞ y ðkÞÞ2 Þ2 k
The L2 error represents the average tracking error. And the L1 error represents the worst case tracking error, which is usually caused by occlusion. Lower values indicate better performance. 4.2. System failure In spite of the reliable performance in the above scenarios, there are some hard situations where our algorithm might fail. First, when the worker’s appearance changes too much because of changing illumination or other reasons, the algorithm might lose track. As shown in Fig. 7(c), when the worker walks into the building and is strongly shadowed, the tracking will eventually fail. One possible solution to this is to update the color model using KPCA incrementally. This is reserved for future effort. Second, for severe occlusion, since the localization is only based on Kalman filter prediction without observation, if the worker undergoes severe occlusion for too many frames, the algorithm might lose track. For static-camera application, it might be possible to recover from lost tracking with the help of background model. Attempting to track a person that is not visible is a difficult situation. 5. Conclusion This paper proposed a robust multiple workforces tracking scheme using either stationary or pan-tilt-zoom cameras. The tracking procedure is based on machine learning. It utilizes kernel principle component analysis to identify statistically relevant joint feature-spatial relationships, which is then used to define an
appearance model for construction personnel. Using a parameterized appearance feature function uniquely estimated for each person, the tracking algorithm is able to track multiple personnel in a given video sequence. Future work seeks to compare visual tracking algorithm with alternative sensor strategies, such as Ultra Wideband (UWB) [23] or Radio Detection and Ranging (RADAR). A Robotic Total Station will be used to serve as reference signal. Further data fusion work between different sensors may give highly reliable data for safety monitoring or progress analysis purposes. References [1] O. Arif, P.A. Vela, Kernel covariance image region description for object tracking, in: IEEE International Conference on Image Processing, 2009. [2] R. Navon, R. Sacks, Assessing research issues in automated project performance control (APPC), Automation in Construction 16 (4) (2007) 474–484. [3] M. Golparvar-Fard, S. Savarese, F. Peña-Mora, Interactive Visual Construction Progress Monitoring with D4R-4D Augmented Reality Models, American Society of Civil Engineers, 2009. [4] C. Gordon, F. Boukamp, D. Huber, E. Latimer, K. Park, B. Akinci, Combining reality capture technologies for construction defect detection: a case study, in: EIA9: E-Activities and Intelligent Support in Design and the Built Environment, 9th EuropIA International Conference, 2003, pp. 99–108. [5] V. Kamat, J. Martinez, et al., Large-scale dynamic terrain in three-dimensional construction process visualizations, Journal of Computing in Civil Engineering 19 (2005) 160. [6] E. Trucco, A. Kaka, A framework for automatic progress assessment on construction sites using computer vision, International Journal of IT in Architecture, Engineering and Construction 2 (2004) 147–164. [7] T. Lukins, E. Trucco, Towards automated visual assessment of progress in construction projects, in: Proceedings of the British Machine Vision Conference, Coventry, 2007, pp. 142–151. [8] N. Choi, H. Son, C. Kin, Rapid 3D object recognition for automatic project progress monitoring using a stereo vision system, in: The 25th International Symposium on Automation and Robotics in Construction, 2009. [9] A. Peddi, L. Huan, Y. Bai, S. Kim, Development of human pose analyzing algorithms for the determination of construction productivity in real-time, in: Construction Research Congress, ASCE, Seattle, WA, 2009, pp. 11–20. [10] J. Gong, C.H. Caldas, A computer vision based video interpretation model for automated productivity analysis of construction operations, Journal of Computing in Civil Engineering (2009) preprint. [11] F. Cordova, I. Brilakis, On-site 3D vision tracking of construction personnel, in: Conference of the International Group for Lean Construction Management, 2008, pp. 809–820. [12] J. Teizer, P. Vela, Personnel tracking on construction sites using video cameras, Advanced Engineering Informatics 23 (4) (2009) 452–462. [13] A. Yilmaz, O. Javed, M. Shah, Object tracking: a survey, ACM Computing Surveys 38 (4) (2007) 1–45. [14] M. Enzweiler, D. Gavrila, Monocular pedestrian detection: Survey and experiments, IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (12) (2009) 2179–2195. [15] A. Elgammal, L. Davis, Probabilistic framework for segmenting people under occlusion, in: International Conference on Computer Vision, vol. 1, 2001, pp. 781–788. [16] F. Porikli, O. Tuzel, P. Meer, Covariance tracking using model update based on Lie algebra, in: IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 728–735. [17] O. Arif, P. Vela, Non-rigid object localization and segmentation using eigenspace representation, in: IEEE International Conference on Computer Vision, 2009. [18] B. Scholkopf, A. Smola, K. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation 10 (5) (1998) 1299–1319. [19] D. Comaniciu, V. Ramesh, P. Meer, Kernel-based object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (5) (2003) 564–577. [20] B. Lucas, T. Kanade, An iterative image registration technique with an application to stereo vision, in: Proceedings of the DARPA Image Understanding Workshop, 1981, pp. 121–130. [21] M. Tipping, C. Bishop, Probabilistic principal component analysis, Journal of the Royal Statistical Society Series B (Statistical Methodology) 61 (3) (1999) 611–622. [22] J. Yang, P.A. Vela, Z. Shi, J. Teizer, Probabilistic multiple people tracking through complex situations, in: 11th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, 2009. [23] J. Teizer, M. Venugopal, A. Walia, Ultrawideband for automated real-time three-dimensional location sensing for workforce, equipment, and material positioning and tracking, Transportation Research Record: Journal of the Transportation Research Board 2081 (2008) 56–64.
Contents lists available at ScienceDirect
Advanced Engineering Informatics journal homepage: www.elsevier.com/locate/aei
Tracking multiple workers on construction sites using video cameras J. Yang a,b, O. Arif b, P.A. Vela b, J. Teizer c,*, Zhongke Shi a a
College of Automation, Northwestern Polytechnical University, China School of Electrical and Computer Engineering, Georgia Tech, Atlanta, GA 30332, USA c School of Civil and Environmental Engineering, Georgia Tech, Atlanta, GA 30332, USA b
a r t i c l e
i n f o
Article history: Received 16 June 2010 Accepted 23 June 2010 Available online 31 July 2010 Keywords: Construction Worker Monitoring Tracking Video surveillance
a b s t r a c t This paper proposes a tracking scheme for tracking multiple workers on construction sites using video cameras. Prior work has compared several contemporary tracking algorithms on construction sites and identified several difficulties, one of which included the existence of interacting workforce. In order to address the challenge of multiple workers within the camera’s field of view, the authors have developed a tracking algorithm based upon machine learning methods. The algorithm requires several sample templates of the tracking target and learns a general model that can be applied to other targets with similar geometry. A parameterized feature bank is proposed to handle the case of variable appearance content. The tracking initialization has been discussed for different types of video cameras. A multiple tracking management module is applied to optimize the system. The principal objective of this paper is to test and demonstrate the feasibility of tracking multiple workers from statically placed and dynamically moving cameras. Published by Elsevier Ltd.
1. Introduction In recent years, arising from the affordable price of video cameras, the placement of cameras on construction sites for surveillance purposes is becoming more commonplace. If monitoring is performed at all, a worker or manager will be tasked to review or monitor the video, which is a time and money consuming task. Alternatively, the imagery will only be utilized to corroborate events after the fact should such a need arise. With the development of computer vision algorithms capable of processing the video stream, an intelligent construction site visual surveillance system would be feasible. Such a system could supply information for workforce productivity analysis, workforce training, and safety monitoring. However, little work has been done for these special applications mainly because of the complex and uncontrolled environment in any construction site. This paper presents a robust multiple workers tracking scheme for the construction site, which is one of the core components in a visual surveillance system. The tracking scheme extends previous work [1]. Considering the similarity among construction workers, a machine learning algorithm is used to train a general spatial model for tracking, which can be rapidly specialized to the appearance model of a given worker. To handle complex situations like
* Corresponding author. Tel.: +1 404 894 8269. E-mail addresses: [email protected] (J. Yang), [email protected] (O. Arif), [email protected] (P.A. Vela), [email protected] (J. Teizer), [email protected] (Z. Shi). 1474-0346/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.aei.2010.06.008
occlusions or interactions, a multiple tracking management module is applied. Initialization methods for static versus dynamic cameras are discussed. The remainder of the paper is organized as follows: a review of related work is given in Section 2, the description of the proposed tracking scheme follows in Section 3, with experimental results in Section 4, and a conclusion in Section 5. 2. Background and motivation Construction safety is a key motivation for this research since the United States has had more than 1100 construction related fatalities in each of the past 16 years. An additional reason for this research is to analyze the work that construction personnel perform on-site. Many management related duties, such as tracking the position, activity, and status of construction resources (personnel, equipment, material), are not in near- nor in real-time, thus limiting the awareness of project status in project management decision making at construction sites. Automated tracking of vital construction resources needs to be realized for timely feedback of construction progress [2]. While much research is devoted to analyzing the application of sensor technologies to the construction site [2], such as Global Positioning Systems or Radio Frequency ID technologies, this paper focuses on visual surveillance systems, which are becoming more common on the work-site. Vision-based monitoring research has principally investigated the use of vision for comparing as-planned versus as-built infrastructure, defect detection, and structural modelling in augmented
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
reality [3–5]. Trucco and Kaka [6] proposed an algorithm for automatically locating objects in unconstrained images of building sites and sketched a computational framework to capture and measure construction progress automatically from images taken on site. Lukins and Trucco [7] proposed a system capable of automatically detecting changes on a building site observed by a fixed camera, and to identify such changes as parts of the construction plan. Choi et al. [8] used a stereo vision system of 3D object recognition to extract specific features for comparison with pre-planned 3D CAD data for progress monitoring. Such works mostly seek to use vision for improved spatial awareness of construction work-site conditions or progress. Furthermore, these methods analyze individual images. Research dealing with video sequences are closer to the research described herein. Peddi et al. [9] proposed a system using blob tracking, pose extraction and pose classification to judge the productivity of the worker by classifying different poses into categories such as effective, ineffective, and contributory. Gong and Caldas [10] presented a study on developing a video interpretation model to interpret videos of construction operations automatically into productivity information. It focused on the detection and tracking of project resources as well as work state classifications and abnormal production scenario identifications. Most related to this paper is the work in Cordova and Brilakis [11], who detailed a static-camera, stereo-based tracking method for construction work-sites. However, the tracking of multiple, interacting personnel is not discussed. In prior work, we discussed the possibility and need for tracking workforce on construction jobsites using video cameras. A comparison among four tracking algorithms and evaluation of their associated results were presented [12]. One pressing need was the ability to track multiple workers with interacting trajectories and similar appearance models. This paper extends existing research efforts to the case of multiple, interacting, and occluding workforce.
[13]. For the PTZ camera, since a consistent background model cannot be generated, a people detector is needed. For people detection algorithm, please refer to [14]. Assuming the existence of a worker’s initial template from the detection module, to relate it with the spatial model, a segmentation method to distinguish the upper and lower body is needed. Inspired by Elgammal and Davis [15], the body is divided into three blobs – head, torso and legs. Each blob is modeled by its own color density and spatial density functions. Assuming every pixel inside every blob has the same color distribution and the vertical location of a blob inside the body is independent of its horizontal location, the joint distribution of pixel located at x with color c given blob A can be expressed as:
PA ðx; cÞ ¼ PA;spatial ðxÞPA;color ðcÞ where PA;spatial ðxÞ is the spatial distribution and P A;color ðcÞ is the color distribution. A pixel X ¼ ðx; cÞ can be classified to one of the three body blobs (head, torso, legs) by maximizing the joint distribution
X 2 Ak ;
such that k ¼ argk max PðXjAk Þ
where k ¼ 1; 2; 3. To learn the color and spatial models, samples pixels are extracted from the high-confidence regions associated to each blob. For a person in upright pose, the confident regions for the head, torso, and legs is chosen empirically. In our experiment, the confidence regions parameter matrix is [0.02, 0.08; 0.3, 0.4; 0.7, 0.8], which means vertically from 0.02 to 0.08 of the body is likely to be head, from 0.3 to 0.4 of the body is likely to be torso, and from 0.7 to 0.8 of the body is likely to be legs. For simplicity, we model the color intensity instead of the pixel RGB color using non-parametric kernel density estimation. Given a sample set S ¼ fci g where i ¼ 1 . . . N and ci is a one dimensional vector since we use color intensity, the probability that a test pixel t is from the same distribution as the sample set is
PðtÞ ¼
3. Visual tracking algorithm
429
N 1 X t ci K Nr i¼1 r
As shown in Fig. 1, depending on the type of camera used, there are two ways to initialize a tracker associated to a single worker. Both a spatial model and a color model are needed for a kernel covariance tracking. To handle occlusion or interaction, there needs to be a multiple tracking management module. The tracking system’s output – workers’ trajectories would then be the input for safety monitoring or progress analysis. The following will illustrate the system in detail.
where the Gaussian kernel is used. Fig. 2 shows the segmentation results of different people. Even though a person’s body is described by three regions, only the torso and legs will be used for tracking because a person’s head appearance changes with viewing angle. After the segmentation, the color distribution of a person’s torso and legs is learned with a certain distribution assumption. In our experiments, the multivariate Gaussian distribution is used for the RGB channels.
3.1. Online color model learning
3.2. Kernel covariance tracking
Both stationary and pan-tilt-zoom (PTZ) cameras are common for video surveillance. There are different ways to handle their output video. For the stationary camera, a background model is learned and workers are detected by background subtraction
Covariance tracking has been shown to provide good general purpose tracking capabilities [16]. This section describes a nonlinear extension to the covariance tracking, called kernel covariance tracking and detailed in [17].
Fig. 1. Flow chart depicting the detection and tracking algorithm.
430
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
bk ¼
N N X X aki pffiffiffiffi ffi kðu; ui Þ ¼ wki kðu; ui Þ: kk i¼1 i¼1
In practice, the KPCA procedure retains only nr < N of the eigenvectors which results in (i) a more compact representation, and (ii) noise rejection. Performing the projection for the nr eigenvectors kept, results in the transformation of the test point u 2 Rnf into the finite-dimensional, reduced Hilbert space b 2 Hnr . The procedure is depicted in Fig. 3, which shows the point u 2 Rnf being mapped to a 2 H, then projected to the reduced Hilbert space b 2 Hnr . When specifying that b arises from the nonlinear projection of u, we will write it as bðuÞ. The mapping from the input space to the KPCA space has one critical benefit. If the test point u agrees with the original set of training data it will align with one or more of the main nonlinear covariant directions, and will thus have maximal magnitude. If the test point does not match the training data, then it will map closer to the origin, and will have a small magnitude. As regards the KPCA space, a test point that agrees with the learned data will map onto the surface of a hyper ellipse in Hnr , whereas one that does not will map close to the origin. Fig. 2. Segmentation results: left image depicts intensity image and right image depicts the resulting segmentation.
3.2.1. Target representation Let f ðxÞ be the np -dimensional feature vector extracted from image I at the spatial location x,
f ðxÞ ¼ CðI; xÞ where C can be any image-to-feature mapping encoding such as color, image gradient, edge, texture, etc. The joint feature-spatial vector ui with dimension nf ¼ np þ 2 is constructed by concatenating the feature and the spatial data xi ; ui ¼ ½f ðxi Þ; xi T . Given N samP ples from a target region R with centered observations Ni¼1 ui ¼ 0, the covariance matrix C is
C¼
N X
Fig. 3. Depiction of KPCA projection.
ui uTi ;
i¼1
which measures the linear dependence between the feature and the spatial values of the samples. Linear dependence may not characterize all of the structure in a given data set. We propose to use the nonlinear covariance matrix obtained by first nonlinearly mapping the target samples into a higher-dimensional Hilbert space using a nonlinear mapping as
CH ¼
N 1X /ðui Þ/ðui ÞT : N i¼1
The nonlinear covariance matrix is used to define a reduced-dimension transformation via Kernel Principal Component Analysis (KPCA) [18]. To perform KPCA, the Gaussian Mercer kernel is employed. A Mercer kernel is a function kðui ; uj Þ allowing for the computation of the inner product in the Hilbert space without computing the mapping / explicitly. Mathematically speaking,
1 h/ðui Þ; /ðuj Þi ¼ kðui ; uj Þ ¼ exp uTi R1 uj ; 2 where R is a diagonal matrix of bandwidths for each feature-spatial entry, rf1 ; . . . ; rfp , rs1 , rs2 .The eigenvectors ai and eigenvalues ki , for i ¼ 1 . . . N, of the covariance matrix C H are computed and the eigenvectors are normalized. The projection of a test point u 2 Rnf onto the kth normalized eigenvector of the nonlinear covariance matrix is given by
Fig. 4. Identifying consistent appearance models by extracting the appearance and spatial models. (a) Person, (b) template models, (c) parameterized appearance model fit to RGB data, (d) spatial model.
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
3.2.2. Target localization From the discussed properties of the KPCA mapping, the squared distance from the origin indicates the amount a test point u aligns with the eigenvectors of the learned covariance matrix C H ,
JðuÞ ¼
nr X
bk ðuÞ
2
;
k¼1
where nr is the number of eigenvectors retained. Summing this calculation for all the test points falling within a region R results in a measure of how well, as a whole, the region R resembles the target model. Thus a measure of region similarity is
JðRÞ ¼
X
JðuÞ:
u2R
A local search for the optimal placement of the target region is accomplished through gradient descent. In our case, the mean shift procedure provides a good estimate of the best gradient step [19]. The update location is given by M P P
^x ¼
u2R^x k¼1
bk ðuÞ
N P i¼1
wki kðu; ui Þðx xi Þ
JðR^x Þ
:
3.2.3. Tracking refinement To ensure temporally consistent tracking of the personnel, a second-order constant velocity Kalman filter is instantiated for
431
each person being tracked. This has the effect of smoothing out the tracking signal and providing dynamically consistent predictions for the gradient descent template search procedure. Also, for the PTZ camera case, the movement between image frames can effect the stability of localization. A Kanade–Lucas– Tomasi (KLT) feature based registration method has been applied for the movement compensation from frame to frame [20]. 3.3. Offline spatial model training and parametrized feature function Since construction workers have very consistent geometry, it is possible to learn one target model if the varying appearance model can be characterized. One manner to achieve this is to have a parameterized feature function,
f ðxÞ ¼ CðTðxÞ; x; qÞ; where q is a parameter vector that alters the feature function and its selectivity. In our case, the feature function specifically targets the appearance model associated to the upper and lower body parts. Fig. 4 depicts such a procedure. Given a person, Fig. 4(a), and several templates of the person in different poses, Fig. 4(b), an appearance model can be generated. For example, using probabilistic PCA (PPCA) [21], an additive Gaussian mixture model can be found for the template appearance model, Fig. 4(c). The Gaussian mixture model parameters become the parameters, q of the feature function Cð:; :; ;qÞ. The feature function essentially generates a seg-
Fig. 5. Flow chart depicting the multiple tracking management module.
Fig. 6. Artificial scenario (the right most figure shows the comparison between tracking result and ground truth, the tracking signal is in black/red/blue line and the ground truth is in green dotted line. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
432
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
Fig. 7. Construction scenario 1.
mentation of the template images as in Fig. 4(d). The segmentations are then used as the model templates to learn. When given a new person to track, the algorithm automatically determines the Gaussian mixture model parameters to use in the feature function. 3.4. Multiple tracking management Considering the fact that most construction sites involve collaborative work, interactions between workers, workers and machines will happen frequently. The interactions will introduce static or dynamic occlusion, which causes difficulties for tracking. Occlusion handling is still an open problem in visual tracking. However, by tracking each worker, it is possible to use global information to tackle occlusion. Hence there is a management module in our scheme who can control every single tracking’s behavior
globally and make them collaborate with each other to reach the optimal performance. The management flow is illustrated in Fig. 5. Based on the assumption that every worker is on the same flat ground plane, we can get the depth ordering by comparing every worker’s location. The lower in the image the worker is located, the closer the worker is to the camera, and thus the more likely this worker will occlude other workers. The multiple track management module takes the output location of each single tracker and computes the presumed depth ordering. For the next frame of tracking, the occlusion evaluation module will evaluate the occlusion level between different workers according to the tracking result from the previous frame. No occlusion, partial occlusion or severe occlusion are the three possible consequences. For the no occlusion case, each worker will be tracked individually without any special procedure. For partial occlusion case, the occlusion seg-
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
mentation module will segment the workers under occlusion by their color models. The worker closer to the camera will be segmented and tracked first. The pixels belonging to this worker will be labeled and removed from consideration during the tracking procedure for the other workers. In this way, an occluded worker’s tracking module will not utilize the occluding worker’s information for localization. Such a measure efficiently prevents the loss of track or identity switches. For more details on occlusion segmentation, see [22]. For the severe occlusion case, such as when a worker is completely occluded by another worker or an object, the management module will not rely on the localization step and will instead rely on the Kalman filter prediction. 4. Experimental results The proposed multiple workers tracking scheme has been applied to several scenarios. The first is an artificial scenario where two of the tracked people pass by each other. The second is a construction scenario with three sequences which monitor the exterior area around a building under construction. The third is a construction scenario with two sequences which are during the initial clearing operations associated to an open work-site. Only the first one was taken by a stationary camera. The rest were from a PTZ camera. A visual comparison shows that the real construction video is noisier and more complicated. Except the artificial scenario, all the construction sequences use the same spatial model learned from a worker in the second scenario. Five templates with differing walking poses were chosen to train the spatial model offline. The covariance matrix formed by the extracted feature vectors was learned using KPCA and the top five eigenvectors were retained. Figs. 6–8 contain snapshots of individual frames associated to the tracking task. Within each
433
frame is depicted the trajectories of the tracked individuals for some time period prior to the depicted frame. As can be seen, the general spatial model works robustly for every worker to track. The artificial sequence contains both trajectory crossing and severe occlusion. In Fig. 6, the first frame depicts the situation where one person is occluded by another person due to a trajectory crossing. The second frame shows the trajectories after the both persons cross paths. In the third frame, the person with the red shirt walks under the tree. The majority of his body is occluded by the leaves, however, the tracking algorithm can still follow the person. The second scenario includes cases where the trajectories of personnel were overlapping and their appearances were similar. In Fig. 7(a), for example, the two workers cross each others’ paths. Both persons wear reflective (green polychloroprene) vests that were both severely occluded in the work process. Fig. 7(b) depicts a situation where one worker crosses the paths of two workers cooperating on a task. One of them walks away from the camera and two of them walk towards the camera resulting in a scale change. The proposed tracking algorithm can handle the scale change well. As seen in Fig. 7(c), both workers that wear the same type of safety vest walk close by each other and cross paths. From their trajectories, we can see that the algorithm did not lose track nor switch tracks in spite of the similar appearance (safety vest). In the last scenario, Fig. 8(a) depicts the tracking results of two workers joining up then walking together. Later, it can be seen that several trajectories overlap at some time during the observation of the construction workers. Fig. 8(b) contains a later sequence where three people meet up and walk off together. From all the shown results, our multiple workers tracking scheme works robustly for the complex situations, like worker interaction, occlusions, and scale change.
Fig. 8. Construction scenario 2.
434
J. Yang et al. / Advanced Engineering Informatics 24 (2010) 428–434
Table 1 Table of tracking error metrics and target size (height width). All units are in pixels. Lower performance values are better. The star above the target ID means this target experienced occlusion. Scenario
Target ID
Target size
L2 error
L1 error
Artificial scenario
1 2 3
76 56 54 38 66 32
7.54 3.40 8.45
18.18 7.63 16.82
Scenario 1(a)
1 2
46 18 44 22
5.00 2.61
17.92 9.20
Scenario 1(b)
1 2 3
48 22 48 22 46 22
2.68 2.41 2.91
4.72 6.46 7.31
Scenario 1(c)
1 2
28 18 28 18
3.15 3.12
6.77 9.37
Scenario 2(a)
1 2
40 20 40 20
2.78 6.07
5.73 24.01
Scenario 2(b)
1 2 3
28 18 28 18 28 18
5.09 5.13 2.50
8.74 21.64 8.91
4.1. Performance evaluation Table 1 contains the performance results of all the scenarios shown above. The ground truth was generated manually. The L2 and L1 errors were computed between the given tracking signal and the ground truth, which is denoted by x ðÞ; y ðÞ. The L2 error is
jjejj2 ¼
N 1X ððxðkÞ x ðkÞÞ2 þ ðyðkÞ y ðkÞÞ2 Þ N k¼1
!12
whereas the L1 error is 1
jjejj1 ¼ maxððxðkÞ x ðkÞÞ2 þ ðyðkÞ y ðkÞÞ2 Þ2 k
The L2 error represents the average tracking error. And the L1 error represents the worst case tracking error, which is usually caused by occlusion. Lower values indicate better performance. 4.2. System failure In spite of the reliable performance in the above scenarios, there are some hard situations where our algorithm might fail. First, when the worker’s appearance changes too much because of changing illumination or other reasons, the algorithm might lose track. As shown in Fig. 7(c), when the worker walks into the building and is strongly shadowed, the tracking will eventually fail. One possible solution to this is to update the color model using KPCA incrementally. This is reserved for future effort. Second, for severe occlusion, since the localization is only based on Kalman filter prediction without observation, if the worker undergoes severe occlusion for too many frames, the algorithm might lose track. For static-camera application, it might be possible to recover from lost tracking with the help of background model. Attempting to track a person that is not visible is a difficult situation. 5. Conclusion This paper proposed a robust multiple workforces tracking scheme using either stationary or pan-tilt-zoom cameras. The tracking procedure is based on machine learning. It utilizes kernel principle component analysis to identify statistically relevant joint feature-spatial relationships, which is then used to define an
appearance model for construction personnel. Using a parameterized appearance feature function uniquely estimated for each person, the tracking algorithm is able to track multiple personnel in a given video sequence. Future work seeks to compare visual tracking algorithm with alternative sensor strategies, such as Ultra Wideband (UWB) [23] or Radio Detection and Ranging (RADAR). A Robotic Total Station will be used to serve as reference signal. Further data fusion work between different sensors may give highly reliable data for safety monitoring or progress analysis purposes. References [1] O. Arif, P.A. Vela, Kernel covariance image region description for object tracking, in: IEEE International Conference on Image Processing, 2009. [2] R. Navon, R. Sacks, Assessing research issues in automated project performance control (APPC), Automation in Construction 16 (4) (2007) 474–484. [3] M. Golparvar-Fard, S. Savarese, F. Peña-Mora, Interactive Visual Construction Progress Monitoring with D4R-4D Augmented Reality Models, American Society of Civil Engineers, 2009. [4] C. Gordon, F. Boukamp, D. Huber, E. Latimer, K. Park, B. Akinci, Combining reality capture technologies for construction defect detection: a case study, in: EIA9: E-Activities and Intelligent Support in Design and the Built Environment, 9th EuropIA International Conference, 2003, pp. 99–108. [5] V. Kamat, J. Martinez, et al., Large-scale dynamic terrain in three-dimensional construction process visualizations, Journal of Computing in Civil Engineering 19 (2005) 160. [6] E. Trucco, A. Kaka, A framework for automatic progress assessment on construction sites using computer vision, International Journal of IT in Architecture, Engineering and Construction 2 (2004) 147–164. [7] T. Lukins, E. Trucco, Towards automated visual assessment of progress in construction projects, in: Proceedings of the British Machine Vision Conference, Coventry, 2007, pp. 142–151. [8] N. Choi, H. Son, C. Kin, Rapid 3D object recognition for automatic project progress monitoring using a stereo vision system, in: The 25th International Symposium on Automation and Robotics in Construction, 2009. [9] A. Peddi, L. Huan, Y. Bai, S. Kim, Development of human pose analyzing algorithms for the determination of construction productivity in real-time, in: Construction Research Congress, ASCE, Seattle, WA, 2009, pp. 11–20. [10] J. Gong, C.H. Caldas, A computer vision based video interpretation model for automated productivity analysis of construction operations, Journal of Computing in Civil Engineering (2009) preprint. [11] F. Cordova, I. Brilakis, On-site 3D vision tracking of construction personnel, in: Conference of the International Group for Lean Construction Management, 2008, pp. 809–820. [12] J. Teizer, P. Vela, Personnel tracking on construction sites using video cameras, Advanced Engineering Informatics 23 (4) (2009) 452–462. [13] A. Yilmaz, O. Javed, M. Shah, Object tracking: a survey, ACM Computing Surveys 38 (4) (2007) 1–45. [14] M. Enzweiler, D. Gavrila, Monocular pedestrian detection: Survey and experiments, IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (12) (2009) 2179–2195. [15] A. Elgammal, L. Davis, Probabilistic framework for segmenting people under occlusion, in: International Conference on Computer Vision, vol. 1, 2001, pp. 781–788. [16] F. Porikli, O. Tuzel, P. Meer, Covariance tracking using model update based on Lie algebra, in: IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 728–735. [17] O. Arif, P. Vela, Non-rigid object localization and segmentation using eigenspace representation, in: IEEE International Conference on Computer Vision, 2009. [18] B. Scholkopf, A. Smola, K. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation 10 (5) (1998) 1299–1319. [19] D. Comaniciu, V. Ramesh, P. Meer, Kernel-based object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (5) (2003) 564–577. [20] B. Lucas, T. Kanade, An iterative image registration technique with an application to stereo vision, in: Proceedings of the DARPA Image Understanding Workshop, 1981, pp. 121–130. [21] M. Tipping, C. Bishop, Probabilistic principal component analysis, Journal of the Royal Statistical Society Series B (Statistical Methodology) 61 (3) (1999) 611–622. [22] J. Yang, P.A. Vela, Z. Shi, J. Teizer, Probabilistic multiple people tracking through complex situations, in: 11th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, 2009. [23] J. Teizer, M. Venugopal, A. Walia, Ultrawideband for automated real-time three-dimensional location sensing for workforce, equipment, and material positioning and tracking, Transportation Research Record: Journal of the Transportation Research Board 2081 (2008) 56–64.