- Email: [email protected]
Automatic tracking of neural stem cells in sequential digital images
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.elsevier.com/locate/bbe
Original Research Article
Automatic tracking of neural stem cells in sequential digital images Tao Zhang a, Wenjing Jia c, Yuemin Zhu b, Jie Yang a,* a
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China Centre National de la Recherche Scientifique of France, Villeurbanne, France c Faculty of Engineering and Information Technology, University of Technology, Australia b
article info
abstract
Article history:
Neural stem cells are the cells that give rise to the main cell types of the nervous system.
Received 21 September 2014
Due to their varying size and shape, and random movement, the tracking of these cells in
Received in revised form
suspension in video sequences is challenging. This paper develops an automatic tracking
21 May 2015
system for neural stem cells. The system first detects and localizes cells in the image
Accepted 7 October 2015
sequence, followed by a feature extraction step for the subsequent cell tracking. Then,
Available online xxx
the system tracks inactive cells using an improved mean shift algorithm, divisive cells through a context-based technique, and active cells by means of dynamic local prediction
Keywords:
(DLP) and gray prediction (GP) algorithms. Experimental results show that the proposed
Cell tracking
system not only improves the accuracy of fast moving tracking, but also constructs
Improved mean shift
accurately the trajectories of the cell movement and reduces the iterations during the
Dynamic local prediction (DLP)
center searching.
Gray prediction (GP)
# 2015 Nałęcz Institute of Biocybernetics and Biomedical Engineering. Published by
Neural stem cells
1.
Introduction
The discovery of neural stem cells is a significant progress in the field of bioscience at the end of the last century. Neural stem cells hide in certain parts of nervous system. Some of these cells stay in original state, some have the potential to divide with multi-aspect, and others have regeneration ability [1,2]. Due to the good plasticity of neural stem cells, we can make it as a therapy tool of nerve damage through genetic or
Elsevier Sp. z o.o. All rights reserved.
cell engineering. Researchers have observed that new nerve cells are produced by neural stem cells in the brain of adult humans and animals [2,3]. However, people know little about the basic development mechanism of neural stem cells. In order to achieve further understanding on regeneration of brain cells, through the research on cultivated cells, scientists hope to get more germiparity characteristics in a certain period of cells. Therefore, specialized image processing techniques for segmentation, object detection and object tracking are required.
* Corresponding author at: Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China. E-mail addresses: [email protected] (T. Zhang), [email protected] (W. Jia), [email protected] (Y. Zhu), [email protected] (J. Yang). http://dx.doi.org/10.1016/j.bbe.2015.10.001 0208-5216/# 2015 Nałęcz Institute of Biocybernetics and Biomedical Engineering. Published by Elsevier Sp. z o.o. All rights reserved.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
2
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
In pharmaceutical process, traditional artificial methods not only depend on a large number of tediously manual labor, but also use dyeing and other chemical technology, whereas these operations influence the motion mode of the cells [4]. Therefore, the traditional cell movement research methods are no longer applicable, and it appears that computer vision technology is an approach for studying cell movement. Moreover, this technology lends itself well to systematization and equipment integration [5]. All that led us to propose a vision-based cell tracking system in this paper, and we also wish to provide a new technical support for biological cell researches. Cell segmentation is an important step in cell tracking. Segmentation is the process of dividing an image into meaningful parts, resulting in a new image containing for each pixel a label indicating to which segment it belongs (such as ‘‘foreground’’ versus ‘‘background’’). In the category of segmentation driven methods, cells are first detected in each frame based on their intensity, texture, or gradient, and then the detected cells are associated in two or more consecutive frames. Korzynska et al. [30] described a segmentation method combining a texture based approach with a contour based approach. The technique is designed to enable the study of cell behavior over time by segmenting bright-field microscope image sequences. However, the performance gains of the method are derived from the initialization procedure performed by the human operator on the first image of the sequence. Iwanowski et al. [31] and Korzynska et al. [32] described a multistage morphological segmentation method (MSMA) for microscopic cell images. The proposed method is based on two types of information, i.e., the cell texture coming from the bright field images and the intensity of light emission. Warowny and Markiewicz [33] presented two methods of texture feature generation for recognizing neoplasm and non-neoplasm cells in cancer diagnosis. The proposed methods have proved to be useful in practice for diagnosing cancer. Koprowski et al. [34] presented an attempt to segmentation of cell structures images. With the employment of the presented decision trees algorithm, biological diagnostic support goes fully automatically. Korzynska [35] improved the neutrophils' movement quantification by extending the cell's activity description to two stages of classification. The proposed new method has been used to describe the differences between normal children and the Chediak-Higashi syndrome patients. Korzynska [36] also examined and compared three microscopic image segmentation methods (reference method, morphological flattening method and watershed method), and showed that the watershed method detects cells' area more precisely than others. More recent examples of cell tracking algorithms include affine transformation invariance [6] and a biological global positioning system [7]. Several different nowcasting algorithms were compared from 2003 to 2011. The computer vision technology provided a way of investigating cell tracking algorithm in a wide variety of applications, such as quantitative motion analysis algorithm [25], epidermal Langerhans cells tracking [8], real-time tumor tracking [9], embryo cell motion tracking [13], tracking fluorescent cells with coupled active surfaces [10], automatic tracking of biological cells in
time-lapse microscopy [14], cell tracking using level sets [11], and cancer cell tracking [15]. Li et al. [12] exploited a fast topology-constrained level-set method in conjunction with a stochastic motion filter with a higher accuration, making it suitable for some specific application. However, the application of these algorithms is limited by their assumptions and constraints. Most existing tracking methods have high computational complexity, and are only effective in limited applications. Hence, there is a great demand for developing automatic cell tracking system, which has attracted increasing research attention. In this paper, based on the characteristics of neural stem cells [1–3], cells are classified into three types, and different tracking techniques are developed to handle cells with different characteristic. These types are (1) the inactive cells that produce only small inter-frame movement, (2) the active cells that do random, large hop movement between frames, and (3) the divisive cells that are in division. Different tracking techniques are developed to address the challenges that each type of cells have. To overcome the adhesion problem of adhesive cells so as to locate them properly, a context-based adhesive cell separation method is proposed. To track inactive cells, we propose an improved mean shift algorithm. To track active cells, we propose a dynamic local prediction (DLP) algorithm to adjust the central position of the candidate movement region, and a new gray prediction (GP) model is also established. When dealing with divisive cells, we firstly track one of the sub-cells by using our improved mean shift algorithm, and then search the other sub-cells using other features. The rest of the paper is organized as follows. Section 2 details the cell tracking methodology. The experimental results of cell tracking are described in Section 3. Section 4 gives some dicussions and analysis. Finally, Section 5 highlights the achieved results.
2.
The proposed method
The proposed method for automatic cell tracking consists of two main modules: i.e., the detection module and the tracking module, as shown in Fig. 1. The detection module mainly detects and localizes cells in the image sequence, and extracts features for the tracking module. The tracking module consists of tracking inactive cells using our improved mean shift algorithm, tracking divisive cells through a context-based technology, and tracking active cells by means of DLP and GP algorithms.
Fig. 1 – The framework of the proposed method.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
2.1.
Cell detection and localization
Cell detection and localization is the first step of the cell tracking system, which also constitutes the basis of the whole detection system. Moving object detection plays an important role in our system. There are three kinds of methods, i.e., optical flow, frame difference and background subtraction methods. Optical flow methods are based on unchanged gray gradient and constant brightness. This kind of methods can detect an object of independent motion without knowing any information of the scene in advance. But when the contrast of the target and background image is too low, or there exists noise in the image, this algorithm will lead to high false alarm rates [16]. The frame difference methods use the difference of two or more consecutive frames in video sequences to detect moving objects. However, the number of continuous frames is often difficult to determine. The basic idea of background subtraction method is to perform pixel-by-pixel subtraction of the current frame and background image stored in advance [24,25]. If the difference is bigger than some threshold value, then the pixel belongs to the moving target; otherwise, the pixel is classified as the background. Compared with the frame difference methods, there exists less influence of the moving target due to relatively fixed background, and the algorithm is simple to design. The above analysis led us to use a background subtraction method to detect cells in sequential digital images. Mathematically, the method can be described as Dk ðx; yÞ ¼ jFk ðx; yÞBk ðx; yÞj
(1)
where Fk and Bk are the current kth frame image and the background image respectively, and Dk denotes the result of background subtraction.
3
Next, image preprocessing operations are performed, including image binarization, area filling, image erosion and image denoising. The Ostu's algorithm [28] is used to do adaptive image binarization. To fill the holes and detect the moving targets in the video sequence, we use an area filling algorithm where the 4-connected area is used. An example of image preprocessing is shown in Fig. 2, where Fig. 2(b) is obtained by performing morphological opening operation. In cell images, there exist non-cellular objects such as tissue fluid and adhesive molecules. These tissue fluid and adhesive molecules can lead to cell adhesion, which makes segmenting the cells very difficult. Therefore, we propose a novel method based on the Bresenham algorithm [29] to solve the above adhesion problem. Bresenham algorithm can efficiently convert line segments because it requires only an integer addition and a sign test for each pixel generated. The process of drawing lines using the Bresenham algorithm is as follows [29]: (1) Input the two line endpoints and store the left endpoint in (x0, y0). (2) Load (x0, y0) into the frame buffer; that is, plot the first point. (3) Calculate constants Dx, Dy, 2Dx, and 2Dy 2Dx, where Dx and Dy are the vertical and horizontal separations of the endpoint positions. These constants are calculated once for each line to be scan converted. (4) Obtain the starting value for the decision parameter as: p0 = 2Dy Dx. Assuming we have determined that the pixel at (xk, yk) is to be displayed, we next need to decide which pixel to plot in column xk+1. (5) At each xk along the line, starting at k = 0. At sampling position xk+1, we label vertical pixel separations from the mathematical line path as d1 and d2. Let pk = Dx(d1 d2).
Fig. 2 – An example of image preprocessing: (a) Original image; (b) Background image; (c) Background subtraction result; (d) Binarization result; (e) Area filling result; (f) Image erosion and denoising result. Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
4
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
(6) If pk < 0, the next point to plot is (xk+1, yk) and pk+1 = pk + 2Dy. Otherwise, the next point to plot is (xk+1, yk+1) and pk+1 = pk + 2Dy 2Dx. (7) Repeat step 5 to 6. More details of drawing lines using the Bresenham algorithm can be found in [29]. Denote the cell number in the previous frame and the set of non-marker regions as m and N respectively. Our adhesive cell separation algorithm is described as follows: Adhesive cell separation algorithm. Input: The ith preprocessed frame image (i = 1, 2, . . ., n). Output: The final separation results. While there exists an input frame in the last loop Step 1. For each region p 2 N, calculate its area S, then we can get the average area vs. Step 2. The region which area is less than 0.3vs is set to 0. Step 3. Calculate the region number n. Step 4. While m > n, search these regions whose areas are bigger than c (here set as 1.8vs), and then determine the centroids of these regions. Otherwise, jump to Step 6. Step 5. Search m n + 1 edge points that are closest to centroids in the above regions. Draw a line between each edge point and centroid, and separate adhesive cells by using the Bresenham algorithm. Then let i = i + 1, go back to Step 1. Step 6. Track the cells. If there exist divisive cells (see 2.3 part), let n = n + l with l denoting the number of divisive cells, m = n, i = i + 1, and go back to Step 1. Otherwise, let m = n, i = i + 1, and go back to Step 1. End
Secondly, we locate each moving target by using the connectivity of binary images. The main idea of this method is to determine four vertices of the target boundary rectangle, and mark the moving target [26]. After getting these target areas, we still cannot guarantee whether each of these targets is a cell or not. In order to solve the problem, we need to consider other feature of the moving target. Since most of the cells are elliptical, the degree of roundness can be used as a criterion. Through the analysis of connected areas, the following parameters of each moving target will be extracted: the gray level histogram, the centroid, the circumference and area of each independent region, and the corresponding roundness degree. The area S of a region R is defined as X f ðx; yÞ (2) S¼ ðx;yÞ 2 R
where f(x, y) = 1. Obviously, area S is rotation and translation invariant. The degree of roundness P of an object is given by P¼
4pS L2
(3)
where the perimeter L is defined as the length of the area outer boundary. It can be derived that, 0 < P < 1, and the bigger the P value, the closer the area to round.
segmenting and recognizing the cell in the image, and then tracking these segmented cells using our improved mean shift algorithm. Compared with other moving target tracking algorithm [9–12], advantages of mean shift algorithm are as follows: (1) less computation, suitable for real-time tracking, (2) insensitive to deformation, rotating, changes in the background of moving target, (3) robust in many scenarios, (4) as a nonparameter estimation algorithm, easily integrate with other algorithms. The classical mean shift tracking algorithm uses color histogram to describe the target appearance. In gray-level image sequences, due to the sensitivity to illumination and the lack of the information about the object representation, the cell tracking based on classical mean shift algorithm is not stable. This led us to propose an improved mean shift tracking algorithm combining pixel probability and orientation. Let {xi} i = 1, 2, . . ., n and b(xi) be the pixel position of a normalized target template area and its gray value, respectively. At the same time, due to the occlusion and changes of light, whether some pixels belong to a certain target or not is unknown. We assign the same weight value to the pixels near the target center. However, when the distance exceeds our preset value, we use a distribution to describe the probability. In the present study, an exponential probability distribution is given by 8 1 < jjXjjTðyÞ jjXjjTðyÞ (4) RðxÞ ¼ jjXjj TðyÞ : l exp l maxfjjXjjTðyÞgni¼1 where T(y) is the threshold, and || || represents the norm of the gray vector X, and l = 0.8. Assume that the target model is quantized into m bins (orientation codes) and n gray levels. The modified gray level probability density function of the target template (gray feature u = 1, . . ., n) is calculated as n X (5) quv ¼ C RðxÞd½b1 ðxi Þud½b2 ðxi Þv i¼1
where the function b1 : R2 ! {0, 1, . . ., m 1} indicates that b(xi) is mapped into the corresponding orientation code indexes, the function b2 : R2 ! {0, 1, . . ., n 1} shows that b(xi) is mapped into the corresponding gray-level quantitative indexes, d is the Delta function, v is the highest gray level in feature space and C is the corresponding normalization constant. Gray probability density function of candidate model from the target candidate region centered at position y is given by n X (6) quv ðyÞ ¼ Ch RðxÞDd½b1 ðxi Þud½b2 ðxi Þv i¼1
where D = k(||y xi/h||2), h is the dimension of the candidate target, and Ch the corresponding normalization constant. Other aspects of the improved tracking algorithm are similar to those in [15,17,18], with some modifications in light of the above formulation.
2.3. 2.2.
Divisive cells tracking
Inactive cells tracking
The cell with a small inter-frame movement is called inactive cell. The basic idea of tracking inactive cells consists of first
Cells have different life periods. If cells are in division, we will not be able to continue tracking them accurately, because the above described method can only track one target.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
5
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
When a cell is dividing, there are several properties [1–3]: (1) After division of metrocyte A, A will be split into two cells (A1 and A2), where one of them still stays at the position of cell A, the other cell is pushed away. (2) The area of the subcells will decrease, and the sum of subcell A1 and subcell A2 is approximately equal to A. (3) The distance between the two subcells is very close. Therefore, we firstly track one sub-cell using our improved mean shift algorithm, and then search the other cell through using the above properties. However, due to the uncertainty of cell movement, when a certain cell is dividing, another cell may jump out of the boundary of the image. In this case, although the total number of cells is the same as that of the previous frame, there still exist divided cells. Therefore, we need consider whether a cell will step out of the boundary or not. In order to solve the problem, we propose a context-based divisive cells tracking algorithm: Context-based divisive cells tracking. Input: The ith preprocessed frame image (i = 1, 2, . . ., n). Output: The final tracking results. While there exists an input frame in the last loop Step 1. Calculate the total number of targets N(i) and N(i 1) in the ith and (i 1)th frames, respectively. Step 2. If N(i) N(i 1) 6¼ 0, then jump to Step 4; otherwise, jump to Step 3. Step 3. If there exist some cells that jump out of boundary, then jump to Step 4; otherwise, track these cells using the improved mean shift, update i = i + 1 and go back to Step 1. Step 4. Firstly, track these cells by our improved mean shift algorithm. Then calculate the area S(i) of each cell that has been tracked successfully. Set these cells which area is reduced to half from the last search as A1, and find the cell A2 which area is closest to A1. Step 5. If A1 and A2 meet preset conditions (area and distance), then locate subcell A2, update i = i + 1, and go back to Step 1; otherwise, it indicates a failure tracking. End
2.4.
Active cells tracking
In the process of cell movement, sometimes certain cells will do random hop movement, which indicates a larger interframe movement. Now we consider such a situation when there are two cells in the direction of the trajectories. In this case, predicting only by trajectory may result in a failure tracking. The phenomenon of ‘‘cell hopping’’ in this section is the general problem of data association. Motion information can be exploited by the pixel's gray value variation between continuous two frames. Cell movement is a random process, namely, the moving trend of current frame cells is related to the following frames [3]. This led us to propose a so-called dynamic local prediction (DLP) algorithm, which uses the local information of known nodes to estimate the position of unknown nodes. More precisely, the algorithm uses the information between several local frames to predict cell movement. The algorithm aims at the validity prediction of one target.
In order to track them more accurately, we introduce the following parameters. (1) The distance of cell movement between two continuous frames. Assume the coordinate of cell A in a certain frame is (x, y), and its coordinate in the next frame is (x1, y1), then the Euclidean distance can be calculated as qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (7) s ¼ ðxx1 Þ2 þ ðyy1 Þ2 (2) The total distance of cell movement, calculated as S¼
n X si ;
n ¼ 200
(8)
i¼1
(3) The speed vi of cells, which reflects the active degree of cells, is given by s vi ¼ ðpixels=minÞ (9) t where t denotes the time interval between two frames, s is the distance of the movement. (4) The average speed v of cells. It is defined by v¼
X 1 n¼N1 v ðpixels=minÞ N1 i¼1 i
(10)
The cells average speed reflects the cells active degree in the whole image sequence. If the value is big, then the active degree is high. The details of the DLP algorithm are given as follows: DLP algorithm. Step 1. Take these positions of cells that are not tracked from the ith to (i + 10)th frame. Step 2. Take a centroid every two, three, five, and seven frames, respectively. For these centroids, calculate the average distance between the two continuous centroids. Step 3. Compute four centroids according to the obtained average distance, then get four prediction positions according to the centroids. Step 4. Seek more accurate positions through weighted summation of the four prediction positions. Step 5. Compute the angles between cell position, candidate position, and prediction position, respectively, and then normalize after computing the difference for the angles. Step 6. Compute the total distance, cell velocity V, and active coefficients of cells. Step 7. Get probability value through weighted summation of the above parameters, and then determine cell positions.
The notable DLP algorithm densely extracts the information of continuous centroids. It captures the trajectory information which enables us to detect the active cells. Weighted summation of random variables is investigated in connection with the law of large numbers and the law of iterated logarithm. The main aim is to eliminate the roughness and inconsistency of data. In what follows, we introduce the gray prediction (GP) algorithm to track active cells. The gray system theory was proposed by Professor Deng Julong in the late 1970s. Relative to the definitions of black
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
6
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
system and white system, a system with only part of information known is called gray system. As a new traverse discipline, it has shown strong science vitality and academic status, and formed a theory system whose main content consists of system analysis, information processing, modeling, forecasting, decision-making and controlling [20]. Gray system theory considers random processes as gray processes varying in a definite range and time-related, and it regards all sorts of random. Variables as gray variables change within a certain limit, so the method of gray data generation was introduced to trim disordered original data [19–21]. Five kinds of gray prediction methods have been proposed for their respective application [20–22]. In general, the gray model is GM(m, n)[19–22], where m is the order of the differential equation, and n is the number of variables. In a real control environment, GM(1, 1) is the most commonly used model in practice [19,22]. Cell movement is a dynamic process that changes constantly. It happens under special condition and it is random and uncertain. But its latency, development and occurrence are continuous, comparable and related [25]. So, its occurrence is relative to its past and present. This is why we can forecast its occurrence with GP algorithm. In our GP algorithm, we chose GM(1, 1). Due to the gray variable of original time series, sometimes the gray model is not stable, and the prediction result is not efficient. In order to improve the accuracy of prediction, we propose the following two improvements at the level of generation of input data series and small error probability. Assume that the original series of data is expressed as ð0Þ
p
ð0Þ
ð0Þ
ð0Þ
¼ ðp ð1Þ; p ð2Þ; . . .; p ðnÞÞ
(11)
(0)
where p denotes the original data series. In engineering practice, the raw data commonly show strong randomicity, so that if the raw data is directly used in the model, the prediction precision is usually not as high as we want, so the raw data need to be pre-processed. There are many data pre-processing methods for the raw strong randomicity data, such as weighted exponent method, policy factor added processing method, sliding average method, etc. In the paper, the linear weighted exponent smoothing method is employed to smooth strong randomicity sequence of raw data. Then the prediction model is established by using increment data series: xð0Þ ðkÞ ¼ apð0Þ ðk þ 1Þpð0Þ ðkÞ;
k ¼ 1; 2; . . .; n1
(12)
where a is the corresponding constant (here a is set as 1.2). Then the input data series of the gray model can be expressed as xð0Þ ¼ ðxð0Þ ð1Þ; xð0Þ ð2Þ; . . .; xð0Þ ðnÞÞ
(13)
In the next step, we will improve p value, which represents small error probability. In our model, p is defined as: p ¼ PfjeðkÞej < vS1 g
(14)
where e(k) denotes residual error, e the average of residual error, S1 the standard deviation of original data series, and v the probability value of a certain distribution (here we use Gaussian distribution to estimate the value), and the value of v depends on the selection of correct rate.
Other details of original gray model operations that are needed for building GP algorithm model can be found in [18–22].
3.
Results
3.1.
Data set
Our data set includes six intravital microscopy video sequences obtained from Johan Degerma at the University of Chalmers. Video recordings were made by a charge-coupled device (CCD) camera attached to an intravital microscope, and cells were not dyed. The video frames were recorded at a spatial resolution of 640 480 pixels. Each frame reflects the state of cells under the influence of chemical catalyst. Note that, in order to activate the cell's normal movement, chemical stimulator catalyst was used. These cells primarily come from neurons, and are self-renewing, multipotent cells that generate the main phenotype of the nervous system. They are characterized by their capability to differentiate into multiple cell types via exogenous stimuli from their environment. They undergo asymmetric cell division into two daughter cells, one non-specialized and the other specialized. These cells were grown at 37 8C and 5% CO2 atmosphere in a humidified incubator.
3.2.
Results on segmentation
Now we will measure the performance of the proposed methods in Section 2. The performance of adhesive cell separation algorithm is shown in Fig. 3(a) and (b). As it is shown, all adhesive cells are separated perfectly. To verify our segmentation algorithm, we compare it with the segmentation results of the Multistage Morphological Method (MO) [32], the morphological flattening (MF) [36], the watershed method (WS) [36], the connectivity based merging and coupled level sets [23,27]. Table 1 shows the comparison results, it can be seen that our algorithm with a higher accuracy is very high in processing speed and is more effective in a real-time environment.
3.3.
Results of cell tracking
Fig. 4(a)–(f) shows the results of inactive cells tracking over different frames, where the final tracking results are indicated
Table 1 – Comparison of different segmentation algorithms. Algorithm
Time
Correct segmentation rate
MO [32] MF [36] WS [36] Connectivity based merging [23] Coupled level sets [27] Ours
769 ms 483 ms 620 ms 465 ms
98.2% 95.7% 97.9% 97.8%
589 ms
98.7%
180 ms
98.9%
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
7
Fig. 3 – Adhesive cell separation: (a) Adhesive cells; (b) Result of adhesive cell separation algorithm.
using green labels. As is shown, we can successfully track them using our improved mean shift algorithm. Fig. 5(a)–(d) shows the detected results using context-based divisive cells tracking algorithm. The results of active cells tracking are given in Fig. 6(a)–(d). As can be seen in Fig. 6(b), the conventional mean shift algorithm has failed to track active cells. That is because that cells do a larger inter-frame movement, in which case, the performance of mean shift algorithm degrades significantly. On the contrary, the DLP and GP algorithms both have a good performance in tracking active cells. Since there are visually few differences between the two results in Fig. 6(c) and (d), we will give more quantitative
analysis of the above results. For the DLP algorithm, we compute the active degree of cells according to the formula in (10). Over ten frames, the total movement distances for the two cells A and B (see Fig. 7) are respectively 79.3972 and 45.6247. The active degree is set as p3, the probability at A1 (candidate position 1) is set as p4, and the probability at A2 (candidate position 2) is set as p5. A3 denotes the predicted position. a1, a2 and a3 denote the angle between the current position and A1, A2, A3, respectively. ja2 a1j is set as p1, and ja3 a3j is set as p2. p1 and p2 indicate the distance between the cell's current position and each of the candidate positions A1 and A2, p3 denotes the active degree of the current cell, and p4 and p5 reflect the probability of the current cell at position 1 and 2
Fig. 4 – Results of inactive cells tracking: (a) Second frame; (b) Forth frame; (c) Sixth frame; (d) Eighth frame; (e) Tenth frame; (f) Twelfth frame. Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
8
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
Fig. 5 – Results of divisive cells tracking over: (a) the second frame; (b) the forth frame; (c) the eighth frame.
Fig. 6 – Results of active cells tracking: (a) Active cells to be tracked; (b)–(d) are tracking results obtained using (b) the mean shift algorithm; (c) the DLP algorithm; (d) the GP algorithm.
respectively. We can then predict the final positions, as shown in Table 2. For example, in the first row of Table 2, p2 < p1 (i.e., candidate position A2 is closer to the current position) and p5 > p4 (i.e., the cell is more likely to be at candidate position 2), then we can conclude position A2 as the predicted position of the cell. While for the GP algorithm, through analyzing and
calculating the gray model, these vertical coordinates of cells centroids can be calculated, which results are given in Table 3. Table 3 gives these predicted positions of cell A, while for the cell B, their positions can be obtained using the same method. Finally, through analyzing the above results, we can obtain the movement trajectory of cells, as shown in Fig. 7. In this denotes the four predicted positions of cell A, figure, denotes the final predicted position obtained using DLP, denotes the final predicted position obtained by GP algorithm. denotes the four predicted positions of cell B, denotes the final predicted position given by DLP, denotes the final predicted position given by GP algorithm. It can be seen that the position given by GP algorithm is clearly more accurate. In order to demonstrate the effectiveness of our algorithm, we have tested thousands of cells. We compare our algorithm with the state-of-the-art approaches, including the fast topology-constrained level-set approach proposed in [11], the affine transformation (AT) invariance approach of [6], the adaptive tracking method combining motion and topological features
Table 2 – Predicting positions for the two cells shown in Fig. 7. Fig. 7 – Schematic diagram of cell trajectory. (For interpretation of the references to color in the text, the reader is referred to the web version of the article.)
Cell A B
p1
p2
p3
p4
p5
Final result
0.3980 0.1943
0.0549 0.3067
0.2647 0.1521
0.1210 0.4637
0.5334 0.1302
A2 A1
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
Table 3 – Simulation of data. Value of model _0
x ð2Þ ¼ 110:4128 x ð3Þ ¼ 105:3364 _ð0Þ x ð4Þ ¼ 102:0891 _ð0Þ x ð5Þ ¼ 98:1656 _ð0Þ x ð6Þ ¼ 94:3929 _ð0Þ x ð7Þ ¼ 100:7652 _ð0Þ x ð8Þ ¼ 87:2769 _ð0Þ x ð9Þ ¼ 83:9227 _ð0Þ x ð10Þ ¼ 80:6974 _ð0Þ x ð11Þ ¼ 80:6974 _ð0Þ
Real value
Residual error E(k)
Relative error e(k)
106.2933 105.3364 104.5028 97.6461 95.8020 101.3993 86.6796 80.6068 75.4976 62.9100
4.1195 0.8330 2.4137 0.5195 1.4091 0.6341 0.5973 3.3159 5.1998 2.6357
0.0388 0.0079 0.0231 0.0053 0.0147 0.0063 0.0069 0.0411 0.0689 0.0902
(ATCMT) [14]. The results are shown in Table 4. As is shown, our algorithm with a higher accuracy has the advantages of high detection speed and strong practicability. The results in [6,14] is very close to our results, that is because that the methods in [6,14] focus on the segmentation of the nucleus instead of on the entire cellular structure, so it is effective even in the case of variable cell morphologies (with different cell cultures), cell partial overlaps, and dynamic changes in the cell-shape during migration. Our tracking system runs at an average speed of 6 frames/ min for processing the images in our experiments on a workstation with a 3 GHz processor and 2 GB memory. So the system is well suited for tracking during acquisition.
4.
Discussion
In this work, we developed an automatic tracking system of neural stem cells and compared its performance with the state-of-the-art. Four performance measures were used to evaluate the proposed tracking system, i.e., the correct segmentation rate, which quantifies the segmentation accuracy, and the residual error, the relative error and tracking accuracy rate, which measure the robustness of tracking. As it shows, our segmentation method is semi-automatic, because the operator initiates it and adjusts its parameters according to the segmented image and object properties, and, therefore, it is much faster, and the results are more
Table 4 – Comparison of cell tracking accuracy and processing time with state-of-the-art. Sequence
Num
1 2 3 4 5 6 Total Tracking accuracy rate Average time
270 320 290 300 270 300 1750
Level-set [11] 265 311 282 289 266 293 1706 97.48%
1630 ms
AT [6] 261 306 280 279 260 290 1676 95.77%
1370 ms
ATCMT [14] 263 312 281 288 267 294 1705 97.42%
1478 ms
Ours 264 313 282 290 265 295 1709 97.66%
1000 ms
9
reproducible than with any manual segmentation results; the method can cope very well with noisy images; the object position error is small, and therefore, the tracking method can be used for observing and quantifying the movement of a nonrigid object. The watershed method and the connectivity based merging demonstrated similar performance for cell segmentation. The multistage morphological method and the coupled level sets had a slightly lower accuracy than our segmentation method. However, our processing time is obviously smallest in all the compared methods. The affine transformation invariance approach and the adaptive tracking approach combining motion and topological features demonstrated similar performance for cell tracking, and these two methods have a slightly lower detection rates than our tracking method. In general, we find that if a feature is sufficiently well defined that it can be tracked or identified visually, the algorithm can also track or identify it. Cell tracking is a relatively independent subject that touches wider areas. There are still many problems yet to be solved perfectly. For example, to reduce the dependency on the human operator by employing a pre-segmentation stage which would preview the sequence and adjust the parameters of the method; to develop methods to deal with clustered cells, which are in contact with each other. Further research will be required to address this problem.
5.
Conclusions
In this paper, we have proposed a complete system for the biological cells detection and tracking in a time-lapse microscopy. The tracking system is capable of segmenting adhesive cells and tracking accurately each cell in video sequences. The proposed system described in this paper combines two approaches: the segmentation-based approach and the tracking-based approach. The system is designed for processing a sequence of images with a single non-rigid object changing position and shape over time. The system exploits temporal and spatial contextual information and utilizes it for reducing any uncertainty in the cell border and increasing the speed of segmentation. When the combined method is used to segment an object in a single image, the method is effective. Segmentation and tracking experiments on real data demonstrated that the proposed system has a superior performance over existing works in both accuracy and processing speed.
Financial support This research is partly supported by NSFC, China (No: 61375048).
Acknowledgment The authors would like to thank Johan Degerma at the University of Chalmers, for providing microscopic image sequences for this research and valuable inputs on biological properties of neural stem cells.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
10
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
references
[1] Li Z, Hu S, Ghosh Z, Han Z, Wu JC. Functional comparison and expression profiling of human induced pluripotent stem cell and embryonic stem cell-derived endothelial cells. Stem Cells Dev 2011;20:1701–10. [2] Gu E, Chen WY, Gu J, Burridge P, Wu JC. Molecular imaging of stem cells: tracking survival, biodistribution, tumorigenicity, and immunogenicity. Theranostics 2012;2 (4):335–45. [3] Schroeder T. Imaging stem-cell-driven regeneration in mammals. Nature 2008;453(7193):345–51. [4] Ozturk Z, Sadettin S. Cell culture technology for pharmaceutical and cell-based therapies. New York: Marcel Dekker Inc; 2005. [5] Stephens DJ, Allan VJ. Light microscopy techniques for live cell imaging. Science 2003;300:82–6. [6] Lin ZL. An affine transformation invariance approach to cell tracking. Comput Med Imaging Graph 2008;32:554–65. [7] Li SC, Tachiki LM, Luo J, Dethlefs BA, Chen Z, Loudon WG. A biological global positioning system: considerations for tracking stem cell behaviors in the whole body. Stem Cell Rev 2010;6(2):317–33. [8] Lee PW, Hsu SH. Multifunctional core–shell polymeric nanoparticles for transdermal DNA delivery and epidermal Langerhans cells tracking. Biomaterials 2010;31:2425–34. [9] Sharp GC, Jiang SB, Shimizu S, Shirato H. Tracking errors in a prototype real-time tumour tracking system. Phys Med Biol 2004;49:5347–56. [10] Dufour A, Shinin V, Tajbakhsh S, Guillén-Aghion N, OlivoMarin JC, Zimmer C. Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces. IEEE Trans Image Process 2005;9:1396–410. [11] Padfield D, Rittscher J, Thomas N, Roysam B. Spatiotemporal cell cycle phase analysis using level sets and fast marching methods. Med Image Anal 2009;13:143–50. [12] Li K, Miller E, Weiss L, Campbell P, Kanade T. Online tracking of migrating and proliferating cells imaged with phase-contrast microscopy. Proceedings of the CVPRW; 2006. p. 65–72. [13] Dunn GA, Jones GE. Cell motility under the microscope: Vorsprung durch technik. Nat Rev 2004;5:667–72. [14] Dewan MAA, Ahmad MO, Swamy MNS. Tracking biological cells in time-lapse microscopy: an adaptive technique combining motion and topological features. IEEE Trans Biomed Eng 2011;58:1637–47. [15] Debeir O, Ham PV, Kiss R, Decaestecker C. Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes. IEEE Trans Med Imaging 2005;24:697–711. [16] Hossein M, Shahriar N. On robustness and localization accuracy of optical flow computation for underwater color images. Comput Vis Image Underst 2006;104:61–76.
[17] Cheng Y. Mean shift, mode seeking, and clustering. EEE Trans Pattern Anal Mach Intell 1995;17:790–9. [18] Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 2002;24:603–19. [19] Deng JL. Gray theory elements. Wu Han: Central China Science and Technology Publishing House; 2002. [20] Deng JL. Gray prediction and gray decision—making. Wu Han: Central China Science and Technology Publishing House; 2002. [21] Tien TL. A new grey prediction model FGM(1, 1). Math Comput Model 2009;49:1416–26. [22] Lin CB, Su SF, Hsu YT. High-precision forecast using grey models. Int J Syst Sci 2001;32:609–19. [23] Umesh Adiga PS, Chaudhuri BB. An efficient method based on watershed and rule-based merging for segmentation of 3-D histopathological images. Pattern Recognit 2001;34:1449–58. [24] Xu XY, Li BX. Adaptive rao-blackwellized particle filter and its evaluation for tracking in surveillance. IEEE Trans Imaging Process 2007;16:838–49. [25] Gerlich D, Mattes J, Eils R. Quantitative motion analysis and visualization of cellular structures. Methods 2003;29:3–13. [26] Sharp GC, Jiang SB, Shimizu S, Shirato H. Prediction of respiratory tumor motion for real-time image-guided radiotherapy. Phys Med Biol 2004;49:425–40. [27] Nath SK, Palaniappan K, Bunyak F. Cell segmentation using coupled level sets and graph-vertex coloring. MICCAI 2006;4190:101–8. [28] Weszka JS. A survey of threshold selection techniques. Comput Graph Image Process 1978;7:259–65. [29] Donald H, Baker PM. Computer graphics with OpenGL. New Jersey: Prentice-Hall International; 2004. [30] Korzynska A, Strojny W, Hoppe A, Wertheim D, Hoser P. Segmentation of microscope images of living cells. Pattern Anal Appl 2007;10(4):301–19. [31] Iwanowski M, Korzyńska A. Segmentation of moving cells in bright field and epi-fluorescent microscopic image sequences. Lecture notes in computer science. Springer-Verlag; 2010. [32] Korzyńska A, Iwanowski M. Multistage morphological segmentation of bright-field and fluorescent microscopy images. Opto-Electron Rev 2012;20(2):174–86. [33] Warowny M, Markiewicz T. Quick texture generation for multiobject image analysis in brain pathology. Przeglad Elektrotechniczny 2010;86(1):50–2. [34] Koprowski R, Izdebska-Straszak G, Wróbel Z, Adamek B. The cell structures segmentation with using of decision trees. Pattern Recognit Image Anal 2005;15(3):1–8. [35] Korzynska A. Neutrophils movement in vitro. Pattern Ann N Y Acad Sci 2002;972:139–43. [36] Korzynska A, Iwanowski M, Neuman U, Dobrowolska E, Hoser P. Comparison of the methods of microscopic image segmentation. World Congress on Medical Physics and Biomedical Engineering, vol. 25(4); 2009. pp. 425–8.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.elsevier.com/locate/bbe
Original Research Article
Automatic tracking of neural stem cells in sequential digital images Tao Zhang a, Wenjing Jia c, Yuemin Zhu b, Jie Yang a,* a
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China Centre National de la Recherche Scientifique of France, Villeurbanne, France c Faculty of Engineering and Information Technology, University of Technology, Australia b
article info
abstract
Article history:
Neural stem cells are the cells that give rise to the main cell types of the nervous system.
Received 21 September 2014
Due to their varying size and shape, and random movement, the tracking of these cells in
Received in revised form
suspension in video sequences is challenging. This paper develops an automatic tracking
21 May 2015
system for neural stem cells. The system first detects and localizes cells in the image
Accepted 7 October 2015
sequence, followed by a feature extraction step for the subsequent cell tracking. Then,
Available online xxx
the system tracks inactive cells using an improved mean shift algorithm, divisive cells through a context-based technique, and active cells by means of dynamic local prediction
Keywords:
(DLP) and gray prediction (GP) algorithms. Experimental results show that the proposed
Cell tracking
system not only improves the accuracy of fast moving tracking, but also constructs
Improved mean shift
accurately the trajectories of the cell movement and reduces the iterations during the
Dynamic local prediction (DLP)
center searching.
Gray prediction (GP)
# 2015 Nałęcz Institute of Biocybernetics and Biomedical Engineering. Published by
Neural stem cells
1.
Introduction
The discovery of neural stem cells is a significant progress in the field of bioscience at the end of the last century. Neural stem cells hide in certain parts of nervous system. Some of these cells stay in original state, some have the potential to divide with multi-aspect, and others have regeneration ability [1,2]. Due to the good plasticity of neural stem cells, we can make it as a therapy tool of nerve damage through genetic or
Elsevier Sp. z o.o. All rights reserved.
cell engineering. Researchers have observed that new nerve cells are produced by neural stem cells in the brain of adult humans and animals [2,3]. However, people know little about the basic development mechanism of neural stem cells. In order to achieve further understanding on regeneration of brain cells, through the research on cultivated cells, scientists hope to get more germiparity characteristics in a certain period of cells. Therefore, specialized image processing techniques for segmentation, object detection and object tracking are required.
* Corresponding author at: Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China. E-mail addresses: [email protected] (T. Zhang), [email protected] (W. Jia), [email protected] (Y. Zhu), [email protected] (J. Yang). http://dx.doi.org/10.1016/j.bbe.2015.10.001 0208-5216/# 2015 Nałęcz Institute of Biocybernetics and Biomedical Engineering. Published by Elsevier Sp. z o.o. All rights reserved.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
2
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
In pharmaceutical process, traditional artificial methods not only depend on a large number of tediously manual labor, but also use dyeing and other chemical technology, whereas these operations influence the motion mode of the cells [4]. Therefore, the traditional cell movement research methods are no longer applicable, and it appears that computer vision technology is an approach for studying cell movement. Moreover, this technology lends itself well to systematization and equipment integration [5]. All that led us to propose a vision-based cell tracking system in this paper, and we also wish to provide a new technical support for biological cell researches. Cell segmentation is an important step in cell tracking. Segmentation is the process of dividing an image into meaningful parts, resulting in a new image containing for each pixel a label indicating to which segment it belongs (such as ‘‘foreground’’ versus ‘‘background’’). In the category of segmentation driven methods, cells are first detected in each frame based on their intensity, texture, or gradient, and then the detected cells are associated in two or more consecutive frames. Korzynska et al. [30] described a segmentation method combining a texture based approach with a contour based approach. The technique is designed to enable the study of cell behavior over time by segmenting bright-field microscope image sequences. However, the performance gains of the method are derived from the initialization procedure performed by the human operator on the first image of the sequence. Iwanowski et al. [31] and Korzynska et al. [32] described a multistage morphological segmentation method (MSMA) for microscopic cell images. The proposed method is based on two types of information, i.e., the cell texture coming from the bright field images and the intensity of light emission. Warowny and Markiewicz [33] presented two methods of texture feature generation for recognizing neoplasm and non-neoplasm cells in cancer diagnosis. The proposed methods have proved to be useful in practice for diagnosing cancer. Koprowski et al. [34] presented an attempt to segmentation of cell structures images. With the employment of the presented decision trees algorithm, biological diagnostic support goes fully automatically. Korzynska [35] improved the neutrophils' movement quantification by extending the cell's activity description to two stages of classification. The proposed new method has been used to describe the differences between normal children and the Chediak-Higashi syndrome patients. Korzynska [36] also examined and compared three microscopic image segmentation methods (reference method, morphological flattening method and watershed method), and showed that the watershed method detects cells' area more precisely than others. More recent examples of cell tracking algorithms include affine transformation invariance [6] and a biological global positioning system [7]. Several different nowcasting algorithms were compared from 2003 to 2011. The computer vision technology provided a way of investigating cell tracking algorithm in a wide variety of applications, such as quantitative motion analysis algorithm [25], epidermal Langerhans cells tracking [8], real-time tumor tracking [9], embryo cell motion tracking [13], tracking fluorescent cells with coupled active surfaces [10], automatic tracking of biological cells in
time-lapse microscopy [14], cell tracking using level sets [11], and cancer cell tracking [15]. Li et al. [12] exploited a fast topology-constrained level-set method in conjunction with a stochastic motion filter with a higher accuration, making it suitable for some specific application. However, the application of these algorithms is limited by their assumptions and constraints. Most existing tracking methods have high computational complexity, and are only effective in limited applications. Hence, there is a great demand for developing automatic cell tracking system, which has attracted increasing research attention. In this paper, based on the characteristics of neural stem cells [1–3], cells are classified into three types, and different tracking techniques are developed to handle cells with different characteristic. These types are (1) the inactive cells that produce only small inter-frame movement, (2) the active cells that do random, large hop movement between frames, and (3) the divisive cells that are in division. Different tracking techniques are developed to address the challenges that each type of cells have. To overcome the adhesion problem of adhesive cells so as to locate them properly, a context-based adhesive cell separation method is proposed. To track inactive cells, we propose an improved mean shift algorithm. To track active cells, we propose a dynamic local prediction (DLP) algorithm to adjust the central position of the candidate movement region, and a new gray prediction (GP) model is also established. When dealing with divisive cells, we firstly track one of the sub-cells by using our improved mean shift algorithm, and then search the other sub-cells using other features. The rest of the paper is organized as follows. Section 2 details the cell tracking methodology. The experimental results of cell tracking are described in Section 3. Section 4 gives some dicussions and analysis. Finally, Section 5 highlights the achieved results.
2.
The proposed method
The proposed method for automatic cell tracking consists of two main modules: i.e., the detection module and the tracking module, as shown in Fig. 1. The detection module mainly detects and localizes cells in the image sequence, and extracts features for the tracking module. The tracking module consists of tracking inactive cells using our improved mean shift algorithm, tracking divisive cells through a context-based technology, and tracking active cells by means of DLP and GP algorithms.
Fig. 1 – The framework of the proposed method.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
2.1.
Cell detection and localization
Cell detection and localization is the first step of the cell tracking system, which also constitutes the basis of the whole detection system. Moving object detection plays an important role in our system. There are three kinds of methods, i.e., optical flow, frame difference and background subtraction methods. Optical flow methods are based on unchanged gray gradient and constant brightness. This kind of methods can detect an object of independent motion without knowing any information of the scene in advance. But when the contrast of the target and background image is too low, or there exists noise in the image, this algorithm will lead to high false alarm rates [16]. The frame difference methods use the difference of two or more consecutive frames in video sequences to detect moving objects. However, the number of continuous frames is often difficult to determine. The basic idea of background subtraction method is to perform pixel-by-pixel subtraction of the current frame and background image stored in advance [24,25]. If the difference is bigger than some threshold value, then the pixel belongs to the moving target; otherwise, the pixel is classified as the background. Compared with the frame difference methods, there exists less influence of the moving target due to relatively fixed background, and the algorithm is simple to design. The above analysis led us to use a background subtraction method to detect cells in sequential digital images. Mathematically, the method can be described as Dk ðx; yÞ ¼ jFk ðx; yÞBk ðx; yÞj
(1)
where Fk and Bk are the current kth frame image and the background image respectively, and Dk denotes the result of background subtraction.
3
Next, image preprocessing operations are performed, including image binarization, area filling, image erosion and image denoising. The Ostu's algorithm [28] is used to do adaptive image binarization. To fill the holes and detect the moving targets in the video sequence, we use an area filling algorithm where the 4-connected area is used. An example of image preprocessing is shown in Fig. 2, where Fig. 2(b) is obtained by performing morphological opening operation. In cell images, there exist non-cellular objects such as tissue fluid and adhesive molecules. These tissue fluid and adhesive molecules can lead to cell adhesion, which makes segmenting the cells very difficult. Therefore, we propose a novel method based on the Bresenham algorithm [29] to solve the above adhesion problem. Bresenham algorithm can efficiently convert line segments because it requires only an integer addition and a sign test for each pixel generated. The process of drawing lines using the Bresenham algorithm is as follows [29]: (1) Input the two line endpoints and store the left endpoint in (x0, y0). (2) Load (x0, y0) into the frame buffer; that is, plot the first point. (3) Calculate constants Dx, Dy, 2Dx, and 2Dy 2Dx, where Dx and Dy are the vertical and horizontal separations of the endpoint positions. These constants are calculated once for each line to be scan converted. (4) Obtain the starting value for the decision parameter as: p0 = 2Dy Dx. Assuming we have determined that the pixel at (xk, yk) is to be displayed, we next need to decide which pixel to plot in column xk+1. (5) At each xk along the line, starting at k = 0. At sampling position xk+1, we label vertical pixel separations from the mathematical line path as d1 and d2. Let pk = Dx(d1 d2).
Fig. 2 – An example of image preprocessing: (a) Original image; (b) Background image; (c) Background subtraction result; (d) Binarization result; (e) Area filling result; (f) Image erosion and denoising result. Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
4
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
(6) If pk < 0, the next point to plot is (xk+1, yk) and pk+1 = pk + 2Dy. Otherwise, the next point to plot is (xk+1, yk+1) and pk+1 = pk + 2Dy 2Dx. (7) Repeat step 5 to 6. More details of drawing lines using the Bresenham algorithm can be found in [29]. Denote the cell number in the previous frame and the set of non-marker regions as m and N respectively. Our adhesive cell separation algorithm is described as follows: Adhesive cell separation algorithm. Input: The ith preprocessed frame image (i = 1, 2, . . ., n). Output: The final separation results. While there exists an input frame in the last loop Step 1. For each region p 2 N, calculate its area S, then we can get the average area vs. Step 2. The region which area is less than 0.3vs is set to 0. Step 3. Calculate the region number n. Step 4. While m > n, search these regions whose areas are bigger than c (here set as 1.8vs), and then determine the centroids of these regions. Otherwise, jump to Step 6. Step 5. Search m n + 1 edge points that are closest to centroids in the above regions. Draw a line between each edge point and centroid, and separate adhesive cells by using the Bresenham algorithm. Then let i = i + 1, go back to Step 1. Step 6. Track the cells. If there exist divisive cells (see 2.3 part), let n = n + l with l denoting the number of divisive cells, m = n, i = i + 1, and go back to Step 1. Otherwise, let m = n, i = i + 1, and go back to Step 1. End
Secondly, we locate each moving target by using the connectivity of binary images. The main idea of this method is to determine four vertices of the target boundary rectangle, and mark the moving target [26]. After getting these target areas, we still cannot guarantee whether each of these targets is a cell or not. In order to solve the problem, we need to consider other feature of the moving target. Since most of the cells are elliptical, the degree of roundness can be used as a criterion. Through the analysis of connected areas, the following parameters of each moving target will be extracted: the gray level histogram, the centroid, the circumference and area of each independent region, and the corresponding roundness degree. The area S of a region R is defined as X f ðx; yÞ (2) S¼ ðx;yÞ 2 R
where f(x, y) = 1. Obviously, area S is rotation and translation invariant. The degree of roundness P of an object is given by P¼
4pS L2
(3)
where the perimeter L is defined as the length of the area outer boundary. It can be derived that, 0 < P < 1, and the bigger the P value, the closer the area to round.
segmenting and recognizing the cell in the image, and then tracking these segmented cells using our improved mean shift algorithm. Compared with other moving target tracking algorithm [9–12], advantages of mean shift algorithm are as follows: (1) less computation, suitable for real-time tracking, (2) insensitive to deformation, rotating, changes in the background of moving target, (3) robust in many scenarios, (4) as a nonparameter estimation algorithm, easily integrate with other algorithms. The classical mean shift tracking algorithm uses color histogram to describe the target appearance. In gray-level image sequences, due to the sensitivity to illumination and the lack of the information about the object representation, the cell tracking based on classical mean shift algorithm is not stable. This led us to propose an improved mean shift tracking algorithm combining pixel probability and orientation. Let {xi} i = 1, 2, . . ., n and b(xi) be the pixel position of a normalized target template area and its gray value, respectively. At the same time, due to the occlusion and changes of light, whether some pixels belong to a certain target or not is unknown. We assign the same weight value to the pixels near the target center. However, when the distance exceeds our preset value, we use a distribution to describe the probability. In the present study, an exponential probability distribution is given by 8 1 < jjXjjTðyÞ jjXjjTðyÞ (4) RðxÞ ¼ jjXjj TðyÞ : l exp l maxfjjXjjTðyÞgni¼1 where T(y) is the threshold, and || || represents the norm of the gray vector X, and l = 0.8. Assume that the target model is quantized into m bins (orientation codes) and n gray levels. The modified gray level probability density function of the target template (gray feature u = 1, . . ., n) is calculated as n X (5) quv ¼ C RðxÞd½b1 ðxi Þud½b2 ðxi Þv i¼1
where the function b1 : R2 ! {0, 1, . . ., m 1} indicates that b(xi) is mapped into the corresponding orientation code indexes, the function b2 : R2 ! {0, 1, . . ., n 1} shows that b(xi) is mapped into the corresponding gray-level quantitative indexes, d is the Delta function, v is the highest gray level in feature space and C is the corresponding normalization constant. Gray probability density function of candidate model from the target candidate region centered at position y is given by n X (6) quv ðyÞ ¼ Ch RðxÞDd½b1 ðxi Þud½b2 ðxi Þv i¼1
where D = k(||y xi/h||2), h is the dimension of the candidate target, and Ch the corresponding normalization constant. Other aspects of the improved tracking algorithm are similar to those in [15,17,18], with some modifications in light of the above formulation.
2.3. 2.2.
Divisive cells tracking
Inactive cells tracking
The cell with a small inter-frame movement is called inactive cell. The basic idea of tracking inactive cells consists of first
Cells have different life periods. If cells are in division, we will not be able to continue tracking them accurately, because the above described method can only track one target.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
5
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
When a cell is dividing, there are several properties [1–3]: (1) After division of metrocyte A, A will be split into two cells (A1 and A2), where one of them still stays at the position of cell A, the other cell is pushed away. (2) The area of the subcells will decrease, and the sum of subcell A1 and subcell A2 is approximately equal to A. (3) The distance between the two subcells is very close. Therefore, we firstly track one sub-cell using our improved mean shift algorithm, and then search the other cell through using the above properties. However, due to the uncertainty of cell movement, when a certain cell is dividing, another cell may jump out of the boundary of the image. In this case, although the total number of cells is the same as that of the previous frame, there still exist divided cells. Therefore, we need consider whether a cell will step out of the boundary or not. In order to solve the problem, we propose a context-based divisive cells tracking algorithm: Context-based divisive cells tracking. Input: The ith preprocessed frame image (i = 1, 2, . . ., n). Output: The final tracking results. While there exists an input frame in the last loop Step 1. Calculate the total number of targets N(i) and N(i 1) in the ith and (i 1)th frames, respectively. Step 2. If N(i) N(i 1) 6¼ 0, then jump to Step 4; otherwise, jump to Step 3. Step 3. If there exist some cells that jump out of boundary, then jump to Step 4; otherwise, track these cells using the improved mean shift, update i = i + 1 and go back to Step 1. Step 4. Firstly, track these cells by our improved mean shift algorithm. Then calculate the area S(i) of each cell that has been tracked successfully. Set these cells which area is reduced to half from the last search as A1, and find the cell A2 which area is closest to A1. Step 5. If A1 and A2 meet preset conditions (area and distance), then locate subcell A2, update i = i + 1, and go back to Step 1; otherwise, it indicates a failure tracking. End
2.4.
Active cells tracking
In the process of cell movement, sometimes certain cells will do random hop movement, which indicates a larger interframe movement. Now we consider such a situation when there are two cells in the direction of the trajectories. In this case, predicting only by trajectory may result in a failure tracking. The phenomenon of ‘‘cell hopping’’ in this section is the general problem of data association. Motion information can be exploited by the pixel's gray value variation between continuous two frames. Cell movement is a random process, namely, the moving trend of current frame cells is related to the following frames [3]. This led us to propose a so-called dynamic local prediction (DLP) algorithm, which uses the local information of known nodes to estimate the position of unknown nodes. More precisely, the algorithm uses the information between several local frames to predict cell movement. The algorithm aims at the validity prediction of one target.
In order to track them more accurately, we introduce the following parameters. (1) The distance of cell movement between two continuous frames. Assume the coordinate of cell A in a certain frame is (x, y), and its coordinate in the next frame is (x1, y1), then the Euclidean distance can be calculated as qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (7) s ¼ ðxx1 Þ2 þ ðyy1 Þ2 (2) The total distance of cell movement, calculated as S¼
n X si ;
n ¼ 200
(8)
i¼1
(3) The speed vi of cells, which reflects the active degree of cells, is given by s vi ¼ ðpixels=minÞ (9) t where t denotes the time interval between two frames, s is the distance of the movement. (4) The average speed v of cells. It is defined by v¼
X 1 n¼N1 v ðpixels=minÞ N1 i¼1 i
(10)
The cells average speed reflects the cells active degree in the whole image sequence. If the value is big, then the active degree is high. The details of the DLP algorithm are given as follows: DLP algorithm. Step 1. Take these positions of cells that are not tracked from the ith to (i + 10)th frame. Step 2. Take a centroid every two, three, five, and seven frames, respectively. For these centroids, calculate the average distance between the two continuous centroids. Step 3. Compute four centroids according to the obtained average distance, then get four prediction positions according to the centroids. Step 4. Seek more accurate positions through weighted summation of the four prediction positions. Step 5. Compute the angles between cell position, candidate position, and prediction position, respectively, and then normalize after computing the difference for the angles. Step 6. Compute the total distance, cell velocity V, and active coefficients of cells. Step 7. Get probability value through weighted summation of the above parameters, and then determine cell positions.
The notable DLP algorithm densely extracts the information of continuous centroids. It captures the trajectory information which enables us to detect the active cells. Weighted summation of random variables is investigated in connection with the law of large numbers and the law of iterated logarithm. The main aim is to eliminate the roughness and inconsistency of data. In what follows, we introduce the gray prediction (GP) algorithm to track active cells. The gray system theory was proposed by Professor Deng Julong in the late 1970s. Relative to the definitions of black
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
6
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
system and white system, a system with only part of information known is called gray system. As a new traverse discipline, it has shown strong science vitality and academic status, and formed a theory system whose main content consists of system analysis, information processing, modeling, forecasting, decision-making and controlling [20]. Gray system theory considers random processes as gray processes varying in a definite range and time-related, and it regards all sorts of random. Variables as gray variables change within a certain limit, so the method of gray data generation was introduced to trim disordered original data [19–21]. Five kinds of gray prediction methods have been proposed for their respective application [20–22]. In general, the gray model is GM(m, n)[19–22], where m is the order of the differential equation, and n is the number of variables. In a real control environment, GM(1, 1) is the most commonly used model in practice [19,22]. Cell movement is a dynamic process that changes constantly. It happens under special condition and it is random and uncertain. But its latency, development and occurrence are continuous, comparable and related [25]. So, its occurrence is relative to its past and present. This is why we can forecast its occurrence with GP algorithm. In our GP algorithm, we chose GM(1, 1). Due to the gray variable of original time series, sometimes the gray model is not stable, and the prediction result is not efficient. In order to improve the accuracy of prediction, we propose the following two improvements at the level of generation of input data series and small error probability. Assume that the original series of data is expressed as ð0Þ
p
ð0Þ
ð0Þ
ð0Þ
¼ ðp ð1Þ; p ð2Þ; . . .; p ðnÞÞ
(11)
(0)
where p denotes the original data series. In engineering practice, the raw data commonly show strong randomicity, so that if the raw data is directly used in the model, the prediction precision is usually not as high as we want, so the raw data need to be pre-processed. There are many data pre-processing methods for the raw strong randomicity data, such as weighted exponent method, policy factor added processing method, sliding average method, etc. In the paper, the linear weighted exponent smoothing method is employed to smooth strong randomicity sequence of raw data. Then the prediction model is established by using increment data series: xð0Þ ðkÞ ¼ apð0Þ ðk þ 1Þpð0Þ ðkÞ;
k ¼ 1; 2; . . .; n1
(12)
where a is the corresponding constant (here a is set as 1.2). Then the input data series of the gray model can be expressed as xð0Þ ¼ ðxð0Þ ð1Þ; xð0Þ ð2Þ; . . .; xð0Þ ðnÞÞ
(13)
In the next step, we will improve p value, which represents small error probability. In our model, p is defined as: p ¼ PfjeðkÞej < vS1 g
(14)
where e(k) denotes residual error, e the average of residual error, S1 the standard deviation of original data series, and v the probability value of a certain distribution (here we use Gaussian distribution to estimate the value), and the value of v depends on the selection of correct rate.
Other details of original gray model operations that are needed for building GP algorithm model can be found in [18–22].
3.
Results
3.1.
Data set
Our data set includes six intravital microscopy video sequences obtained from Johan Degerma at the University of Chalmers. Video recordings were made by a charge-coupled device (CCD) camera attached to an intravital microscope, and cells were not dyed. The video frames were recorded at a spatial resolution of 640 480 pixels. Each frame reflects the state of cells under the influence of chemical catalyst. Note that, in order to activate the cell's normal movement, chemical stimulator catalyst was used. These cells primarily come from neurons, and are self-renewing, multipotent cells that generate the main phenotype of the nervous system. They are characterized by their capability to differentiate into multiple cell types via exogenous stimuli from their environment. They undergo asymmetric cell division into two daughter cells, one non-specialized and the other specialized. These cells were grown at 37 8C and 5% CO2 atmosphere in a humidified incubator.
3.2.
Results on segmentation
Now we will measure the performance of the proposed methods in Section 2. The performance of adhesive cell separation algorithm is shown in Fig. 3(a) and (b). As it is shown, all adhesive cells are separated perfectly. To verify our segmentation algorithm, we compare it with the segmentation results of the Multistage Morphological Method (MO) [32], the morphological flattening (MF) [36], the watershed method (WS) [36], the connectivity based merging and coupled level sets [23,27]. Table 1 shows the comparison results, it can be seen that our algorithm with a higher accuracy is very high in processing speed and is more effective in a real-time environment.
3.3.
Results of cell tracking
Fig. 4(a)–(f) shows the results of inactive cells tracking over different frames, where the final tracking results are indicated
Table 1 – Comparison of different segmentation algorithms. Algorithm
Time
Correct segmentation rate
MO [32] MF [36] WS [36] Connectivity based merging [23] Coupled level sets [27] Ours
769 ms 483 ms 620 ms 465 ms
98.2% 95.7% 97.9% 97.8%
589 ms
98.7%
180 ms
98.9%
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
7
Fig. 3 – Adhesive cell separation: (a) Adhesive cells; (b) Result of adhesive cell separation algorithm.
using green labels. As is shown, we can successfully track them using our improved mean shift algorithm. Fig. 5(a)–(d) shows the detected results using context-based divisive cells tracking algorithm. The results of active cells tracking are given in Fig. 6(a)–(d). As can be seen in Fig. 6(b), the conventional mean shift algorithm has failed to track active cells. That is because that cells do a larger inter-frame movement, in which case, the performance of mean shift algorithm degrades significantly. On the contrary, the DLP and GP algorithms both have a good performance in tracking active cells. Since there are visually few differences between the two results in Fig. 6(c) and (d), we will give more quantitative
analysis of the above results. For the DLP algorithm, we compute the active degree of cells according to the formula in (10). Over ten frames, the total movement distances for the two cells A and B (see Fig. 7) are respectively 79.3972 and 45.6247. The active degree is set as p3, the probability at A1 (candidate position 1) is set as p4, and the probability at A2 (candidate position 2) is set as p5. A3 denotes the predicted position. a1, a2 and a3 denote the angle between the current position and A1, A2, A3, respectively. ja2 a1j is set as p1, and ja3 a3j is set as p2. p1 and p2 indicate the distance between the cell's current position and each of the candidate positions A1 and A2, p3 denotes the active degree of the current cell, and p4 and p5 reflect the probability of the current cell at position 1 and 2
Fig. 4 – Results of inactive cells tracking: (a) Second frame; (b) Forth frame; (c) Sixth frame; (d) Eighth frame; (e) Tenth frame; (f) Twelfth frame. Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
8
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
Fig. 5 – Results of divisive cells tracking over: (a) the second frame; (b) the forth frame; (c) the eighth frame.
Fig. 6 – Results of active cells tracking: (a) Active cells to be tracked; (b)–(d) are tracking results obtained using (b) the mean shift algorithm; (c) the DLP algorithm; (d) the GP algorithm.
respectively. We can then predict the final positions, as shown in Table 2. For example, in the first row of Table 2, p2 < p1 (i.e., candidate position A2 is closer to the current position) and p5 > p4 (i.e., the cell is more likely to be at candidate position 2), then we can conclude position A2 as the predicted position of the cell. While for the GP algorithm, through analyzing and
calculating the gray model, these vertical coordinates of cells centroids can be calculated, which results are given in Table 3. Table 3 gives these predicted positions of cell A, while for the cell B, their positions can be obtained using the same method. Finally, through analyzing the above results, we can obtain the movement trajectory of cells, as shown in Fig. 7. In this denotes the four predicted positions of cell A, figure, denotes the final predicted position obtained using DLP, denotes the final predicted position obtained by GP algorithm. denotes the four predicted positions of cell B, denotes the final predicted position given by DLP, denotes the final predicted position given by GP algorithm. It can be seen that the position given by GP algorithm is clearly more accurate. In order to demonstrate the effectiveness of our algorithm, we have tested thousands of cells. We compare our algorithm with the state-of-the-art approaches, including the fast topology-constrained level-set approach proposed in [11], the affine transformation (AT) invariance approach of [6], the adaptive tracking method combining motion and topological features
Table 2 – Predicting positions for the two cells shown in Fig. 7. Fig. 7 – Schematic diagram of cell trajectory. (For interpretation of the references to color in the text, the reader is referred to the web version of the article.)
Cell A B
p1
p2
p3
p4
p5
Final result
0.3980 0.1943
0.0549 0.3067
0.2647 0.1521
0.1210 0.4637
0.5334 0.1302
A2 A1
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10 biocybernetics and biomedical engineering xxx (2015) xxx–xxx
Table 3 – Simulation of data. Value of model _0
x ð2Þ ¼ 110:4128 x ð3Þ ¼ 105:3364 _ð0Þ x ð4Þ ¼ 102:0891 _ð0Þ x ð5Þ ¼ 98:1656 _ð0Þ x ð6Þ ¼ 94:3929 _ð0Þ x ð7Þ ¼ 100:7652 _ð0Þ x ð8Þ ¼ 87:2769 _ð0Þ x ð9Þ ¼ 83:9227 _ð0Þ x ð10Þ ¼ 80:6974 _ð0Þ x ð11Þ ¼ 80:6974 _ð0Þ
Real value
Residual error E(k)
Relative error e(k)
106.2933 105.3364 104.5028 97.6461 95.8020 101.3993 86.6796 80.6068 75.4976 62.9100
4.1195 0.8330 2.4137 0.5195 1.4091 0.6341 0.5973 3.3159 5.1998 2.6357
0.0388 0.0079 0.0231 0.0053 0.0147 0.0063 0.0069 0.0411 0.0689 0.0902
(ATCMT) [14]. The results are shown in Table 4. As is shown, our algorithm with a higher accuracy has the advantages of high detection speed and strong practicability. The results in [6,14] is very close to our results, that is because that the methods in [6,14] focus on the segmentation of the nucleus instead of on the entire cellular structure, so it is effective even in the case of variable cell morphologies (with different cell cultures), cell partial overlaps, and dynamic changes in the cell-shape during migration. Our tracking system runs at an average speed of 6 frames/ min for processing the images in our experiments on a workstation with a 3 GHz processor and 2 GB memory. So the system is well suited for tracking during acquisition.
4.
Discussion
In this work, we developed an automatic tracking system of neural stem cells and compared its performance with the state-of-the-art. Four performance measures were used to evaluate the proposed tracking system, i.e., the correct segmentation rate, which quantifies the segmentation accuracy, and the residual error, the relative error and tracking accuracy rate, which measure the robustness of tracking. As it shows, our segmentation method is semi-automatic, because the operator initiates it and adjusts its parameters according to the segmented image and object properties, and, therefore, it is much faster, and the results are more
Table 4 – Comparison of cell tracking accuracy and processing time with state-of-the-art. Sequence
Num
1 2 3 4 5 6 Total Tracking accuracy rate Average time
270 320 290 300 270 300 1750
Level-set [11] 265 311 282 289 266 293 1706 97.48%
1630 ms
AT [6] 261 306 280 279 260 290 1676 95.77%
1370 ms
ATCMT [14] 263 312 281 288 267 294 1705 97.42%
1478 ms
Ours 264 313 282 290 265 295 1709 97.66%
1000 ms
9
reproducible than with any manual segmentation results; the method can cope very well with noisy images; the object position error is small, and therefore, the tracking method can be used for observing and quantifying the movement of a nonrigid object. The watershed method and the connectivity based merging demonstrated similar performance for cell segmentation. The multistage morphological method and the coupled level sets had a slightly lower accuracy than our segmentation method. However, our processing time is obviously smallest in all the compared methods. The affine transformation invariance approach and the adaptive tracking approach combining motion and topological features demonstrated similar performance for cell tracking, and these two methods have a slightly lower detection rates than our tracking method. In general, we find that if a feature is sufficiently well defined that it can be tracked or identified visually, the algorithm can also track or identify it. Cell tracking is a relatively independent subject that touches wider areas. There are still many problems yet to be solved perfectly. For example, to reduce the dependency on the human operator by employing a pre-segmentation stage which would preview the sequence and adjust the parameters of the method; to develop methods to deal with clustered cells, which are in contact with each other. Further research will be required to address this problem.
5.
Conclusions
In this paper, we have proposed a complete system for the biological cells detection and tracking in a time-lapse microscopy. The tracking system is capable of segmenting adhesive cells and tracking accurately each cell in video sequences. The proposed system described in this paper combines two approaches: the segmentation-based approach and the tracking-based approach. The system is designed for processing a sequence of images with a single non-rigid object changing position and shape over time. The system exploits temporal and spatial contextual information and utilizes it for reducing any uncertainty in the cell border and increasing the speed of segmentation. When the combined method is used to segment an object in a single image, the method is effective. Segmentation and tracking experiments on real data demonstrated that the proposed system has a superior performance over existing works in both accuracy and processing speed.
Financial support This research is partly supported by NSFC, China (No: 61375048).
Acknowledgment The authors would like to thank Johan Degerma at the University of Chalmers, for providing microscopic image sequences for this research and valuable inputs on biological properties of neural stem cells.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001
BBE-89; No. of Pages 10
10
biocybernetics and biomedical engineering xxx (2015) xxx–xxx
references
[1] Li Z, Hu S, Ghosh Z, Han Z, Wu JC. Functional comparison and expression profiling of human induced pluripotent stem cell and embryonic stem cell-derived endothelial cells. Stem Cells Dev 2011;20:1701–10. [2] Gu E, Chen WY, Gu J, Burridge P, Wu JC. Molecular imaging of stem cells: tracking survival, biodistribution, tumorigenicity, and immunogenicity. Theranostics 2012;2 (4):335–45. [3] Schroeder T. Imaging stem-cell-driven regeneration in mammals. Nature 2008;453(7193):345–51. [4] Ozturk Z, Sadettin S. Cell culture technology for pharmaceutical and cell-based therapies. New York: Marcel Dekker Inc; 2005. [5] Stephens DJ, Allan VJ. Light microscopy techniques for live cell imaging. Science 2003;300:82–6. [6] Lin ZL. An affine transformation invariance approach to cell tracking. Comput Med Imaging Graph 2008;32:554–65. [7] Li SC, Tachiki LM, Luo J, Dethlefs BA, Chen Z, Loudon WG. A biological global positioning system: considerations for tracking stem cell behaviors in the whole body. Stem Cell Rev 2010;6(2):317–33. [8] Lee PW, Hsu SH. Multifunctional core–shell polymeric nanoparticles for transdermal DNA delivery and epidermal Langerhans cells tracking. Biomaterials 2010;31:2425–34. [9] Sharp GC, Jiang SB, Shimizu S, Shirato H. Tracking errors in a prototype real-time tumour tracking system. Phys Med Biol 2004;49:5347–56. [10] Dufour A, Shinin V, Tajbakhsh S, Guillén-Aghion N, OlivoMarin JC, Zimmer C. Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces. IEEE Trans Image Process 2005;9:1396–410. [11] Padfield D, Rittscher J, Thomas N, Roysam B. Spatiotemporal cell cycle phase analysis using level sets and fast marching methods. Med Image Anal 2009;13:143–50. [12] Li K, Miller E, Weiss L, Campbell P, Kanade T. Online tracking of migrating and proliferating cells imaged with phase-contrast microscopy. Proceedings of the CVPRW; 2006. p. 65–72. [13] Dunn GA, Jones GE. Cell motility under the microscope: Vorsprung durch technik. Nat Rev 2004;5:667–72. [14] Dewan MAA, Ahmad MO, Swamy MNS. Tracking biological cells in time-lapse microscopy: an adaptive technique combining motion and topological features. IEEE Trans Biomed Eng 2011;58:1637–47. [15] Debeir O, Ham PV, Kiss R, Decaestecker C. Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes. IEEE Trans Med Imaging 2005;24:697–711. [16] Hossein M, Shahriar N. On robustness and localization accuracy of optical flow computation for underwater color images. Comput Vis Image Underst 2006;104:61–76.
[17] Cheng Y. Mean shift, mode seeking, and clustering. EEE Trans Pattern Anal Mach Intell 1995;17:790–9. [18] Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 2002;24:603–19. [19] Deng JL. Gray theory elements. Wu Han: Central China Science and Technology Publishing House; 2002. [20] Deng JL. Gray prediction and gray decision—making. Wu Han: Central China Science and Technology Publishing House; 2002. [21] Tien TL. A new grey prediction model FGM(1, 1). Math Comput Model 2009;49:1416–26. [22] Lin CB, Su SF, Hsu YT. High-precision forecast using grey models. Int J Syst Sci 2001;32:609–19. [23] Umesh Adiga PS, Chaudhuri BB. An efficient method based on watershed and rule-based merging for segmentation of 3-D histopathological images. Pattern Recognit 2001;34:1449–58. [24] Xu XY, Li BX. Adaptive rao-blackwellized particle filter and its evaluation for tracking in surveillance. IEEE Trans Imaging Process 2007;16:838–49. [25] Gerlich D, Mattes J, Eils R. Quantitative motion analysis and visualization of cellular structures. Methods 2003;29:3–13. [26] Sharp GC, Jiang SB, Shimizu S, Shirato H. Prediction of respiratory tumor motion for real-time image-guided radiotherapy. Phys Med Biol 2004;49:425–40. [27] Nath SK, Palaniappan K, Bunyak F. Cell segmentation using coupled level sets and graph-vertex coloring. MICCAI 2006;4190:101–8. [28] Weszka JS. A survey of threshold selection techniques. Comput Graph Image Process 1978;7:259–65. [29] Donald H, Baker PM. Computer graphics with OpenGL. New Jersey: Prentice-Hall International; 2004. [30] Korzynska A, Strojny W, Hoppe A, Wertheim D, Hoser P. Segmentation of microscope images of living cells. Pattern Anal Appl 2007;10(4):301–19. [31] Iwanowski M, Korzyńska A. Segmentation of moving cells in bright field and epi-fluorescent microscopic image sequences. Lecture notes in computer science. Springer-Verlag; 2010. [32] Korzyńska A, Iwanowski M. Multistage morphological segmentation of bright-field and fluorescent microscopy images. Opto-Electron Rev 2012;20(2):174–86. [33] Warowny M, Markiewicz T. Quick texture generation for multiobject image analysis in brain pathology. Przeglad Elektrotechniczny 2010;86(1):50–2. [34] Koprowski R, Izdebska-Straszak G, Wróbel Z, Adamek B. The cell structures segmentation with using of decision trees. Pattern Recognit Image Anal 2005;15(3):1–8. [35] Korzynska A. Neutrophils movement in vitro. Pattern Ann N Y Acad Sci 2002;972:139–43. [36] Korzynska A, Iwanowski M, Neuman U, Dobrowolska E, Hoser P. Comparison of the methods of microscopic image segmentation. World Congress on Medical Physics and Biomedical Engineering, vol. 25(4); 2009. pp. 425–8.
Please cite this article in press as: Zhang T, et al. Automatic tracking of neural stem cells in sequential digital images. Biocybern Biomed Eng (2015), http://dx.doi.org/10.1016/j.bbe.2015.10.001