- Email: [email protected]
Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS Optik xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system Hanshan Li ∗ , Junchai Gao School of Electronic Information Engineering, Xi’an Technological University, Xi’an 710032, China
a r t i c l e
i n f o
Article history: Received 14 July 2013 Accepted 6 January 2014 Available online xxx Keywords: Difference image Detection system Image processing arithmetic Target region extraction Ballistic trajectory
a b s t r a c t To solve the problem that the complexity background affects the dynamic target detection performance, which causes detection performance instability in dynamic target track system, this paper is to study target photoelectricity track method based on revolving image sensor, analyze dynamic targets track principle and track geometry relation on optical image track instrument, put forward the improved Mean Shift target track arithmetic and the improved difference image processing arithmetic to eliminate the background effect; research the positive and negative difference image processing algorithm and image target region extraction, analyze the flow of image processing arithmetic and derivate their calculation method by gathering target image in track detection system. Through experimentation gathering and processing target sequence image, the results show the target track method and processing arithmetic are accurate and feasible. © 2014 Elsevier GmbH. All rights reserved.
1. Introduction Dynamic target detection is an important part in aviation, aerospace, weapons guidance and control system, precise target location and detection will effectively improve the efficiency of the intelligent control system [1]. In dynamic target detection system, detection control platform is mainly to control core, which consists of synchronized control, target detection and recognition processing, and so on [2]. Due to environmental variability, target diversity that makes all kinds of dynamic target detection is not one and only, there are unpredictable factors, especially, when the target dynamic parameter changes in long-distance outside ballistic trajectory, it exists target motion acceleration, deceleration, the background environment variety, the loading platform inconsistency, which makes remote track type target detection also brings certain difficulty [3,4]. To make track target accurate in test system, the target image processing is very important, but, conventional processing algorithms have many shortcomings, for example, track system is instable, to solve those problems, this paper researched a new dynamic targets track detection system and improved image processing algorithm.
∗ Corresponding author. E-mail address: [email protected] (H. Li).
2. The dynamic targets photoelectricity track detection system and its track arithmetic 2.1. Dynamic targets track principle The dynamic targets track principle can be shown by Fig. 1. An denotes n optical image track detection instrument that forms continuum track view in scheduled orbit. In dynamic targets track detection system, every optical image track detection instrument has its own detection view, when the flying target enters their detection view, we use computer to gather and process targets image in order to achieve synchronization track target in scheduled orbit.
2.2. The track photoelectricity geometry relation of optical image track instrument in its detection view In dynamic targets track detection system, when the target is moving, the image sensor will revolve according to target displacement variety in orbit, their geometry relation can be shown by Fig. 2. Suppose, O1 O2 is scheduled orbit in an optical image track instrument. H is the vertical distance between optical image track detection instrument and O1 O2 , OO1 and OO2 are their view sides, d1 and d2 are side length, i and i+1 are revolving unit angles under even Si and Si+1 , here, we make Si+1 = Si when target moving in orbit, ˇ is the angle between OO1 and O1 O2 , the revolving angle
http://dx.doi.org/10.1016/j.ijleo.2014.01.089 0030-4026/© 2014 Elsevier GmbH. All rights reserved.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model
ARTICLE IN PRESS
IJLEO-54372; No. of Pages 6
H. Li, J. Gao / Optik xxx (2014) xxx–xxx
2
The characteristic of the first nucleus function is the gene summation, and the second nucleus function is ensured by spatial location and target center. We use nucleus function method to set up target characteristic probability density distributing. Suppose, u is denoted the eigenvalue probability density function in target search window, we use qˆ to denote their functions qˆ = C
n x −x 0 i
k ||
i=1
Fig. 1. The sketch map of dynamic targets track system.
h
||2 ı[b(xi ) − u]
(5)
In (5), x0 is central pixel coordinate in search window, xi is the number i pixel coordinate, k(||x||2 ) is nucleus function, h is bandwidth of nucleus function, namely, the target radius, b(xi ) is characteristic value, and ı is standard equation, C is unitary function [6]. In scheduled orbit dynamic targets track detection image system, the dynamic target image is sequence image, we may use the common characteristic in two neighborhood images to analyze their sequence image and ensure central position, suppose, y is central coordinate and nucleus function central position, {xi }i=1,...,nj denotes the number i pixel and u is their characteristic value, and then, its probability density function of pre-election target is pˆ u (y) = C
Fig. 2. The geometry relation on optical image track instrument.
n y−x i
k ||
i=1
is controlled by displacement variety in orbit, its moving range is decided by S, according to their geometry relation, when the moving target velocity is v, we can gain the expression (1) and (2) S=
n
(1)
Si
i = arctan
i
(d1 /
sin ˇ
S ) − cos k=1 k
ˇ
−
(y) ˆ ≡ (ˆp(y), qˆ ) = n
(2)
n=0
S is whole track distance in scheduled orbit, i is unit in S1 , the every image view in optical image track instrument is sum of every i . To realize synchronization real track in dynamic targets track detection system, we can gain the need control time ti , ti = Si /v, when we know ti , the revolving image sensor may be controlled accurately, and we can realize synchronization of real track target by combining the track arithmetic and image processing technology. Image processing technology is to look for target central point based on track arithmetic. In this paper, we research the improved Mean Shift target track arithmetic and the improved different image algorithm.
(6)
We use similar function to ensure image target by comparing with preelection target function, if the two functions are similar, and then, we may ensure the dynamic target positioning track system. Here, we use Bhattacharyya coefficient as similar function, it can be expressed by formula (7)
i=1 i−1
h
||2 ı[b(xi ) − u]
m
pˆ u (y) · qˆ u
(7)
u=1
The value of (y) ˆ is between 0 and 1, if the (y) ˆ is higher, the ˆ and Pˆ u (y) will more similar, which ensures track degree on (y) credibility. To make the (y) ˆ most, when we select y0 as previous frame image central coordinate, from this frame to look for the most matching target image, and then, we calculate beforehand choice model and carry thaler formula expanded in (y ˆ 0 ), and gain the most value of (y), ˆ the position is target. Because dynamic targets track detection system adopts side track method, the moved target has start, accelerate and decelerate state in the whole orbit, we use Mean Shift track method to gain image target central coordinate to reach dynamic track availably. 3. Target image processing algorithm
2.3. The improved Mean Shift target track arithmetic 3.1. The basic principle of difference image Based on the dynamic targets track detection system, we use the improved Mean Shift arithmetic to realize track target in every optical image track detection instrument [5]. The target model is probability density function on gray value and part standard difference, we use nucleus density to estimate and select Epanechnikov nucleus function to form multi-nucleus function, which can be expressed by formula (3) and (4)
K1 (x) =
K2 (x) =
3(h2 − x2 )/(4h3 ),
xh
0,
else
2(h2 − xT x)/(h3 ),
xT xh2
0,
else
(3)
(4)
In target detection system, if we need accurate synchronization track target, it is necessary to select moving target image feature. The previous frame and rear frame image will have obvious differences in target image, difference image is the premise that the dynamic target feature extract, the image on motion target was represented as previous and rear image intensity gray variation characteristics in whole integral time [7], we can identify motion target image gray feature changes on the basis of the image moving target characteristics, which can be described by adjacent serial image gray variation, suppose, dynamic target image was defined by formula (8) f (x, t1 , t2 ) = f2 (x, t2 ) − f1 (x, t1 )
(8)
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
3
under the target shape information without restriction conditions, we consider this is target when image gray and shape information model are near [11]. Suppose, there are n possible area in detection target image that gain from track system, according to the principle of combination, the total combined number C can be calculated by formula (9) C = n(n − 1)/2 Fig. 3. Difference image motion description.
In (8), f(x, t1 , t2 ) is a difference image. If the scenes have several independent motion targets, which move in optical image system, so, the difference image is motion target the combination effect. In dynamic target image, difference image can be seen to a kind of approximation derivative function, two point finite difference in image is middle point that locate df(x,y)/dt approximation derivative function in time interval of t2 − t1 ; the effect of gain difference image and static edge images are the same in a real dynamic image background, and the feature extraction represents image intensity variation process [8]. In order to describe the motion of the image in dynamic target detection, suppose, image of f1 (x, t1 ) contains a gray square area that its strength is g in time t1 , it can be shown by Fig. 3, this area in imaging system is based on a constant horizontal velocity v moving to the right, we make the image background intensity value “0”, the gray value will more than “0” in other position [9]. And then, f2 (x, t1 ) is an image in time t2 , when it moves to right, we can gain the result: the time interval of t2 − t1 , background relative intensity is “0” region will cover the original square surface, the processing results will contain a strength of −g region; the right square region will cover the background that intensity is “0”, the difference processing result will form g area, then, the two images exist the same square area that its strength after image difference processing is “0”, here, T = t2 − t1 . 3.2. Improved different target image algorithm Based on the difference image principle, we propose an improved method that not only adds background compensation mechanism but also will make dynamic background turn to static background, which can improve detection effect. In image processing, there are two compensation methods, one is using known background, for example, using difference image method to detect motion targets sequence image, and the other is based on an unknown background motion estimation algorithm, which will be the next state of the background image motion prediction, and then compensate the adjacent image background, which can eliminate the difficulty of target detection under complex background [10]. Positive and negative difference image method is based on the difference image compensation principle and the detection algorithm, under the condition of fixing field optical imaging target detection problem. First, we gain a goal image as a background, second, containing target image is compared with the background image, we can gain difference image by two image subtract operation algorithm. The core problem of this algorithm is that environmental background illumination changes caused image gray changes, which will cause some shadow interference. The whole track processing system, optical imaging system will be changed with the distance between target and lens, which will make the image definition have difference, so, the detection image needs modifying and processing. According to the difference image operation rules of fsub = |ft2 − ft1 |, ft2 and ft1 are two images corresponding to the gray value [8], they have the same gray and shape distribution mode. In order to find the target area in gain image,
(9)
If the target image exists ten regions, then the total combination number is forty-five based on formula (9). In these combinations, there may exist two noise regions or a noise region with a target region matching image, which causes mistake detection target. This kind of mistake detection with the combination number increase will be more likely to occur, which will cause detection rate low. In order to eliminate the number of combinations causing too much wrong judgment, we put forward the positive and negative difference image method to the detection moving targets in track system, the positive and negative difference image symmetry was used to omit the background difference compensation, and improve reliability and adaptability on dynamic target detection, so that the whole detection system is very stable. The algorithm of positive and negative difference image is the target image which was divided into two difference images according to image gray difference value, one is positive difference image Q(x, y), the other is negative difference image G(x, y), their representations can be expressed by formula (10) and (11)
Q (x, y) =
G(x, y) =
I1 (x, y) − I2 (x, y)
if (I1 (x, y) − I2 (x, y)) > 0
0
if (I1 (x, y) − I2 (x, y)) ≤ 0
I1 (x, y) − I2 (x, y)
if (I1 (x, y) − I2 (x, y)) < 0
0
if (I1 (x, y) − I2 (x, y))≥0
(10)
(11)
Here, I1 (x, y) and I2 (x, y) denote two moments of the image respectively in optical system. Positive and negative difference image are target region’s two properties, one is corresponding to the same target area that is located in the positive and negative difference image, and it distributes in a positive or negative image of the two regions that are not likely to appear corresponding in the same target in dynamic target detection [12]; the other is the number of target area in the absolute value of difference image for m, positive difference image is m1 , negative difference image is m2 , and then, m equals m1 adding to m2 . Based on two properties, when we use positive and negative difference image algorithm to detect moving target, the combination only needs matching the area of positive and negative image that will possibly appear target, and we do not need to consider the positive and negative difference image inside each image area, so, the total combination number k equals m1 × m2 . When m1 = 1 or m2 = 1, k takes the least value m − 1, if m1 = m2 /2, k takes the max m2 /4, k can be expressed by formula (12) and (13) m−1≤k ≤
m2 4
k m 2 ≤ ≤ m C 2(m − 1)
(12)
(13)
When m increases, then, m/2(m − 1) ≈ 1/2, so, the method of positive and negative difference image algorithm can reduce the number of combinations in dynamic target detection image, and improve the processing speed and enhance target detection rate.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model
ARTICLE IN PRESS
IJLEO-54372; No. of Pages 6
H. Li, J. Gao / Optik xxx (2014) xxx–xxx
4
4. Based on positive and negative difference image of moving target detection algorithm
preprocessing, edge detection, region labeling, Bi (x, y) denotes the processing result in fi (x, y), it can be described by formula (19)
4.1. Positive and negative difference image
Bi (x, y) =
Suppose, fi (x, y) is the gathering target sequence image fi (x, y) = (ai (x, y), bi (x, y), ci (x, y)), x ≤ W, y ≤ H, i = 0, 1, 2, . . .
(14)
Here, ai (x, y), bi (x, y), ci (x, y) respectively are the pixel point (x, y) in number i frame image, W and H respectively are width and height of target image, i is frame number of target sequence image [13]. We process the acquisition of sequence image based on the gathered image to determine the dynamic target field area. The difference image sequence di (x, y) can be expressed by formula (15) di (x, y) = (ai (x, y), bi (x, y), ci (x, y)), x ≤ W, y ≤ H, i = 0, 1, 2, . . .
(15)
In (15), ai (x, y), bi (x, y), ci (x, y) respectively are the pixel point (x, y) in number i frame image, their expression can be shown by formula (16)–(18) ai (x, y) = |ai (x, y) − ai−1 (x, y)|
(16)
bi (x, y) = |bi (x, y) − bi−1 (x, y)|
(17)
ci (x, y) = |ci (x, y) − ci−1 (x, y)|
(18)
Through the real-time acquisition the i and i + 1 frame image that comes from the original image in track system, Fig. 4 are the original image, Fig. 5 is processing image on positive and negative difference arithmetic. From the results of differential image processing, when the background image is relatively stable in detection system, we may use two adjacent frame image that gained from optical system to become true target location. In the process, according to difference image processing flow, we set a appropriate threshold value Ti , if the image gray is higher than this threshold, the pixel is set to “1”, or else, the pixel is set to “0” in target image [14]. By choosing the proper threshold, we gain sequence image by difference image
Fig. 4. Original image.
1,
||di (x, y)||∞ ≥T
0,
||di (x, y)||∞ < T
(19)
T is threshold value of gray in image, when the target image background occurs disturbance, the difference image not only reflects dynamic target edge and region, but also the background that exists perturbations will be detected, so, the result cannot accurately determine the position of the target. To remove these interferences, we make “AND” operation between i and i + 1 frame based on difference symmetry processing, symmetric difference images except the dynamic target edge points, there are still some isolated points, these points mainly are due to not completely overcoming the dithering of optical image camera, to eliminate the effects of dithering, and we must filter the symmetric difference image. The method of filtering detects each nonzero pixel point in neighborhood area whether there are other non-zero pixels, if there exists non-zero pixel point, which is more than a certain threshold, this pixel point is considered that not being caused by camera shaking or background jitter, but it is caused by moving target boundaries point, otherwise, it is considered noise. 4.2. Target images center extraction The difference image was processed to horizontal and vertical projection image in the moving target area; projective vector can be expressed by formula (20) and (21) Qi,X (x) =
W
di (x, y)
(20)
x=1
Qi,Y (y) =
H
di (x, y)
(21)
y=1
Here, Qi,X (x) and Qi,Y (y) respectively are number i frame in the horizontal direction coordinate x and in the vertical direction coordinate y for the projection value, W and H respectively are width and height of target image, di (x, y) is number i frame of difference image gray value after image filtering. The particular processing flow can be described as follows. To horizontal direction, first, suppose, flag is initialization marking variable, flag = 0,z is horizontal motion block of the target region, z = 0, startx[i] is horizontal boundary starting point, startx[i] = 0, x = 0, endx[i] is ending point, we contrast the value of Qi,X (x) and the threshold value of T, if Qi,X (x) ≥ T and flag = 0, x[i] is the dynamic target region in horizontal direction and the starting point of the boundary, and then, we make flag = 1,z = z + 1, startx[i] = x, if Qi,X (x) < T and flag = 1,x[i] is a dynamic target area in horizontal direction and the termination of boundary point. Second, suppose, flag = 0, endx[z] = x; if x = x + 1, we repeat judgment and calculate according to above arithmetic, when x = W, Qi,X (x) ≥ T and flag = 1, x is moving target area in horizontal direction and the termination of boundary point, the same principle, we can determine the direction perpendicular to the boundary point. 5. Experiments and processing
Fig. 5. Difference processing image.
According to the above calculation principle of different image arithmetic, we process the target sequence image. The optical image system was located in ballistic trajectory orbit side, the distance about 600 m between the orbit and image system, the gathered frequency is 10 Hz, the optical view is about 45◦ , the scope of turning angle of lens is about 68◦ , computer was used
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
5
Fig. 6. The result of processing image.
Fig. 9. The histogram of gradient by Sobel operator.
Fig. 7. Detection and processing sequence images in start a certain time.
to memorize target sequence image, according to the track measurement principle, we gather different time sequence images and apply positive and negative difference image algorithm and target images region extraction to processing image under complexity background. Fig. 6 is to gather and process the image result in orbit for a certain period of time, Fig. 6(a) is to gather six frames original image comprising dynamic target, Fig. 6(b) is processing results by positive and negative difference image algorithm, and positive and negative difference image. In order to verify the target detection algorithm, according to difference image algorithm and target images region extraction, we gather the starting of a certain time target sequence image in middle ballistic trajectory, such as Fig. 7, the right image is the first frame, Fig. 7(a) is the acquisition of the original infrared sequence image, Fig. 7(b) is image of filtering processing, and positive and negative difference image. Fig. 8 is the middle of a certain time target sequence image in ballistic trajectory, the right image is the first frame, Fig. 8(a) is the acquisition of the original infrared sequence image, Fig. 8(b) is image of filtering processing and positive and negative difference image. By processing and analysis, we know that different target point image size is inconsistent in the whole of the track rail, the longer the distance between target and optical system, the smaller the
Fig. 8. Detection and processing sequence images in middle a certain time.
Fig. 10. The histogram of gradient by OTSU operator.
image size in image detector, this mainly is detection turntable in the rotating process and orbital distance is different, which will lead to different imaging distance in optical image system, so, the image size is different.we use Sobel operator and OTSU operator respectively to process the original image with threshold segmentation based on gradient magnitude, the histogram of gradient mean can be shown in Figs. 9 and 10. According to the image process result, the pixels number in adjacent frame can be calculated by using computer image processing and gain the target position in dynamic targets track detection system. Through calculation and analysis, the dynamic targets photoelectricity track detection system and its track arithmetic are scientific and feasible, which can improve photoelectricity track detection system on the photoelectricity track detection system. 6. Conclusions Based on orbit target detection platform, this paper put forward the improved image difference method processing arithmetic and analyzed the detection target image processing method, studied the positive and negative difference image processing algorithm and image target region extraction. At last, we combine detection system, the improved processing method was validated; this illuminates that the improved method is accurate and feasible. However, the difference image method also has some disadvantages, for example, when the environmental background is more complex and detection platform brings dithering, the target has some slur that will affect the target information extraction, if the background changes relatively obvious, the measurement system will be thought that it is the target and interference, which is not conducive to the target detection.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
6
Acknowledgement This work has been supported by Project of the National Natural Science Foundation of China (No. 61107079). References [1] R.H. Zhan, Q. Xin, J. Wan, Modified unscented particle filter for nonlinear Bayesian tracking, J. Syst. Eng. Electron. 19 (2008) 7–14. [2] S.K. Jung, K.Y. Wohn, A model-based 3-D tracking of rigid objects from a sequence of multiple perspective views, Pattern Recognit. Lett. 20 (1998) 499–512. [3] H.S. Li, Z.H. Yaun, Z.Y. Lei, Infrared radiation characteristics analysis of high velocity projectile in detection screen, Infrared Laser Eng. 38 (2009) 777–781. [4] D. Comaniciu, V. Ramesh, P. Meer, Real-time tracking of non-rigid objects using mean shift, IEEE Int. Conf. Comput. Vision Pattern Recognit. 2 (2000) 142–149. [5] D. Comanieiu, P. Meer, A robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach. Intell. 24 (5) (2002) 603–619.
[6] H.S. Li, L.P. Lu, Z.Y. Lei, Infrared heat radiation detection performance influence of screen thickness and atmosphere on projectile, J. Nanjing Univ. Sci. Technol. 35 (2011) 298–303. [7] J.S. Hu, C.W. Juan, J.J. Wang, A spatial-color mean-shift object tracking algorithm with scale and orientation estimation, Pattern Recognit. Lett. 29 (2008) 2165–2173. [8] D. Yan, Z.Y. Lei, Algorithm research in high-speed object parameter validation of exterior trajectory, Opt. Tech. 33 (2007) 816–818. [9] G.W. Gao, S.Q. Liu, H.L. Qin, Auto-tracking algorithm of infrared target under complex background, Opto-Electron. Eng. 37 (2010) 78–84. [10] J. Yan, M.Y. Wu, S.Z. Chen, Camouflaged target tracking based on mean shift, Engineering 36 (2009) 11–15. [11] N. Aleixos, J. Blasco, E. Motto, F. Navarron, Assessment of citrus fruit quality using a real-time machine vision system, Pattern Recognit. 30 (2000) 482–485. [12] B. Wei, L. Zheng, K.Y. Wang, A new method for detecting plane target by imaging in laser fuze, Opt. Eng. 32 (2005) 36–40. [13] X.P. Luo, J. Tian, Z.Y. Zhu, A survey of image segmentation, Pattern Recognit. Artif. Intell. 12 (1999) 300–312. [14] Q.S. Li, Research on miss distance in optical measurement processing, Acta Photon. Sin. 28 (1999) 255–259.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
ARTICLE IN PRESS Optik xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system Hanshan Li ∗ , Junchai Gao School of Electronic Information Engineering, Xi’an Technological University, Xi’an 710032, China
a r t i c l e
i n f o
Article history: Received 14 July 2013 Accepted 6 January 2014 Available online xxx Keywords: Difference image Detection system Image processing arithmetic Target region extraction Ballistic trajectory
a b s t r a c t To solve the problem that the complexity background affects the dynamic target detection performance, which causes detection performance instability in dynamic target track system, this paper is to study target photoelectricity track method based on revolving image sensor, analyze dynamic targets track principle and track geometry relation on optical image track instrument, put forward the improved Mean Shift target track arithmetic and the improved difference image processing arithmetic to eliminate the background effect; research the positive and negative difference image processing algorithm and image target region extraction, analyze the flow of image processing arithmetic and derivate their calculation method by gathering target image in track detection system. Through experimentation gathering and processing target sequence image, the results show the target track method and processing arithmetic are accurate and feasible. © 2014 Elsevier GmbH. All rights reserved.
1. Introduction Dynamic target detection is an important part in aviation, aerospace, weapons guidance and control system, precise target location and detection will effectively improve the efficiency of the intelligent control system [1]. In dynamic target detection system, detection control platform is mainly to control core, which consists of synchronized control, target detection and recognition processing, and so on [2]. Due to environmental variability, target diversity that makes all kinds of dynamic target detection is not one and only, there are unpredictable factors, especially, when the target dynamic parameter changes in long-distance outside ballistic trajectory, it exists target motion acceleration, deceleration, the background environment variety, the loading platform inconsistency, which makes remote track type target detection also brings certain difficulty [3,4]. To make track target accurate in test system, the target image processing is very important, but, conventional processing algorithms have many shortcomings, for example, track system is instable, to solve those problems, this paper researched a new dynamic targets track detection system and improved image processing algorithm.
∗ Corresponding author. E-mail address: [email protected] (H. Li).
2. The dynamic targets photoelectricity track detection system and its track arithmetic 2.1. Dynamic targets track principle The dynamic targets track principle can be shown by Fig. 1. An denotes n optical image track detection instrument that forms continuum track view in scheduled orbit. In dynamic targets track detection system, every optical image track detection instrument has its own detection view, when the flying target enters their detection view, we use computer to gather and process targets image in order to achieve synchronization track target in scheduled orbit.
2.2. The track photoelectricity geometry relation of optical image track instrument in its detection view In dynamic targets track detection system, when the target is moving, the image sensor will revolve according to target displacement variety in orbit, their geometry relation can be shown by Fig. 2. Suppose, O1 O2 is scheduled orbit in an optical image track instrument. H is the vertical distance between optical image track detection instrument and O1 O2 , OO1 and OO2 are their view sides, d1 and d2 are side length, i and i+1 are revolving unit angles under even Si and Si+1 , here, we make Si+1 = Si when target moving in orbit, ˇ is the angle between OO1 and O1 O2 , the revolving angle
http://dx.doi.org/10.1016/j.ijleo.2014.01.089 0030-4026/© 2014 Elsevier GmbH. All rights reserved.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model
ARTICLE IN PRESS
IJLEO-54372; No. of Pages 6
H. Li, J. Gao / Optik xxx (2014) xxx–xxx
2
The characteristic of the first nucleus function is the gene summation, and the second nucleus function is ensured by spatial location and target center. We use nucleus function method to set up target characteristic probability density distributing. Suppose, u is denoted the eigenvalue probability density function in target search window, we use qˆ to denote their functions qˆ = C
n x −x 0 i
k ||
i=1
Fig. 1. The sketch map of dynamic targets track system.
h
||2 ı[b(xi ) − u]
(5)
In (5), x0 is central pixel coordinate in search window, xi is the number i pixel coordinate, k(||x||2 ) is nucleus function, h is bandwidth of nucleus function, namely, the target radius, b(xi ) is characteristic value, and ı is standard equation, C is unitary function [6]. In scheduled orbit dynamic targets track detection image system, the dynamic target image is sequence image, we may use the common characteristic in two neighborhood images to analyze their sequence image and ensure central position, suppose, y is central coordinate and nucleus function central position, {xi }i=1,...,nj denotes the number i pixel and u is their characteristic value, and then, its probability density function of pre-election target is pˆ u (y) = C
Fig. 2. The geometry relation on optical image track instrument.
n y−x i
k ||
i=1
is controlled by displacement variety in orbit, its moving range is decided by S, according to their geometry relation, when the moving target velocity is v, we can gain the expression (1) and (2) S=
n
(1)
Si
i = arctan
i
(d1 /
sin ˇ
S ) − cos k=1 k
ˇ
−
(y) ˆ ≡ (ˆp(y), qˆ ) = n
(2)
n=0
S is whole track distance in scheduled orbit, i is unit in S1 , the every image view in optical image track instrument is sum of every i . To realize synchronization real track in dynamic targets track detection system, we can gain the need control time ti , ti = Si /v, when we know ti , the revolving image sensor may be controlled accurately, and we can realize synchronization of real track target by combining the track arithmetic and image processing technology. Image processing technology is to look for target central point based on track arithmetic. In this paper, we research the improved Mean Shift target track arithmetic and the improved different image algorithm.
(6)
We use similar function to ensure image target by comparing with preelection target function, if the two functions are similar, and then, we may ensure the dynamic target positioning track system. Here, we use Bhattacharyya coefficient as similar function, it can be expressed by formula (7)
i=1 i−1
h
||2 ı[b(xi ) − u]
m
pˆ u (y) · qˆ u
(7)
u=1
The value of (y) ˆ is between 0 and 1, if the (y) ˆ is higher, the ˆ and Pˆ u (y) will more similar, which ensures track degree on (y) credibility. To make the (y) ˆ most, when we select y0 as previous frame image central coordinate, from this frame to look for the most matching target image, and then, we calculate beforehand choice model and carry thaler formula expanded in (y ˆ 0 ), and gain the most value of (y), ˆ the position is target. Because dynamic targets track detection system adopts side track method, the moved target has start, accelerate and decelerate state in the whole orbit, we use Mean Shift track method to gain image target central coordinate to reach dynamic track availably. 3. Target image processing algorithm
2.3. The improved Mean Shift target track arithmetic 3.1. The basic principle of difference image Based on the dynamic targets track detection system, we use the improved Mean Shift arithmetic to realize track target in every optical image track detection instrument [5]. The target model is probability density function on gray value and part standard difference, we use nucleus density to estimate and select Epanechnikov nucleus function to form multi-nucleus function, which can be expressed by formula (3) and (4)
K1 (x) =
K2 (x) =
3(h2 − x2 )/(4h3 ),
xh
0,
else
2(h2 − xT x)/(h3 ),
xT xh2
0,
else
(3)
(4)
In target detection system, if we need accurate synchronization track target, it is necessary to select moving target image feature. The previous frame and rear frame image will have obvious differences in target image, difference image is the premise that the dynamic target feature extract, the image on motion target was represented as previous and rear image intensity gray variation characteristics in whole integral time [7], we can identify motion target image gray feature changes on the basis of the image moving target characteristics, which can be described by adjacent serial image gray variation, suppose, dynamic target image was defined by formula (8) f (x, t1 , t2 ) = f2 (x, t2 ) − f1 (x, t1 )
(8)
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
3
under the target shape information without restriction conditions, we consider this is target when image gray and shape information model are near [11]. Suppose, there are n possible area in detection target image that gain from track system, according to the principle of combination, the total combined number C can be calculated by formula (9) C = n(n − 1)/2 Fig. 3. Difference image motion description.
In (8), f(x, t1 , t2 ) is a difference image. If the scenes have several independent motion targets, which move in optical image system, so, the difference image is motion target the combination effect. In dynamic target image, difference image can be seen to a kind of approximation derivative function, two point finite difference in image is middle point that locate df(x,y)/dt approximation derivative function in time interval of t2 − t1 ; the effect of gain difference image and static edge images are the same in a real dynamic image background, and the feature extraction represents image intensity variation process [8]. In order to describe the motion of the image in dynamic target detection, suppose, image of f1 (x, t1 ) contains a gray square area that its strength is g in time t1 , it can be shown by Fig. 3, this area in imaging system is based on a constant horizontal velocity v moving to the right, we make the image background intensity value “0”, the gray value will more than “0” in other position [9]. And then, f2 (x, t1 ) is an image in time t2 , when it moves to right, we can gain the result: the time interval of t2 − t1 , background relative intensity is “0” region will cover the original square surface, the processing results will contain a strength of −g region; the right square region will cover the background that intensity is “0”, the difference processing result will form g area, then, the two images exist the same square area that its strength after image difference processing is “0”, here, T = t2 − t1 . 3.2. Improved different target image algorithm Based on the difference image principle, we propose an improved method that not only adds background compensation mechanism but also will make dynamic background turn to static background, which can improve detection effect. In image processing, there are two compensation methods, one is using known background, for example, using difference image method to detect motion targets sequence image, and the other is based on an unknown background motion estimation algorithm, which will be the next state of the background image motion prediction, and then compensate the adjacent image background, which can eliminate the difficulty of target detection under complex background [10]. Positive and negative difference image method is based on the difference image compensation principle and the detection algorithm, under the condition of fixing field optical imaging target detection problem. First, we gain a goal image as a background, second, containing target image is compared with the background image, we can gain difference image by two image subtract operation algorithm. The core problem of this algorithm is that environmental background illumination changes caused image gray changes, which will cause some shadow interference. The whole track processing system, optical imaging system will be changed with the distance between target and lens, which will make the image definition have difference, so, the detection image needs modifying and processing. According to the difference image operation rules of fsub = |ft2 − ft1 |, ft2 and ft1 are two images corresponding to the gray value [8], they have the same gray and shape distribution mode. In order to find the target area in gain image,
(9)
If the target image exists ten regions, then the total combination number is forty-five based on formula (9). In these combinations, there may exist two noise regions or a noise region with a target region matching image, which causes mistake detection target. This kind of mistake detection with the combination number increase will be more likely to occur, which will cause detection rate low. In order to eliminate the number of combinations causing too much wrong judgment, we put forward the positive and negative difference image method to the detection moving targets in track system, the positive and negative difference image symmetry was used to omit the background difference compensation, and improve reliability and adaptability on dynamic target detection, so that the whole detection system is very stable. The algorithm of positive and negative difference image is the target image which was divided into two difference images according to image gray difference value, one is positive difference image Q(x, y), the other is negative difference image G(x, y), their representations can be expressed by formula (10) and (11)
Q (x, y) =
G(x, y) =
I1 (x, y) − I2 (x, y)
if (I1 (x, y) − I2 (x, y)) > 0
0
if (I1 (x, y) − I2 (x, y)) ≤ 0
I1 (x, y) − I2 (x, y)
if (I1 (x, y) − I2 (x, y)) < 0
0
if (I1 (x, y) − I2 (x, y))≥0
(10)
(11)
Here, I1 (x, y) and I2 (x, y) denote two moments of the image respectively in optical system. Positive and negative difference image are target region’s two properties, one is corresponding to the same target area that is located in the positive and negative difference image, and it distributes in a positive or negative image of the two regions that are not likely to appear corresponding in the same target in dynamic target detection [12]; the other is the number of target area in the absolute value of difference image for m, positive difference image is m1 , negative difference image is m2 , and then, m equals m1 adding to m2 . Based on two properties, when we use positive and negative difference image algorithm to detect moving target, the combination only needs matching the area of positive and negative image that will possibly appear target, and we do not need to consider the positive and negative difference image inside each image area, so, the total combination number k equals m1 × m2 . When m1 = 1 or m2 = 1, k takes the least value m − 1, if m1 = m2 /2, k takes the max m2 /4, k can be expressed by formula (12) and (13) m−1≤k ≤
m2 4
k m 2 ≤ ≤ m C 2(m − 1)
(12)
(13)
When m increases, then, m/2(m − 1) ≈ 1/2, so, the method of positive and negative difference image algorithm can reduce the number of combinations in dynamic target detection image, and improve the processing speed and enhance target detection rate.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model
ARTICLE IN PRESS
IJLEO-54372; No. of Pages 6
H. Li, J. Gao / Optik xxx (2014) xxx–xxx
4
4. Based on positive and negative difference image of moving target detection algorithm
preprocessing, edge detection, region labeling, Bi (x, y) denotes the processing result in fi (x, y), it can be described by formula (19)
4.1. Positive and negative difference image
Bi (x, y) =
Suppose, fi (x, y) is the gathering target sequence image fi (x, y) = (ai (x, y), bi (x, y), ci (x, y)), x ≤ W, y ≤ H, i = 0, 1, 2, . . .
(14)
Here, ai (x, y), bi (x, y), ci (x, y) respectively are the pixel point (x, y) in number i frame image, W and H respectively are width and height of target image, i is frame number of target sequence image [13]. We process the acquisition of sequence image based on the gathered image to determine the dynamic target field area. The difference image sequence di (x, y) can be expressed by formula (15) di (x, y) = (ai (x, y), bi (x, y), ci (x, y)), x ≤ W, y ≤ H, i = 0, 1, 2, . . .
(15)
In (15), ai (x, y), bi (x, y), ci (x, y) respectively are the pixel point (x, y) in number i frame image, their expression can be shown by formula (16)–(18) ai (x, y) = |ai (x, y) − ai−1 (x, y)|
(16)
bi (x, y) = |bi (x, y) − bi−1 (x, y)|
(17)
ci (x, y) = |ci (x, y) − ci−1 (x, y)|
(18)
Through the real-time acquisition the i and i + 1 frame image that comes from the original image in track system, Fig. 4 are the original image, Fig. 5 is processing image on positive and negative difference arithmetic. From the results of differential image processing, when the background image is relatively stable in detection system, we may use two adjacent frame image that gained from optical system to become true target location. In the process, according to difference image processing flow, we set a appropriate threshold value Ti , if the image gray is higher than this threshold, the pixel is set to “1”, or else, the pixel is set to “0” in target image [14]. By choosing the proper threshold, we gain sequence image by difference image
Fig. 4. Original image.
1,
||di (x, y)||∞ ≥T
0,
||di (x, y)||∞ < T
(19)
T is threshold value of gray in image, when the target image background occurs disturbance, the difference image not only reflects dynamic target edge and region, but also the background that exists perturbations will be detected, so, the result cannot accurately determine the position of the target. To remove these interferences, we make “AND” operation between i and i + 1 frame based on difference symmetry processing, symmetric difference images except the dynamic target edge points, there are still some isolated points, these points mainly are due to not completely overcoming the dithering of optical image camera, to eliminate the effects of dithering, and we must filter the symmetric difference image. The method of filtering detects each nonzero pixel point in neighborhood area whether there are other non-zero pixels, if there exists non-zero pixel point, which is more than a certain threshold, this pixel point is considered that not being caused by camera shaking or background jitter, but it is caused by moving target boundaries point, otherwise, it is considered noise. 4.2. Target images center extraction The difference image was processed to horizontal and vertical projection image in the moving target area; projective vector can be expressed by formula (20) and (21) Qi,X (x) =
W
di (x, y)
(20)
x=1
Qi,Y (y) =
H
di (x, y)
(21)
y=1
Here, Qi,X (x) and Qi,Y (y) respectively are number i frame in the horizontal direction coordinate x and in the vertical direction coordinate y for the projection value, W and H respectively are width and height of target image, di (x, y) is number i frame of difference image gray value after image filtering. The particular processing flow can be described as follows. To horizontal direction, first, suppose, flag is initialization marking variable, flag = 0,z is horizontal motion block of the target region, z = 0, startx[i] is horizontal boundary starting point, startx[i] = 0, x = 0, endx[i] is ending point, we contrast the value of Qi,X (x) and the threshold value of T, if Qi,X (x) ≥ T and flag = 0, x[i] is the dynamic target region in horizontal direction and the starting point of the boundary, and then, we make flag = 1,z = z + 1, startx[i] = x, if Qi,X (x) < T and flag = 1,x[i] is a dynamic target area in horizontal direction and the termination of boundary point. Second, suppose, flag = 0, endx[z] = x; if x = x + 1, we repeat judgment and calculate according to above arithmetic, when x = W, Qi,X (x) ≥ T and flag = 1, x is moving target area in horizontal direction and the termination of boundary point, the same principle, we can determine the direction perpendicular to the boundary point. 5. Experiments and processing
Fig. 5. Difference processing image.
According to the above calculation principle of different image arithmetic, we process the target sequence image. The optical image system was located in ballistic trajectory orbit side, the distance about 600 m between the orbit and image system, the gathered frequency is 10 Hz, the optical view is about 45◦ , the scope of turning angle of lens is about 68◦ , computer was used
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
5
Fig. 6. The result of processing image.
Fig. 9. The histogram of gradient by Sobel operator.
Fig. 7. Detection and processing sequence images in start a certain time.
to memorize target sequence image, according to the track measurement principle, we gather different time sequence images and apply positive and negative difference image algorithm and target images region extraction to processing image under complexity background. Fig. 6 is to gather and process the image result in orbit for a certain period of time, Fig. 6(a) is to gather six frames original image comprising dynamic target, Fig. 6(b) is processing results by positive and negative difference image algorithm, and positive and negative difference image. In order to verify the target detection algorithm, according to difference image algorithm and target images region extraction, we gather the starting of a certain time target sequence image in middle ballistic trajectory, such as Fig. 7, the right image is the first frame, Fig. 7(a) is the acquisition of the original infrared sequence image, Fig. 7(b) is image of filtering processing, and positive and negative difference image. Fig. 8 is the middle of a certain time target sequence image in ballistic trajectory, the right image is the first frame, Fig. 8(a) is the acquisition of the original infrared sequence image, Fig. 8(b) is image of filtering processing and positive and negative difference image. By processing and analysis, we know that different target point image size is inconsistent in the whole of the track rail, the longer the distance between target and optical system, the smaller the
Fig. 8. Detection and processing sequence images in middle a certain time.
Fig. 10. The histogram of gradient by OTSU operator.
image size in image detector, this mainly is detection turntable in the rotating process and orbital distance is different, which will lead to different imaging distance in optical image system, so, the image size is different.we use Sobel operator and OTSU operator respectively to process the original image with threshold segmentation based on gradient magnitude, the histogram of gradient mean can be shown in Figs. 9 and 10. According to the image process result, the pixels number in adjacent frame can be calculated by using computer image processing and gain the target position in dynamic targets track detection system. Through calculation and analysis, the dynamic targets photoelectricity track detection system and its track arithmetic are scientific and feasible, which can improve photoelectricity track detection system on the photoelectricity track detection system. 6. Conclusions Based on orbit target detection platform, this paper put forward the improved image difference method processing arithmetic and analyzed the detection target image processing method, studied the positive and negative difference image processing algorithm and image target region extraction. At last, we combine detection system, the improved processing method was validated; this illuminates that the improved method is accurate and feasible. However, the difference image method also has some disadvantages, for example, when the environmental background is more complex and detection platform brings dithering, the target has some slur that will affect the target information extraction, if the background changes relatively obvious, the measurement system will be thought that it is the target and interference, which is not conducive to the target detection.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089
G Model IJLEO-54372; No. of Pages 6
ARTICLE IN PRESS H. Li, J. Gao / Optik xxx (2014) xxx–xxx
6
Acknowledgement This work has been supported by Project of the National Natural Science Foundation of China (No. 61107079). References [1] R.H. Zhan, Q. Xin, J. Wan, Modified unscented particle filter for nonlinear Bayesian tracking, J. Syst. Eng. Electron. 19 (2008) 7–14. [2] S.K. Jung, K.Y. Wohn, A model-based 3-D tracking of rigid objects from a sequence of multiple perspective views, Pattern Recognit. Lett. 20 (1998) 499–512. [3] H.S. Li, Z.H. Yaun, Z.Y. Lei, Infrared radiation characteristics analysis of high velocity projectile in detection screen, Infrared Laser Eng. 38 (2009) 777–781. [4] D. Comaniciu, V. Ramesh, P. Meer, Real-time tracking of non-rigid objects using mean shift, IEEE Int. Conf. Comput. Vision Pattern Recognit. 2 (2000) 142–149. [5] D. Comanieiu, P. Meer, A robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach. Intell. 24 (5) (2002) 603–619.
[6] H.S. Li, L.P. Lu, Z.Y. Lei, Infrared heat radiation detection performance influence of screen thickness and atmosphere on projectile, J. Nanjing Univ. Sci. Technol. 35 (2011) 298–303. [7] J.S. Hu, C.W. Juan, J.J. Wang, A spatial-color mean-shift object tracking algorithm with scale and orientation estimation, Pattern Recognit. Lett. 29 (2008) 2165–2173. [8] D. Yan, Z.Y. Lei, Algorithm research in high-speed object parameter validation of exterior trajectory, Opt. Tech. 33 (2007) 816–818. [9] G.W. Gao, S.Q. Liu, H.L. Qin, Auto-tracking algorithm of infrared target under complex background, Opto-Electron. Eng. 37 (2010) 78–84. [10] J. Yan, M.Y. Wu, S.Z. Chen, Camouflaged target tracking based on mean shift, Engineering 36 (2009) 11–15. [11] N. Aleixos, J. Blasco, E. Motto, F. Navarron, Assessment of citrus fruit quality using a real-time machine vision system, Pattern Recognit. 30 (2000) 482–485. [12] B. Wei, L. Zheng, K.Y. Wang, A new method for detecting plane target by imaging in laser fuze, Opt. Eng. 32 (2005) 36–40. [13] X.P. Luo, J. Tian, Z.Y. Zhu, A survey of image segmentation, Pattern Recognit. Artif. Intell. 12 (1999) 300–312. [14] Q.S. Li, Research on miss distance in optical measurement processing, Acta Photon. Sin. 28 (1999) 255–259.
Please cite this article in press as: H. Li, J. Gao, Research on target photoelectricity track method and improved image processing arithmetic in dynamic targets detection system, Optik - Int. J. Light Electron Opt. (2014), http://dx.doi.org/10.1016/j.ijleo.2014.01.089