- Email: [email protected]
Image quality optimization towards lidar registration based on iterative termination
J. Vis. Commun. Image R. 64 (2019) 102634
Contents lists available at ScienceDirect
J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci
Image quality optimization towards lidar registration based on iterative termination q Wanyi Zhang, Xiuhua Fu ⇑, Chunyang Wang School of OptoElectronic Engineering, Changchun University of Science and Technology, Changchun, China
a r t i c l e
i n f o
Article history: Received 26 May 2019 Revised 30 August 2019 Accepted 30 August 2019 Available online 30 August 2019 Keywords: Laser radar Registration image Quality optimization Image processing
a b s t r a c t Image quality optimization is a key technique in image processing, whose goal is to improve image quality by image enhancement or image format transform. This paper aims at optimizing image acquisition using Lidar registration, which can cope with disadvantages of conventional algorithms such as lowresolution. Specifically, we propose an iterative termination optimization strategy based on image quality perception features and local mean estimation. First, fuzzy images with different types and degrees of distortion are incorporated to form a representative natural image set, and feature maps of fuzzy images are extracted by the natural scene statistical method in the spatial domain. Noticeably, the proposed algorithm which performs iterative deblurring operation records the optimal iteration point based on recording the quality value FSIM of the restored image, and calibrates the corresponding feature vector in the sample library with the optimal iteration point (step number). Afterwards, we leverage LME method to implement an estimate of the number of iteration steps. Based on these two steps, the estimation of the initial iterative monitoring point is completed, so that the subsequent adaptive iterative termination work is more purposeful to monitor the defuzzification metric. The optimization operation can be completed faster effectively. Ó 2019 Elsevier Inc. All rights reserved.
1. Introduction The lidar imaging system emits echo signals on the surface of the target object by emitting laser pulses of a certain wavelength, and receives the echo signals according to different reflectances of the surface of the object, and the target becomes an intensity image [1,2,24–30]. At the same time, according to the flight time of the laser pulse, the distance information of the target can be obtained. Therefore, the active imaging system of the lidar can simultaneously form the intensity image and the distance image, realize the three-dimensional imaging of the target, characterize the geometric structure of the object, and have higher distance precision. And the contrast of the lidar image is relatively high, which greatly reduces the complexity of target recognition and extrac-
Abbreviations: NCSR, Nonlocally Centralized Sparse Representation; FSIM, Feature SIMilarity; LIME, Low-light IMage Enhancement; SSM, single stimulus method; DSM, double stimulus method; MSE, mean square error; PSNR, peak signal-to-noise ratio; VSNR, visual signal-to-noise ratio; IFC, information fidelity criterion; VIF, visual information fidelity; GCV, generalized cross validation. q
This paper has been recommended for acceptance by Maofu Liu.
⇑ Corresponding author.
E-mail address: [email protected] (X. Fu). https://doi.org/10.1016/j.jvcir.2019.102634 1047-3203/Ó 2019 Elsevier Inc. All rights reserved.
tion, and is an effective means to achieve accurate positioning and recognition [3–5]. However, the Lidar system is an active imaging system with a limited range of motion and insufficient concealment, so it is not possible to perform a wide range of search and detection [6]. Passive infrared detection is the passive imaging using the infrared radiation emitted by the object itself. Since the target object and the background have different emissivity and temperature, the wavelength and energy of the emitted infrared light are different, so the infrared image of the target and the background is formed. There are different gray levels in it, and effective target detection can be performed [7]. At the same time, due to the strong penetrability of long-wave infrared light, passive infrared detection has strong anti-interference ability against bad weather such as clouds, and is not blinded by glare and glare, and has a long working distance. In addition, because it is a passive mode work, it has better concealment and can scan all over the weather in a large field of view. However, the disadvantages of passive infrared imaging are also obvious. Passive infrared images have the disadvantages of low contrast, blurred edges, low signal-to-noise ratio, high false alarm probability and inability to form distance images [8–10].
2
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
With the development of science and technology, the requirements for target detection are getting higher and higher [31–36]. Although the performance of a single detector is constantly improving, the limitations of the single detection mode are also due to the different sensor types and response bands. Exposure is becoming more and more obvious [5,6,11]. Lidar detection can not only form an intensity registration image, but also can be a distance image, and the registration image resolution is higher, but because Lidar detection is active detection, its concealment is poor, and it is not possible to perform large-scale scanning [12]. The passive infrared imaging detection has a long distance, strong antiinterference ability, can detect all-weather and wide range, and can effectively reflect the target thermal information, but the edge of the registration image is blurred, and accurate distance information cannot be obtained [2,13,14]. Therefore, the active and passive images are effectively fused, and the advantages of the active and passive images are integrated. The fused image can reflect the thermal information of the target and reflect the high resolution information and distance information of the target, and the fused image is more in line with the human eye. The visual habits provide a more comprehensive and complete information for the further analysis of the target scene, while improving the accuracy and reliability of target recognition and tracking [3,15,16]. The Optical Center of the University of Massachusetts has successfully implemented synthetic aperture laser radar imaging of quantum cascade lasers, introducing synthetic apertures into the terahertz band [17]. The transmitting device adopts a 2.408 THz quantum cascade laser, and receives the heterodyne receiving method, and uses a CO2 laser pumping molecular laser with a frequency of 2409.293 GHz as the local oscillator light source [18]. The target rotates in two dimensions, and the resolution of the azimuth and distance directions is related to the angle at which the target turns. In this experiment, the target aperture is 5.5 m, and the azimuth and distance resolutions are 0.4 mm and 0.6 mm, respectively [19,20]. According to Liu Liren and others of Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, according to the principle of synthetic aperture laser radar, the azimuth and distance of synthetic aperture laser radar are simultaneously realized in China [21]. The light source adopts a 1550 nm single-mode ytterbium YAG laser with a laser output power of 2 mW, a laser 啁啾 range of 1540–1542 nm and a time of 20 ms. The distance between the target and the radar is 3.2 m. The point target diameter is approximately 1 mm [22]. The distance image width is about 0.6 mm and the azimuth width is about 2.0 mm. The State Key Laboratory of Radar Signal Processing, Xi’an University of Electronic Science and Technology, designed an experimental system for indoor inverse synthetic aperture laser radar [23]. It is unique in that it uses a system in which the transceiver lens is split, effectively overcoming the end reflection. The above experimental system is directed to a turntable model of inverse synthetic aperture, the laser outputs a laser signal whose wavelength frequency changes linearly, and 90% of the signal is emitted to the collimator lens output to illuminate the target. The remaining signals are divided into three paths, one output to the wavelength reference, used to track the initial wavelength for azimuth processing. All the way as the local oscillator for the difference frequency processing. The other signal is connected to the photoelectric receiving device as a reference signal to compensate for the phase error generated by the laser output nonlinear frequency modulated signal. This paper proposes an iterative termination optimization strategy. Specifically, the two steps are implemented by initial monitoring point estimation and adaptive iteration termination. In the initial monitoring point estimation, an iterative estimation algorithm based on image quality perception features is proposed. The algorithm is based on image quality perception features and
local mean estimation methods. Firstly, fuzzy images with different types and degrees of distortion are added to a representative natural image set, and the feature samples of the fuzzy images are extracted by the natural scene statistical method in the spatial domain. The non-localized sparse representation is used. The algorithm performs iterative deblurring operation, records the optimal iteration point by recording the quality value FSIM of the restored image after each iteration of the NCSR algorithm, and calibrates the corresponding feature vector in the sample library with the optimal iteration point (step number); finally, utilizes The LME method implements an estimate of the number of iteration steps. Through the above steps, the estimation of the initial iterative monitoring point is completed, so that the subsequent adaptive iterative termination work is more purposeful to monitor the defuzzification metric, and complete the fast and effective algorithm optimization.
2. Proposed method 2.1. Image quality evaluation 2.1.1. Subjective evaluation of image quality The so-called subjective evaluation means that the observer makes a subjective judgment on the image quality through the naked eye, and scores the observed image according to a certain standard. However, because different evaluation subjects have very complex features, different people’s visual perception characteristics have certain differences in contrast sensitivity and masking effects. In addition, the psychological characteristics of the observer and the test environment at the time are also important factors that interfere with subjective evaluation. Therefore, in order to ensure the scientific and effective subjective evaluation, many scientific research institutions and image and video research organizations have issued a series of evaluation criteria. For example, the subjective evaluation standard P.910 for multimedia and the subjective evaluation standard BT.500-11 for digital television images, wherein the BT.500-11 standard specifies specific scoring standards in order to perform numerical values on subjective scoring of observers. The uniformity of the above includes the 5-point system, the 9-point system, the 11-point system and the percentage system. The higher classification system corresponds to higher evaluation accuracy. Commonly used subjective evaluation methods are single stimulation and double stimulation. The first type is Single Stimulus (SSM), which means that a single image of a test image or video is scored according to certain rules by the evaluation subject if the reference image or video cannot be obtained. When the evaluation subject observes the image to be measured, the image in the image library is randomly displayed one or more times, and the observer subjectively evaluates based on the perceived image effect. The second common method is the double stimulation method (DSM). The double stimulation method requires the evaluation subject to use the original image as a reference and basis for scoring when observing the distorted image. Since the original image is used as the comparison, this evaluation method is used. More reliable and effective than single stimulation. However, in actual experimental conditions, the cost of obtaining reference images is relatively difficult, and the use of the dual stimulation method has great limitations. In contrast, the single stimulation method is more widely used. The reference image is the original image and the default reference image is the undistorted image from the source. The image to be tested is the image to be tested, and there is a certain distortion. The image team is composed of the reference image and the image to be tested, and the reference image is compared to observe the damage of the image to be
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
tested, and the level of the image to be tested is selected according to the subjective quality level 5 score table (see Table 1). 2.1.2. Objective evaluation of images 2.1.2.1. Full reference method. The full reference method is to use the entire information of the original image, calculate the perceptual error between the original image and the distorted image, and combine these errors to obtain the evaluation value of the distorted image quality. The default original image is an undistorted image. There are a wide variety of full reference methods. The algorithmic features of these methods are basically mathematical models, which can be divided into the following categories: algorithms based on pixel error statistics; algorithms based on structural phase; information fidelity based on information theory; based on human visual systems and other algorithms Combine. An algorithm based on pixel error statistics. Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). These two methods measure the quality of the image by calculating the error between the gray values of the corresponding pixel points. The disadvantage of this type of algorithm is that only the difference between pixels is calculated, and the evaluation results do not reflect the subjective feelings of the human eye on image quality. Some people combine PSNR and SSIM to form a new algorithm. Peak signal-to-noise ratio (PSNR) is a commonly used indicator to measure signal distortion, but PSNR does not involve the characteristics of the signal’s own content. When evaluating the quality of some images or video sequences, it will have a large deviation from the quality of subjective perception. Structural similarity (SSIM) is a method based on structural information to measure the similarity between the original signal and the processed signal. The calculation is simple, and the correlation with the subjective quality evaluation is strong. First, the cluster analysis method is used according to the PSNR value. And the SSIM output value is used to regularly cluster the sample images, and then different quality evaluation rules are applied to different types of images. The evaluation rules are determined by binary regression analysis; the images to be tested are classified by the support vector machine (SVM) classifier. This method can effectively reflect the subjective quality of the image. An algorithm based on structural phase. In 2004, it was proposed by Wang et al., which is an algorithm for measuring the structural similarity (SSIM) between reference images and distorted images. It is believed that the main function of the human eye is to extract structural information in the field of view, and the human eye has a signal structure in the field of view. The change is highly adaptive. Since then, many people have improved the SSIM algorithm and introduced the characteristics of the human visual system. For example, a Wavelet-based visual signal-to-noise ratio natural image quality evaluation method (VSNR) based on visual perception threshold. The SSIM algorithm is improved based on the comprehensive perceptual error of the visual attention area. SSIM has applications in rate-distortion optimized coding, bit allocation of H.264 encoders, etc. Information fidelity based on information theory. From the perspective of information theory, Hamid et al. calculated the image quality by calculating the mutual information between the original image and the distorted image, and proposed two algorithms:
Table 1 Subjective quality evaluation score sheet. Level
Absolute measure
1 2 3 4 5
Best Good General Failed Worst
3
information fidelity criterion (IFC) and visual information fidelity (VIF). The information fidelity method develops a link between image information and visual quality. Based on the human visual system and combined with other algorithms. A typical representative of the quality evaluation method based on HVS characteristics is the just noticeable difference JND model; Yang Wei et al. combines the visual characteristics of human eyes with the structural similarity of images to obtain an image that conforms to human visual characteristics. A new method of quality evaluation. The method can distinguish image features of different regions in the image and conform to human visual characteristics. 2.1.2.2. Partial reference. Part of the reference method uses only partial information of the original image to estimate the visual perceptual quality of the distorted image. The advantage of the partial reference method is that on the basis of reducing the amount of transmitted data, a better evaluation effect is obtained. The disadvantage is that the algorithm is very sensitive to the extracted features. Feature extraction and feature comparison are the key factors affecting the performance of some reference methods. The main part is to extract the partial information of the original image and the distorted image first, and compare the partial information to evaluate the quality of the distorted image. Part of the reference method can be divided into an original image feature method, a digital watermark based method, and a Wavelet domain statistical model based method. 2.2. Regularization method 2.2.1. Regularization model The regularization method is a commonly used and reliable method for solving ill-posed problems. In the regularizationbased deblurring algorithm, the regularized prior knowledge about the original image f is usually used to solve the following optimal objective function values achieve.
min C k ðf ; hÞ ¼
1 2 k g f h k2 þ kUðf Þ 2
ð1Þ
The first term in formula (1) is the data fidelity term, U(f) is the regular term representing the image prior knowledge, also known as the constraint term or penalty term, and k is the regularization parameter. Regularization parameters are used to control data fidelity and regular item weights. Since the optimal solution of the regularization model is very sensitive to regularization parameters, how to choose appropriate regularization parameters is also part of the research topic. Generally speaking, if the value is too large, the image is too smooth, and the texture and edge structure are lost. When the setting is too small, image blur information is likely to remain, and the smooth region of the image still has noise and the like. For the selected values, the two commonly used standards in linear methods are generalized cross-validation (GCV) and Lcurve; while the non-linear methods are widely used in the SURE (Stein’s unbiased risk estimate) standard. 2.2.2. Classical regularization method In the regularization method, the most classic is the Tikhonov regularization method. It is a linear regularization method whose regular term is generally defined as a Laplacian, the L2 norm. Its mathematical model can be expressed as:
1 2 min u k g f g k2 þ kjLðuÞj2 2 where L (.) is a linear operation.
ð2Þ
4
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
This method generally assumes that the original image is smooth as a priori knowledge of the image. Although the Tikhonov regularization method is fast, it is easy to cause the edge information of the restored image to be lost and ringing. 2.2.3. Iterative regularization method The total variation regularization method is one of the most successful and popular iterative regularization methods. At first, the method was applied to the research of image denoising, and later applied by scholars in other directions in the field of image restoration processing. This method can be defined as follows:
1 2 k g f h k2 þ kjujBVðXÞ min u2BVðXÞ 2
ð3Þ
where k > 0 is the regularization parameter, BVðXÞ is the bounded variation (BV) function in space, and |.| represents the semi-norm of the BV space. Usually the regular term is written as:
jujBVðXÞ ¼ f X jrujdx
ð4Þ
The all-variation model has a piecewise smoothing feature that allows smoothing only in a direction perpendicular to the gradient, so it can well protect the edge information of the image. Full variation regularization is widely used because it eliminates high frequency noise while maintaining good edges. However, the model is very poorly separated from texture and noise; when the regularization parameter is too large, the restored image will be over-smoothed, losing detail or texture, and the regularization parameters are too small, and the restored image still has a lot of noise. 2.3. Sparse non-local regularization weighted coding algorithm There have been many denoising algorithms for single noise such as Gaussian noise, salt and pepper noise, and periodic noise. For example: dictionary learning algorithm based on sparse representation for Gaussian noise, based on Gaussian scale mixed model and non-local filtering algorithm; mean filtering algorithm for salt and pepper noise, adaptive median filtering algorithm, etc.; band rejection filtering algorithm for periodic noise, notch filter algorithm, etc. However, in general, these single noise denoising methods are difficult to achieve the best denoising effect when applied to mixed noise. Therefore, research work on mixed noise is particularly important. In recent years, research methods for mixed noise have been proposed, such as: sparse representation based methods, weighted dictionary learning algorithms, partial differential equation based methods, PCNN based models and wavelet transform based algorithms, etc. Noise has a significant inhibitory effect, especially with the sparse non-local regularization weighted coding (WESNR) method. 2.3.1. Sparse non-local regularization model pffiffiffi pffiffiffi Suppose x 2 RN is an image. The n n sized image block n extracted by the xi ¼ Ri x 2 R from the image, where Ri is a way of extracting a block using a matrix, and Ri x represents a block xi extracted from the i-th position of the image x. According to the sparse representation theory, if xi is sparsely encoded, then xi can be expressed as xi ¼ Uai , and ai is a sparse coding vector with few non-zero atoms, so x can be solved by least squares method:
x¼
X i
!1 RTi R
X
! RTi Uai
ð5Þ
i
For convenience of expression, the above formula is usually rewritten as a in x ¼ Ua, where U is a set of sparse coding vectors
ai . Thus, in the case of Gaussian noise, the sparse coding model can generally be written as:
^ ¼ arg min k y Ua k22 þ kRðaÞ a
ð6Þ
a
where R(a) is a regular term for a and k is a coefficient for a regular term. 2.3.2. Sparse non-local regularization weighted coding model For images contaminated by mixed noise, the distribution of mixed noise is far from Gaussian noise, so the data fidelity term k y Ua k22 in Eq. (6) is not convenient for removing mixed noise. Because the analysis shows that weighting the data fidelity term can make the distribution of the residuals close to the Gaussian distribution, so the sparse coding model is modified into the following weighted sparse coding model: 1
2
a ¼ argmin k W 2 ðy UaÞ k2 þ kRðaÞ
ð7Þ
a
where W is a weighted diagonal matrix whose element Wii on the diagonal is given by:
W ii ¼ expðae2t Þ
ð8Þ
e ¼ ½e1 ; e2 ; :::; eN ¼ y Ua
ð9Þ
On the other hand, the regular term R(a) is used to describe the a priori of natural images. The a priori widely used for image denoising generally has two: local sparse and non-local selfsimilarity (NSS). However, the sparsely encoded local sparse solution a is usually represented by the l1 norm, and the non-local selfsimilarity (NSS) is usually described by the minimum error of the prediction block and its similar blocks, thus combining the two a priorities into a sparse regular term. The final sparse non-local regularization weighted coding (WESNR) model is:
( 2 ) X 1 2 a ¼ argmin W ðy UaÞ þ k ai ui a 2
i
ð10Þ
1
In order to solve the model (10), the weighting matrix Wa can be determined first, because the relationship between Wii and ei is inversely proportional, and the pixels contaminated by the impulse noise are sparsely represented, there is a large coding residual, and the weight is not polluted by the impulse noise. Approaching l, so we set both Wii 2 (0,1]. 3. Experiments 3.1. Data source For the target simulation, the on-board measurement platform travels at a speed of 60 km/h on the highway in a straight line. The platform is accompanied by a random vibration with an amplitude of 0.5 m during driving; the focal length of the imaging optical system is 0.1 m; and the image intensifier has a quantum efficiency of 10%, the noise factor is 1.4; the distance between the start and end of the gate strobe is 950 m and 1050 m respectively; the constant gain of channel 1 is 300, the gain of channel 2 is linearly changed from 50 to 500 from the start to the end of the gate; the CCD pixel spacing is 200 lm; A total of 25 distance images with ranging error caused by shot noise are generated on the ground along the route; the pixels in each CCD field of view obtain an average of 1000 photons, and the average signal-to-noise ratio of the signal received by the CCD can be calculated to be 8.5. The average error of calculating the single-frame distance image is 9.1 m. Obtain the original registration image as shown in Fig. 1.
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
5
Fig. 1. Original registration image.
3.2. Extraction and identification of parameters Parameter extraction and identification. The main parameter extraction is number of pixels obtained based on different parts of the video image characteristic parameter extraction, and obtained a corresponding image based on the parameter change for the better the relationship between the actual pixel size. And the video object identification is to use the extracted characteristic parameter of the target video, embodiments of the automatic recognition of the target by different methods. The computer automatically recognizes a key factor in determining good and bad effects parameter extraction. 4. Discussion 4.1. Lidar registration image quality improvement effect For the sake of simplicity, the parameter r controls the number of iterations. In order to balance the iteration number with the denoising effect, after a number of experiments, finally set f = 0.003: the parameter a of the control weight decay speed in Eq. (8) is set to 0.0007. The parameter k is initially set to a smaller number of 0.0006 to weaken the role of the non-local regular term in the algorithm; from the second cycle, the impulse noise is largely attenuated, at this time non-local similarity search The ability of the block becomes very strong, we set k to a relatively large value to remove Gaussian noise. If the standard deviation of Gaussian noise is greater than 10, we set k = 1; otherwise, set k = 0.5, the purpose is to suppress Gaussian noise, so that the image retains the detail as much as possible. As shown in Figs. 2 and 3, it is easy to see from the experimental results that the improved algorithm can make
full use of the prior knowledge of sparsity and non-local similarity in the image, and has good denoising ability for high-intensity mixed noise. 4.2. Adaptive iterative termination strategy analysis based on registration image In the first iteration, it is evident that there is a clear image outline and key image edge information in the registration image. In the 20-step iteration, some less obvious structural information can still be seen in the lower half of the image. As the number of iterations advances, the image structure information gradually disappears, that is, the difference between the pixels of the registration image gradually becomes smaller. When the number of iteration steps is 50, it is no longer possible to visually observe information about the image content in the registration image. Explain that the quality of the image after iteration is very close to the optimal value. 4 is a histogram corresponding to the luminance values of the registered image pixel points in the above three iteration steps. It can be seen from Fig. 4 that as the number of iterations increases the distribution range of the luminance values in the residual image, the contraction gradually narrows, indicating that the intermediate estimated image is gradually approaching the original image. 4.3. Time efficiency analysis of the algorithm The improved NCSR algorithm increases the execution time of the other two aspects: one is the execution time of the IE-IQF algorithm, and the other is the deblurring metric execution time based on the statistical features of the residual image. In order to test the effect of algorithm optimization on the execution time of the NCSR
Fig. 2. Image enhancement effect (1).
6
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
Fig. 3. Image enhancement effect (2).
Fig. 4. Histogram of the resulting registration image under key iteration steps.
Fig. 5. NCSR algorithm improvement before and after execution time (unit: second).
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
algorithm, an experimental analysis was performed on an image of size 256 256. The experimental results are shown in Fig. 5. As can be seen from Fig. 5, the improved NCSR algorithm execution efficiency is significantly improved, an average increase of 31.3%. Specifically, in the case of uniform, Gaussian, and defocused blur, the average is increased by 38.2%, 29.8%, and 25.9%, respectively, and the best case is even increased by about 50%. Although the estimation of the degree of image blurring has affected the execution efficiency, the worst case has an increase of more than 10%. It can be proved that the iterative termination optimization strategy proposed in this paper has good practicability and feasibility. 5. Conclusions (1) For the mixed noise, an improved non-locally centralized sparse representation model (MNCSR) is proposed. The improved algorithm is divided into two phases. In the first stage, adaptive median filtering is used as preprocessing to remove the sparse noise part of the mixed noise. In the second stage, the original NCSR algorithm is used to remove the residual noise in the upper stage. The experimental results show that the proposed MNCSR algorithm improves the ability of the NCSR algorithm to deal with sparse noise. (2) Based on the two-channel gain modulation imaging lidar ranging error model, a method of integrating multi-frame distance image to improve the ranging accuracy is proposed by applying weighted registration and weighted superposition method. Simulation and experiments show that the method has higher registration accuracy and reduces the distance error to about 1/n of a single frame image. The method can obtain the improvement of the ranging accuracy by the image processing method without increasing the light source intensity, improving the radar receiving aperture and switching to the high-performance detector. (3) Through the imaging simulation of the simulated lattice target, the resolution of the synthetic aperture laser radar fold line scanning is analyzed. The results show that the fused image obtained by the synthetic aperture laser radar fold line scanning method and the traditional synthetic aperture laser radar image Compared with the image resolution, the resolution of the original aperture of the synthetic aperture laser radar is increased to the level consistent with the original orientation, which improves the image quality.
Declaration of Competing Interest The authors declared that there is no conflict of interest. References [1] H. Yu, E. Li, W. Gong, et al., Structured image reconstruction for threedimensional ghost imaging lidar, Opt. Exp. 23 (11) (2015) 14541–14551. [2] L. Yuan, H. Yan, S. Hu, et al., Three-dimensional video imaging lidar system based on acousto-optic laser scanning, Chin. J. Lasers 41 (8) (2014) 0802005. [3] Z. Tian, Z. Cui, L. Zhang, et al., Control and image processing for streak tube imaging lidar based on VB and MATLAB, Chin. Opt. Lett. 12 (6) (2014) 67–70. [4] A. Muttini, V. Russo, E. Rossi, et al., An integrated multi-medial approach to cultural heritage conservation and documentation: from remotely-sensed lidar imaging to historical archive data, in: SPIE Remote Sensing, 2015, p. 13. [5] J. Sun, J. Liu, W. Qi, Experimental research on multiple-slit streak tube imaging lidar, Optik – Int. J. Light Electr. Opt. 126 (21) (2015) 3181–3184. [6] C. Deng, L. Pan, C. Wang, et al., Performance analysis of ghost imaging lidar in background light environment, Photo. Res. 5 (5) (2017) 431–435. [7] L. Xu, Y. Zhang, Y. Zhang, et al., Four Gm-APDs photon counting imaging lidar to improve detection performances, Infrar. Laser Eng. 44 (9) (2015).
7
[8] G. Jian, J. Sun, W. Qi, Experiments on the range resolution measurement of a slit Streak Tube Imaging Lidar, Optik – Int. J. Light Electr. Opt. 126 (21) (2015) 3084–3087. [9] J. Gao, J. Sun, Q. Wang, Experiments of ocean surface waves and underwater target detection imaging using a slit Streak Tube Imaging Lidar, Optik – Int. J. Light Electr. Opt. 125 (18) (2014) 5199–5201. [10] G. Ye, R. Fan, Z. Chen, et al., Range accuracy analysis of streak tube imaging lidar systems, Opt. Commun. 360 (2016) 7–14. [11] W. Xiang, C. Miao, L. Lu, et al., Accelerated extraction technology research on damaged building information by earthquake based on LiDAR image, Commun. Comput. Inform. Sci. 482 (2015) 117–132. [12] Y. Ling, G. Gu, W. He, et al., Adaptive target profile acquiring method for photon counting 3-D imaging lidar, IEEE Photo. J. 8 (6) (2016) 1–10. [13] F.Q. Qu, C. Shu, J.H. Tu, Modeling and simulation research of lidar tomography imaging, Appl. Mech. Mater. 721 (2014) 468–471. [14] J.B. Blair, D. Rabine, M.A. Hofton, et al., Geodetic Imaging Lidar: Applications for high-accuracy, large area mapping with NASA’s upcoming high-altitude waveform-based airborne laser altimetry Facility, in: AGU Fall Meeting, 2015, p. 07. [15] C. Wang, X. Mei, P. Long, et al., Airborne near infrared three-dimensional ghost imaging LiDAR via sparsity constraint, Remote Sens. 10 (5) (2018) 732. [16] C. Wang, X. Lee, T. Cui, et al., Reducing the influence of direct reflection on return signal detection in a 3D imaging lidar system by rotating the polarizing beam splitter., Appl. Opt. 55 (7) (2016) 1559. [17] S.C. Henry, L.M. Zurk, D.D. Duncan, Three-dimensional broadband terahertz synthetic aperture imaging, Opt. Eng. 51 (9) (2012) 1603. [18] B. Li, D. Wang, H. Huang, et al., Synthetic aperture in terahertz in-line digital holography for resolution enhancement, Appl. Opt. 55 (3) (2016) A43. [19] Ana Maria Pusßcasßu, Special phenomena of the shadow region in the high resolution synthetic aperture radar image due to synthetic aperture, J. Infrar. Millim. Terahertz Waves 33 (10) (2012) 1052–1070. [20] L.M. Zurk, S.C. Henry, 3D terahertz synthetic aperture imaging of objects with arbitrary boundaries, Proc. SPIE – Int. Soc. Opt. Eng. 8846 (2013) 49–52. [21] L.R. Liu, N. Xu, Z. Yan, et al., Two-dimensional imaging experiment of scaledown synthetic aperture laser radar, Acta Opt. Sin. 29 (7) (2009) 2030– 2032. [22] Y.W. Liu, L.R. Liu, C.H. Zhou, et al., Experimental study of non-volatile holographic storage of doubly- and triply-doped lithium niobate crystals, Opt. Commun. 181 (1) (2001) 47–52. [23] L. Guo, M. Xing, X. Zeng, et al., Inverse synthetic aperture lidar imaging of indoor real data, Infrar. Laser Eng. 40 (4) (2011) 637–642. [24] Xu. Mingliang, Lu. Fuhai Chen, Chen Shen Li, Pei Lv, Bing Zhou, Rongrong Ji, Bio-inspired deep attribute learning towards facial aesthetic prediction, IEEE Trans. Affect. Comput. (2018), https://doi.org/10.1109/TAFFC.2018. 2868651. [25] Masud Ibn Afjal, Md Al Mamun, Md Palash Uddin, Band reordering heuristics for lossless satellite image compression with 3D CALIC and CCSDS, J. Vis. Commun. Image Repres. 59 (2019) 514–526. [26] Junwei Han, Dingwen Zhang, Gong Cheng, Lei Guo, Jinchang Ren, Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning, IEEE Trans. Geosci. Remote Sens. 53 (6) (2015) 3325–3337. [27] Yildiray Anagün, Sahin Isik, Erol Seke, SRLibrary: Comparing different loss function for super-resolution over various convolutional architectures, J. Vis. Commun. Image Repres. 61 (2019) 178–187. [28] Xu. Mingliang, Jiejie Zhu, Pei Lv, Bing Zhou, Marshall Tappen, Rongrong Ji, Learning-based shadow recognition and removal from monochromatic natural images, IEEE Trans. Image Process. 26 (12) (2017) 5811–5824. [29] D. Nair, P. Sankaran, Color image dehazing using surround filter and dark channel prior, J. Vis. Commun. Image Repres. 50 (2017) 9–15. [30] R. Pramanik, S. Bag, Shape decomposition-based handwritten compound character recognition for bangla ocr, J. Vis. Commun. Image Repres. 50 (2018) 123–134. [31] Junwei Han, Dingwen Zhang, Hu Xintao, Lei Guo, Jinchang Ren, Feng Wu Background prior-based salient object detection via deep reconstruction residual, IEEE Trans. Circ. Syst. Video Technol. 25 (8) (2015) 1309–1321. [32] Junwei Han, Xiang Ji, Hu Xintao, Dajiang Zhu, Kaiming Li, Xi Jiang, Guangbin Cui, Lei Guo, Tianming Liu, Representing and retrieving video shots in humancentric brain imaging space, IEEE Trans. Image Process. 22 (7) (2013) 2723– 2736. [33] Xu Mingliang, Mingyuan Li, Xu Weiwei, Zhigang Deng, Yin Yang, Kun Zhou, Interactive mechanism modeling from multi-view images, ACM Trans. Graph. 35 (6) (2016), Article 236. [34] C. Bai, J.N. Chen, L. Huang, K. Kpalma, S. Chen, Saliency-based multi-feature modeling for semantic image retrieval, J. Vis. Commun. Image Repres. 50 (2018) 199–204. [35] Junwei Han, King Ngi Ngan, Mingjing Li, Hong-Jiang Zhang, Unsupervised extraction of visual attention objects in color images, IEEE Trans. Circ. Syst. Video Technol. 16 (1) (2006) 141–145. [36] Xu Mingliang, Wu Yunpeng, Yangdong Ye, Illes Farkas, Hao Jiang, Zhigang Deng, Collective crowd formation transform with mutual information-based runtime feedback, Comput. Graph. Forum 34 (1) (2015) 60–73.
8
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634 Wanyi Zhang was born in Changchun, Jilin, P.R. China, in 1985. She received the Master degree from Changchun University of Science and Technology, P.R. China. Now, she studies in School of OptoElectronic Engineering, Changchun University of Science and Technology. Her research interests include contemporary optical measurement technology and optical image processing. E-mail: [email protected].
Xiuhua Fu was born in Binzhou, Shandong, P.R. China, in 1963. She received the Doctorate from Changchun University of Science and Technology, P.R. China. Now, she works in School of OptoElectronic Engineering, Changchun University of Science and Technology. Her research interests include cloud optical filming and optical manufacturing technology. E-mail: [email protected].
Chunyang Wang was born in Changchun, Jilin, P.R. China, in 1964. She received the Doctorate from Jilin University, P.R. China. Now, she works in School of Electronics and Information Engineering, Changchun University of Science and Technology. Her research interests include photoelectric information processing and detection technology. E-mail: [email protected].
Contents lists available at ScienceDirect
J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci
Image quality optimization towards lidar registration based on iterative termination q Wanyi Zhang, Xiuhua Fu ⇑, Chunyang Wang School of OptoElectronic Engineering, Changchun University of Science and Technology, Changchun, China
a r t i c l e
i n f o
Article history: Received 26 May 2019 Revised 30 August 2019 Accepted 30 August 2019 Available online 30 August 2019 Keywords: Laser radar Registration image Quality optimization Image processing
a b s t r a c t Image quality optimization is a key technique in image processing, whose goal is to improve image quality by image enhancement or image format transform. This paper aims at optimizing image acquisition using Lidar registration, which can cope with disadvantages of conventional algorithms such as lowresolution. Specifically, we propose an iterative termination optimization strategy based on image quality perception features and local mean estimation. First, fuzzy images with different types and degrees of distortion are incorporated to form a representative natural image set, and feature maps of fuzzy images are extracted by the natural scene statistical method in the spatial domain. Noticeably, the proposed algorithm which performs iterative deblurring operation records the optimal iteration point based on recording the quality value FSIM of the restored image, and calibrates the corresponding feature vector in the sample library with the optimal iteration point (step number). Afterwards, we leverage LME method to implement an estimate of the number of iteration steps. Based on these two steps, the estimation of the initial iterative monitoring point is completed, so that the subsequent adaptive iterative termination work is more purposeful to monitor the defuzzification metric. The optimization operation can be completed faster effectively. Ó 2019 Elsevier Inc. All rights reserved.
1. Introduction The lidar imaging system emits echo signals on the surface of the target object by emitting laser pulses of a certain wavelength, and receives the echo signals according to different reflectances of the surface of the object, and the target becomes an intensity image [1,2,24–30]. At the same time, according to the flight time of the laser pulse, the distance information of the target can be obtained. Therefore, the active imaging system of the lidar can simultaneously form the intensity image and the distance image, realize the three-dimensional imaging of the target, characterize the geometric structure of the object, and have higher distance precision. And the contrast of the lidar image is relatively high, which greatly reduces the complexity of target recognition and extrac-
Abbreviations: NCSR, Nonlocally Centralized Sparse Representation; FSIM, Feature SIMilarity; LIME, Low-light IMage Enhancement; SSM, single stimulus method; DSM, double stimulus method; MSE, mean square error; PSNR, peak signal-to-noise ratio; VSNR, visual signal-to-noise ratio; IFC, information fidelity criterion; VIF, visual information fidelity; GCV, generalized cross validation. q
This paper has been recommended for acceptance by Maofu Liu.
⇑ Corresponding author.
E-mail address: [email protected] (X. Fu). https://doi.org/10.1016/j.jvcir.2019.102634 1047-3203/Ó 2019 Elsevier Inc. All rights reserved.
tion, and is an effective means to achieve accurate positioning and recognition [3–5]. However, the Lidar system is an active imaging system with a limited range of motion and insufficient concealment, so it is not possible to perform a wide range of search and detection [6]. Passive infrared detection is the passive imaging using the infrared radiation emitted by the object itself. Since the target object and the background have different emissivity and temperature, the wavelength and energy of the emitted infrared light are different, so the infrared image of the target and the background is formed. There are different gray levels in it, and effective target detection can be performed [7]. At the same time, due to the strong penetrability of long-wave infrared light, passive infrared detection has strong anti-interference ability against bad weather such as clouds, and is not blinded by glare and glare, and has a long working distance. In addition, because it is a passive mode work, it has better concealment and can scan all over the weather in a large field of view. However, the disadvantages of passive infrared imaging are also obvious. Passive infrared images have the disadvantages of low contrast, blurred edges, low signal-to-noise ratio, high false alarm probability and inability to form distance images [8–10].
2
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
With the development of science and technology, the requirements for target detection are getting higher and higher [31–36]. Although the performance of a single detector is constantly improving, the limitations of the single detection mode are also due to the different sensor types and response bands. Exposure is becoming more and more obvious [5,6,11]. Lidar detection can not only form an intensity registration image, but also can be a distance image, and the registration image resolution is higher, but because Lidar detection is active detection, its concealment is poor, and it is not possible to perform large-scale scanning [12]. The passive infrared imaging detection has a long distance, strong antiinterference ability, can detect all-weather and wide range, and can effectively reflect the target thermal information, but the edge of the registration image is blurred, and accurate distance information cannot be obtained [2,13,14]. Therefore, the active and passive images are effectively fused, and the advantages of the active and passive images are integrated. The fused image can reflect the thermal information of the target and reflect the high resolution information and distance information of the target, and the fused image is more in line with the human eye. The visual habits provide a more comprehensive and complete information for the further analysis of the target scene, while improving the accuracy and reliability of target recognition and tracking [3,15,16]. The Optical Center of the University of Massachusetts has successfully implemented synthetic aperture laser radar imaging of quantum cascade lasers, introducing synthetic apertures into the terahertz band [17]. The transmitting device adopts a 2.408 THz quantum cascade laser, and receives the heterodyne receiving method, and uses a CO2 laser pumping molecular laser with a frequency of 2409.293 GHz as the local oscillator light source [18]. The target rotates in two dimensions, and the resolution of the azimuth and distance directions is related to the angle at which the target turns. In this experiment, the target aperture is 5.5 m, and the azimuth and distance resolutions are 0.4 mm and 0.6 mm, respectively [19,20]. According to Liu Liren and others of Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, according to the principle of synthetic aperture laser radar, the azimuth and distance of synthetic aperture laser radar are simultaneously realized in China [21]. The light source adopts a 1550 nm single-mode ytterbium YAG laser with a laser output power of 2 mW, a laser 啁啾 range of 1540–1542 nm and a time of 20 ms. The distance between the target and the radar is 3.2 m. The point target diameter is approximately 1 mm [22]. The distance image width is about 0.6 mm and the azimuth width is about 2.0 mm. The State Key Laboratory of Radar Signal Processing, Xi’an University of Electronic Science and Technology, designed an experimental system for indoor inverse synthetic aperture laser radar [23]. It is unique in that it uses a system in which the transceiver lens is split, effectively overcoming the end reflection. The above experimental system is directed to a turntable model of inverse synthetic aperture, the laser outputs a laser signal whose wavelength frequency changes linearly, and 90% of the signal is emitted to the collimator lens output to illuminate the target. The remaining signals are divided into three paths, one output to the wavelength reference, used to track the initial wavelength for azimuth processing. All the way as the local oscillator for the difference frequency processing. The other signal is connected to the photoelectric receiving device as a reference signal to compensate for the phase error generated by the laser output nonlinear frequency modulated signal. This paper proposes an iterative termination optimization strategy. Specifically, the two steps are implemented by initial monitoring point estimation and adaptive iteration termination. In the initial monitoring point estimation, an iterative estimation algorithm based on image quality perception features is proposed. The algorithm is based on image quality perception features and
local mean estimation methods. Firstly, fuzzy images with different types and degrees of distortion are added to a representative natural image set, and the feature samples of the fuzzy images are extracted by the natural scene statistical method in the spatial domain. The non-localized sparse representation is used. The algorithm performs iterative deblurring operation, records the optimal iteration point by recording the quality value FSIM of the restored image after each iteration of the NCSR algorithm, and calibrates the corresponding feature vector in the sample library with the optimal iteration point (step number); finally, utilizes The LME method implements an estimate of the number of iteration steps. Through the above steps, the estimation of the initial iterative monitoring point is completed, so that the subsequent adaptive iterative termination work is more purposeful to monitor the defuzzification metric, and complete the fast and effective algorithm optimization.
2. Proposed method 2.1. Image quality evaluation 2.1.1. Subjective evaluation of image quality The so-called subjective evaluation means that the observer makes a subjective judgment on the image quality through the naked eye, and scores the observed image according to a certain standard. However, because different evaluation subjects have very complex features, different people’s visual perception characteristics have certain differences in contrast sensitivity and masking effects. In addition, the psychological characteristics of the observer and the test environment at the time are also important factors that interfere with subjective evaluation. Therefore, in order to ensure the scientific and effective subjective evaluation, many scientific research institutions and image and video research organizations have issued a series of evaluation criteria. For example, the subjective evaluation standard P.910 for multimedia and the subjective evaluation standard BT.500-11 for digital television images, wherein the BT.500-11 standard specifies specific scoring standards in order to perform numerical values on subjective scoring of observers. The uniformity of the above includes the 5-point system, the 9-point system, the 11-point system and the percentage system. The higher classification system corresponds to higher evaluation accuracy. Commonly used subjective evaluation methods are single stimulation and double stimulation. The first type is Single Stimulus (SSM), which means that a single image of a test image or video is scored according to certain rules by the evaluation subject if the reference image or video cannot be obtained. When the evaluation subject observes the image to be measured, the image in the image library is randomly displayed one or more times, and the observer subjectively evaluates based on the perceived image effect. The second common method is the double stimulation method (DSM). The double stimulation method requires the evaluation subject to use the original image as a reference and basis for scoring when observing the distorted image. Since the original image is used as the comparison, this evaluation method is used. More reliable and effective than single stimulation. However, in actual experimental conditions, the cost of obtaining reference images is relatively difficult, and the use of the dual stimulation method has great limitations. In contrast, the single stimulation method is more widely used. The reference image is the original image and the default reference image is the undistorted image from the source. The image to be tested is the image to be tested, and there is a certain distortion. The image team is composed of the reference image and the image to be tested, and the reference image is compared to observe the damage of the image to be
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
tested, and the level of the image to be tested is selected according to the subjective quality level 5 score table (see Table 1). 2.1.2. Objective evaluation of images 2.1.2.1. Full reference method. The full reference method is to use the entire information of the original image, calculate the perceptual error between the original image and the distorted image, and combine these errors to obtain the evaluation value of the distorted image quality. The default original image is an undistorted image. There are a wide variety of full reference methods. The algorithmic features of these methods are basically mathematical models, which can be divided into the following categories: algorithms based on pixel error statistics; algorithms based on structural phase; information fidelity based on information theory; based on human visual systems and other algorithms Combine. An algorithm based on pixel error statistics. Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). These two methods measure the quality of the image by calculating the error between the gray values of the corresponding pixel points. The disadvantage of this type of algorithm is that only the difference between pixels is calculated, and the evaluation results do not reflect the subjective feelings of the human eye on image quality. Some people combine PSNR and SSIM to form a new algorithm. Peak signal-to-noise ratio (PSNR) is a commonly used indicator to measure signal distortion, but PSNR does not involve the characteristics of the signal’s own content. When evaluating the quality of some images or video sequences, it will have a large deviation from the quality of subjective perception. Structural similarity (SSIM) is a method based on structural information to measure the similarity between the original signal and the processed signal. The calculation is simple, and the correlation with the subjective quality evaluation is strong. First, the cluster analysis method is used according to the PSNR value. And the SSIM output value is used to regularly cluster the sample images, and then different quality evaluation rules are applied to different types of images. The evaluation rules are determined by binary regression analysis; the images to be tested are classified by the support vector machine (SVM) classifier. This method can effectively reflect the subjective quality of the image. An algorithm based on structural phase. In 2004, it was proposed by Wang et al., which is an algorithm for measuring the structural similarity (SSIM) between reference images and distorted images. It is believed that the main function of the human eye is to extract structural information in the field of view, and the human eye has a signal structure in the field of view. The change is highly adaptive. Since then, many people have improved the SSIM algorithm and introduced the characteristics of the human visual system. For example, a Wavelet-based visual signal-to-noise ratio natural image quality evaluation method (VSNR) based on visual perception threshold. The SSIM algorithm is improved based on the comprehensive perceptual error of the visual attention area. SSIM has applications in rate-distortion optimized coding, bit allocation of H.264 encoders, etc. Information fidelity based on information theory. From the perspective of information theory, Hamid et al. calculated the image quality by calculating the mutual information between the original image and the distorted image, and proposed two algorithms:
Table 1 Subjective quality evaluation score sheet. Level
Absolute measure
1 2 3 4 5
Best Good General Failed Worst
3
information fidelity criterion (IFC) and visual information fidelity (VIF). The information fidelity method develops a link between image information and visual quality. Based on the human visual system and combined with other algorithms. A typical representative of the quality evaluation method based on HVS characteristics is the just noticeable difference JND model; Yang Wei et al. combines the visual characteristics of human eyes with the structural similarity of images to obtain an image that conforms to human visual characteristics. A new method of quality evaluation. The method can distinguish image features of different regions in the image and conform to human visual characteristics. 2.1.2.2. Partial reference. Part of the reference method uses only partial information of the original image to estimate the visual perceptual quality of the distorted image. The advantage of the partial reference method is that on the basis of reducing the amount of transmitted data, a better evaluation effect is obtained. The disadvantage is that the algorithm is very sensitive to the extracted features. Feature extraction and feature comparison are the key factors affecting the performance of some reference methods. The main part is to extract the partial information of the original image and the distorted image first, and compare the partial information to evaluate the quality of the distorted image. Part of the reference method can be divided into an original image feature method, a digital watermark based method, and a Wavelet domain statistical model based method. 2.2. Regularization method 2.2.1. Regularization model The regularization method is a commonly used and reliable method for solving ill-posed problems. In the regularizationbased deblurring algorithm, the regularized prior knowledge about the original image f is usually used to solve the following optimal objective function values achieve.
min C k ðf ; hÞ ¼
1 2 k g f h k2 þ kUðf Þ 2
ð1Þ
The first term in formula (1) is the data fidelity term, U(f) is the regular term representing the image prior knowledge, also known as the constraint term or penalty term, and k is the regularization parameter. Regularization parameters are used to control data fidelity and regular item weights. Since the optimal solution of the regularization model is very sensitive to regularization parameters, how to choose appropriate regularization parameters is also part of the research topic. Generally speaking, if the value is too large, the image is too smooth, and the texture and edge structure are lost. When the setting is too small, image blur information is likely to remain, and the smooth region of the image still has noise and the like. For the selected values, the two commonly used standards in linear methods are generalized cross-validation (GCV) and Lcurve; while the non-linear methods are widely used in the SURE (Stein’s unbiased risk estimate) standard. 2.2.2. Classical regularization method In the regularization method, the most classic is the Tikhonov regularization method. It is a linear regularization method whose regular term is generally defined as a Laplacian, the L2 norm. Its mathematical model can be expressed as:
1 2 min u k g f g k2 þ kjLðuÞj2 2 where L (.) is a linear operation.
ð2Þ
4
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
This method generally assumes that the original image is smooth as a priori knowledge of the image. Although the Tikhonov regularization method is fast, it is easy to cause the edge information of the restored image to be lost and ringing. 2.2.3. Iterative regularization method The total variation regularization method is one of the most successful and popular iterative regularization methods. At first, the method was applied to the research of image denoising, and later applied by scholars in other directions in the field of image restoration processing. This method can be defined as follows:
1 2 k g f h k2 þ kjujBVðXÞ min u2BVðXÞ 2
ð3Þ
where k > 0 is the regularization parameter, BVðXÞ is the bounded variation (BV) function in space, and |.| represents the semi-norm of the BV space. Usually the regular term is written as:
jujBVðXÞ ¼ f X jrujdx
ð4Þ
The all-variation model has a piecewise smoothing feature that allows smoothing only in a direction perpendicular to the gradient, so it can well protect the edge information of the image. Full variation regularization is widely used because it eliminates high frequency noise while maintaining good edges. However, the model is very poorly separated from texture and noise; when the regularization parameter is too large, the restored image will be over-smoothed, losing detail or texture, and the regularization parameters are too small, and the restored image still has a lot of noise. 2.3. Sparse non-local regularization weighted coding algorithm There have been many denoising algorithms for single noise such as Gaussian noise, salt and pepper noise, and periodic noise. For example: dictionary learning algorithm based on sparse representation for Gaussian noise, based on Gaussian scale mixed model and non-local filtering algorithm; mean filtering algorithm for salt and pepper noise, adaptive median filtering algorithm, etc.; band rejection filtering algorithm for periodic noise, notch filter algorithm, etc. However, in general, these single noise denoising methods are difficult to achieve the best denoising effect when applied to mixed noise. Therefore, research work on mixed noise is particularly important. In recent years, research methods for mixed noise have been proposed, such as: sparse representation based methods, weighted dictionary learning algorithms, partial differential equation based methods, PCNN based models and wavelet transform based algorithms, etc. Noise has a significant inhibitory effect, especially with the sparse non-local regularization weighted coding (WESNR) method. 2.3.1. Sparse non-local regularization model pffiffiffi pffiffiffi Suppose x 2 RN is an image. The n n sized image block n extracted by the xi ¼ Ri x 2 R from the image, where Ri is a way of extracting a block using a matrix, and Ri x represents a block xi extracted from the i-th position of the image x. According to the sparse representation theory, if xi is sparsely encoded, then xi can be expressed as xi ¼ Uai , and ai is a sparse coding vector with few non-zero atoms, so x can be solved by least squares method:
x¼
X i
!1 RTi R
X
! RTi Uai
ð5Þ
i
For convenience of expression, the above formula is usually rewritten as a in x ¼ Ua, where U is a set of sparse coding vectors
ai . Thus, in the case of Gaussian noise, the sparse coding model can generally be written as:
^ ¼ arg min k y Ua k22 þ kRðaÞ a
ð6Þ
a
where R(a) is a regular term for a and k is a coefficient for a regular term. 2.3.2. Sparse non-local regularization weighted coding model For images contaminated by mixed noise, the distribution of mixed noise is far from Gaussian noise, so the data fidelity term k y Ua k22 in Eq. (6) is not convenient for removing mixed noise. Because the analysis shows that weighting the data fidelity term can make the distribution of the residuals close to the Gaussian distribution, so the sparse coding model is modified into the following weighted sparse coding model: 1
2
a ¼ argmin k W 2 ðy UaÞ k2 þ kRðaÞ
ð7Þ
a
where W is a weighted diagonal matrix whose element Wii on the diagonal is given by:
W ii ¼ expðae2t Þ
ð8Þ
e ¼ ½e1 ; e2 ; :::; eN ¼ y Ua
ð9Þ
On the other hand, the regular term R(a) is used to describe the a priori of natural images. The a priori widely used for image denoising generally has two: local sparse and non-local selfsimilarity (NSS). However, the sparsely encoded local sparse solution a is usually represented by the l1 norm, and the non-local selfsimilarity (NSS) is usually described by the minimum error of the prediction block and its similar blocks, thus combining the two a priorities into a sparse regular term. The final sparse non-local regularization weighted coding (WESNR) model is:
( 2 ) X 1 2 a ¼ argmin W ðy UaÞ þ k ai ui a 2
i
ð10Þ
1
In order to solve the model (10), the weighting matrix Wa can be determined first, because the relationship between Wii and ei is inversely proportional, and the pixels contaminated by the impulse noise are sparsely represented, there is a large coding residual, and the weight is not polluted by the impulse noise. Approaching l, so we set both Wii 2 (0,1]. 3. Experiments 3.1. Data source For the target simulation, the on-board measurement platform travels at a speed of 60 km/h on the highway in a straight line. The platform is accompanied by a random vibration with an amplitude of 0.5 m during driving; the focal length of the imaging optical system is 0.1 m; and the image intensifier has a quantum efficiency of 10%, the noise factor is 1.4; the distance between the start and end of the gate strobe is 950 m and 1050 m respectively; the constant gain of channel 1 is 300, the gain of channel 2 is linearly changed from 50 to 500 from the start to the end of the gate; the CCD pixel spacing is 200 lm; A total of 25 distance images with ranging error caused by shot noise are generated on the ground along the route; the pixels in each CCD field of view obtain an average of 1000 photons, and the average signal-to-noise ratio of the signal received by the CCD can be calculated to be 8.5. The average error of calculating the single-frame distance image is 9.1 m. Obtain the original registration image as shown in Fig. 1.
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
5
Fig. 1. Original registration image.
3.2. Extraction and identification of parameters Parameter extraction and identification. The main parameter extraction is number of pixels obtained based on different parts of the video image characteristic parameter extraction, and obtained a corresponding image based on the parameter change for the better the relationship between the actual pixel size. And the video object identification is to use the extracted characteristic parameter of the target video, embodiments of the automatic recognition of the target by different methods. The computer automatically recognizes a key factor in determining good and bad effects parameter extraction. 4. Discussion 4.1. Lidar registration image quality improvement effect For the sake of simplicity, the parameter r controls the number of iterations. In order to balance the iteration number with the denoising effect, after a number of experiments, finally set f = 0.003: the parameter a of the control weight decay speed in Eq. (8) is set to 0.0007. The parameter k is initially set to a smaller number of 0.0006 to weaken the role of the non-local regular term in the algorithm; from the second cycle, the impulse noise is largely attenuated, at this time non-local similarity search The ability of the block becomes very strong, we set k to a relatively large value to remove Gaussian noise. If the standard deviation of Gaussian noise is greater than 10, we set k = 1; otherwise, set k = 0.5, the purpose is to suppress Gaussian noise, so that the image retains the detail as much as possible. As shown in Figs. 2 and 3, it is easy to see from the experimental results that the improved algorithm can make
full use of the prior knowledge of sparsity and non-local similarity in the image, and has good denoising ability for high-intensity mixed noise. 4.2. Adaptive iterative termination strategy analysis based on registration image In the first iteration, it is evident that there is a clear image outline and key image edge information in the registration image. In the 20-step iteration, some less obvious structural information can still be seen in the lower half of the image. As the number of iterations advances, the image structure information gradually disappears, that is, the difference between the pixels of the registration image gradually becomes smaller. When the number of iteration steps is 50, it is no longer possible to visually observe information about the image content in the registration image. Explain that the quality of the image after iteration is very close to the optimal value. 4 is a histogram corresponding to the luminance values of the registered image pixel points in the above three iteration steps. It can be seen from Fig. 4 that as the number of iterations increases the distribution range of the luminance values in the residual image, the contraction gradually narrows, indicating that the intermediate estimated image is gradually approaching the original image. 4.3. Time efficiency analysis of the algorithm The improved NCSR algorithm increases the execution time of the other two aspects: one is the execution time of the IE-IQF algorithm, and the other is the deblurring metric execution time based on the statistical features of the residual image. In order to test the effect of algorithm optimization on the execution time of the NCSR
Fig. 2. Image enhancement effect (1).
6
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
Fig. 3. Image enhancement effect (2).
Fig. 4. Histogram of the resulting registration image under key iteration steps.
Fig. 5. NCSR algorithm improvement before and after execution time (unit: second).
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634
algorithm, an experimental analysis was performed on an image of size 256 256. The experimental results are shown in Fig. 5. As can be seen from Fig. 5, the improved NCSR algorithm execution efficiency is significantly improved, an average increase of 31.3%. Specifically, in the case of uniform, Gaussian, and defocused blur, the average is increased by 38.2%, 29.8%, and 25.9%, respectively, and the best case is even increased by about 50%. Although the estimation of the degree of image blurring has affected the execution efficiency, the worst case has an increase of more than 10%. It can be proved that the iterative termination optimization strategy proposed in this paper has good practicability and feasibility. 5. Conclusions (1) For the mixed noise, an improved non-locally centralized sparse representation model (MNCSR) is proposed. The improved algorithm is divided into two phases. In the first stage, adaptive median filtering is used as preprocessing to remove the sparse noise part of the mixed noise. In the second stage, the original NCSR algorithm is used to remove the residual noise in the upper stage. The experimental results show that the proposed MNCSR algorithm improves the ability of the NCSR algorithm to deal with sparse noise. (2) Based on the two-channel gain modulation imaging lidar ranging error model, a method of integrating multi-frame distance image to improve the ranging accuracy is proposed by applying weighted registration and weighted superposition method. Simulation and experiments show that the method has higher registration accuracy and reduces the distance error to about 1/n of a single frame image. The method can obtain the improvement of the ranging accuracy by the image processing method without increasing the light source intensity, improving the radar receiving aperture and switching to the high-performance detector. (3) Through the imaging simulation of the simulated lattice target, the resolution of the synthetic aperture laser radar fold line scanning is analyzed. The results show that the fused image obtained by the synthetic aperture laser radar fold line scanning method and the traditional synthetic aperture laser radar image Compared with the image resolution, the resolution of the original aperture of the synthetic aperture laser radar is increased to the level consistent with the original orientation, which improves the image quality.
Declaration of Competing Interest The authors declared that there is no conflict of interest. References [1] H. Yu, E. Li, W. Gong, et al., Structured image reconstruction for threedimensional ghost imaging lidar, Opt. Exp. 23 (11) (2015) 14541–14551. [2] L. Yuan, H. Yan, S. Hu, et al., Three-dimensional video imaging lidar system based on acousto-optic laser scanning, Chin. J. Lasers 41 (8) (2014) 0802005. [3] Z. Tian, Z. Cui, L. Zhang, et al., Control and image processing for streak tube imaging lidar based on VB and MATLAB, Chin. Opt. Lett. 12 (6) (2014) 67–70. [4] A. Muttini, V. Russo, E. Rossi, et al., An integrated multi-medial approach to cultural heritage conservation and documentation: from remotely-sensed lidar imaging to historical archive data, in: SPIE Remote Sensing, 2015, p. 13. [5] J. Sun, J. Liu, W. Qi, Experimental research on multiple-slit streak tube imaging lidar, Optik – Int. J. Light Electr. Opt. 126 (21) (2015) 3181–3184. [6] C. Deng, L. Pan, C. Wang, et al., Performance analysis of ghost imaging lidar in background light environment, Photo. Res. 5 (5) (2017) 431–435. [7] L. Xu, Y. Zhang, Y. Zhang, et al., Four Gm-APDs photon counting imaging lidar to improve detection performances, Infrar. Laser Eng. 44 (9) (2015).
7
[8] G. Jian, J. Sun, W. Qi, Experiments on the range resolution measurement of a slit Streak Tube Imaging Lidar, Optik – Int. J. Light Electr. Opt. 126 (21) (2015) 3084–3087. [9] J. Gao, J. Sun, Q. Wang, Experiments of ocean surface waves and underwater target detection imaging using a slit Streak Tube Imaging Lidar, Optik – Int. J. Light Electr. Opt. 125 (18) (2014) 5199–5201. [10] G. Ye, R. Fan, Z. Chen, et al., Range accuracy analysis of streak tube imaging lidar systems, Opt. Commun. 360 (2016) 7–14. [11] W. Xiang, C. Miao, L. Lu, et al., Accelerated extraction technology research on damaged building information by earthquake based on LiDAR image, Commun. Comput. Inform. Sci. 482 (2015) 117–132. [12] Y. Ling, G. Gu, W. He, et al., Adaptive target profile acquiring method for photon counting 3-D imaging lidar, IEEE Photo. J. 8 (6) (2016) 1–10. [13] F.Q. Qu, C. Shu, J.H. Tu, Modeling and simulation research of lidar tomography imaging, Appl. Mech. Mater. 721 (2014) 468–471. [14] J.B. Blair, D. Rabine, M.A. Hofton, et al., Geodetic Imaging Lidar: Applications for high-accuracy, large area mapping with NASA’s upcoming high-altitude waveform-based airborne laser altimetry Facility, in: AGU Fall Meeting, 2015, p. 07. [15] C. Wang, X. Mei, P. Long, et al., Airborne near infrared three-dimensional ghost imaging LiDAR via sparsity constraint, Remote Sens. 10 (5) (2018) 732. [16] C. Wang, X. Lee, T. Cui, et al., Reducing the influence of direct reflection on return signal detection in a 3D imaging lidar system by rotating the polarizing beam splitter., Appl. Opt. 55 (7) (2016) 1559. [17] S.C. Henry, L.M. Zurk, D.D. Duncan, Three-dimensional broadband terahertz synthetic aperture imaging, Opt. Eng. 51 (9) (2012) 1603. [18] B. Li, D. Wang, H. Huang, et al., Synthetic aperture in terahertz in-line digital holography for resolution enhancement, Appl. Opt. 55 (3) (2016) A43. [19] Ana Maria Pusßcasßu, Special phenomena of the shadow region in the high resolution synthetic aperture radar image due to synthetic aperture, J. Infrar. Millim. Terahertz Waves 33 (10) (2012) 1052–1070. [20] L.M. Zurk, S.C. Henry, 3D terahertz synthetic aperture imaging of objects with arbitrary boundaries, Proc. SPIE – Int. Soc. Opt. Eng. 8846 (2013) 49–52. [21] L.R. Liu, N. Xu, Z. Yan, et al., Two-dimensional imaging experiment of scaledown synthetic aperture laser radar, Acta Opt. Sin. 29 (7) (2009) 2030– 2032. [22] Y.W. Liu, L.R. Liu, C.H. Zhou, et al., Experimental study of non-volatile holographic storage of doubly- and triply-doped lithium niobate crystals, Opt. Commun. 181 (1) (2001) 47–52. [23] L. Guo, M. Xing, X. Zeng, et al., Inverse synthetic aperture lidar imaging of indoor real data, Infrar. Laser Eng. 40 (4) (2011) 637–642. [24] Xu. Mingliang, Lu. Fuhai Chen, Chen Shen Li, Pei Lv, Bing Zhou, Rongrong Ji, Bio-inspired deep attribute learning towards facial aesthetic prediction, IEEE Trans. Affect. Comput. (2018), https://doi.org/10.1109/TAFFC.2018. 2868651. [25] Masud Ibn Afjal, Md Al Mamun, Md Palash Uddin, Band reordering heuristics for lossless satellite image compression with 3D CALIC and CCSDS, J. Vis. Commun. Image Repres. 59 (2019) 514–526. [26] Junwei Han, Dingwen Zhang, Gong Cheng, Lei Guo, Jinchang Ren, Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning, IEEE Trans. Geosci. Remote Sens. 53 (6) (2015) 3325–3337. [27] Yildiray Anagün, Sahin Isik, Erol Seke, SRLibrary: Comparing different loss function for super-resolution over various convolutional architectures, J. Vis. Commun. Image Repres. 61 (2019) 178–187. [28] Xu. Mingliang, Jiejie Zhu, Pei Lv, Bing Zhou, Marshall Tappen, Rongrong Ji, Learning-based shadow recognition and removal from monochromatic natural images, IEEE Trans. Image Process. 26 (12) (2017) 5811–5824. [29] D. Nair, P. Sankaran, Color image dehazing using surround filter and dark channel prior, J. Vis. Commun. Image Repres. 50 (2017) 9–15. [30] R. Pramanik, S. Bag, Shape decomposition-based handwritten compound character recognition for bangla ocr, J. Vis. Commun. Image Repres. 50 (2018) 123–134. [31] Junwei Han, Dingwen Zhang, Hu Xintao, Lei Guo, Jinchang Ren, Feng Wu Background prior-based salient object detection via deep reconstruction residual, IEEE Trans. Circ. Syst. Video Technol. 25 (8) (2015) 1309–1321. [32] Junwei Han, Xiang Ji, Hu Xintao, Dajiang Zhu, Kaiming Li, Xi Jiang, Guangbin Cui, Lei Guo, Tianming Liu, Representing and retrieving video shots in humancentric brain imaging space, IEEE Trans. Image Process. 22 (7) (2013) 2723– 2736. [33] Xu Mingliang, Mingyuan Li, Xu Weiwei, Zhigang Deng, Yin Yang, Kun Zhou, Interactive mechanism modeling from multi-view images, ACM Trans. Graph. 35 (6) (2016), Article 236. [34] C. Bai, J.N. Chen, L. Huang, K. Kpalma, S. Chen, Saliency-based multi-feature modeling for semantic image retrieval, J. Vis. Commun. Image Repres. 50 (2018) 199–204. [35] Junwei Han, King Ngi Ngan, Mingjing Li, Hong-Jiang Zhang, Unsupervised extraction of visual attention objects in color images, IEEE Trans. Circ. Syst. Video Technol. 16 (1) (2006) 141–145. [36] Xu Mingliang, Wu Yunpeng, Yangdong Ye, Illes Farkas, Hao Jiang, Zhigang Deng, Collective crowd formation transform with mutual information-based runtime feedback, Comput. Graph. Forum 34 (1) (2015) 60–73.
8
W. Zhang et al. / J. Vis. Commun. Image R. 64 (2019) 102634 Wanyi Zhang was born in Changchun, Jilin, P.R. China, in 1985. She received the Master degree from Changchun University of Science and Technology, P.R. China. Now, she studies in School of OptoElectronic Engineering, Changchun University of Science and Technology. Her research interests include contemporary optical measurement technology and optical image processing. E-mail: [email protected].
Xiuhua Fu was born in Binzhou, Shandong, P.R. China, in 1963. She received the Doctorate from Changchun University of Science and Technology, P.R. China. Now, she works in School of OptoElectronic Engineering, Changchun University of Science and Technology. Her research interests include cloud optical filming and optical manufacturing technology. E-mail: [email protected].
Chunyang Wang was born in Changchun, Jilin, P.R. China, in 1964. She received the Doctorate from Jilin University, P.R. China. Now, she works in School of Electronics and Information Engineering, Changchun University of Science and Technology. Her research interests include photoelectric information processing and detection technology. E-mail: [email protected].